2023-12-02T00:16:13.038 INFO:root:teuthology version: 0.0.1.dev214+gf2de108 2023-12-02T00:16:13.039 DEBUG:teuthology.run:Teuthology command: teuthology --name yuriw-2023-12-01_17:57:53-fs-main-distro-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/pool-perm} --archive /home/teuthworker/archive/yuriw-2023-12-01_17:57:53-fs-main-distro-default-smithi/7474581 --owner scheduled_yuriw@teuthology --verbose -- /home/teuthworker/archive/yuriw-2023-12-01_17:57:53-fs-main-distro-default-smithi/7474581/orig.config.yaml 2023-12-02T00:16:13.113 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-02T00:16:13.205 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-12-01_17:57:53-fs-main-distro-default-smithi/7474581 branch: main description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/pool-perm} email: yweinste@redhat.com first_in_suite: false job_id: '7474581' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2023-12-01_17:57:53-fs-main-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: main ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: bed7681480ccb0192b91a35e691658867e1e96b7 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: bed7681480ccb0192b91a35e691658867e1e96b7 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: main sha1: bed7681480ccb0192b91a35e691658867e1e96b7 owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 1991 sha1: bed7681480ccb0192b91a35e691658867e1e96b7 sleep_before_teardown: 0 subset: 111/12000 suite: fs suite_branch: main suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_bed7681480ccb0192b91a35e691658867e1e96b7/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: bed7681480ccb0192b91a35e691658867e1e96b7 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi046.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKZy/FA+EREo0z8VJi9Rp6/BsV8NFlDNytLV8JJkAdXZb9ZW5B/l6lq8qWNioT2ApsmOClzljsrv9/+hKUss4Ag= smithi162.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD+84dNHyVg7BDh7SUQ8Op99UGvFszg9wn8IU0wSp3Htbhuzt5b5g5WRjXDiOjAh4yuQB30H0+XZrpmNFRD4KmQ= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_pool_perm teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: f2de108beb0a6dde7f3e6c96039071828a1f6269 timestamp: 2023-12-01_17:57:53 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.232157 2023-12-02T00:16:13.206 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_bed7681480ccb0192b91a35e691658867e1e96b7/qa; will attempt to use it 2023-12-02T00:16:13.207 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_bed7681480ccb0192b91a35e691658867e1e96b7/qa/tasks 2023-12-02T00:16:13.207 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-12-02T00:16:13.278 INFO:teuthology.task.internal:Checking packages... 2023-12-02T00:16:13.308 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'bed7681480ccb0192b91a35e691658867e1e96b7' 2023-12-02T00:16:13.308 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-12-02T00:16:13.308 INFO:teuthology.packaging:ref: None 2023-12-02T00:16:13.308 INFO:teuthology.packaging:tag: None 2023-12-02T00:16:13.308 INFO:teuthology.packaging:branch: main 2023-12-02T00:16:13.308 INFO:teuthology.packaging:sha1: bed7681480ccb0192b91a35e691658867e1e96b7 2023-12-02T00:16:13.308 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=main 2023-12-02T00:16:13.548 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-7660.gb27938f5 2023-12-02T00:16:13.548 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-12-02T00:16:13.678 INFO:teuthology.task.internal:no buildpackages task found 2023-12-02T00:16:13.678 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-12-02T00:16:13.697 INFO:teuthology.task.internal:Saving configuration 2023-12-02T00:16:13.715 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-12-02T00:16:13.731 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-02T00:16:13.751 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi046.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-12-01_17:57:53-fs-main-distro-default-smithi/7474581', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-02 00:11:05.202741', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKZy/FA+EREo0z8VJi9Rp6/BsV8NFlDNytLV8JJkAdXZb9ZW5B/l6lq8qWNioT2ApsmOClzljsrv9/+hKUss4Ag='} 2023-12-02T00:16:13.771 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi162.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-12-01_17:57:53-fs-main-distro-default-smithi/7474581', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-02 00:11:05.204051', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD+84dNHyVg7BDh7SUQ8Op99UGvFszg9wn8IU0wSp3Htbhuzt5b5g5WRjXDiOjAh4yuQB30H0+XZrpmNFRD4KmQ='} 2023-12-02T00:16:13.771 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-12-02T00:16:13.781 INFO:teuthology.task.internal:roles: ubuntu@smithi046.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-12-02T00:16:13.782 INFO:teuthology.task.internal:roles: ubuntu@smithi162.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-12-02T00:16:13.782 INFO:teuthology.run_tasks:Running task console_log... 2023-12-02T00:16:13.857 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f4d9143d040>, signals=[15]) 2023-12-02T00:16:13.857 INFO:teuthology.run_tasks:Running task internal.connect... 2023-12-02T00:16:13.867 INFO:teuthology.task.internal:Opening connections... 2023-12-02T00:16:13.867 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi046.front.sepia.ceph.com 2023-12-02T00:16:13.869 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-02T00:16:14.007 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi162.front.sepia.ceph.com 2023-12-02T00:16:14.008 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-02T00:16:14.090 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-12-02T00:16:14.148 DEBUG:teuthology.orchestra.run.smithi046:> uname -m 2023-12-02T00:16:14.166 INFO:teuthology.orchestra.run.smithi046.stdout:x86_64 2023-12-02T00:16:14.167 DEBUG:teuthology.orchestra.run.smithi046:> cat /etc/os-release 2023-12-02T00:16:14.227 INFO:teuthology.orchestra.run.smithi046.stdout:NAME="CentOS Stream" 2023-12-02T00:16:14.227 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION="9" 2023-12-02T00:16:14.227 INFO:teuthology.orchestra.run.smithi046.stdout:ID="centos" 2023-12-02T00:16:14.227 INFO:teuthology.orchestra.run.smithi046.stdout:ID_LIKE="rhel fedora" 2023-12-02T00:16:14.227 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION_ID="9" 2023-12-02T00:16:14.227 INFO:teuthology.orchestra.run.smithi046.stdout:PLATFORM_ID="platform:el9" 2023-12-02T00:16:14.227 INFO:teuthology.orchestra.run.smithi046.stdout:PRETTY_NAME="CentOS Stream 9" 2023-12-02T00:16:14.227 INFO:teuthology.orchestra.run.smithi046.stdout:ANSI_COLOR="0;31" 2023-12-02T00:16:14.227 INFO:teuthology.orchestra.run.smithi046.stdout:LOGO="fedora-logo-icon" 2023-12-02T00:16:14.228 INFO:teuthology.orchestra.run.smithi046.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-12-02T00:16:14.228 INFO:teuthology.orchestra.run.smithi046.stdout:HOME_URL="https://centos.org/" 2023-12-02T00:16:14.228 INFO:teuthology.orchestra.run.smithi046.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-02T00:16:14.228 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-12-02T00:16:14.228 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-12-02T00:16:14.228 INFO:teuthology.lock.ops:Updating smithi046.front.sepia.ceph.com on lock server 2023-12-02T00:16:14.275 DEBUG:teuthology.orchestra.run.smithi162:> uname -m 2023-12-02T00:16:14.292 INFO:teuthology.orchestra.run.smithi162.stdout:x86_64 2023-12-02T00:16:14.293 DEBUG:teuthology.orchestra.run.smithi162:> cat /etc/os-release 2023-12-02T00:16:14.350 INFO:teuthology.orchestra.run.smithi162.stdout:NAME="CentOS Stream" 2023-12-02T00:16:14.350 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION="9" 2023-12-02T00:16:14.350 INFO:teuthology.orchestra.run.smithi162.stdout:ID="centos" 2023-12-02T00:16:14.350 INFO:teuthology.orchestra.run.smithi162.stdout:ID_LIKE="rhel fedora" 2023-12-02T00:16:14.350 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION_ID="9" 2023-12-02T00:16:14.350 INFO:teuthology.orchestra.run.smithi162.stdout:PLATFORM_ID="platform:el9" 2023-12-02T00:16:14.350 INFO:teuthology.orchestra.run.smithi162.stdout:PRETTY_NAME="CentOS Stream 9" 2023-12-02T00:16:14.350 INFO:teuthology.orchestra.run.smithi162.stdout:ANSI_COLOR="0;31" 2023-12-02T00:16:14.350 INFO:teuthology.orchestra.run.smithi162.stdout:LOGO="fedora-logo-icon" 2023-12-02T00:16:14.350 INFO:teuthology.orchestra.run.smithi162.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-12-02T00:16:14.351 INFO:teuthology.orchestra.run.smithi162.stdout:HOME_URL="https://centos.org/" 2023-12-02T00:16:14.351 INFO:teuthology.orchestra.run.smithi162.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-02T00:16:14.351 INFO:teuthology.orchestra.run.smithi162.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-12-02T00:16:14.351 INFO:teuthology.orchestra.run.smithi162.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-12-02T00:16:14.351 INFO:teuthology.lock.ops:Updating smithi162.front.sepia.ceph.com on lock server 2023-12-02T00:16:14.387 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-12-02T00:16:14.464 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-12-02T00:16:14.474 INFO:teuthology.task.internal:Checking for old test directory... 2023-12-02T00:16:14.475 DEBUG:teuthology.orchestra.run.smithi046:> test '!' -e /home/ubuntu/cephtest 2023-12-02T00:16:14.477 DEBUG:teuthology.orchestra.run.smithi162:> test '!' -e /home/ubuntu/cephtest 2023-12-02T00:16:14.493 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-12-02T00:16:14.505 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-12-02T00:16:14.505 DEBUG:teuthology.orchestra.run.smithi046:> test -z $(ls -A /var/lib/ceph) 2023-12-02T00:16:14.535 DEBUG:teuthology.orchestra.run.smithi162:> test -z $(ls -A /var/lib/ceph) 2023-12-02T00:16:14.629 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-12-02T00:16:14.722 INFO:teuthology.run_tasks:Running task kernel... 2023-12-02T00:16:14.781 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-12-02T00:16:14.781 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-12-02T00:16:14.781 DEBUG:teuthology.orchestra.run.smithi046:> test -f /run/.containerenv -o -f /.dockerenv 2023-12-02T00:16:14.782 DEBUG:teuthology.orchestra.run.smithi162:> test -f /run/.containerenv -o -f /.dockerenv 2023-12-02T00:16:14.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-02T00:16:14.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-02T00:16:14.802 DEBUG:teuthology.orchestra.run.smithi046:> uname -r 2023-12-02T00:16:14.802 DEBUG:teuthology.orchestra.run.smithi162:> uname -r 2023-12-02T00:16:14.859 INFO:teuthology.orchestra.run.smithi162.stdout:5.14.0-361.el9.x86_64 2023-12-02T00:16:14.859 INFO:teuthology.task.kernel:Running kernel on smithi162: 5.14.0-361.el9.x86_64 2023-12-02T00:16:14.859 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum install -y kernel 2023-12-02T00:16:14.860 INFO:teuthology.orchestra.run.smithi046.stdout:5.14.0-361.el9.x86_64 2023-12-02T00:16:14.860 INFO:teuthology.task.kernel:Running kernel on smithi046: 5.14.0-361.el9.x86_64 2023-12-02T00:16:14.860 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum install -y kernel 2023-12-02T00:16:17.225 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS-9 - AppStream 10 MB/s | 6.4 MB 00:00 2023-12-02T00:16:17.256 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS-9 - AppStream 8.5 MB/s | 6.4 MB 00:00 2023-12-02T00:16:18.809 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS-9 - BaseOS 4.1 MB/s | 2.0 MB 00:00 2023-12-02T00:16:18.923 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS-9 - BaseOS 3.5 MB/s | 2.0 MB 00:00 2023-12-02T00:16:20.502 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - BaseOS 5.8 MB/s | 7.9 MB 00:01 2023-12-02T00:16:21.606 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - BaseOS 3.5 MB/s | 7.9 MB 00:02 2023-12-02T00:16:23.050 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - AppStream 13 MB/s | 18 MB 00:01 2023-12-02T00:16:24.641 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - AppStream 9.3 MB/s | 18 MB 00:01 2023-12-02T00:16:28.278 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - CRB 4.0 MB/s | 5.8 MB 00:01 2023-12-02T00:16:29.273 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - CRB 4.6 MB/s | 5.8 MB 00:01 2023-12-02T00:16:30.731 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - Extras packages 44 kB/s | 15 kB 00:00 2023-12-02T00:16:30.826 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - Extras packages 11 kB/s | 15 kB 00:01 2023-12-02T00:16:31.860 INFO:teuthology.orchestra.run.smithi162.stdout:Extra Packages for Enterprise Linux 20 MB/s | 20 MB 00:00 2023-12-02T00:16:32.455 INFO:teuthology.orchestra.run.smithi046.stdout:Extra Packages for Enterprise Linux 13 MB/s | 20 MB 00:01 2023-12-02T00:16:36.616 INFO:teuthology.orchestra.run.smithi162.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2023-12-02T00:16:37.419 INFO:teuthology.orchestra.run.smithi046.stdout:lab-extras 42 kB/s | 1.7 kB 00:00 2023-12-02T00:16:38.897 INFO:teuthology.orchestra.run.smithi162.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2023-12-02T00:16:38.897 INFO:teuthology.orchestra.run.smithi162.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2023-12-02T00:16:39.008 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2023-12-02T00:16:39.020 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2023-12-02T00:16:39.020 INFO:teuthology.orchestra.run.smithi162.stdout: Package Architecture Version Repository Size 2023-12-02T00:16:39.020 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2023-12-02T00:16:39.020 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2023-12-02T00:16:39.020 INFO:teuthology.orchestra.run.smithi162.stdout: kernel x86_64 5.14.0-388.el9 baseos 5.6 M 2023-12-02T00:16:39.020 INFO:teuthology.orchestra.run.smithi162.stdout:Installing dependencies: 2023-12-02T00:16:39.020 INFO:teuthology.orchestra.run.smithi162.stdout: kernel-core x86_64 5.14.0-388.el9 baseos 20 M 2023-12-02T00:16:39.020 INFO:teuthology.orchestra.run.smithi162.stdout: kernel-modules x86_64 5.14.0-388.el9 baseos 38 M 2023-12-02T00:16:39.020 INFO:teuthology.orchestra.run.smithi162.stdout: kernel-modules-core x86_64 5.14.0-388.el9 baseos 32 M 2023-12-02T00:16:39.021 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-02T00:16:39.021 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2023-12-02T00:16:39.021 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2023-12-02T00:16:39.021 INFO:teuthology.orchestra.run.smithi162.stdout:Install 4 Packages 2023-12-02T00:16:39.021 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-02T00:16:39.021 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 96 M 2023-12-02T00:16:39.021 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 121 M 2023-12-02T00:16:39.022 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2023-12-02T00:16:39.792 INFO:teuthology.orchestra.run.smithi162.stdout:(1/4): kernel-5.14.0-388.el9.x86_64.rpm 14 MB/s | 5.6 MB 00:00 2023-12-02T00:16:39.851 INFO:teuthology.orchestra.run.smithi046.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2023-12-02T00:16:39.851 INFO:teuthology.orchestra.run.smithi046.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2023-12-02T00:16:39.943 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2023-12-02T00:16:39.947 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2023-12-02T00:16:39.947 INFO:teuthology.orchestra.run.smithi046.stdout: Package Architecture Version Repository Size 2023-12-02T00:16:39.947 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2023-12-02T00:16:39.947 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2023-12-02T00:16:39.947 INFO:teuthology.orchestra.run.smithi046.stdout: kernel x86_64 5.14.0-388.el9 baseos 5.6 M 2023-12-02T00:16:39.947 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2023-12-02T00:16:39.947 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-core x86_64 5.14.0-388.el9 baseos 20 M 2023-12-02T00:16:39.947 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-modules x86_64 5.14.0-388.el9 baseos 38 M 2023-12-02T00:16:39.947 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-modules-core x86_64 5.14.0-388.el9 baseos 32 M 2023-12-02T00:16:39.947 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:16:39.947 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2023-12-02T00:16:39.948 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2023-12-02T00:16:39.948 INFO:teuthology.orchestra.run.smithi046.stdout:Install 4 Packages 2023-12-02T00:16:39.948 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:16:39.948 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 96 M 2023-12-02T00:16:39.948 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 121 M 2023-12-02T00:16:39.948 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2023-12-02T00:16:40.467 INFO:teuthology.orchestra.run.smithi162.stdout:(2/4): kernel-core-5.14.0-388.el9.x86_64.rpm 19 MB/s | 20 MB 00:01 2023-12-02T00:16:40.859 INFO:teuthology.orchestra.run.smithi162.stdout:(3/4): kernel-modules-5.14.0-388.el9.x86_64.rpm 26 MB/s | 38 MB 00:01 2023-12-02T00:16:41.259 INFO:teuthology.orchestra.run.smithi162.stdout:(4/4): kernel-modules-core-5.14.0-388.el9.x86_6 22 MB/s | 32 MB 00:01 2023-12-02T00:16:41.259 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2023-12-02T00:16:41.259 INFO:teuthology.orchestra.run.smithi162.stdout:Total 43 MB/s | 96 MB 00:02 2023-12-02T00:16:42.118 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2023-12-02T00:16:42.498 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2023-12-02T00:16:42.499 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2023-12-02T00:16:42.662 INFO:teuthology.orchestra.run.smithi046.stdout:(1/4): kernel-5.14.0-388.el9.x86_64.rpm 2.2 MB/s | 5.6 MB 00:02 2023-12-02T00:16:43.819 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2023-12-02T00:16:43.820 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2023-12-02T00:16:46.730 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2023-12-02T00:16:47.364 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : kernel-modules-core-5.14.0-388.el9.x86_64 1/4 2023-12-02T00:16:47.500 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : kernel-core-5.14.0-388.el9.x86_64 2/4 2023-12-02T00:16:49.853 INFO:teuthology.orchestra.run.smithi046.stdout:(2/4): kernel-core-5.14.0-388.el9.x86_64.rpm 2.1 MB/s | 20 MB 00:09 2023-12-02T00:16:50.287 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: kernel-core-5.14.0-388.el9.x86_64 2/4 2023-12-02T00:16:50.467 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : kernel-modules-5.14.0-388.el9.x86_64 3/4 2023-12-02T00:16:51.428 INFO:teuthology.orchestra.run.smithi046.stdout:(3/4): kernel-modules-5.14.0-388.el9.x86_64.rpm 3.4 MB/s | 38 MB 00:11 2023-12-02T00:16:55.074 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: kernel-modules-5.14.0-388.el9.x86_64 3/4 2023-12-02T00:16:55.273 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : kernel-5.14.0-388.el9.x86_64 4/4 2023-12-02T00:16:58.661 INFO:teuthology.orchestra.run.smithi046.stdout:(4/4): kernel-modules-core-5.14.0-388.el9.x86_6 2.0 MB/s | 32 MB 00:15 2023-12-02T00:16:58.661 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2023-12-02T00:16:58.661 INFO:teuthology.orchestra.run.smithi046.stdout:Total 5.1 MB/s | 96 MB 00:18 2023-12-02T00:16:59.552 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2023-12-02T00:16:59.883 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: kernel-modules-core-5.14.0-388.el9.x86_64 4/4 2023-12-02T00:16:59.971 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2023-12-02T00:16:59.971 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2023-12-02T00:17:01.343 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2023-12-02T00:17:01.343 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2023-12-02T00:17:04.140 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2023-12-02T00:17:04.828 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : kernel-modules-core-5.14.0-388.el9.x86_64 1/4 2023-12-02T00:17:04.952 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : kernel-core-5.14.0-388.el9.x86_64 2/4 2023-12-02T00:17:07.878 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: kernel-core-5.14.0-388.el9.x86_64 2/4 2023-12-02T00:17:08.053 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : kernel-modules-5.14.0-388.el9.x86_64 3/4 2023-12-02T00:17:13.020 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: kernel-modules-5.14.0-388.el9.x86_64 3/4 2023-12-02T00:17:13.145 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : kernel-5.14.0-388.el9.x86_64 4/4 2023-12-02T00:17:18.122 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: kernel-modules-core-5.14.0-388.el9.x86_64 4/4 2023-12-02T00:17:28.853 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: kernel-core-5.14.0-388.el9.x86_64 4/4 2023-12-02T00:17:28.895 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: kernel-modules-5.14.0-388.el9.x86_64 4/4 2023-12-02T00:17:29.344 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: kernel-5.14.0-388.el9.x86_64 4/4 2023-12-02T00:17:29.344 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : kernel-5.14.0-388.el9.x86_64 1/4 2023-12-02T00:17:29.344 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : kernel-core-5.14.0-388.el9.x86_64 2/4 2023-12-02T00:17:29.344 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : kernel-modules-5.14.0-388.el9.x86_64 3/4 2023-12-02T00:17:29.763 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : kernel-modules-core-5.14.0-388.el9.x86_64 4/4 2023-12-02T00:17:29.763 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-02T00:17:29.763 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2023-12-02T00:17:29.763 INFO:teuthology.orchestra.run.smithi162.stdout: kernel-5.14.0-388.el9.x86_64 2023-12-02T00:17:29.763 INFO:teuthology.orchestra.run.smithi162.stdout: kernel-core-5.14.0-388.el9.x86_64 2023-12-02T00:17:29.764 INFO:teuthology.orchestra.run.smithi162.stdout: kernel-modules-5.14.0-388.el9.x86_64 2023-12-02T00:17:29.764 INFO:teuthology.orchestra.run.smithi162.stdout: kernel-modules-core-5.14.0-388.el9.x86_64 2023-12-02T00:17:29.764 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-02T00:17:29.764 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2023-12-02T00:17:30.231 DEBUG:teuthology.orchestra.run.smithi162:> rpm -q kernel | sort -rV | head -n 1 2023-12-02T00:17:30.385 INFO:teuthology.orchestra.run.smithi162.stdout:kernel-5.14.0-388.el9.x86_64 2023-12-02T00:17:30.386 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-388.el9.x86_64 2023-12-02T00:17:30.386 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-388.el9.x86_64 2023-12-02T00:17:30.386 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-12-02T00:17:30.386 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-12-02T00:17:30.386 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2023-12-02T00:17:30.386 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi162.front.sepia.ceph.com, path=None, version=distro) 2023-12-02T00:17:30.386 DEBUG:teuthology.orchestra.run.smithi162:> rpm -q kernel | sort -rV | head -n 1 2023-12-02T00:17:30.505 INFO:teuthology.orchestra.run.smithi162.stdout:kernel-5.14.0-388.el9.x86_64 2023-12-02T00:17:30.505 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-388.el9.x86_64 2023-12-02T00:17:30.505 DEBUG:teuthology.orchestra.run.smithi162:> sudo rpm -qi grub2-tools 2023-12-02T00:17:30.549 INFO:teuthology.orchestra.run.smithi162.stdout:Name : grub2-tools 2023-12-02T00:17:30.550 INFO:teuthology.orchestra.run.smithi162.stdout:Epoch : 1 2023-12-02T00:17:30.550 INFO:teuthology.orchestra.run.smithi162.stdout:Version : 2.06 2023-12-02T00:17:30.550 INFO:teuthology.orchestra.run.smithi162.stdout:Release : 68.el9 2023-12-02T00:17:30.550 INFO:teuthology.orchestra.run.smithi162.stdout:Architecture: x86_64 2023-12-02T00:17:30.550 INFO:teuthology.orchestra.run.smithi162.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-12-02T00:17:30.550 INFO:teuthology.orchestra.run.smithi162.stdout:Group : Unspecified 2023-12-02T00:17:30.550 INFO:teuthology.orchestra.run.smithi162.stdout:Size : 8264254 2023-12-02T00:17:30.550 INFO:teuthology.orchestra.run.smithi162.stdout:License : GPLv3+ 2023-12-02T00:17:30.550 INFO:teuthology.orchestra.run.smithi162.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-12-02T00:17:30.550 INFO:teuthology.orchestra.run.smithi162.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-12-02T00:17:30.550 INFO:teuthology.orchestra.run.smithi162.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-12-02T00:17:30.550 INFO:teuthology.orchestra.run.smithi162.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-12-02T00:17:30.550 INFO:teuthology.orchestra.run.smithi162.stdout:Packager : builder@centos.org 2023-12-02T00:17:30.550 INFO:teuthology.orchestra.run.smithi162.stdout:Vendor : CentOS 2023-12-02T00:17:30.551 INFO:teuthology.orchestra.run.smithi162.stdout:URL : http://www.gnu.org/software/grub/ 2023-12-02T00:17:30.551 INFO:teuthology.orchestra.run.smithi162.stdout:Summary : Support tools for GRUB. 2023-12-02T00:17:30.551 INFO:teuthology.orchestra.run.smithi162.stdout:Description : 2023-12-02T00:17:30.551 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-02T00:17:30.551 INFO:teuthology.orchestra.run.smithi162.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-12-02T00:17:30.551 INFO:teuthology.orchestra.run.smithi162.stdout:customizable bootloader with modular architecture. It supports a rich 2023-12-02T00:17:30.551 INFO:teuthology.orchestra.run.smithi162.stdout:variety of kernel formats, file systems, computer architectures and 2023-12-02T00:17:30.551 INFO:teuthology.orchestra.run.smithi162.stdout:hardware devices. 2023-12-02T00:17:30.551 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-02T00:17:30.551 INFO:teuthology.orchestra.run.smithi162.stdout:This subpackage provides tools for support of all platforms. 2023-12-02T00:17:30.552 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-12-02T00:17:30.552 INFO:teuthology.task.kernel:Updating grub on smithi162 to boot 5.14.0-388.el9.x86_64 2023-12-02T00:17:30.552 DEBUG:teuthology.orchestra.run.smithi162:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-12-02T00:17:31.156 INFO:teuthology.orchestra.run.smithi162.stderr:Generating grub configuration file ... 2023-12-02T00:17:32.495 INFO:teuthology.orchestra.run.smithi162.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-12-02T00:17:32.516 INFO:teuthology.orchestra.run.smithi162.stderr:done 2023-12-02T00:17:32.518 DEBUG:teuthology.orchestra.run.smithi162:> mktemp 2023-12-02T00:17:32.534 INFO:teuthology.orchestra.run.smithi162.stdout:/tmp/tmp.EGQYstC0oI 2023-12-02T00:17:32.534 DEBUG:teuthology.orchestra.run.smithi162:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.EGQYstC0oI 2023-12-02T00:17:32.604 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 0666 /tmp/tmp.EGQYstC0oI 2023-12-02T00:17:32.744 DEBUG:teuthology.orchestra.remote:smithi162:/tmp/tmp.EGQYstC0oI is 6KB 2023-12-02T00:17:32.792 DEBUG:teuthology.orchestra.run.smithi162:> rm -fr /tmp/tmp.EGQYstC0oI 2023-12-02T00:17:32.806 DEBUG:teuthology.orchestra.run.smithi162:> sudo /bin/ls /boot/loader/entries || true 2023-12-02T00:17:32.870 INFO:teuthology.orchestra.run.smithi162.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-12-02T00:17:32.871 INFO:teuthology.orchestra.run.smithi162.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-12-02T00:17:32.871 INFO:teuthology.orchestra.run.smithi162.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-12-02T00:17:32.871 INFO:teuthology.orchestra.run.smithi162.stdout:15949e3c88704300882482eddfd7baae-5.14.0-388.el9.x86_64.conf 2023-12-02T00:17:32.872 DEBUG:teuthology.orchestra.run.smithi162:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-388.el9.x86_64 2023-12-02T00:17:33.031 DEBUG:teuthology.orchestra.run.smithi162:> sudo shutdown -r now 2023-12-02T00:17:48.668 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: kernel-core-5.14.0-388.el9.x86_64 4/4 2023-12-02T00:17:48.713 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: kernel-modules-5.14.0-388.el9.x86_64 4/4 2023-12-02T00:17:49.101 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: kernel-5.14.0-388.el9.x86_64 4/4 2023-12-02T00:17:49.101 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : kernel-5.14.0-388.el9.x86_64 1/4 2023-12-02T00:17:49.102 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : kernel-core-5.14.0-388.el9.x86_64 2/4 2023-12-02T00:17:49.102 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : kernel-modules-5.14.0-388.el9.x86_64 3/4 2023-12-02T00:17:49.516 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : kernel-modules-core-5.14.0-388.el9.x86_64 4/4 2023-12-02T00:17:49.516 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:17:49.516 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2023-12-02T00:17:49.516 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-5.14.0-388.el9.x86_64 2023-12-02T00:17:49.516 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-core-5.14.0-388.el9.x86_64 2023-12-02T00:17:49.516 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-modules-5.14.0-388.el9.x86_64 2023-12-02T00:17:49.516 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-modules-core-5.14.0-388.el9.x86_64 2023-12-02T00:17:49.516 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:17:49.516 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2023-12-02T00:17:50.043 DEBUG:teuthology.orchestra.run.smithi046:> rpm -q kernel | sort -rV | head -n 1 2023-12-02T00:17:50.196 INFO:teuthology.orchestra.run.smithi046.stdout:kernel-5.14.0-388.el9.x86_64 2023-12-02T00:17:50.196 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-388.el9.x86_64 2023-12-02T00:17:50.196 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-388.el9.x86_64 2023-12-02T00:17:50.196 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-12-02T00:17:50.196 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-12-02T00:17:50.196 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-12-02T00:17:50.197 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi046.front.sepia.ceph.com, path=None, version=distro) 2023-12-02T00:17:50.197 DEBUG:teuthology.orchestra.run.smithi046:> rpm -q kernel | sort -rV | head -n 1 2023-12-02T00:17:50.319 INFO:teuthology.orchestra.run.smithi046.stdout:kernel-5.14.0-388.el9.x86_64 2023-12-02T00:17:50.320 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-388.el9.x86_64 2023-12-02T00:17:50.320 DEBUG:teuthology.orchestra.run.smithi046:> sudo rpm -qi grub2-tools 2023-12-02T00:17:50.361 INFO:teuthology.orchestra.run.smithi046.stdout:Name : grub2-tools 2023-12-02T00:17:50.362 INFO:teuthology.orchestra.run.smithi046.stdout:Epoch : 1 2023-12-02T00:17:50.362 INFO:teuthology.orchestra.run.smithi046.stdout:Version : 2.06 2023-12-02T00:17:50.362 INFO:teuthology.orchestra.run.smithi046.stdout:Release : 68.el9 2023-12-02T00:17:50.362 INFO:teuthology.orchestra.run.smithi046.stdout:Architecture: x86_64 2023-12-02T00:17:50.362 INFO:teuthology.orchestra.run.smithi046.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-12-02T00:17:50.363 INFO:teuthology.orchestra.run.smithi046.stdout:Group : Unspecified 2023-12-02T00:17:50.363 INFO:teuthology.orchestra.run.smithi046.stdout:Size : 8264254 2023-12-02T00:17:50.363 INFO:teuthology.orchestra.run.smithi046.stdout:License : GPLv3+ 2023-12-02T00:17:50.363 INFO:teuthology.orchestra.run.smithi046.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-12-02T00:17:50.363 INFO:teuthology.orchestra.run.smithi046.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-12-02T00:17:50.363 INFO:teuthology.orchestra.run.smithi046.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-12-02T00:17:50.364 INFO:teuthology.orchestra.run.smithi046.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-12-02T00:17:50.364 INFO:teuthology.orchestra.run.smithi046.stdout:Packager : builder@centos.org 2023-12-02T00:17:50.364 INFO:teuthology.orchestra.run.smithi046.stdout:Vendor : CentOS 2023-12-02T00:17:50.364 INFO:teuthology.orchestra.run.smithi046.stdout:URL : http://www.gnu.org/software/grub/ 2023-12-02T00:17:50.365 INFO:teuthology.orchestra.run.smithi046.stdout:Summary : Support tools for GRUB. 2023-12-02T00:17:50.365 INFO:teuthology.orchestra.run.smithi046.stdout:Description : 2023-12-02T00:17:50.365 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:17:50.365 INFO:teuthology.orchestra.run.smithi046.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-12-02T00:17:50.365 INFO:teuthology.orchestra.run.smithi046.stdout:customizable bootloader with modular architecture. It supports a rich 2023-12-02T00:17:50.366 INFO:teuthology.orchestra.run.smithi046.stdout:variety of kernel formats, file systems, computer architectures and 2023-12-02T00:17:50.366 INFO:teuthology.orchestra.run.smithi046.stdout:hardware devices. 2023-12-02T00:17:50.366 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:17:50.366 INFO:teuthology.orchestra.run.smithi046.stdout:This subpackage provides tools for support of all platforms. 2023-12-02T00:17:50.368 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-12-02T00:17:50.368 INFO:teuthology.task.kernel:Updating grub on smithi046 to boot 5.14.0-388.el9.x86_64 2023-12-02T00:17:50.368 DEBUG:teuthology.orchestra.run.smithi046:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-12-02T00:17:51.004 INFO:teuthology.orchestra.run.smithi046.stderr:Generating grub configuration file ... 2023-12-02T00:17:52.394 INFO:teuthology.orchestra.run.smithi046.stderr:File descriptor 3 (pipe:[36985]) leaked on vgs invocation. Parent PID 18173: grub2-probe 2023-12-02T00:17:52.394 INFO:teuthology.orchestra.run.smithi046.stderr:File descriptor 9 (pipe:[29140]) leaked on vgs invocation. Parent PID 18173: grub2-probe 2023-12-02T00:17:52.424 INFO:teuthology.orchestra.run.smithi046.stderr:File descriptor 3 (pipe:[36985]) leaked on vgs invocation. Parent PID 18173: grub2-probe 2023-12-02T00:17:52.424 INFO:teuthology.orchestra.run.smithi046.stderr:File descriptor 9 (pipe:[29140]) leaked on vgs invocation. Parent PID 18173: grub2-probe 2023-12-02T00:17:52.595 INFO:teuthology.orchestra.run.smithi046.stderr:File descriptor 3 (pipe:[36985]) leaked on vgs invocation. Parent PID 18221: grub2-probe 2023-12-02T00:17:52.596 INFO:teuthology.orchestra.run.smithi046.stderr:File descriptor 9 (pipe:[29140]) leaked on vgs invocation. Parent PID 18221: grub2-probe 2023-12-02T00:17:52.633 INFO:teuthology.orchestra.run.smithi046.stderr:File descriptor 3 (pipe:[36985]) leaked on vgs invocation. Parent PID 18221: grub2-probe 2023-12-02T00:17:52.633 INFO:teuthology.orchestra.run.smithi046.stderr:File descriptor 9 (pipe:[29140]) leaked on vgs invocation. Parent PID 18221: grub2-probe 2023-12-02T00:17:52.769 INFO:teuthology.orchestra.run.smithi046.stderr:File descriptor 3 (pipe:[36985]) leaked on vgs invocation. Parent PID 18269: grub2-probe 2023-12-02T00:17:52.769 INFO:teuthology.orchestra.run.smithi046.stderr:File descriptor 9 (pipe:[29140]) leaked on vgs invocation. Parent PID 18269: grub2-probe 2023-12-02T00:17:52.807 INFO:teuthology.orchestra.run.smithi046.stderr:File descriptor 3 (pipe:[36985]) leaked on vgs invocation. Parent PID 18269: grub2-probe 2023-12-02T00:17:52.807 INFO:teuthology.orchestra.run.smithi046.stderr:File descriptor 9 (pipe:[29140]) leaked on vgs invocation. Parent PID 18269: grub2-probe 2023-12-02T00:17:52.938 INFO:teuthology.orchestra.run.smithi046.stderr:File descriptor 3 (pipe:[36985]) leaked on vgs invocation. Parent PID 18317: grub2-probe 2023-12-02T00:17:52.938 INFO:teuthology.orchestra.run.smithi046.stderr:File descriptor 9 (pipe:[29140]) leaked on vgs invocation. Parent PID 18317: grub2-probe 2023-12-02T00:17:52.971 INFO:teuthology.orchestra.run.smithi046.stderr:File descriptor 3 (pipe:[36985]) leaked on vgs invocation. Parent PID 18317: grub2-probe 2023-12-02T00:17:52.971 INFO:teuthology.orchestra.run.smithi046.stderr:File descriptor 9 (pipe:[29140]) leaked on vgs invocation. Parent PID 18317: grub2-probe 2023-12-02T00:17:53.135 INFO:teuthology.orchestra.run.smithi046.stderr:File descriptor 3 (pipe:[36985]) leaked on vgs invocation. Parent PID 18365: grub2-probe 2023-12-02T00:17:53.135 INFO:teuthology.orchestra.run.smithi046.stderr:File descriptor 9 (pipe:[29140]) leaked on vgs invocation. Parent PID 18365: grub2-probe 2023-12-02T00:17:53.167 INFO:teuthology.orchestra.run.smithi046.stderr:File descriptor 3 (pipe:[36985]) leaked on vgs invocation. Parent PID 18365: grub2-probe 2023-12-02T00:17:53.167 INFO:teuthology.orchestra.run.smithi046.stderr:File descriptor 9 (pipe:[29140]) leaked on vgs invocation. Parent PID 18365: grub2-probe 2023-12-02T00:17:53.298 INFO:teuthology.orchestra.run.smithi046.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-12-02T00:17:53.312 INFO:teuthology.orchestra.run.smithi046.stderr:done 2023-12-02T00:17:53.314 DEBUG:teuthology.orchestra.run.smithi046:> mktemp 2023-12-02T00:17:53.329 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/tmp.E9G2YZ1w6r 2023-12-02T00:17:53.329 DEBUG:teuthology.orchestra.run.smithi046:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.E9G2YZ1w6r 2023-12-02T00:17:53.406 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0666 /tmp/tmp.E9G2YZ1w6r 2023-12-02T00:17:53.545 DEBUG:teuthology.orchestra.remote:smithi046:/tmp/tmp.E9G2YZ1w6r is 6KB 2023-12-02T00:17:53.594 DEBUG:teuthology.orchestra.run.smithi046:> rm -fr /tmp/tmp.E9G2YZ1w6r 2023-12-02T00:17:53.608 DEBUG:teuthology.orchestra.run.smithi046:> sudo /bin/ls /boot/loader/entries || true 2023-12-02T00:17:53.684 INFO:teuthology.orchestra.run.smithi046.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-12-02T00:17:53.684 INFO:teuthology.orchestra.run.smithi046.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-12-02T00:17:53.684 INFO:teuthology.orchestra.run.smithi046.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-12-02T00:17:53.684 INFO:teuthology.orchestra.run.smithi046.stdout:15949e3c88704300882482eddfd7baae-5.14.0-388.el9.x86_64.conf 2023-12-02T00:17:53.685 DEBUG:teuthology.orchestra.run.smithi046:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-388.el9.x86_64 2023-12-02T00:17:53.837 DEBUG:teuthology.orchestra.run.smithi046:> sudo shutdown -r now 2023-12-02T00:18:03.043 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-12-02T00:18:03.043 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2023-12-02T00:18:03.044 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-02T00:18:23.858 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-12-02T00:18:23.858 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2023-12-02T00:18:23.859 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-02T00:18:37.538 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.162 2023-12-02T00:18:46.549 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2023-12-02T00:18:46.550 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-02T00:18:58.534 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.46 2023-12-02T00:19:07.542 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2023-12-02T00:19:07.543 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-02T00:19:25.922 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.46 2023-12-02T00:19:37.934 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2023-12-02T00:19:37.935 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-02T00:19:38.616 DEBUG:teuthology.orchestra.run.smithi046:> true 2023-12-02T00:19:39.083 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi046.front.sepia.ceph.com' 2023-12-02T00:19:39.083 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-388.el9.x86_64"... 2023-12-02T00:19:39.083 DEBUG:teuthology.orchestra.run.smithi046:> uname -r 2023-12-02T00:19:39.098 INFO:teuthology.orchestra.run.smithi046.stdout:5.14.0-388.el9.x86_64 2023-12-02T00:19:39.099 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-388.el9.x86_64 vs 5.14.0-388.el9.x86_64 2023-12-02T00:19:39.099 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-12-02T00:19:39.099 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-12-02T00:19:40.100 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-12-02T00:19:40.101 DEBUG:teuthology.orchestra.run.smithi046:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-12-02T00:19:40.183 INFO:teuthology.orchestra.run.smithi046.stdout:ttyS1 2023-12-02T00:19:40.227 DEBUG:teuthology.parallel:result is None 2023-12-02T00:19:46.552 DEBUG:teuthology.orchestra.remote:timed out 2023-12-02T00:19:58.553 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2023-12-02T00:19:58.554 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-02T00:19:58.988 DEBUG:teuthology.orchestra.run.smithi162:> true 2023-12-02T00:19:59.355 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi162.front.sepia.ceph.com' 2023-12-02T00:19:59.356 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-388.el9.x86_64"... 2023-12-02T00:19:59.356 DEBUG:teuthology.orchestra.run.smithi162:> uname -r 2023-12-02T00:19:59.410 INFO:teuthology.orchestra.run.smithi162.stdout:5.14.0-388.el9.x86_64 2023-12-02T00:19:59.411 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-388.el9.x86_64 vs 5.14.0-388.el9.x86_64 2023-12-02T00:19:59.411 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-12-02T00:19:59.411 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-12-02T00:20:00.412 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-12-02T00:20:00.412 DEBUG:teuthology.orchestra.run.smithi162:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-12-02T00:20:00.490 INFO:teuthology.orchestra.run.smithi162.stdout:ttyS1 2023-12-02T00:20:00.531 DEBUG:teuthology.parallel:result is None 2023-12-02T00:20:00.532 INFO:teuthology.run_tasks:Running task internal.base... 2023-12-02T00:20:00.540 INFO:teuthology.task.internal:Creating test directory... 2023-12-02T00:20:00.540 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-12-02T00:20:00.544 DEBUG:teuthology.orchestra.run.smithi162:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-12-02T00:20:00.563 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-12-02T00:20:00.570 INFO:teuthology.run_tasks:Running task internal.archive... 2023-12-02T00:20:00.580 INFO:teuthology.task.internal:Creating archive directory... 2023-12-02T00:20:00.580 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-12-02T00:20:00.605 DEBUG:teuthology.orchestra.run.smithi162:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-12-02T00:20:00.646 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-12-02T00:20:00.654 INFO:teuthology.task.internal:Enabling coredump saving... 2023-12-02T00:20:00.655 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-12-02T00:20:00.683 DEBUG:teuthology.orchestra.run.smithi162:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-12-02T00:20:00.719 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-02T00:20:00.725 INFO:teuthology.orchestra.run.smithi162.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-02T00:20:00.737 INFO:teuthology.orchestra.run.smithi162.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-02T00:20:00.737 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-02T00:20:00.739 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-12-02T00:20:00.747 INFO:teuthology.task.internal:Configuring sudo... 2023-12-02T00:20:00.747 DEBUG:teuthology.orchestra.run.smithi046:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-12-02T00:20:00.781 DEBUG:teuthology.orchestra.run.smithi162:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-12-02T00:20:00.813 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-12-02T00:20:00.822 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-12-02T00:20:00.822 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-12-02T00:20:00.855 DEBUG:teuthology.orchestra.run.smithi162:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-12-02T00:20:00.871 DEBUG:teuthology.orchestra.run.smithi046:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-02T00:20:00.956 DEBUG:teuthology.orchestra.run.smithi046:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-02T00:20:01.006 DEBUG:teuthology.orchestra.run.smithi046:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-02T00:20:01.088 DEBUG:teuthology.orchestra.run.smithi046:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-02T00:20:01.138 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-02T00:20:01.138 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-12-02T00:20:01.209 DEBUG:teuthology.orchestra.run.smithi162:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-02T00:20:01.254 DEBUG:teuthology.orchestra.run.smithi162:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-02T00:20:01.319 DEBUG:teuthology.orchestra.run.smithi162:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-02T00:20:01.363 DEBUG:teuthology.orchestra.run.smithi162:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-02T00:20:01.405 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-12-02T00:20:01.405 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-12-02T00:20:01.474 DEBUG:teuthology.orchestra.run.smithi046:> sudo service rsyslog restart 2023-12-02T00:20:01.476 DEBUG:teuthology.orchestra.run.smithi162:> sudo service rsyslog restart 2023-12-02T00:20:01.532 INFO:teuthology.orchestra.run.smithi046.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-02T00:20:01.560 INFO:teuthology.orchestra.run.smithi162.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-02T00:20:01.901 INFO:teuthology.run_tasks:Running task internal.timer... 2023-12-02T00:20:01.992 INFO:teuthology.task.internal:Starting timer... 2023-12-02T00:20:01.992 INFO:teuthology.run_tasks:Running task pcp... 2023-12-02T00:20:02.002 INFO:teuthology.run_tasks:Running task selinux... 2023-12-02T00:20:02.012 DEBUG:teuthology.orchestra.run.smithi046:> sudo service auditd rotate 2023-12-02T00:20:02.071 INFO:teuthology.orchestra.run.smithi046.stdout:Rotating logs: 2023-12-02T00:20:02.073 DEBUG:teuthology.orchestra.run.smithi162:> sudo service auditd rotate 2023-12-02T00:20:02.143 INFO:teuthology.orchestra.run.smithi162.stdout:Rotating logs: 2023-12-02T00:20:02.145 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-12-02T00:20:02.145 DEBUG:teuthology.orchestra.run.smithi046:> /usr/sbin/getenforce 2023-12-02T00:20:02.167 INFO:teuthology.orchestra.run.smithi046.stdout:Permissive 2023-12-02T00:20:02.168 DEBUG:teuthology.orchestra.run.smithi162:> /usr/sbin/getenforce 2023-12-02T00:20:02.193 INFO:teuthology.orchestra.run.smithi162.stdout:Permissive 2023-12-02T00:20:02.194 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi046.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi162.front.sepia.ceph.com': 'permissive'} 2023-12-02T00:20:02.194 DEBUG:teuthology.orchestra.run.smithi046:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-02T00:20:02.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-02T00:20:02.241 DEBUG:teuthology.orchestra.run.smithi162:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-02T00:20:02.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-02T00:20:02.271 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-12-02T00:20:02.271 DEBUG:teuthology.orchestra.run.smithi046:> sudo /usr/sbin/setenforce permissive 2023-12-02T00:20:02.319 DEBUG:teuthology.orchestra.run.smithi162:> sudo /usr/sbin/setenforce permissive 2023-12-02T00:20:02.370 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-12-02T00:20:02.381 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-12-02T00:20:02.381 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-12-02T00:20:02.398 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-12-02T00:20:02.399 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi046.front.sepia.ceph.com,smithi162.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-12-02T00:26:06.416 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi046.front.sepia.ceph.com'), Remote(name='ubuntu@smithi162.front.sepia.ceph.com')] 2023-12-02T00:26:06.417 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2023-12-02T00:26:06.418 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-02T00:26:06.516 DEBUG:teuthology.orchestra.run.smithi046:> true 2023-12-02T00:26:06.590 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi046.front.sepia.ceph.com' 2023-12-02T00:26:06.591 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2023-12-02T00:26:06.592 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-02T00:26:06.674 DEBUG:teuthology.orchestra.run.smithi162:> true 2023-12-02T00:26:06.744 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi162.front.sepia.ceph.com' 2023-12-02T00:26:06.745 INFO:teuthology.run_tasks:Running task clock... 2023-12-02T00:26:06.757 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-12-02T00:26:06.757 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-12-02T00:26:06.757 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-02T00:26:06.760 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-12-02T00:26:06.760 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-02T00:26:06.794 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-12-02T00:26:06.814 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-12-02T00:26:06.816 INFO:teuthology.orchestra.run.smithi162.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-12-02T00:26:06.833 INFO:teuthology.orchestra.run.smithi162.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-12-02T00:26:06.851 INFO:teuthology.orchestra.run.smithi046.stderr:sudo: ntpd: command not found 2023-12-02T00:26:06.862 INFO:teuthology.orchestra.run.smithi162.stderr:sudo: ntpd: command not found 2023-12-02T00:26:06.868 INFO:teuthology.orchestra.run.smithi046.stdout:506 Cannot talk to daemon 2023-12-02T00:26:06.876 INFO:teuthology.orchestra.run.smithi162.stdout:506 Cannot talk to daemon 2023-12-02T00:26:06.888 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-12-02T00:26:06.892 INFO:teuthology.orchestra.run.smithi162.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-12-02T00:26:06.909 INFO:teuthology.orchestra.run.smithi162.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-12-02T00:26:06.909 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-12-02T00:26:06.951 INFO:teuthology.orchestra.run.smithi162.stderr:bash: line 1: ntpq: command not found 2023-12-02T00:26:06.955 INFO:teuthology.orchestra.run.smithi162.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-02T00:26:06.955 INFO:teuthology.orchestra.run.smithi162.stdout:=============================================================================== 2023-12-02T00:26:06.955 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-02T00:26:06.955 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-02T00:26:06.955 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-02T00:26:06.955 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-02T00:26:06.956 INFO:teuthology.orchestra.run.smithi046.stderr:bash: line 1: ntpq: command not found 2023-12-02T00:26:06.959 INFO:teuthology.orchestra.run.smithi046.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-02T00:26:06.959 INFO:teuthology.orchestra.run.smithi046.stdout:=============================================================================== 2023-12-02T00:26:06.959 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-02T00:26:06.959 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-02T00:26:06.959 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-02T00:26:06.959 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-02T00:26:06.960 INFO:teuthology.run_tasks:Running task install... 2023-12-02T00:26:06.975 DEBUG:teuthology.task.install:project ceph 2023-12-02T00:26:06.975 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'bed7681480ccb0192b91a35e691658867e1e96b7'}} 2023-12-02T00:26:06.975 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bed7681480ccb0192b91a35e691658867e1e96b7'} 2023-12-02T00:26:06.976 INFO:teuthology.task.install:Using flavor: default 2023-12-02T00:26:06.987 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-12-02T00:26:06.987 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-12-02T00:26:06.987 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bed7681480ccb0192b91a35e691658867e1e96b7', 'tag': None, 'wait_for_package': False} 2023-12-02T00:26:06.988 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bed7681480ccb0192b91a35e691658867e1e96b7 2023-12-02T00:26:06.989 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bed7681480ccb0192b91a35e691658867e1e96b7', 'tag': None, 'wait_for_package': False} 2023-12-02T00:26:06.990 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bed7681480ccb0192b91a35e691658867e1e96b7 2023-12-02T00:26:07.148 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/main/bed7681480ccb0192b91a35e691658867e1e96b7/centos/9/flavors/default/ 2023-12-02T00:26:07.148 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7631.gbed76814 2023-12-02T00:26:07.218 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/main/bed7681480ccb0192b91a35e691658867e1e96b7/centos/9/flavors/default/ 2023-12-02T00:26:07.219 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7631.gbed76814 2023-12-02T00:26:07.264 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/main/bed7681480ccb0192b91a35e691658867e1e96b7/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/main/bed7681480ccb0192b91a35e691658867e1e96b7/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/main/bed7681480ccb0192b91a35e691658867e1e96b7/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-12-02T00:26:07.264 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-12-02T00:26:07.264 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-12-02T00:26:07.291 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-12-02T00:26:07.292 DEBUG:teuthology.orchestra.run.smithi162:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bed7681480ccb0192b91a35e691658867e1e96b7/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-12-02T00:26:07.324 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/main/bed7681480ccb0192b91a35e691658867e1e96b7/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/main/bed7681480ccb0192b91a35e691658867e1e96b7/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/main/bed7681480ccb0192b91a35e691658867e1e96b7/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-12-02T00:26:07.324 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-02T00:26:07.324 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-12-02T00:26:07.357 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-12-02T00:26:07.358 DEBUG:teuthology.orchestra.run.smithi046:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bed7681480ccb0192b91a35e691658867e1e96b7/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-12-02T00:26:07.359 DEBUG:teuthology.orchestra.run.smithi162:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-12-02T00:26:07.430 DEBUG:teuthology.orchestra.run.smithi046:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-12-02T00:26:07.439 DEBUG:teuthology.orchestra.run.smithi162:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-12-02T00:26:07.506 INFO:teuthology.orchestra.run.smithi162.stdout:check_obsoletes = 1 2023-12-02T00:26:07.507 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum clean all 2023-12-02T00:26:07.517 DEBUG:teuthology.orchestra.run.smithi046:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-12-02T00:26:07.547 INFO:teuthology.orchestra.run.smithi046.stdout:check_obsoletes = 1 2023-12-02T00:26:07.548 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum clean all 2023-12-02T00:26:07.750 INFO:teuthology.orchestra.run.smithi162.stdout:73 files removed 2023-12-02T00:26:07.778 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-12-02T00:26:07.801 INFO:teuthology.orchestra.run.smithi046.stdout:73 files removed 2023-12-02T00:26:07.828 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-12-02T00:26:08.491 INFO:teuthology.orchestra.run.smithi046.stdout:ceph packages for x86_64 240 kB/s | 78 kB 00:00 2023-12-02T00:26:08.577 INFO:teuthology.orchestra.run.smithi162.stdout:ceph packages for x86_64 163 kB/s | 78 kB 00:00 2023-12-02T00:26:08.858 INFO:teuthology.orchestra.run.smithi046.stdout:ceph noarch packages 30 kB/s | 9.5 kB 00:00 2023-12-02T00:26:09.019 INFO:teuthology.orchestra.run.smithi162.stdout:ceph noarch packages 24 kB/s | 9.5 kB 00:00 2023-12-02T00:26:09.276 INFO:teuthology.orchestra.run.smithi046.stdout:ceph source packages 4.9 kB/s | 1.9 kB 00:00 2023-12-02T00:26:09.454 INFO:teuthology.orchestra.run.smithi162.stdout:ceph source packages 4.7 kB/s | 1.9 kB 00:00 2023-12-02T00:26:09.945 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS-9 - AppStream 10 MB/s | 6.4 MB 00:00 2023-12-02T00:26:09.956 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS-9 - AppStream 14 MB/s | 6.4 MB 00:00 2023-12-02T00:26:11.503 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS-9 - BaseOS 4.4 MB/s | 2.0 MB 00:00 2023-12-02T00:26:11.513 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS-9 - BaseOS 3.9 MB/s | 2.0 MB 00:00 2023-12-02T00:26:12.377 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 7.9 MB 00:00 2023-12-02T00:26:13.220 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - BaseOS 5.7 MB/s | 7.9 MB 00:01 2023-12-02T00:26:16.096 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - AppStream 10 MB/s | 18 MB 00:01 2023-12-02T00:26:16.495 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - AppStream 6.1 MB/s | 18 MB 00:02 2023-12-02T00:26:20.913 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - CRB 6.4 MB/s | 5.8 MB 00:00 2023-12-02T00:26:21.647 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - CRB 2.6 MB/s | 5.8 MB 00:02 2023-12-02T00:26:22.413 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - Extras packages 53 kB/s | 15 kB 00:00 2023-12-02T00:26:23.639 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - Extras packages 24 kB/s | 15 kB 00:00 2023-12-02T00:26:24.541 INFO:teuthology.orchestra.run.smithi046.stdout:Extra Packages for Enterprise Linux 10 MB/s | 20 MB 00:01 2023-12-02T00:26:24.807 INFO:teuthology.orchestra.run.smithi162.stdout:Extra Packages for Enterprise Linux 19 MB/s | 20 MB 00:01 2023-12-02T00:26:29.500 INFO:teuthology.orchestra.run.smithi046.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2023-12-02T00:26:29.632 INFO:teuthology.orchestra.run.smithi162.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2023-12-02T00:26:32.287 INFO:teuthology.orchestra.run.smithi162.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-12-02T00:26:32.288 INFO:teuthology.orchestra.run.smithi162.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-12-02T00:26:32.296 INFO:teuthology.orchestra.run.smithi046.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-12-02T00:26:32.297 INFO:teuthology.orchestra.run.smithi046.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-12-02T00:26:32.301 INFO:teuthology.orchestra.run.smithi162.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-12-02T00:26:32.302 INFO:teuthology.orchestra.run.smithi162.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-12-02T00:26:32.303 INFO:teuthology.orchestra.run.smithi162.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-12-02T00:26:32.304 INFO:teuthology.orchestra.run.smithi162.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-12-02T00:26:32.306 INFO:teuthology.orchestra.run.smithi162.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-12-02T00:26:32.309 INFO:teuthology.orchestra.run.smithi162.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-12-02T00:26:32.310 INFO:teuthology.orchestra.run.smithi162.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-12-02T00:26:32.311 INFO:teuthology.orchestra.run.smithi046.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-12-02T00:26:32.312 INFO:teuthology.orchestra.run.smithi046.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-12-02T00:26:32.313 INFO:teuthology.orchestra.run.smithi046.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-12-02T00:26:32.315 INFO:teuthology.orchestra.run.smithi046.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-12-02T00:26:32.317 INFO:teuthology.orchestra.run.smithi046.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-12-02T00:26:32.320 INFO:teuthology.orchestra.run.smithi046.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-12-02T00:26:32.321 INFO:teuthology.orchestra.run.smithi046.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-12-02T00:26:32.383 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2023-12-02T00:26:32.393 INFO:teuthology.orchestra.run.smithi162.stdout:============================================================================================ 2023-12-02T00:26:32.393 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repository Size 2023-12-02T00:26:32.393 INFO:teuthology.orchestra.run.smithi162.stdout:============================================================================================ 2023-12-02T00:26:32.393 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2023-12-02T00:26:32.393 INFO:teuthology.orchestra.run.smithi162.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-12-02T00:26:32.394 INFO:teuthology.orchestra.run.smithi162.stdout: ceph x86_64 2:18.0.0-7631.gbed76814.el9 ceph 6.6 k 2023-12-02T00:26:32.394 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-base x86_64 2:18.0.0-7631.gbed76814.el9 ceph 5.2 M 2023-12-02T00:26:32.394 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-fuse x86_64 2:18.0.0-7631.gbed76814.el9 ceph 859 k 2023-12-02T00:26:32.394 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7631.gbed76814.el9 ceph 141 k 2023-12-02T00:26:32.394 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr x86_64 2:18.0.0-7631.gbed76814.el9 ceph 1.5 M 2023-12-02T00:26:32.394 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7631.gbed76814.el9 ceph-noarch 133 k 2023-12-02T00:26:32.394 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7631.gbed76814.el9 ceph-noarch 1.8 M 2023-12-02T00:26:32.394 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7631.gbed76814.el9 ceph-noarch 7.4 M 2023-12-02T00:26:32.394 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-rook noarch 2:18.0.0-7631.gbed76814.el9 ceph-noarch 50 k 2023-12-02T00:26:32.394 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-radosgw x86_64 2:18.0.0-7631.gbed76814.el9 ceph 8.6 M 2023-12-02T00:26:32.394 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-test x86_64 2:18.0.0-7631.gbed76814.el9 ceph 41 M 2023-12-02T00:26:32.394 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-volume noarch 2:18.0.0-7631.gbed76814.el9 ceph-noarch 262 k 2023-12-02T00:26:32.394 INFO:teuthology.orchestra.run.smithi162.stdout: cephadm noarch 2:18.0.0-7631.gbed76814.el9 ceph-noarch 776 k 2023-12-02T00:26:32.395 INFO:teuthology.orchestra.run.smithi162.stdout: cephfs-mirror x86_64 2:18.0.0-7631.gbed76814.el9 ceph 218 k 2023-12-02T00:26:32.395 INFO:teuthology.orchestra.run.smithi162.stdout: cephfs-top noarch 2:18.0.0-7631.gbed76814.el9 ceph-noarch 18 k 2023-12-02T00:26:32.395 INFO:teuthology.orchestra.run.smithi162.stdout: elfutils-libelf-devel x86_64 0.190-1.el9 CentOS-AppStream 21 k 2023-12-02T00:26:32.395 INFO:teuthology.orchestra.run.smithi162.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-12-02T00:26:32.395 INFO:teuthology.orchestra.run.smithi162.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-12-02T00:26:32.395 INFO:teuthology.orchestra.run.smithi162.stdout: libcephfs-devel x86_64 2:18.0.0-7631.gbed76814.el9 ceph 31 k 2023-12-02T00:26:32.395 INFO:teuthology.orchestra.run.smithi162.stdout: libcephfs2 x86_64 2:18.0.0-7631.gbed76814.el9 ceph 718 k 2023-12-02T00:26:32.395 INFO:teuthology.orchestra.run.smithi162.stdout: librados-devel x86_64 2:18.0.0-7631.gbed76814.el9 ceph 127 k 2023-12-02T00:26:32.395 INFO:teuthology.orchestra.run.smithi162.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-12-02T00:26:32.395 INFO:teuthology.orchestra.run.smithi162.stdout: libuuid-devel x86_64 2.37.4-15.el9 CentOS-AppStream 24 k 2023-12-02T00:26:32.395 INFO:teuthology.orchestra.run.smithi162.stdout: openssl-devel x86_64 1:3.0.7-25.el9 appstream 4.1 M 2023-12-02T00:26:32.395 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-12-02T00:26:32.395 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-12-02T00:26:32.395 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-12-02T00:26:32.395 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cephfs x86_64 2:18.0.0-7631.gbed76814.el9 ceph 162 k 2023-12-02T00:26:32.396 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rados x86_64 2:18.0.0-7631.gbed76814.el9 ceph 320 k 2023-12-02T00:26:32.396 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rbd x86_64 2:18.0.0-7631.gbed76814.el9 ceph 298 k 2023-12-02T00:26:32.396 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rgw x86_64 2:18.0.0-7631.gbed76814.el9 ceph 99 k 2023-12-02T00:26:32.396 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-fuse x86_64 2:18.0.0-7631.gbed76814.el9 ceph 85 k 2023-12-02T00:26:32.396 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-mirror x86_64 2:18.0.0-7631.gbed76814.el9 ceph 3.0 M 2023-12-02T00:26:32.396 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-nbd x86_64 2:18.0.0-7631.gbed76814.el9 ceph 170 k 2023-12-02T00:26:32.396 INFO:teuthology.orchestra.run.smithi162.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 CentOS-AppStream 85 k 2023-12-02T00:26:32.396 INFO:teuthology.orchestra.run.smithi162.stdout:Upgrading: 2023-12-02T00:26:32.396 INFO:teuthology.orchestra.run.smithi162.stdout: NetworkManager x86_64 1:1.45.7-1.el9 baseos 2.3 M 2023-12-02T00:26:32.396 INFO:teuthology.orchestra.run.smithi162.stdout: NetworkManager-libnm x86_64 1:1.45.7-1.el9 baseos 1.8 M 2023-12-02T00:26:32.396 INFO:teuthology.orchestra.run.smithi162.stdout: NetworkManager-team x86_64 1:1.45.7-1.el9 baseos 43 k 2023-12-02T00:26:32.396 INFO:teuthology.orchestra.run.smithi162.stdout: NetworkManager-tui x86_64 1:1.45.7-1.el9 baseos 246 k 2023-12-02T00:26:32.396 INFO:teuthology.orchestra.run.smithi162.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2023-12-02T00:26:32.396 INFO:teuthology.orchestra.run.smithi162.stdout: elfutils-debuginfod-client x86_64 0.190-1.el9 CentOS-BaseOS 36 k 2023-12-02T00:26:32.397 INFO:teuthology.orchestra.run.smithi162.stdout: elfutils-libelf x86_64 0.190-1.el9 CentOS-BaseOS 192 k 2023-12-02T00:26:32.397 INFO:teuthology.orchestra.run.smithi162.stdout: elfutils-libs x86_64 0.190-1.el9 CentOS-BaseOS 255 k 2023-12-02T00:26:32.397 INFO:teuthology.orchestra.run.smithi162.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2023-12-02T00:26:32.397 INFO:teuthology.orchestra.run.smithi162.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2023-12-02T00:26:32.397 INFO:teuthology.orchestra.run.smithi162.stdout: librados2 x86_64 2:18.0.0-7631.gbed76814.el9 ceph 3.3 M 2023-12-02T00:26:32.397 INFO:teuthology.orchestra.run.smithi162.stdout: librbd1 x86_64 2:18.0.0-7631.gbed76814.el9 ceph 3.0 M 2023-12-02T00:26:32.397 INFO:teuthology.orchestra.run.smithi162.stdout: openssl x86_64 1:3.0.7-25.el9 baseos 1.2 M 2023-12-02T00:26:32.397 INFO:teuthology.orchestra.run.smithi162.stdout: openssl-libs x86_64 1:3.0.7-25.el9 baseos 2.2 M 2023-12-02T00:26:32.397 INFO:teuthology.orchestra.run.smithi162.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2023-12-02T00:26:32.397 INFO:teuthology.orchestra.run.smithi162.stdout:Installing dependencies: 2023-12-02T00:26:32.397 INFO:teuthology.orchestra.run.smithi162.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2023-12-02T00:26:32.397 INFO:teuthology.orchestra.run.smithi162.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-12-02T00:26:32.397 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-common x86_64 2:18.0.0-7631.gbed76814.el9 ceph 19 M 2023-12-02T00:26:32.397 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7631.gbed76814.el9 ceph-noarch 28 k 2023-12-02T00:26:32.397 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mds x86_64 2:18.0.0-7631.gbed76814.el9 ceph 2.1 M 2023-12-02T00:26:32.398 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7631.gbed76814.el9 ceph-noarch 243 k 2023-12-02T00:26:32.398 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mon x86_64 2:18.0.0-7631.gbed76814.el9 ceph 4.4 M 2023-12-02T00:26:32.398 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-osd x86_64 2:18.0.0-7631.gbed76814.el9 ceph 18 M 2023-12-02T00:26:32.398 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7631.gbed76814.el9 ceph-noarch 15 k 2023-12-02T00:26:32.398 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-selinux x86_64 2:18.0.0-7631.gbed76814.el9 ceph 24 k 2023-12-02T00:26:32.398 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-12-02T00:26:32.398 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-12-02T00:26:32.398 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-12-02T00:26:32.398 INFO:teuthology.orchestra.run.smithi162.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-12-02T00:26:32.398 INFO:teuthology.orchestra.run.smithi162.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-12-02T00:26:32.398 INFO:teuthology.orchestra.run.smithi162.stdout: jq x86_64 1.6-16.el9 CentOS-AppStream 186 k 2023-12-02T00:26:32.398 INFO:teuthology.orchestra.run.smithi162.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-12-02T00:26:32.398 INFO:teuthology.orchestra.run.smithi162.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-12-02T00:26:32.398 INFO:teuthology.orchestra.run.smithi162.stdout: libcephsqlite x86_64 2:18.0.0-7631.gbed76814.el9 ceph 164 k 2023-12-02T00:26:32.398 INFO:teuthology.orchestra.run.smithi162.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-12-02T00:26:32.399 INFO:teuthology.orchestra.run.smithi162.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-12-02T00:26:32.399 INFO:teuthology.orchestra.run.smithi162.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-12-02T00:26:32.399 INFO:teuthology.orchestra.run.smithi162.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-12-02T00:26:32.399 INFO:teuthology.orchestra.run.smithi162.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2023-12-02T00:26:32.399 INFO:teuthology.orchestra.run.smithi162.stdout: libradosstriper1 x86_64 2:18.0.0-7631.gbed76814.el9 ceph 475 k 2023-12-02T00:26:32.399 INFO:teuthology.orchestra.run.smithi162.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-12-02T00:26:32.399 INFO:teuthology.orchestra.run.smithi162.stdout: librgw2 x86_64 2:18.0.0-7631.gbed76814.el9 ceph 4.8 M 2023-12-02T00:26:32.399 INFO:teuthology.orchestra.run.smithi162.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-12-02T00:26:32.399 INFO:teuthology.orchestra.run.smithi162.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-12-02T00:26:32.399 INFO:teuthology.orchestra.run.smithi162.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-12-02T00:26:32.399 INFO:teuthology.orchestra.run.smithi162.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2023-12-02T00:26:32.399 INFO:teuthology.orchestra.run.smithi162.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2023-12-02T00:26:32.399 INFO:teuthology.orchestra.run.smithi162.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2023-12-02T00:26:32.400 INFO:teuthology.orchestra.run.smithi162.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-12-02T00:26:32.400 INFO:teuthology.orchestra.run.smithi162.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-12-02T00:26:32.400 INFO:teuthology.orchestra.run.smithi162.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-12-02T00:26:32.400 INFO:teuthology.orchestra.run.smithi162.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-12-02T00:26:32.400 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-12-02T00:26:32.400 INFO:teuthology.orchestra.run.smithi162.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-12-02T00:26:32.400 INFO:teuthology.orchestra.run.smithi162.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-12-02T00:26:32.400 INFO:teuthology.orchestra.run.smithi162.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-12-02T00:26:32.400 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-12-02T00:26:32.400 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ceph-argparse x86_64 2:18.0.0-7631.gbed76814.el9 ceph 45 k 2023-12-02T00:26:32.400 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ceph-common x86_64 2:18.0.0-7631.gbed76814.el9 ceph 129 k 2023-12-02T00:26:32.400 INFO:teuthology.orchestra.run.smithi162.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-12-02T00:26:32.400 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-12-02T00:26:32.400 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-12-02T00:26:32.400 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-12-02T00:26:32.401 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2023-12-02T00:26:32.401 INFO:teuthology.orchestra.run.smithi162.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2023-12-02T00:26:32.401 INFO:teuthology.orchestra.run.smithi162.stdout: python3-google-auth noarch 1:2.23.4-1.el9 epel 197 k 2023-12-02T00:26:32.401 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-12-02T00:26:32.401 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-12-02T00:26:32.401 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-12-02T00:26:32.401 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-12-02T00:26:32.401 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-12-02T00:26:32.401 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-12-02T00:26:32.401 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-12-02T00:26:32.401 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-12-02T00:26:32.401 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-12-02T00:26:32.401 INFO:teuthology.orchestra.run.smithi162.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-12-02T00:26:32.401 INFO:teuthology.orchestra.run.smithi162.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-12-02T00:26:32.402 INFO:teuthology.orchestra.run.smithi162.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-12-02T00:26:32.402 INFO:teuthology.orchestra.run.smithi162.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-12-02T00:26:32.402 INFO:teuthology.orchestra.run.smithi162.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-12-02T00:26:32.402 INFO:teuthology.orchestra.run.smithi162.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-12-02T00:26:32.402 INFO:teuthology.orchestra.run.smithi162.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-12-02T00:26:32.402 INFO:teuthology.orchestra.run.smithi162.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-12-02T00:26:32.402 INFO:teuthology.orchestra.run.smithi162.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-12-02T00:26:32.402 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-12-02T00:26:32.402 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-12-02T00:26:32.402 INFO:teuthology.orchestra.run.smithi162.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-12-02T00:26:32.402 INFO:teuthology.orchestra.run.smithi162.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-12-02T00:26:32.402 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-12-02T00:26:32.403 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-12-02T00:26:32.403 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-12-02T00:26:32.403 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-12-02T00:26:32.403 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2023-12-02T00:26:32.403 INFO:teuthology.orchestra.run.smithi162.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-12-02T00:26:32.403 INFO:teuthology.orchestra.run.smithi162.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-12-02T00:26:32.403 INFO:teuthology.orchestra.run.smithi162.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-12-02T00:26:32.403 INFO:teuthology.orchestra.run.smithi162.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-12-02T00:26:32.403 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-12-02T00:26:32.403 INFO:teuthology.orchestra.run.smithi162.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-12-02T00:26:32.403 INFO:teuthology.orchestra.run.smithi162.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-12-02T00:26:32.403 INFO:teuthology.orchestra.run.smithi162.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-12-02T00:26:32.403 INFO:teuthology.orchestra.run.smithi162.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-12-02T00:26:32.403 INFO:teuthology.orchestra.run.smithi162.stdout: python3-urllib3 noarch 1.26.5-4.el9 CentOS-BaseOS 187 k 2023-12-02T00:26:32.404 INFO:teuthology.orchestra.run.smithi162.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-12-02T00:26:32.404 INFO:teuthology.orchestra.run.smithi162.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-12-02T00:26:32.404 INFO:teuthology.orchestra.run.smithi162.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-12-02T00:26:32.404 INFO:teuthology.orchestra.run.smithi162.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-12-02T00:26:32.404 INFO:teuthology.orchestra.run.smithi162.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-12-02T00:26:32.404 INFO:teuthology.orchestra.run.smithi162.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-12-02T00:26:32.404 INFO:teuthology.orchestra.run.smithi162.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-12-02T00:26:32.404 INFO:teuthology.orchestra.run.smithi162.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-12-02T00:26:32.404 INFO:teuthology.orchestra.run.smithi162.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-12-02T00:26:32.404 INFO:teuthology.orchestra.run.smithi162.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2023-12-02T00:26:32.404 INFO:teuthology.orchestra.run.smithi162.stdout:Installing weak dependencies: 2023-12-02T00:26:32.404 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-12-02T00:26:32.404 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-02T00:26:32.405 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2023-12-02T00:26:32.405 INFO:teuthology.orchestra.run.smithi162.stdout:============================================================================================ 2023-12-02T00:26:32.405 INFO:teuthology.orchestra.run.smithi162.stdout:Install 135 Packages 2023-12-02T00:26:32.405 INFO:teuthology.orchestra.run.smithi162.stdout:Upgrade 15 Packages 2023-12-02T00:26:32.405 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-02T00:26:32.405 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2023-12-02T00:26:32.406 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 204 M 2023-12-02T00:26:32.406 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2023-12-02T00:26:32.427 INFO:teuthology.orchestra.run.smithi046.stdout:============================================================================================ 2023-12-02T00:26:32.427 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repository Size 2023-12-02T00:26:32.427 INFO:teuthology.orchestra.run.smithi046.stdout:============================================================================================ 2023-12-02T00:26:32.427 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2023-12-02T00:26:32.427 INFO:teuthology.orchestra.run.smithi046.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-12-02T00:26:32.427 INFO:teuthology.orchestra.run.smithi046.stdout: ceph x86_64 2:18.0.0-7631.gbed76814.el9 ceph 6.6 k 2023-12-02T00:26:32.427 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-base x86_64 2:18.0.0-7631.gbed76814.el9 ceph 5.2 M 2023-12-02T00:26:32.427 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-fuse x86_64 2:18.0.0-7631.gbed76814.el9 ceph 859 k 2023-12-02T00:26:32.427 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7631.gbed76814.el9 ceph 141 k 2023-12-02T00:26:32.427 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr x86_64 2:18.0.0-7631.gbed76814.el9 ceph 1.5 M 2023-12-02T00:26:32.428 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7631.gbed76814.el9 ceph-noarch 133 k 2023-12-02T00:26:32.428 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7631.gbed76814.el9 ceph-noarch 1.8 M 2023-12-02T00:26:32.428 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7631.gbed76814.el9 ceph-noarch 7.4 M 2023-12-02T00:26:32.428 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-rook noarch 2:18.0.0-7631.gbed76814.el9 ceph-noarch 50 k 2023-12-02T00:26:32.428 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-radosgw x86_64 2:18.0.0-7631.gbed76814.el9 ceph 8.6 M 2023-12-02T00:26:32.428 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-test x86_64 2:18.0.0-7631.gbed76814.el9 ceph 41 M 2023-12-02T00:26:32.428 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-volume noarch 2:18.0.0-7631.gbed76814.el9 ceph-noarch 262 k 2023-12-02T00:26:32.428 INFO:teuthology.orchestra.run.smithi046.stdout: cephadm noarch 2:18.0.0-7631.gbed76814.el9 ceph-noarch 776 k 2023-12-02T00:26:32.428 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-mirror x86_64 2:18.0.0-7631.gbed76814.el9 ceph 218 k 2023-12-02T00:26:32.428 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-top noarch 2:18.0.0-7631.gbed76814.el9 ceph-noarch 18 k 2023-12-02T00:26:32.428 INFO:teuthology.orchestra.run.smithi046.stdout: elfutils-libelf-devel x86_64 0.190-1.el9 CentOS-AppStream 21 k 2023-12-02T00:26:32.428 INFO:teuthology.orchestra.run.smithi046.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-12-02T00:26:32.428 INFO:teuthology.orchestra.run.smithi046.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-12-02T00:26:32.428 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs-devel x86_64 2:18.0.0-7631.gbed76814.el9 ceph 31 k 2023-12-02T00:26:32.428 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs2 x86_64 2:18.0.0-7631.gbed76814.el9 ceph 718 k 2023-12-02T00:26:32.429 INFO:teuthology.orchestra.run.smithi046.stdout: librados-devel x86_64 2:18.0.0-7631.gbed76814.el9 ceph 127 k 2023-12-02T00:26:32.429 INFO:teuthology.orchestra.run.smithi046.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-12-02T00:26:32.429 INFO:teuthology.orchestra.run.smithi046.stdout: libuuid-devel x86_64 2.37.4-15.el9 CentOS-AppStream 24 k 2023-12-02T00:26:32.429 INFO:teuthology.orchestra.run.smithi046.stdout: openssl-devel x86_64 1:3.0.7-25.el9 appstream 4.1 M 2023-12-02T00:26:32.429 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-12-02T00:26:32.429 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-12-02T00:26:32.429 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-12-02T00:26:32.429 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cephfs x86_64 2:18.0.0-7631.gbed76814.el9 ceph 162 k 2023-12-02T00:26:32.429 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rados x86_64 2:18.0.0-7631.gbed76814.el9 ceph 320 k 2023-12-02T00:26:32.429 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rbd x86_64 2:18.0.0-7631.gbed76814.el9 ceph 298 k 2023-12-02T00:26:32.429 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rgw x86_64 2:18.0.0-7631.gbed76814.el9 ceph 99 k 2023-12-02T00:26:32.429 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-fuse x86_64 2:18.0.0-7631.gbed76814.el9 ceph 85 k 2023-12-02T00:26:32.429 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-mirror x86_64 2:18.0.0-7631.gbed76814.el9 ceph 3.0 M 2023-12-02T00:26:32.429 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-nbd x86_64 2:18.0.0-7631.gbed76814.el9 ceph 170 k 2023-12-02T00:26:32.430 INFO:teuthology.orchestra.run.smithi046.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 CentOS-AppStream 85 k 2023-12-02T00:26:32.430 INFO:teuthology.orchestra.run.smithi046.stdout:Upgrading: 2023-12-02T00:26:32.430 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager x86_64 1:1.45.7-1.el9 baseos 2.3 M 2023-12-02T00:26:32.430 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager-libnm x86_64 1:1.45.7-1.el9 baseos 1.8 M 2023-12-02T00:26:32.430 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager-team x86_64 1:1.45.7-1.el9 baseos 43 k 2023-12-02T00:26:32.430 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager-tui x86_64 1:1.45.7-1.el9 baseos 246 k 2023-12-02T00:26:32.430 INFO:teuthology.orchestra.run.smithi046.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2023-12-02T00:26:32.430 INFO:teuthology.orchestra.run.smithi046.stdout: elfutils-debuginfod-client x86_64 0.190-1.el9 CentOS-BaseOS 36 k 2023-12-02T00:26:32.430 INFO:teuthology.orchestra.run.smithi046.stdout: elfutils-libelf x86_64 0.190-1.el9 CentOS-BaseOS 192 k 2023-12-02T00:26:32.430 INFO:teuthology.orchestra.run.smithi046.stdout: elfutils-libs x86_64 0.190-1.el9 CentOS-BaseOS 255 k 2023-12-02T00:26:32.430 INFO:teuthology.orchestra.run.smithi046.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2023-12-02T00:26:32.430 INFO:teuthology.orchestra.run.smithi046.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2023-12-02T00:26:32.430 INFO:teuthology.orchestra.run.smithi046.stdout: librados2 x86_64 2:18.0.0-7631.gbed76814.el9 ceph 3.3 M 2023-12-02T00:26:32.430 INFO:teuthology.orchestra.run.smithi046.stdout: librbd1 x86_64 2:18.0.0-7631.gbed76814.el9 ceph 3.0 M 2023-12-02T00:26:32.431 INFO:teuthology.orchestra.run.smithi046.stdout: openssl x86_64 1:3.0.7-25.el9 baseos 1.2 M 2023-12-02T00:26:32.431 INFO:teuthology.orchestra.run.smithi046.stdout: openssl-libs x86_64 1:3.0.7-25.el9 baseos 2.2 M 2023-12-02T00:26:32.431 INFO:teuthology.orchestra.run.smithi046.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2023-12-02T00:26:32.431 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2023-12-02T00:26:32.431 INFO:teuthology.orchestra.run.smithi046.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2023-12-02T00:26:32.431 INFO:teuthology.orchestra.run.smithi046.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-12-02T00:26:32.431 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-common x86_64 2:18.0.0-7631.gbed76814.el9 ceph 19 M 2023-12-02T00:26:32.431 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7631.gbed76814.el9 ceph-noarch 28 k 2023-12-02T00:26:32.431 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mds x86_64 2:18.0.0-7631.gbed76814.el9 ceph 2.1 M 2023-12-02T00:26:32.431 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7631.gbed76814.el9 ceph-noarch 243 k 2023-12-02T00:26:32.431 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mon x86_64 2:18.0.0-7631.gbed76814.el9 ceph 4.4 M 2023-12-02T00:26:32.431 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-osd x86_64 2:18.0.0-7631.gbed76814.el9 ceph 18 M 2023-12-02T00:26:32.431 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7631.gbed76814.el9 ceph-noarch 15 k 2023-12-02T00:26:32.431 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-selinux x86_64 2:18.0.0-7631.gbed76814.el9 ceph 24 k 2023-12-02T00:26:32.432 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-12-02T00:26:32.432 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-12-02T00:26:32.432 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-12-02T00:26:32.432 INFO:teuthology.orchestra.run.smithi046.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-12-02T00:26:32.432 INFO:teuthology.orchestra.run.smithi046.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-12-02T00:26:32.432 INFO:teuthology.orchestra.run.smithi046.stdout: jq x86_64 1.6-16.el9 CentOS-AppStream 186 k 2023-12-02T00:26:32.432 INFO:teuthology.orchestra.run.smithi046.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-12-02T00:26:32.432 INFO:teuthology.orchestra.run.smithi046.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-12-02T00:26:32.432 INFO:teuthology.orchestra.run.smithi046.stdout: libcephsqlite x86_64 2:18.0.0-7631.gbed76814.el9 ceph 164 k 2023-12-02T00:26:32.432 INFO:teuthology.orchestra.run.smithi046.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-12-02T00:26:32.432 INFO:teuthology.orchestra.run.smithi046.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-12-02T00:26:32.432 INFO:teuthology.orchestra.run.smithi046.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-12-02T00:26:32.432 INFO:teuthology.orchestra.run.smithi046.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-12-02T00:26:32.432 INFO:teuthology.orchestra.run.smithi046.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2023-12-02T00:26:32.433 INFO:teuthology.orchestra.run.smithi046.stdout: libradosstriper1 x86_64 2:18.0.0-7631.gbed76814.el9 ceph 475 k 2023-12-02T00:26:32.433 INFO:teuthology.orchestra.run.smithi046.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-12-02T00:26:32.433 INFO:teuthology.orchestra.run.smithi046.stdout: librgw2 x86_64 2:18.0.0-7631.gbed76814.el9 ceph 4.8 M 2023-12-02T00:26:32.433 INFO:teuthology.orchestra.run.smithi046.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-12-02T00:26:32.433 INFO:teuthology.orchestra.run.smithi046.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-12-02T00:26:32.433 INFO:teuthology.orchestra.run.smithi046.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-12-02T00:26:32.433 INFO:teuthology.orchestra.run.smithi046.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2023-12-02T00:26:32.433 INFO:teuthology.orchestra.run.smithi046.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2023-12-02T00:26:32.433 INFO:teuthology.orchestra.run.smithi046.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2023-12-02T00:26:32.433 INFO:teuthology.orchestra.run.smithi046.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-12-02T00:26:32.433 INFO:teuthology.orchestra.run.smithi046.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-12-02T00:26:32.433 INFO:teuthology.orchestra.run.smithi046.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-12-02T00:26:32.433 INFO:teuthology.orchestra.run.smithi046.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-12-02T00:26:32.433 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-12-02T00:26:32.434 INFO:teuthology.orchestra.run.smithi046.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-12-02T00:26:32.434 INFO:teuthology.orchestra.run.smithi046.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-12-02T00:26:32.434 INFO:teuthology.orchestra.run.smithi046.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-12-02T00:26:32.434 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-12-02T00:26:32.434 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-argparse x86_64 2:18.0.0-7631.gbed76814.el9 ceph 45 k 2023-12-02T00:26:32.434 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-common x86_64 2:18.0.0-7631.gbed76814.el9 ceph 129 k 2023-12-02T00:26:32.434 INFO:teuthology.orchestra.run.smithi046.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-12-02T00:26:32.434 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-12-02T00:26:32.434 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-12-02T00:26:32.434 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-12-02T00:26:32.434 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2023-12-02T00:26:32.434 INFO:teuthology.orchestra.run.smithi046.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2023-12-02T00:26:32.434 INFO:teuthology.orchestra.run.smithi046.stdout: python3-google-auth noarch 1:2.23.4-1.el9 epel 197 k 2023-12-02T00:26:32.434 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-12-02T00:26:32.434 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-12-02T00:26:32.435 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-12-02T00:26:32.435 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-12-02T00:26:32.435 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-12-02T00:26:32.435 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-12-02T00:26:32.435 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-12-02T00:26:32.435 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-12-02T00:26:32.435 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-12-02T00:26:32.435 INFO:teuthology.orchestra.run.smithi046.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-12-02T00:26:32.435 INFO:teuthology.orchestra.run.smithi046.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-12-02T00:26:32.435 INFO:teuthology.orchestra.run.smithi046.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-12-02T00:26:32.435 INFO:teuthology.orchestra.run.smithi046.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-12-02T00:26:32.435 INFO:teuthology.orchestra.run.smithi046.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-12-02T00:26:32.435 INFO:teuthology.orchestra.run.smithi046.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-12-02T00:26:32.435 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-12-02T00:26:32.435 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-12-02T00:26:32.436 INFO:teuthology.orchestra.run.smithi046.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-12-02T00:26:32.436 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-12-02T00:26:32.436 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-12-02T00:26:32.436 INFO:teuthology.orchestra.run.smithi046.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-12-02T00:26:32.436 INFO:teuthology.orchestra.run.smithi046.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-12-02T00:26:32.436 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-12-02T00:26:32.436 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-12-02T00:26:32.436 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-12-02T00:26:32.436 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-12-02T00:26:32.436 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2023-12-02T00:26:32.436 INFO:teuthology.orchestra.run.smithi046.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-12-02T00:26:32.436 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-12-02T00:26:32.436 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-12-02T00:26:32.436 INFO:teuthology.orchestra.run.smithi046.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-12-02T00:26:32.436 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-12-02T00:26:32.437 INFO:teuthology.orchestra.run.smithi046.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-12-02T00:26:32.437 INFO:teuthology.orchestra.run.smithi046.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-12-02T00:26:32.437 INFO:teuthology.orchestra.run.smithi046.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-12-02T00:26:32.437 INFO:teuthology.orchestra.run.smithi046.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-12-02T00:26:32.437 INFO:teuthology.orchestra.run.smithi046.stdout: python3-urllib3 noarch 1.26.5-4.el9 CentOS-BaseOS 187 k 2023-12-02T00:26:32.437 INFO:teuthology.orchestra.run.smithi046.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-12-02T00:26:32.437 INFO:teuthology.orchestra.run.smithi046.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-12-02T00:26:32.437 INFO:teuthology.orchestra.run.smithi046.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-12-02T00:26:32.437 INFO:teuthology.orchestra.run.smithi046.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-12-02T00:26:32.437 INFO:teuthology.orchestra.run.smithi046.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-12-02T00:26:32.437 INFO:teuthology.orchestra.run.smithi046.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-12-02T00:26:32.437 INFO:teuthology.orchestra.run.smithi046.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-12-02T00:26:32.437 INFO:teuthology.orchestra.run.smithi046.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-12-02T00:26:32.438 INFO:teuthology.orchestra.run.smithi046.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-12-02T00:26:32.438 INFO:teuthology.orchestra.run.smithi046.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2023-12-02T00:26:32.438 INFO:teuthology.orchestra.run.smithi046.stdout:Installing weak dependencies: 2023-12-02T00:26:32.438 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-12-02T00:26:32.438 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:26:32.438 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2023-12-02T00:26:32.438 INFO:teuthology.orchestra.run.smithi046.stdout:============================================================================================ 2023-12-02T00:26:32.438 INFO:teuthology.orchestra.run.smithi046.stdout:Install 135 Packages 2023-12-02T00:26:32.438 INFO:teuthology.orchestra.run.smithi046.stdout:Upgrade 15 Packages 2023-12-02T00:26:32.438 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:26:32.439 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 204 M 2023-12-02T00:26:32.439 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2023-12-02T00:26:33.359 INFO:teuthology.orchestra.run.smithi046.stdout:(1/150): ceph-18.0.0-7631.gbed76814.el9.x86_64. 30 kB/s | 6.6 kB 00:00 2023-12-02T00:26:33.475 INFO:teuthology.orchestra.run.smithi162.stdout:(1/150): ceph-18.0.0-7631.gbed76814.el9.x86_64. 27 kB/s | 6.6 kB 00:00 2023-12-02T00:26:33.999 INFO:teuthology.orchestra.run.smithi162.stdout:(2/150): ceph-base-18.0.0-7631.gbed76814.el9.x8 6.8 MB/s | 5.2 MB 00:00 2023-12-02T00:26:34.027 INFO:teuthology.orchestra.run.smithi046.stdout:(2/150): ceph-base-18.0.0-7631.gbed76814.el9.x8 5.9 MB/s | 5.2 MB 00:00 2023-12-02T00:26:34.050 INFO:teuthology.orchestra.run.smithi162.stdout:(3/150): ceph-immutable-object-cache-18.0.0-763 2.8 MB/s | 141 kB 00:00 2023-12-02T00:26:34.068 INFO:teuthology.orchestra.run.smithi046.stdout:(3/150): ceph-fuse-18.0.0-7631.gbed76814.el9.x8 1.2 MB/s | 859 kB 00:00 2023-12-02T00:26:34.118 INFO:teuthology.orchestra.run.smithi046.stdout:(4/150): ceph-immutable-object-cache-18.0.0-763 1.5 MB/s | 141 kB 00:00 2023-12-02T00:26:34.258 INFO:teuthology.orchestra.run.smithi162.stdout:(4/150): ceph-mds-18.0.0-7631.gbed76814.el9.x86 10 MB/s | 2.1 MB 00:00 2023-12-02T00:26:34.367 INFO:teuthology.orchestra.run.smithi162.stdout:(5/150): ceph-fuse-18.0.0-7631.gbed76814.el9.x8 963 kB/s | 859 kB 00:00 2023-12-02T00:26:34.409 INFO:teuthology.orchestra.run.smithi162.stdout:(6/150): ceph-mgr-18.0.0-7631.gbed76814.el9.x86 9.7 MB/s | 1.5 MB 00:00 2023-12-02T00:26:34.435 INFO:teuthology.orchestra.run.smithi046.stdout:(5/150): ceph-mgr-18.0.0-7631.gbed76814.el9.x86 4.6 MB/s | 1.5 MB 00:00 2023-12-02T00:26:34.860 INFO:teuthology.orchestra.run.smithi046.stdout:(6/150): ceph-mds-18.0.0-7631.gbed76814.el9.x86 2.6 MB/s | 2.1 MB 00:00 2023-12-02T00:26:35.917 INFO:teuthology.orchestra.run.smithi162.stdout:(7/150): ceph-osd-18.0.0-7631.gbed76814.el9.x86 12 MB/s | 18 MB 00:01 2023-12-02T00:26:36.219 INFO:teuthology.orchestra.run.smithi046.stdout:(7/150): ceph-mon-18.0.0-7631.gbed76814.el9.x86 2.5 MB/s | 4.4 MB 00:01 2023-12-02T00:26:36.451 INFO:teuthology.orchestra.run.smithi162.stdout:(8/150): ceph-radosgw-18.0.0-7631.gbed76814.el9 16 MB/s | 8.6 MB 00:00 2023-12-02T00:26:36.484 INFO:teuthology.orchestra.run.smithi162.stdout:(9/150): ceph-selinux-18.0.0-7631.gbed76814.el9 730 kB/s | 24 kB 00:00 2023-12-02T00:26:38.102 INFO:teuthology.orchestra.run.smithi046.stdout:(8/150): ceph-common-18.0.0-7631.gbed76814.el9. 3.8 MB/s | 19 MB 00:04 2023-12-02T00:26:38.186 INFO:teuthology.orchestra.run.smithi046.stdout:(9/150): ceph-selinux-18.0.0-7631.gbed76814.el9 290 kB/s | 24 kB 00:00 2023-12-02T00:26:39.127 INFO:teuthology.orchestra.run.smithi162.stdout:(10/150): ceph-test-18.0.0-7631.gbed76814.el9.x 16 MB/s | 41 MB 00:02 2023-12-02T00:26:39.176 INFO:teuthology.orchestra.run.smithi162.stdout:(11/150): cephfs-mirror-18.0.0-7631.gbed76814.e 4.3 MB/s | 218 kB 00:00 2023-12-02T00:26:39.243 INFO:teuthology.orchestra.run.smithi162.stdout:(12/150): ceph-mon-18.0.0-7631.gbed76814.el9.x8 921 kB/s | 4.4 MB 00:04 2023-12-02T00:26:39.271 INFO:teuthology.orchestra.run.smithi162.stdout:(13/150): libcephfs-devel-18.0.0-7631.gbed76814 339 kB/s | 31 kB 00:00 2023-12-02T00:26:39.319 INFO:teuthology.orchestra.run.smithi162.stdout:(14/150): libcephsqlite-18.0.0-7631.gbed76814.e 3.2 MB/s | 164 kB 00:00 2023-12-02T00:26:39.369 INFO:teuthology.orchestra.run.smithi162.stdout:(15/150): librados-devel-18.0.0-7631.gbed76814. 2.5 MB/s | 127 kB 00:00 2023-12-02T00:26:39.428 INFO:teuthology.orchestra.run.smithi162.stdout:(16/150): libradosstriper1-18.0.0-7631.gbed7681 8.0 MB/s | 475 kB 00:00 2023-12-02T00:26:39.478 INFO:teuthology.orchestra.run.smithi046.stdout:(10/150): ceph-radosgw-18.0.0-7631.gbed76814.el 2.6 MB/s | 8.6 MB 00:03 2023-12-02T00:26:39.661 INFO:teuthology.orchestra.run.smithi046.stdout:(11/150): cephfs-mirror-18.0.0-7631.gbed76814.e 1.2 MB/s | 218 kB 00:00 2023-12-02T00:26:39.736 INFO:teuthology.orchestra.run.smithi046.stdout:(12/150): libcephfs-devel-18.0.0-7631.gbed76814 415 kB/s | 31 kB 00:00 2023-12-02T00:26:39.747 INFO:teuthology.orchestra.run.smithi162.stdout:(17/150): librgw2-18.0.0-7631.gbed76814.el9.x86 15 MB/s | 4.8 MB 00:00 2023-12-02T00:26:39.788 INFO:teuthology.orchestra.run.smithi162.stdout:(18/150): python3-ceph-argparse-18.0.0-7631.gbe 1.1 MB/s | 45 kB 00:00 2023-12-02T00:26:39.839 INFO:teuthology.orchestra.run.smithi162.stdout:(19/150): python3-ceph-common-18.0.0-7631.gbed7 2.5 MB/s | 129 kB 00:00 2023-12-02T00:26:39.881 INFO:teuthology.orchestra.run.smithi162.stdout:(20/150): libcephfs2-18.0.0-7631.gbed76814.el9. 1.1 MB/s | 718 kB 00:00 2023-12-02T00:26:39.915 INFO:teuthology.orchestra.run.smithi162.stdout:(21/150): python3-cephfs-18.0.0-7631.gbed76814. 2.1 MB/s | 162 kB 00:00 2023-12-02T00:26:39.965 INFO:teuthology.orchestra.run.smithi162.stdout:(22/150): python3-rbd-18.0.0-7631.gbed76814.el9 5.8 MB/s | 298 kB 00:00 2023-12-02T00:26:39.978 INFO:teuthology.orchestra.run.smithi046.stdout:(13/150): libcephfs2-18.0.0-7631.gbed76814.el9. 2.9 MB/s | 718 kB 00:00 2023-12-02T00:26:40.015 INFO:teuthology.orchestra.run.smithi162.stdout:(23/150): python3-rgw-18.0.0-7631.gbed76814.el9 1.9 MB/s | 99 kB 00:00 2023-12-02T00:26:40.065 INFO:teuthology.orchestra.run.smithi162.stdout:(24/150): rbd-fuse-18.0.0-7631.gbed76814.el9.x8 1.7 MB/s | 85 kB 00:00 2023-12-02T00:26:40.100 INFO:teuthology.orchestra.run.smithi162.stdout:(25/150): python3-rados-18.0.0-7631.gbed76814.e 1.4 MB/s | 320 kB 00:00 2023-12-02T00:26:40.154 INFO:teuthology.orchestra.run.smithi046.stdout:(14/150): ceph-osd-18.0.0-7631.gbed76814.el9.x8 3.3 MB/s | 18 MB 00:05 2023-12-02T00:26:40.171 INFO:teuthology.orchestra.run.smithi046.stdout:(15/150): libcephsqlite-18.0.0-7631.gbed76814.e 853 kB/s | 164 kB 00:00 2023-12-02T00:26:40.263 INFO:teuthology.orchestra.run.smithi046.stdout:(16/150): librados-devel-18.0.0-7631.gbed76814. 1.1 MB/s | 127 kB 00:00 2023-12-02T00:26:40.266 INFO:teuthology.orchestra.run.smithi162.stdout:(26/150): rbd-mirror-18.0.0-7631.gbed76814.el9. 15 MB/s | 3.0 MB 00:00 2023-12-02T00:26:40.291 INFO:teuthology.orchestra.run.smithi162.stdout:(27/150): rbd-nbd-18.0.0-7631.gbed76814.el9.x86 886 kB/s | 170 kB 00:00 2023-12-02T00:26:40.317 INFO:teuthology.orchestra.run.smithi162.stdout:(28/150): ceph-grafana-dashboards-18.0.0-7631.g 561 kB/s | 28 kB 00:00 2023-12-02T00:26:40.430 INFO:teuthology.orchestra.run.smithi046.stdout:(17/150): libradosstriper1-18.0.0-7631.gbed7681 1.8 MB/s | 475 kB 00:00 2023-12-02T00:26:40.450 INFO:teuthology.orchestra.run.smithi162.stdout:(29/150): ceph-mgr-dashboard-18.0.0-7631.gbed76 13 MB/s | 1.8 MB 00:00 2023-12-02T00:26:40.476 INFO:teuthology.orchestra.run.smithi162.stdout:(30/150): ceph-mgr-cephadm-18.0.0-7631.gbed7681 722 kB/s | 133 kB 00:00 2023-12-02T00:26:40.513 INFO:teuthology.orchestra.run.smithi046.stdout:(18/150): python3-ceph-argparse-18.0.0-7631.gbe 542 kB/s | 45 kB 00:00 2023-12-02T00:26:40.614 INFO:teuthology.orchestra.run.smithi046.stdout:(19/150): python3-ceph-common-18.0.0-7631.gbed7 1.3 MB/s | 129 kB 00:00 2023-12-02T00:26:40.701 INFO:teuthology.orchestra.run.smithi162.stdout:(31/150): ceph-mgr-modules-core-18.0.0-7631.gbe 1.1 MB/s | 243 kB 00:00 2023-12-02T00:26:40.714 INFO:teuthology.orchestra.run.smithi046.stdout:(20/150): python3-cephfs-18.0.0-7631.gbed76814. 1.6 MB/s | 162 kB 00:00 2023-12-02T00:26:40.810 INFO:teuthology.orchestra.run.smithi162.stdout:(32/150): ceph-mgr-rook-18.0.0-7631.gbed76814.e 462 kB/s | 50 kB 00:00 2023-12-02T00:26:40.906 INFO:teuthology.orchestra.run.smithi046.stdout:(21/150): python3-rados-18.0.0-7631.gbed76814.e 1.6 MB/s | 320 kB 00:00 2023-12-02T00:26:40.935 INFO:teuthology.orchestra.run.smithi162.stdout:(33/150): ceph-mgr-diskprediction-local-18.0.0- 15 MB/s | 7.4 MB 00:00 2023-12-02T00:26:40.960 INFO:teuthology.orchestra.run.smithi162.stdout:(34/150): ceph-prometheus-alerts-18.0.0-7631.gb 100 kB/s | 15 kB 00:00 2023-12-02T00:26:41.027 INFO:teuthology.orchestra.run.smithi162.stdout:(35/150): cephadm-18.0.0-7631.gbed76814.el9.noa 11 MB/s | 776 kB 00:00 2023-12-02T00:26:41.069 INFO:teuthology.orchestra.run.smithi162.stdout:(36/150): cephfs-top-18.0.0-7631.gbed76814.el9. 439 kB/s | 18 kB 00:00 2023-12-02T00:26:41.116 INFO:teuthology.orchestra.run.smithi046.stdout:(22/150): python3-rbd-18.0.0-7631.gbed76814.el9 1.4 MB/s | 298 kB 00:00 2023-12-02T00:26:41.161 INFO:teuthology.orchestra.run.smithi162.stdout:(37/150): ceph-volume-18.0.0-7631.gbed76814.el9 1.1 MB/s | 262 kB 00:00 2023-12-02T00:26:41.190 INFO:teuthology.orchestra.run.smithi046.stdout:(23/150): python3-rgw-18.0.0-7631.gbed76814.el9 1.3 MB/s | 99 kB 00:00 2023-12-02T00:26:41.273 INFO:teuthology.orchestra.run.smithi046.stdout:(24/150): rbd-fuse-18.0.0-7631.gbed76814.el9.x8 1.0 MB/s | 85 kB 00:00 2023-12-02T00:26:41.370 INFO:teuthology.orchestra.run.smithi162.stdout:(38/150): automake-1.16.2-8.el9.noarch.rpm 2.2 MB/s | 662 kB 00:00 2023-12-02T00:26:41.403 INFO:teuthology.orchestra.run.smithi162.stdout:(39/150): bison-3.7.4-5.el9.x86_64.rpm 3.7 MB/s | 920 kB 00:00 2023-12-02T00:26:41.428 INFO:teuthology.orchestra.run.smithi162.stdout:(40/150): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 106 kB 00:00 2023-12-02T00:26:41.454 INFO:teuthology.orchestra.run.smithi162.stdout:(41/150): elfutils-libelf-devel-0.190-1.el9.x86 430 kB/s | 21 kB 00:00 2023-12-02T00:26:41.465 INFO:teuthology.orchestra.run.smithi046.stdout:(25/150): librgw2-18.0.0-7631.gbed76814.el9.x86 4.0 MB/s | 4.8 MB 00:01 2023-12-02T00:26:41.479 INFO:teuthology.orchestra.run.smithi162.stdout:(42/150): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2023-12-02T00:26:41.505 INFO:teuthology.orchestra.run.smithi162.stdout:(43/150): flexiblas-3.0.4-8.el9.x86_64.rpm 639 kB/s | 32 kB 00:00 2023-12-02T00:26:41.540 INFO:teuthology.orchestra.run.smithi046.stdout:(26/150): rbd-nbd-18.0.0-7631.gbed76814.el9.x86 2.2 MB/s | 170 kB 00:00 2023-12-02T00:26:41.588 INFO:teuthology.orchestra.run.smithi162.stdout:(44/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 28 MB/s | 3.0 MB 00:00 2023-12-02T00:26:41.599 INFO:teuthology.orchestra.run.smithi046.stdout:(27/150): ceph-grafana-dashboards-18.0.0-7631.g 484 kB/s | 28 kB 00:00 2023-12-02T00:26:41.605 INFO:teuthology.orchestra.run.smithi162.stdout:(45/150): flexiblas-openblas-openmp-3.0.4-8.el9 175 kB/s | 17 kB 00:00 2023-12-02T00:26:41.639 INFO:teuthology.orchestra.run.smithi162.stdout:(46/150): libaio-devel-0.3.111-13.el9.x86_64.rp 306 kB/s | 10 kB 00:00 2023-12-02T00:26:41.664 INFO:teuthology.orchestra.run.smithi162.stdout:(47/150): jq-1.6-16.el9.x86_64.rpm 3.1 MB/s | 186 kB 00:00 2023-12-02T00:26:41.666 INFO:teuthology.orchestra.run.smithi046.stdout:(28/150): ceph-mgr-cephadm-18.0.0-7631.gbed7681 1.9 MB/s | 133 kB 00:00 2023-12-02T00:26:41.689 INFO:teuthology.orchestra.run.smithi162.stdout:(48/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2023-12-02T00:26:41.706 INFO:teuthology.orchestra.run.smithi162.stdout:(49/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2023-12-02T00:26:41.748 INFO:teuthology.orchestra.run.smithi162.stdout:(50/150): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 661 kB 00:00 2023-12-02T00:26:41.774 INFO:teuthology.orchestra.run.smithi162.stdout:(51/150): libtool-2.4.6-45.el9.x86_64.rpm 8.6 MB/s | 578 kB 00:00 2023-12-02T00:26:41.790 INFO:teuthology.orchestra.run.smithi162.stdout:(52/150): libuuid-devel-2.37.4-15.el9.x86_64.rp 598 kB/s | 24 kB 00:00 2023-12-02T00:26:41.807 INFO:teuthology.orchestra.run.smithi162.stdout:(53/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 46 kB 00:00 2023-12-02T00:26:41.833 INFO:teuthology.orchestra.run.smithi162.stdout:(54/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.6 MB/s | 282 kB 00:00 2023-12-02T00:26:41.858 INFO:teuthology.orchestra.run.smithi162.stdout:(55/150): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2023-12-02T00:26:41.883 INFO:teuthology.orchestra.run.smithi162.stdout:(56/150): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2023-12-02T00:26:41.908 INFO:teuthology.orchestra.run.smithi162.stdout:(57/150): openblas-0.3.21-2.el9.x86_64.rpm 687 kB/s | 34 kB 00:00 2023-12-02T00:26:42.008 INFO:teuthology.orchestra.run.smithi046.stdout:(29/150): ceph-mgr-dashboard-18.0.0-7631.gbed76 5.2 MB/s | 1.8 MB 00:00 2023-12-02T00:26:42.051 INFO:teuthology.orchestra.run.smithi162.stdout:(58/150): openblas-openmp-0.3.21-2.el9.x86_64.r 28 MB/s | 4.7 MB 00:00 2023-12-02T00:26:42.093 INFO:teuthology.orchestra.run.smithi162.stdout:(59/150): postgresql-13.11-1.el9.x86_64.rpm 8.4 MB/s | 1.5 MB 00:00 2023-12-02T00:26:42.135 INFO:teuthology.orchestra.run.smithi162.stdout:(60/150): postgresql-contrib-13.11-1.el9.x86_64 9.8 MB/s | 811 kB 00:00 2023-12-02T00:26:42.160 INFO:teuthology.orchestra.run.smithi162.stdout:(61/150): postgresql-private-libs-13.11-1.el9.x 2.0 MB/s | 134 kB 00:00 2023-12-02T00:26:42.233 INFO:teuthology.orchestra.run.smithi046.stdout:(30/150): rbd-mirror-18.0.0-7631.gbed76814.el9. 3.1 MB/s | 3.0 MB 00:00 2023-12-02T00:26:42.344 INFO:teuthology.orchestra.run.smithi162.stdout:(62/150): postgresql-server-13.11-1.el9.x86_64. 27 MB/s | 5.7 MB 00:00 2023-12-02T00:26:42.383 INFO:teuthology.orchestra.run.smithi046.stdout:(31/150): ceph-mgr-modules-core-18.0.0-7631.gbe 1.6 MB/s | 243 kB 00:00 2023-12-02T00:26:42.450 INFO:teuthology.orchestra.run.smithi046.stdout:(32/150): ceph-mgr-rook-18.0.0-7631.gbed76814.e 754 kB/s | 50 kB 00:00 2023-12-02T00:26:42.494 INFO:teuthology.orchestra.run.smithi162.stdout:(63/150): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 5.8 MB 00:00 2023-12-02T00:26:42.517 INFO:teuthology.orchestra.run.smithi046.stdout:(33/150): ceph-prometheus-alerts-18.0.0-7631.gb 228 kB/s | 15 kB 00:00 2023-12-02T00:26:42.519 INFO:teuthology.orchestra.run.smithi162.stdout:(64/150): python3-jinja2-2.11.3-4.el9.noarch.rp 1.3 MB/s | 228 kB 00:00 2023-12-02T00:26:42.544 INFO:teuthology.orchestra.run.smithi162.stdout:(65/150): python3-jsonpatch-1.21-16.el9.noarch. 492 kB/s | 24 kB 00:00 2023-12-02T00:26:42.570 INFO:teuthology.orchestra.run.smithi162.stdout:(66/150): python3-jsonpointer-2.0-4.el9.noarch. 361 kB/s | 18 kB 00:00 2023-12-02T00:26:42.595 INFO:teuthology.orchestra.run.smithi162.stdout:(67/150): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2023-12-02T00:26:42.620 INFO:teuthology.orchestra.run.smithi162.stdout:(68/150): python3-markupsafe-1.1.1-12.el9.x86_6 631 kB/s | 32 kB 00:00 2023-12-02T00:26:42.634 INFO:teuthology.orchestra.run.smithi046.stdout:(34/150): ceph-volume-18.0.0-7631.gbed76814.el9 2.2 MB/s | 262 kB 00:00 2023-12-02T00:26:42.796 INFO:teuthology.orchestra.run.smithi162.stdout:(69/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 24 MB/s | 4.8 MB 00:00 2023-12-02T00:26:42.822 INFO:teuthology.orchestra.run.smithi162.stdout:(70/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.1 MB/s | 217 kB 00:00 2023-12-02T00:26:42.846 INFO:teuthology.orchestra.run.smithi162.stdout:(71/150): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2023-12-02T00:26:42.880 INFO:teuthology.orchestra.run.smithi162.stdout:(72/150): python3-prettytable-0.7.2-27.el9.noar 703 kB/s | 41 kB 00:00 2023-12-02T00:26:42.913 INFO:teuthology.orchestra.run.smithi162.stdout:(73/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 132 kB 00:00 2023-12-02T00:26:42.926 INFO:teuthology.orchestra.run.smithi046.stdout:(35/150): cephadm-18.0.0-7631.gbed76814.el9.noa 2.6 MB/s | 776 kB 00:00 2023-12-02T00:26:42.939 INFO:teuthology.orchestra.run.smithi162.stdout:(74/150): python3-pyasn1-modules-0.4.8-6.el9.no 3.5 MB/s | 211 kB 00:00 2023-12-02T00:26:42.964 INFO:teuthology.orchestra.run.smithi162.stdout:(75/150): python3-pytz-2021.1-5.el9.noarch.rpm 951 kB/s | 47 kB 00:00 2023-12-02T00:26:42.993 INFO:teuthology.orchestra.run.smithi046.stdout:(36/150): cephfs-top-18.0.0-7631.gbed76814.el9. 274 kB/s | 18 kB 00:00 2023-12-02T00:26:43.014 INFO:teuthology.orchestra.run.smithi162.stdout:(76/150): python3-requests-oauthlib-1.3.0-12.el 570 kB/s | 43 kB 00:00 2023-12-02T00:26:43.323 INFO:teuthology.orchestra.run.smithi162.stdout:(77/150): python3-toml-0.10.2-6.el9.noarch.rpm 126 kB/s | 39 kB 00:00 2023-12-02T00:26:43.385 INFO:teuthology.orchestra.run.smithi046.stdout:(37/150): automake-1.16.2-8.el9.noarch.rpm 1.6 MB/s | 662 kB 00:00 2023-12-02T00:26:43.515 INFO:teuthology.orchestra.run.smithi162.stdout:(78/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 28 MB/s | 16 MB 00:00 2023-12-02T00:26:43.527 INFO:teuthology.orchestra.run.smithi046.stdout:(38/150): ceph-mgr-diskprediction-local-18.0.0- 4.9 MB/s | 7.4 MB 00:01 2023-12-02T00:26:43.540 INFO:teuthology.orchestra.run.smithi162.stdout:(79/150): socat-1.7.4.1-5.el9.x86_64.rpm 1.4 MB/s | 300 kB 00:00 2023-12-02T00:26:43.561 INFO:teuthology.orchestra.run.smithi046.stdout:(39/150): bison-3.7.4-5.el9.x86_64.rpm 5.1 MB/s | 920 kB 00:00 2023-12-02T00:26:43.582 INFO:teuthology.orchestra.run.smithi162.stdout:(80/150): uuid-1.6.2-55.el9.x86_64.rpm 852 kB/s | 56 kB 00:00 2023-12-02T00:26:43.586 INFO:teuthology.orchestra.run.smithi046.stdout:(40/150): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 106 kB 00:00 2023-12-02T00:26:43.603 INFO:teuthology.orchestra.run.smithi046.stdout:(41/150): elfutils-libelf-devel-0.190-1.el9.x86 519 kB/s | 21 kB 00:00 2023-12-02T00:26:43.607 INFO:teuthology.orchestra.run.smithi162.stdout:(81/150): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 1.2 MB/s | 85 kB 00:00 2023-12-02T00:26:43.634 INFO:teuthology.orchestra.run.smithi162.stdout:(82/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2023-12-02T00:26:43.645 INFO:teuthology.orchestra.run.smithi046.stdout:(42/150): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 307 kB 00:00 2023-12-02T00:26:43.662 INFO:teuthology.orchestra.run.smithi046.stdout:(43/150): flexiblas-3.0.4-8.el9.x86_64.rpm 545 kB/s | 32 kB 00:00 2023-12-02T00:26:43.666 INFO:teuthology.orchestra.run.smithi162.stdout:(83/150): zlib-devel-1.2.11-40.el9.x86_64.rpm 752 kB/s | 44 kB 00:00 2023-12-02T00:26:43.692 INFO:teuthology.orchestra.run.smithi162.stdout:(84/150): fuse-2.9.9-15.el9.x86_64.rpm 1.3 MB/s | 79 kB 00:00 2023-12-02T00:26:43.742 INFO:teuthology.orchestra.run.smithi162.stdout:(85/150): libgfortran-11.4.1-2.3.el9.x86_64.rpm 10 MB/s | 803 kB 00:00 2023-12-02T00:26:43.767 INFO:teuthology.orchestra.run.smithi162.stdout:(86/150): libquadmath-11.4.1-2.3.el9.x86_64.rpm 2.6 MB/s | 195 kB 00:00 2023-12-02T00:26:43.770 INFO:teuthology.orchestra.run.smithi046.stdout:(44/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2023-12-02T00:26:43.796 INFO:teuthology.orchestra.run.smithi046.stdout:(45/150): flexiblas-openblas-openmp-3.0.4-8.el9 129 kB/s | 17 kB 00:00 2023-12-02T00:26:43.802 INFO:teuthology.orchestra.run.smithi162.stdout:(87/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.0 MB/s | 241 kB 00:00 2023-12-02T00:26:43.821 INFO:teuthology.orchestra.run.smithi046.stdout:(46/150): jq-1.6-16.el9.x86_64.rpm 3.6 MB/s | 186 kB 00:00 2023-12-02T00:26:43.846 INFO:teuthology.orchestra.run.smithi162.stdout:(88/150): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.1 MB 00:00 2023-12-02T00:26:43.847 INFO:teuthology.orchestra.run.smithi046.stdout:(47/150): libaio-devel-0.3.111-13.el9.x86_64.rp 203 kB/s | 10 kB 00:00 2023-12-02T00:26:43.869 INFO:teuthology.orchestra.run.smithi162.stdout:(89/150): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 103 kB 00:00 2023-12-02T00:26:43.871 INFO:teuthology.orchestra.run.smithi046.stdout:(48/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2023-12-02T00:26:43.894 INFO:teuthology.orchestra.run.smithi162.stdout:(90/150): python3-pycparser-2.20-6.el9.noarch.r 2.4 MB/s | 124 kB 00:00 2023-12-02T00:26:43.896 INFO:teuthology.orchestra.run.smithi046.stdout:(49/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 886 kB/s | 44 kB 00:00 2023-12-02T00:26:43.922 INFO:teuthology.orchestra.run.smithi046.stdout:(50/150): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2023-12-02T00:26:43.927 INFO:teuthology.orchestra.run.smithi162.stdout:(91/150): python3-requests-2.25.1-7.el9.noarch. 1.9 MB/s | 113 kB 00:00 2023-12-02T00:26:43.948 INFO:teuthology.orchestra.run.smithi046.stdout:(51/150): libtool-2.4.6-45.el9.x86_64.rpm 11 MB/s | 578 kB 00:00 2023-12-02T00:26:43.961 INFO:teuthology.orchestra.run.smithi162.stdout:(92/150): python3-urllib3-1.26.5-4.el9.noarch.r 2.7 MB/s | 187 kB 00:00 2023-12-02T00:26:43.973 INFO:teuthology.orchestra.run.smithi046.stdout:(52/150): libuuid-devel-2.37.4-15.el9.x86_64.rp 496 kB/s | 24 kB 00:00 2023-12-02T00:26:43.998 INFO:teuthology.orchestra.run.smithi046.stdout:(53/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 925 kB/s | 46 kB 00:00 2023-12-02T00:26:44.023 INFO:teuthology.orchestra.run.smithi046.stdout:(54/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2023-12-02T00:26:44.049 INFO:teuthology.orchestra.run.smithi046.stdout:(55/150): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2023-12-02T00:26:44.074 INFO:teuthology.orchestra.run.smithi046.stdout:(56/150): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2023-12-02T00:26:44.095 INFO:teuthology.orchestra.run.smithi162.stdout:(93/150): python3-devel-3.9.17-2.el9.x86_64.rpm 1.8 MB/s | 245 kB 00:00 2023-12-02T00:26:44.099 INFO:teuthology.orchestra.run.smithi046.stdout:(57/150): openblas-0.3.21-2.el9.x86_64.rpm 688 kB/s | 34 kB 00:00 2023-12-02T00:26:44.212 INFO:teuthology.orchestra.run.smithi162.stdout:(94/150): openssl-devel-3.0.7-25.el9.x86_64.rpm 15 MB/s | 4.1 MB 00:00 2023-12-02T00:26:44.225 INFO:teuthology.orchestra.run.smithi046.stdout:(58/150): openblas-openmp-0.3.21-2.el9.x86_64.r 32 MB/s | 4.7 MB 00:00 2023-12-02T00:26:44.267 INFO:teuthology.orchestra.run.smithi046.stdout:(59/150): postgresql-13.11-1.el9.x86_64.rpm 9.2 MB/s | 1.5 MB 00:00 2023-12-02T00:26:44.300 INFO:teuthology.orchestra.run.smithi046.stdout:(60/150): postgresql-contrib-13.11-1.el9.x86_64 11 MB/s | 811 kB 00:00 2023-12-02T00:26:44.325 INFO:teuthology.orchestra.run.smithi046.stdout:(61/150): postgresql-private-libs-13.11-1.el9.x 2.3 MB/s | 134 kB 00:00 2023-12-02T00:26:44.475 INFO:teuthology.orchestra.run.smithi046.stdout:(62/150): postgresql-server-13.11-1.el9.x86_64. 33 MB/s | 5.7 MB 00:00 2023-12-02T00:26:44.592 INFO:teuthology.orchestra.run.smithi046.stdout:(63/150): python3-babel-2.9.1-2.el9.noarch.rpm 22 MB/s | 5.8 MB 00:00 2023-12-02T00:26:44.618 INFO:teuthology.orchestra.run.smithi046.stdout:(64/150): python3-jinja2-2.11.3-4.el9.noarch.rp 1.6 MB/s | 228 kB 00:00 2023-12-02T00:26:44.643 INFO:teuthology.orchestra.run.smithi046.stdout:(65/150): python3-jsonpatch-1.21-16.el9.noarch. 492 kB/s | 24 kB 00:00 2023-12-02T00:26:44.668 INFO:teuthology.orchestra.run.smithi046.stdout:(66/150): python3-jsonpointer-2.0-4.el9.noarch. 361 kB/s | 18 kB 00:00 2023-12-02T00:26:44.687 INFO:teuthology.orchestra.run.smithi162.stdout:(95/150): gperftools-libs-2.9.1-2.el9.x86_64.rp 650 kB/s | 309 kB 00:00 2023-12-02T00:26:44.693 INFO:teuthology.orchestra.run.smithi046.stdout:(67/150): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2023-12-02T00:26:44.719 INFO:teuthology.orchestra.run.smithi046.stdout:(68/150): python3-markupsafe-1.1.1-12.el9.x86_6 633 kB/s | 32 kB 00:00 2023-12-02T00:26:44.754 INFO:teuthology.orchestra.run.smithi162.stdout:(96/150): lua-devel-5.4.4-4.el9.x86_64.rpm 34 kB/s | 22 kB 00:00 2023-12-02T00:26:44.854 INFO:teuthology.orchestra.run.smithi046.stdout:(69/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 30 MB/s | 4.8 MB 00:00 2023-12-02T00:26:44.878 INFO:teuthology.orchestra.run.smithi046.stdout:(70/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.3 MB/s | 217 kB 00:00 2023-12-02T00:26:44.903 INFO:teuthology.orchestra.run.smithi046.stdout:(71/150): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2023-12-02T00:26:44.928 INFO:teuthology.orchestra.run.smithi046.stdout:(72/150): python3-prettytable-0.7.2-27.el9.noar 824 kB/s | 41 kB 00:00 2023-12-02T00:26:44.953 INFO:teuthology.orchestra.run.smithi046.stdout:(73/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2023-12-02T00:26:44.979 INFO:teuthology.orchestra.run.smithi046.stdout:(74/150): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2023-12-02T00:26:45.006 INFO:teuthology.orchestra.run.smithi046.stdout:(75/150): python3-pytz-2021.1-5.el9.noarch.rpm 949 kB/s | 47 kB 00:00 2023-12-02T00:26:45.021 INFO:teuthology.orchestra.run.smithi046.stdout:(76/150): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 43 kB 00:00 2023-12-02T00:26:45.021 INFO:teuthology.orchestra.run.smithi162.stdout:(97/150): libarrow-doc-9.0.0-9.el9.noarch.rpm 91 kB/s | 24 kB 00:00 2023-12-02T00:26:45.082 INFO:teuthology.orchestra.run.smithi162.stdout:(98/150): libarrow-9.0.0-9.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2023-12-02T00:26:45.163 INFO:teuthology.orchestra.run.smithi162.stdout:(99/150): libunwind-1.6.2-1.el9.x86_64.rpm 811 kB/s | 67 kB 00:00 2023-12-02T00:26:45.188 INFO:teuthology.orchestra.run.smithi162.stdout:(100/150): liboath-2.6.7-2.el9.x86_64.rpm 291 kB/s | 49 kB 00:00 2023-12-02T00:26:45.196 INFO:teuthology.orchestra.run.smithi046.stdout:(77/150): python3-toml-0.10.2-6.el9.noarch.rpm 221 kB/s | 39 kB 00:00 2023-12-02T00:26:45.255 INFO:teuthology.orchestra.run.smithi162.stdout:(101/150): luarocks-3.9.2-1.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2023-12-02T00:26:45.356 INFO:teuthology.orchestra.run.smithi162.stdout:(102/150): python3-asyncssh-2.13.2-3.el9.noarch 5.4 MB/s | 548 kB 00:00 2023-12-02T00:26:45.438 INFO:teuthology.orchestra.run.smithi046.stdout:(78/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 36 MB/s | 16 MB 00:00 2023-12-02T00:26:45.463 INFO:teuthology.orchestra.run.smithi046.stdout:(79/150): socat-1.7.4.1-5.el9.x86_64.rpm 1.1 MB/s | 300 kB 00:00 2023-12-02T00:26:45.480 INFO:teuthology.orchestra.run.smithi046.stdout:(80/150): uuid-1.6.2-55.el9.x86_64.rpm 1.3 MB/s | 56 kB 00:00 2023-12-02T00:26:45.522 INFO:teuthology.orchestra.run.smithi046.stdout:(81/150): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 1.4 MB/s | 85 kB 00:00 2023-12-02T00:26:45.531 INFO:teuthology.orchestra.run.smithi162.stdout:(103/150): ceph-common-18.0.0-7631.gbed76814.el 1.5 MB/s | 19 MB 00:12 2023-12-02T00:26:45.548 INFO:teuthology.orchestra.run.smithi046.stdout:(82/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 936 kB/s | 63 kB 00:00 2023-12-02T00:26:45.565 INFO:teuthology.orchestra.run.smithi162.stdout:(104/150): parquet-libs-9.0.0-9.el9.x86_64.rpm 2.2 MB/s | 837 kB 00:00 2023-12-02T00:26:45.573 INFO:teuthology.orchestra.run.smithi046.stdout:(83/150): zlib-devel-1.2.11-40.el9.x86_64.rpm 879 kB/s | 44 kB 00:00 2023-12-02T00:26:45.598 INFO:teuthology.orchestra.run.smithi046.stdout:(84/150): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2023-12-02T00:26:45.640 INFO:teuthology.orchestra.run.smithi162.stdout:(105/150): python3-certifi-2021.10.8-2.el9.noar 200 kB/s | 15 kB 00:00 2023-12-02T00:26:45.640 INFO:teuthology.orchestra.run.smithi046.stdout:(85/150): libgfortran-11.4.1-2.3.el9.x86_64.rpm 12 MB/s | 803 kB 00:00 2023-12-02T00:26:45.666 INFO:teuthology.orchestra.run.smithi046.stdout:(86/150): libquadmath-11.4.1-2.3.el9.x86_64.rpm 2.9 MB/s | 195 kB 00:00 2023-12-02T00:26:45.690 INFO:teuthology.orchestra.run.smithi162.stdout:(106/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 130 kB/s | 43 kB 00:00 2023-12-02T00:26:45.691 INFO:teuthology.orchestra.run.smithi046.stdout:(87/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2023-12-02T00:26:45.733 INFO:teuthology.orchestra.run.smithi046.stdout:(88/150): python3-cryptography-36.0.1-4.el9.x86 17 MB/s | 1.1 MB 00:00 2023-12-02T00:26:45.758 INFO:teuthology.orchestra.run.smithi046.stdout:(89/150): python3-ply-3.11-14.el9.noarch.rpm 1.5 MB/s | 103 kB 00:00 2023-12-02T00:26:45.775 INFO:teuthology.orchestra.run.smithi046.stdout:(90/150): python3-pycparser-2.20-6.el9.noarch.r 2.9 MB/s | 124 kB 00:00 2023-12-02T00:26:45.790 INFO:teuthology.orchestra.run.smithi162.stdout:(107/150): python3-cachetools-4.2.4-1.el9.noarc 124 kB/s | 32 kB 00:00 2023-12-02T00:26:45.800 INFO:teuthology.orchestra.run.smithi046.stdout:(91/150): python3-requests-2.25.1-7.el9.noarch. 2.6 MB/s | 113 kB 00:00 2023-12-02T00:26:45.826 INFO:teuthology.orchestra.run.smithi046.stdout:(92/150): python3-urllib3-1.26.5-4.el9.noarch.r 3.6 MB/s | 187 kB 00:00 2023-12-02T00:26:45.866 INFO:teuthology.orchestra.run.smithi162.stdout:(108/150): python3-cheroot-8.6.0-4.el9.noarch.r 762 kB/s | 172 kB 00:00 2023-12-02T00:26:45.891 INFO:teuthology.orchestra.run.smithi162.stdout:(109/150): python3-cherrypy-18.6.1-2.el9.noarch 1.7 MB/s | 358 kB 00:00 2023-12-02T00:26:46.016 INFO:teuthology.orchestra.run.smithi162.stdout:(110/150): python3-google-auth-2.23.4-1.el9.noa 875 kB/s | 197 kB 00:00 2023-12-02T00:26:46.033 INFO:teuthology.orchestra.run.smithi162.stdout:(111/150): python3-jaraco-8.2.1-3.el9.noarch.rp 64 kB/s | 11 kB 00:00 2023-12-02T00:26:46.050 INFO:teuthology.orchestra.run.smithi162.stdout:(112/150): python3-jaraco-classes-3.2.1-5.el9.n 111 kB/s | 18 kB 00:00 2023-12-02T00:26:46.084 INFO:teuthology.orchestra.run.smithi162.stdout:(113/150): python3-jaraco-collections-3.0.0-8.e 345 kB/s | 23 kB 00:00 2023-12-02T00:26:46.109 INFO:teuthology.orchestra.run.smithi162.stdout:(114/150): python3-jaraco-functools-3.5.0-2.el9 257 kB/s | 19 kB 00:00 2023-12-02T00:26:46.126 INFO:teuthology.orchestra.run.smithi162.stdout:(115/150): python3-jaraco-text-3.2.0-6.el9.noar 260 kB/s | 20 kB 00:00 2023-12-02T00:26:46.160 INFO:teuthology.orchestra.run.smithi162.stdout:(116/150): python3-jwt+crypto-2.4.0-1.el9.noarc 119 kB/s | 9.0 kB 00:00 2023-12-02T00:26:46.185 INFO:teuthology.orchestra.run.smithi162.stdout:(117/150): python3-jwt-2.4.0-1.el9.noarch.rpm 542 kB/s | 41 kB 00:00 2023-12-02T00:26:46.268 INFO:teuthology.orchestra.run.smithi162.stdout:(118/150): python3-logutils-0.3.5-21.el9.noarch 427 kB/s | 46 kB 00:00 2023-12-02T00:26:46.302 INFO:teuthology.orchestra.run.smithi162.stdout:(119/150): python3-kubernetes-26.1.0-2.el9.noar 5.8 MB/s | 1.0 MB 00:00 2023-12-02T00:26:46.334 INFO:teuthology.orchestra.run.smithi046.stdout:(93/150): python3-devel-3.9.17-2.el9.x86_64.rpm 483 kB/s | 245 kB 00:00 2023-12-02T00:26:46.335 INFO:teuthology.orchestra.run.smithi162.stdout:(120/150): python3-more-itertools-8.12.0-2.el9. 524 kB/s | 79 kB 00:00 2023-12-02T00:26:46.361 INFO:teuthology.orchestra.run.smithi162.stdout:(121/150): python3-natsort-7.1.1-5.el9.noarch.r 625 kB/s | 58 kB 00:00 2023-12-02T00:26:46.403 INFO:teuthology.orchestra.run.smithi162.stdout:(122/150): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2023-12-02T00:26:46.436 INFO:teuthology.orchestra.run.smithi162.stdout:(123/150): python3-portend-3.1.0-2.el9.noarch.r 163 kB/s | 16 kB 00:00 2023-12-02T00:26:46.459 INFO:teuthology.orchestra.run.smithi046.stdout:(94/150): lua-devel-5.4.4-4.el9.x86_64.rpm 179 kB/s | 22 kB 00:00 2023-12-02T00:26:46.478 INFO:teuthology.orchestra.run.smithi162.stdout:(124/150): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2023-12-02T00:26:46.521 INFO:teuthology.orchestra.run.smithi162.stdout:(125/150): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2023-12-02T00:26:46.546 INFO:teuthology.orchestra.run.smithi162.stdout:(126/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 487 kB/s | 90 kB 00:00 2023-12-02T00:26:46.571 INFO:teuthology.orchestra.run.smithi162.stdout:(127/150): python3-rsa-4.9-2.el9.noarch.rpm 640 kB/s | 59 kB 00:00 2023-12-02T00:26:46.638 INFO:teuthology.orchestra.run.smithi162.stdout:(128/150): python3-tempora-5.0.0-2.el9.noarch.r 305 kB/s | 36 kB 00:00 2023-12-02T00:26:46.671 INFO:teuthology.orchestra.run.smithi162.stdout:(129/150): python3-typing-extensions-4.4.0-2.el 407 kB/s | 51 kB 00:00 2023-12-02T00:26:46.722 INFO:teuthology.orchestra.run.smithi162.stdout:(130/150): python3-webob-1.8.7-6.el9.noarch.rpm 1.5 MB/s | 230 kB 00:00 2023-12-02T00:26:46.747 INFO:teuthology.orchestra.run.smithi162.stdout:(131/150): python3-websocket-client-1.2.3-2.el9 825 kB/s | 90 kB 00:00 2023-12-02T00:26:46.751 INFO:teuthology.orchestra.run.smithi046.stdout:(95/150): openssl-devel-3.0.7-25.el9.x86_64.rpm 4.3 MB/s | 4.1 MB 00:00 2023-12-02T00:26:46.822 INFO:teuthology.orchestra.run.smithi162.stdout:(132/150): python3-zc-lockfile-2.0-10.el9.noarc 199 kB/s | 20 kB 00:00 2023-12-02T00:26:46.848 INFO:teuthology.orchestra.run.smithi162.stdout:(133/150): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2023-12-02T00:26:46.881 INFO:teuthology.orchestra.run.smithi162.stdout:(134/150): python3-werkzeug-2.0.3-3.el9.1.noarc 2.0 MB/s | 427 kB 00:00 2023-12-02T00:26:46.957 INFO:teuthology.orchestra.run.smithi162.stdout:(135/150): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2023-12-02T00:26:46.998 INFO:teuthology.orchestra.run.smithi162.stdout:(136/150): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.6 MB/s | 68 kB 00:00 2023-12-02T00:26:47.049 INFO:teuthology.orchestra.run.smithi162.stdout:(137/150): acl-2.3.1-4.el9.x86_64.rpm 1.3 MB/s | 69 kB 00:00 2023-12-02T00:26:47.068 INFO:teuthology.orchestra.run.smithi046.stdout:(96/150): gperftools-libs-2.9.1-2.el9.x86_64.rp 507 kB/s | 309 kB 00:00 2023-12-02T00:26:47.083 INFO:teuthology.orchestra.run.smithi162.stdout:(138/150): elfutils-debuginfod-client-0.190-1.e 1.1 MB/s | 36 kB 00:00 2023-12-02T00:26:47.124 INFO:teuthology.orchestra.run.smithi162.stdout:(139/150): elfutils-libelf-0.190-1.el9.x86_64.r 4.5 MB/s | 192 kB 00:00 2023-12-02T00:26:47.166 INFO:teuthology.orchestra.run.smithi162.stdout:(140/150): elfutils-libs-0.190-1.el9.x86_64.rpm 6.0 MB/s | 255 kB 00:00 2023-12-02T00:26:47.177 INFO:teuthology.orchestra.run.smithi046.stdout:(97/150): libarrow-doc-9.0.0-9.el9.noarch.rpm 225 kB/s | 24 kB 00:00 2023-12-02T00:26:47.252 INFO:teuthology.orchestra.run.smithi162.stdout:(141/150): libacl-2.3.1-4.el9.x86_64.rpm 255 kB/s | 22 kB 00:00 2023-12-02T00:26:47.285 INFO:teuthology.orchestra.run.smithi046.stdout:(98/150): liboath-2.6.7-2.el9.x86_64.rpm 448 kB/s | 49 kB 00:00 2023-12-02T00:26:47.367 INFO:teuthology.orchestra.run.smithi162.stdout:(142/150): librbd1-18.0.0-7631.gbed76814.el9.x8 6.2 MB/s | 3.0 MB 00:00 2023-12-02T00:26:47.394 INFO:teuthology.orchestra.run.smithi046.stdout:(99/150): libunwind-1.6.2-1.el9.x86_64.rpm 623 kB/s | 67 kB 00:00 2023-12-02T00:26:47.401 INFO:teuthology.orchestra.run.smithi162.stdout:(143/150): xfsdump-3.1.12-4.el9.x86_64.rpm 2.1 MB/s | 321 kB 00:00 2023-12-02T00:26:47.503 INFO:teuthology.orchestra.run.smithi046.stdout:(100/150): luarocks-3.9.2-1.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2023-12-02T00:26:47.560 INFO:teuthology.orchestra.run.smithi162.stdout:(144/150): NetworkManager-1.45.7-1.el9.x86_64.r 12 MB/s | 2.3 MB 00:00 2023-12-02T00:26:47.626 INFO:teuthology.orchestra.run.smithi162.stdout:(145/150): NetworkManager-libnm-1.45.7-1.el9.x8 7.9 MB/s | 1.8 MB 00:00 2023-12-02T00:26:47.644 INFO:teuthology.orchestra.run.smithi162.stdout:(146/150): NetworkManager-team-1.45.7-1.el9.x86 517 kB/s | 43 kB 00:00 2023-12-02T00:26:47.669 INFO:teuthology.orchestra.run.smithi162.stdout:(147/150): NetworkManager-tui-1.45.7-1.el9.x86_ 5.8 MB/s | 246 kB 00:00 2023-12-02T00:26:47.711 INFO:teuthology.orchestra.run.smithi162.stdout:(148/150): openssl-3.0.7-25.el9.x86_64.rpm 18 MB/s | 1.2 MB 00:00 2023-12-02T00:26:47.711 INFO:teuthology.orchestra.run.smithi046.stdout:(101/150): parquet-libs-9.0.0-9.el9.x86_64.rpm 3.9 MB/s | 837 kB 00:00 2023-12-02T00:26:47.761 INFO:teuthology.orchestra.run.smithi162.stdout:(149/150): openssl-libs-3.0.7-25.el9.x86_64.rpm 23 MB/s | 2.2 MB 00:00 2023-12-02T00:26:47.803 INFO:teuthology.orchestra.run.smithi046.stdout:(102/150): libarrow-9.0.0-9.el9.x86_64.rpm 4.2 MB/s | 4.4 MB 00:01 2023-12-02T00:26:47.837 INFO:teuthology.orchestra.run.smithi046.stdout:(103/150): python3-asyncssh-2.13.2-3.el9.noarch 4.3 MB/s | 548 kB 00:00 2023-12-02T00:26:47.895 INFO:teuthology.orchestra.run.smithi046.stdout:(104/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 473 kB/s | 43 kB 00:00 2023-12-02T00:26:47.938 INFO:teuthology.orchestra.run.smithi046.stdout:(105/150): python3-cachetools-4.2.4-1.el9.noarc 320 kB/s | 32 kB 00:00 2023-12-02T00:26:48.329 INFO:teuthology.orchestra.run.smithi046.stdout:(106/150): ceph-test-18.0.0-7631.gbed76814.el9. 4.1 MB/s | 41 MB 00:10 2023-12-02T00:26:48.346 INFO:teuthology.orchestra.run.smithi046.stdout:(107/150): python3-certifi-2021.10.8-2.el9.noar 33 kB/s | 15 kB 00:00 2023-12-02T00:26:48.430 INFO:teuthology.orchestra.run.smithi046.stdout:(108/150): python3-cheroot-8.6.0-4.el9.noarch.r 349 kB/s | 172 kB 00:00 2023-12-02T00:26:48.469 INFO:teuthology.orchestra.run.smithi162.stdout:(150/150): librados2-18.0.0-7631.gbed76814.el9. 2.0 MB/s | 3.3 MB 00:01 2023-12-02T00:26:48.475 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2023-12-02T00:26:48.475 INFO:teuthology.orchestra.run.smithi162.stdout:Total 13 MB/s | 204 MB 00:16 2023-12-02T00:26:48.480 INFO:teuthology.orchestra.run.smithi046.stdout:(109/150): python3-google-auth-2.23.4-1.el9.noa 1.4 MB/s | 197 kB 00:00 2023-12-02T00:26:48.564 INFO:teuthology.orchestra.run.smithi046.stdout:(110/150): python3-jaraco-8.2.1-3.el9.noarch.rp 100 kB/s | 11 kB 00:00 2023-12-02T00:26:48.581 INFO:teuthology.orchestra.run.smithi046.stdout:(111/150): python3-jaraco-classes-3.2.1-5.el9.n 175 kB/s | 18 kB 00:00 2023-12-02T00:26:48.654 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2023-12-02T00:26:48.664 INFO:teuthology.orchestra.run.smithi046.stdout:(112/150): python3-jaraco-collections-3.0.0-8.e 233 kB/s | 23 kB 00:00 2023-12-02T00:26:48.681 INFO:teuthology.orchestra.run.smithi046.stdout:(113/150): python3-jaraco-functools-3.5.0-2.el9 194 kB/s | 19 kB 00:00 2023-12-02T00:26:48.756 INFO:teuthology.orchestra.run.smithi046.stdout:(114/150): python3-jaraco-text-3.2.0-6.el9.noar 214 kB/s | 20 kB 00:00 2023-12-02T00:26:48.774 INFO:teuthology.orchestra.run.smithi046.stdout:(115/150): python3-jwt+crypto-2.4.0-1.el9.noarc 98 kB/s | 9.0 kB 00:00 2023-12-02T00:26:48.842 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2023-12-02T00:26:48.842 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2023-12-02T00:26:48.857 INFO:teuthology.orchestra.run.smithi046.stdout:(116/150): python3-jwt-2.4.0-1.el9.noarch.rpm 408 kB/s | 41 kB 00:00 2023-12-02T00:26:48.957 INFO:teuthology.orchestra.run.smithi046.stdout:(117/150): python3-logutils-0.3.5-21.el9.noarch 463 kB/s | 46 kB 00:00 2023-12-02T00:26:48.991 INFO:teuthology.orchestra.run.smithi046.stdout:(118/150): python3-kubernetes-26.1.0-2.el9.noar 4.7 MB/s | 1.0 MB 00:00 2023-12-02T00:26:49.025 INFO:teuthology.orchestra.run.smithi046.stdout:(119/150): python3-cherrypy-18.6.1-2.el9.noarch 516 kB/s | 358 kB 00:00 2023-12-02T00:26:49.058 INFO:teuthology.orchestra.run.smithi046.stdout:(120/150): python3-more-itertools-8.12.0-2.el9. 785 kB/s | 79 kB 00:00 2023-12-02T00:26:49.083 INFO:teuthology.orchestra.run.smithi046.stdout:(121/150): python3-natsort-7.1.1-5.el9.noarch.r 626 kB/s | 58 kB 00:00 2023-12-02T00:26:49.134 INFO:teuthology.orchestra.run.smithi046.stdout:(122/150): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2023-12-02T00:26:49.151 INFO:teuthology.orchestra.run.smithi046.stdout:(123/150): python3-portend-3.1.0-2.el9.noarch.r 178 kB/s | 16 kB 00:00 2023-12-02T00:26:49.185 INFO:teuthology.orchestra.run.smithi046.stdout:(124/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 895 kB/s | 90 kB 00:00 2023-12-02T00:26:49.226 INFO:teuthology.orchestra.run.smithi046.stdout:(125/150): python3-repoze-lru-0.7-16.el9.noarch 334 kB/s | 31 kB 00:00 2023-12-02T00:26:49.260 INFO:teuthology.orchestra.run.smithi046.stdout:(126/150): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2023-12-02T00:26:49.285 INFO:teuthology.orchestra.run.smithi046.stdout:(127/150): python3-rsa-4.9-2.el9.noarch.rpm 588 kB/s | 59 kB 00:00 2023-12-02T00:26:49.327 INFO:teuthology.orchestra.run.smithi046.stdout:(128/150): python3-tempora-5.0.0-2.el9.noarch.r 355 kB/s | 36 kB 00:00 2023-12-02T00:26:49.360 INFO:teuthology.orchestra.run.smithi046.stdout:(129/150): python3-typing-extensions-4.4.0-2.el 507 kB/s | 51 kB 00:00 2023-12-02T00:26:49.386 INFO:teuthology.orchestra.run.smithi046.stdout:(130/150): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2023-12-02T00:26:49.436 INFO:teuthology.orchestra.run.smithi046.stdout:(131/150): python3-websocket-client-1.2.3-2.el9 824 kB/s | 90 kB 00:00 2023-12-02T00:26:49.486 INFO:teuthology.orchestra.run.smithi046.stdout:(132/150): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2023-12-02T00:26:49.512 INFO:teuthology.orchestra.run.smithi046.stdout:(133/150): python3-zc-lockfile-2.0-10.el9.noarc 159 kB/s | 20 kB 00:00 2023-12-02T00:26:49.537 INFO:teuthology.orchestra.run.smithi046.stdout:(134/150): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2023-12-02T00:26:49.712 INFO:teuthology.orchestra.run.smithi046.stdout:(135/150): thrift-0.15.0-2.el9.x86_64.rpm 7.0 MB/s | 1.6 MB 00:00 2023-12-02T00:26:49.763 INFO:teuthology.orchestra.run.smithi046.stdout:(136/150): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.3 MB/s | 68 kB 00:00 2023-12-02T00:26:49.804 INFO:teuthology.orchestra.run.smithi046.stdout:(137/150): acl-2.3.1-4.el9.x86_64.rpm 1.6 MB/s | 69 kB 00:00 2023-12-02T00:26:49.855 INFO:teuthology.orchestra.run.smithi046.stdout:(138/150): elfutils-debuginfod-client-0.190-1.e 732 kB/s | 36 kB 00:00 2023-12-02T00:26:49.896 INFO:teuthology.orchestra.run.smithi046.stdout:(139/150): elfutils-libelf-0.190-1.el9.x86_64.r 4.5 MB/s | 192 kB 00:00 2023-12-02T00:26:49.947 INFO:teuthology.orchestra.run.smithi046.stdout:(140/150): elfutils-libs-0.190-1.el9.x86_64.rpm 5.0 MB/s | 255 kB 00:00 2023-12-02T00:26:49.988 INFO:teuthology.orchestra.run.smithi046.stdout:(141/150): libacl-2.3.1-4.el9.x86_64.rpm 528 kB/s | 22 kB 00:00 2023-12-02T00:26:50.039 INFO:teuthology.orchestra.run.smithi046.stdout:(142/150): xfsdump-3.1.12-4.el9.x86_64.rpm 6.3 MB/s | 321 kB 00:00 2023-12-02T00:26:50.264 INFO:teuthology.orchestra.run.smithi046.stdout:(143/150): librados2-18.0.0-7631.gbed76814.el9. 4.4 MB/s | 3.3 MB 00:00 2023-12-02T00:26:50.581 INFO:teuthology.orchestra.run.smithi046.stdout:(144/150): librbd1-18.0.0-7631.gbed76814.el9.x8 2.9 MB/s | 3.0 MB 00:01 2023-12-02T00:26:50.708 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2023-12-02T00:26:50.708 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2023-12-02T00:26:50.740 INFO:teuthology.orchestra.run.smithi046.stdout:(145/150): NetworkManager-1.45.7-1.el9.x86_64.r 3.2 MB/s | 2.3 MB 00:00 2023-12-02T00:26:50.798 INFO:teuthology.orchestra.run.smithi046.stdout:(146/150): NetworkManager-team-1.45.7-1.el9.x86 199 kB/s | 43 kB 00:00 2023-12-02T00:26:50.824 INFO:teuthology.orchestra.run.smithi046.stdout:(147/150): NetworkManager-tui-1.45.7-1.el9.x86_ 2.9 MB/s | 246 kB 00:00 2023-12-02T00:26:51.166 INFO:teuthology.orchestra.run.smithi046.stdout:(148/150): openssl-libs-3.0.7-25.el9.x86_64.rpm 6.3 MB/s | 2.2 MB 00:00 2023-12-02T00:26:51.682 INFO:teuthology.orchestra.run.smithi046.stdout:(149/150): openssl-3.0.7-25.el9.x86_64.rpm 1.4 MB/s | 1.2 MB 00:00 2023-12-02T00:26:51.724 INFO:teuthology.orchestra.run.smithi046.stdout:(150/150): NetworkManager-libnm-1.45.7-1.el9.x8 1.2 MB/s | 1.8 MB 00:01 2023-12-02T00:26:51.732 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2023-12-02T00:26:51.732 INFO:teuthology.orchestra.run.smithi046.stdout:Total 11 MB/s | 204 MB 00:19 2023-12-02T00:26:51.913 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2023-12-02T00:26:52.117 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2023-12-02T00:26:52.118 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2023-12-02T00:26:54.226 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2023-12-02T00:26:54.226 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2023-12-02T00:26:56.847 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2023-12-02T00:26:56.973 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/165 2023-12-02T00:26:57.015 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : thrift-0.15.0-2.el9.x86_64 2/165 2023-12-02T00:26:57.059 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/165 2023-12-02T00:26:57.099 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 4/165 2023-12-02T00:26:57.126 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2023-12-02T00:26:57.178 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2023-12-02T00:26:57.516 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2023-12-02T00:26:57.578 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : librados2-2:18.0.0-7631.gbed76814.el9.x86_64 8/165 2023-12-02T00:26:57.679 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librados2-2:18.0.0-7631.gbed76814.el9.x86_64 8/165 2023-12-02T00:26:57.703 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libcephfs2-2:18.0.0-7631.gbed76814.el9.x86_64 9/165 2023-12-02T00:26:57.795 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: libcephfs2-2:18.0.0-7631.gbed76814.el9.x86_64 9/165 2023-12-02T00:26:57.831 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-rados-2:18.0.0-7631.gbed76814.el9.x86_64 10/165 2023-12-02T00:26:57.877 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/165 2023-12-02T00:26:58.052 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : elfutils-libelf-0.190-1.el9.x86_64 12/165 2023-12-02T00:26:58.098 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : postgresql-13.11-1.el9.x86_64 13/165 2023-12-02T00:26:58.125 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libcephsqlite-2:18.0.0-7631.gbed76814.el9.x86_64 14/165 2023-12-02T00:26:58.219 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7631.gbed76814.el9.x86_64 14/165 2023-12-02T00:26:58.243 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libradosstriper1-2:18.0.0-7631.gbed76814.el9.x86 15/165 2023-12-02T00:26:58.454 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7631.gbed76814.el9.x86 15/165 2023-12-02T00:26:58.580 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : NetworkManager-libnm-1:1.45.7-1.el9.x86_64 16/165 2023-12-02T00:26:58.649 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.7-1.el9.x86_64 16/165 2023-12-02T00:26:58.747 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: NetworkManager-1:1.45.7-1.el9.x86_64 17/165 2023-12-02T00:26:58.817 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : NetworkManager-1:1.45.7-1.el9.x86_64 17/165 2023-12-02T00:26:59.025 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: NetworkManager-1:1.45.7-1.el9.x86_64 17/165 2023-12-02T00:26:59.128 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 18/165 2023-12-02T00:26:59.172 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/165 2023-12-02T00:26:59.240 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/165 2023-12-02T00:26:59.294 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-urllib3-1.26.5-4.el9.noarch 21/165 2023-12-02T00:26:59.325 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 22/165 2023-12-02T00:26:59.370 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 23/165 2023-12-02T00:26:59.409 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 24/165 2023-12-02T00:26:59.473 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 25/165 2023-12-02T00:26:59.508 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 26/165 2023-12-02T00:26:59.550 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/165 2023-12-02T00:26:59.587 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2023-12-02T00:26:59.655 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2023-12-02T00:26:59.708 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-ceph-common-2:18.0.0-7631.gbed76814.el9. 30/165 2023-12-02T00:26:59.752 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-ceph-argparse-2:18.0.0-7631.gbed76814.el 31/165 2023-12-02T00:26:59.780 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cephfs-2:18.0.0-7631.gbed76814.el9.x86_6 32/165 2023-12-02T00:26:59.849 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/165 2023-12-02T00:26:59.927 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/165 2023-12-02T00:27:00.058 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 35/165 2023-12-02T00:27:00.122 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 36/165 2023-12-02T00:27:00.389 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2023-12-02T00:27:00.513 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/165 2023-12-02T00:27:00.635 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-rsa-4.9-2.el9.noarch 37/165 2023-12-02T00:27:00.644 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : thrift-0.15.0-2.el9.x86_64 2/165 2023-12-02T00:27:00.709 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 38/165 2023-12-02T00:27:00.748 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 39/165 2023-12-02T00:27:00.807 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : elfutils-libs-0.190-1.el9.x86_64 40/165 2023-12-02T00:27:00.811 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/165 2023-12-02T00:27:00.845 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : librados-devel-2:18.0.0-7631.gbed76814.el9.x86_6 41/165 2023-12-02T00:27:00.847 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 4/165 2023-12-02T00:27:00.871 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2023-12-02T00:27:00.886 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 42/165 2023-12-02T00:27:00.923 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2023-12-02T00:27:01.041 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 43/165 2023-12-02T00:27:01.087 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 44/165 2023-12-02T00:27:01.194 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : re2-1:20211101-3.el9.x86_64 45/165 2023-12-02T00:27:01.232 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 46/165 2023-12-02T00:27:01.286 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/165 2023-12-02T00:27:01.289 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2023-12-02T00:27:01.332 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : librados2-2:18.0.0-7631.gbed76814.el9.x86_64 8/165 2023-12-02T00:27:01.358 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/165 2023-12-02T00:27:01.389 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/165 2023-12-02T00:27:01.426 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/165 2023-12-02T00:27:01.463 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librados2-2:18.0.0-7631.gbed76814.el9.x86_64 8/165 2023-12-02T00:27:01.472 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/165 2023-12-02T00:27:01.500 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libcephfs2-2:18.0.0-7631.gbed76814.el9.x86_64 9/165 2023-12-02T00:27:01.517 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/165 2023-12-02T00:27:01.560 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 53/165 2023-12-02T00:27:01.592 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libcephfs2-2:18.0.0-7631.gbed76814.el9.x86_64 9/165 2023-12-02T00:27:01.631 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rados-2:18.0.0-7631.gbed76814.el9.x86_64 10/165 2023-12-02T00:27:01.681 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/165 2023-12-02T00:27:01.693 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 54/165 2023-12-02T00:27:01.746 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 55/165 2023-12-02T00:27:01.787 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 56/165 2023-12-02T00:27:01.877 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : elfutils-libelf-0.190-1.el9.x86_64 12/165 2023-12-02T00:27:01.879 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 57/165 2023-12-02T00:27:01.921 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-google-auth-1:2.23.4-1.el9.noarch 58/165 2023-12-02T00:27:01.936 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : postgresql-13.11-1.el9.x86_64 13/165 2023-12-02T00:27:01.962 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 59/165 2023-12-02T00:27:01.963 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libcephsqlite-2:18.0.0-7631.gbed76814.el9.x86_64 14/165 2023-12-02T00:27:01.995 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 60/165 2023-12-02T00:27:02.067 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7631.gbed76814.el9.x86_64 14/165 2023-12-02T00:27:02.089 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libradosstriper1-2:18.0.0-7631.gbed76814.el9.x86 15/165 2023-12-02T00:27:02.090 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 61/165 2023-12-02T00:27:02.139 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 62/165 2023-12-02T00:27:02.187 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-ply-3.11-14.el9.noarch 63/165 2023-12-02T00:27:02.252 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 64/165 2023-12-02T00:27:02.323 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7631.gbed76814.el9.x86 15/165 2023-12-02T00:27:02.434 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : NetworkManager-libnm-1:1.45.7-1.el9.x86_64 16/165 2023-12-02T00:27:02.443 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 65/165 2023-12-02T00:27:02.494 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 66/165 2023-12-02T00:27:02.495 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.7-1.el9.x86_64 16/165 2023-12-02T00:27:02.598 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: NetworkManager-1:1.45.7-1.el9.x86_64 17/165 2023-12-02T00:27:02.629 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 67/165 2023-12-02T00:27:02.646 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : NetworkManager-1:1.45.7-1.el9.x86_64 17/165 2023-12-02T00:27:02.839 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 68/165 2023-12-02T00:27:02.846 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: NetworkManager-1:1.45.7-1.el9.x86_64 17/165 2023-12-02T00:27:02.868 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 69/165 2023-12-02T00:27:02.901 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 70/165 2023-12-02T00:27:02.934 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 71/165 2023-12-02T00:27:02.948 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 18/165 2023-12-02T00:27:02.964 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : fuse-2.9.9-15.el9.x86_64 72/165 2023-12-02T00:27:02.983 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/165 2023-12-02T00:27:03.000 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 73/165 2023-12-02T00:27:03.054 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 74/165 2023-12-02T00:27:03.064 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/165 2023-12-02T00:27:03.097 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : uuid-1.6.2-55.el9.x86_64 75/165 2023-12-02T00:27:03.117 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-urllib3-1.26.5-4.el9.noarch 21/165 2023-12-02T00:27:03.127 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 76/165 2023-12-02T00:27:03.145 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 22/165 2023-12-02T00:27:03.182 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 23/165 2023-12-02T00:27:03.203 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 77/165 2023-12-02T00:27:03.221 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 24/165 2023-12-02T00:27:03.237 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 78/165 2023-12-02T00:27:03.271 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 79/165 2023-12-02T00:27:03.297 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 25/165 2023-12-02T00:27:03.314 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 80/165 2023-12-02T00:27:03.338 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 26/165 2023-12-02T00:27:03.338 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 81/165 2023-12-02T00:27:03.371 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/165 2023-12-02T00:27:03.407 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2023-12-02T00:27:03.428 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 82/165 2023-12-02T00:27:03.466 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 83/165 2023-12-02T00:27:03.479 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2023-12-02T00:27:03.512 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-ceph-common-2:18.0.0-7631.gbed76814.el9. 30/165 2023-12-02T00:27:03.556 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-ceph-argparse-2:18.0.0-7631.gbed76814.el 31/165 2023-12-02T00:27:03.568 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 84/165 2023-12-02T00:27:03.584 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cephfs-2:18.0.0-7631.gbed76814.el9.x86_6 32/165 2023-12-02T00:27:03.625 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 85/165 2023-12-02T00:27:03.655 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/165 2023-12-02T00:27:03.733 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/165 2023-12-02T00:27:04.037 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 35/165 2023-12-02T00:27:04.119 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 36/165 2023-12-02T00:27:04.347 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/165 2023-12-02T00:27:04.398 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 87/165 2023-12-02T00:27:04.432 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 88/165 2023-12-02T00:27:04.473 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 89/165 2023-12-02T00:27:04.661 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : openblas-0.3.21-2.el9.x86_64 90/165 2023-12-02T00:27:04.673 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rsa-4.9-2.el9.noarch 37/165 2023-12-02T00:27:04.696 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 91/165 2023-12-02T00:27:04.749 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 38/165 2023-12-02T00:27:04.775 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 92/165 2023-12-02T00:27:04.789 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 39/165 2023-12-02T00:27:04.840 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : elfutils-libs-0.190-1.el9.x86_64 40/165 2023-12-02T00:27:04.873 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librados-devel-2:18.0.0-7631.gbed76814.el9.x86_6 41/165 2023-12-02T00:27:04.915 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 42/165 2023-12-02T00:27:05.062 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 43/165 2023-12-02T00:27:05.115 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 44/165 2023-12-02T00:27:05.226 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : re2-1:20211101-3.el9.x86_64 45/165 2023-12-02T00:27:05.260 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 46/165 2023-12-02T00:27:05.325 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/165 2023-12-02T00:27:05.399 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/165 2023-12-02T00:27:05.417 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 93/165 2023-12-02T00:27:05.436 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/165 2023-12-02T00:27:05.498 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/165 2023-12-02T00:27:05.527 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 94/165 2023-12-02T00:27:05.545 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/165 2023-12-02T00:27:05.598 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/165 2023-12-02T00:27:05.640 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 53/165 2023-12-02T00:27:05.779 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 54/165 2023-12-02T00:27:05.825 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 55/165 2023-12-02T00:27:05.866 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 56/165 2023-12-02T00:27:05.962 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 57/165 2023-12-02T00:27:06.002 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-google-auth-1:2.23.4-1.el9.noarch 58/165 2023-12-02T00:27:06.044 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 59/165 2023-12-02T00:27:06.075 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 60/165 2023-12-02T00:27:06.174 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 61/165 2023-12-02T00:27:06.220 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 62/165 2023-12-02T00:27:06.279 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-ply-3.11-14.el9.noarch 63/165 2023-12-02T00:27:06.342 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 64/165 2023-12-02T00:27:06.537 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 65/165 2023-12-02T00:27:06.582 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 66/165 2023-12-02T00:27:06.735 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 67/165 2023-12-02T00:27:06.772 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 95/165 2023-12-02T00:27:06.786 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 68/165 2023-12-02T00:27:06.814 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 69/165 2023-12-02T00:27:06.848 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 70/165 2023-12-02T00:27:06.848 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 96/165 2023-12-02T00:27:06.879 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 97/165 2023-12-02T00:27:06.880 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 71/165 2023-12-02T00:27:06.910 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : fuse-2.9.9-15.el9.x86_64 72/165 2023-12-02T00:27:06.919 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : jq-1.6-16.el9.x86_64 98/165 2023-12-02T00:27:06.939 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 73/165 2023-12-02T00:27:06.948 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2023-12-02T00:27:06.981 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 100/165 2023-12-02T00:27:06.985 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 74/165 2023-12-02T00:27:07.019 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : uuid-1.6.2-55.el9.x86_64 75/165 2023-12-02T00:27:07.056 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 76/165 2023-12-02T00:27:07.136 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 77/165 2023-12-02T00:27:07.167 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 78/165 2023-12-02T00:27:07.200 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 79/165 2023-12-02T00:27:07.242 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 80/165 2023-12-02T00:27:07.276 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 81/165 2023-12-02T00:27:07.285 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2023-12-02T00:27:07.319 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : librbd1-2:18.0.0-7631.gbed76814.el9.x86_64 102/165 2023-12-02T00:27:07.370 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 82/165 2023-12-02T00:27:07.391 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librbd1-2:18.0.0-7631.gbed76814.el9.x86_64 102/165 2023-12-02T00:27:07.404 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 83/165 2023-12-02T00:27:07.424 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-rbd-2:18.0.0-7631.gbed76814.el9.x86_64 103/165 2023-12-02T00:27:07.460 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2023-12-02T00:27:07.509 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 84/165 2023-12-02T00:27:07.580 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 85/165 2023-12-02T00:27:07.889 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 105/165 2023-12-02T00:27:07.914 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : librgw2-2:18.0.0-7631.gbed76814.el9.x86_64 106/165 2023-12-02T00:27:07.985 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librgw2-2:18.0.0-7631.gbed76814.el9.x86_64 106/165 2023-12-02T00:27:08.007 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-rgw-2:18.0.0-7631.gbed76814.el9.x86_64 107/165 2023-12-02T00:27:08.345 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/165 2023-12-02T00:27:08.394 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 87/165 2023-12-02T00:27:08.428 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 88/165 2023-12-02T00:27:08.460 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 89/165 2023-12-02T00:27:08.645 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : openblas-0.3.21-2.el9.x86_64 90/165 2023-12-02T00:27:08.674 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 91/165 2023-12-02T00:27:08.755 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 92/165 2023-12-02T00:27:09.470 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 93/165 2023-12-02T00:27:09.582 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 94/165 2023-12-02T00:27:10.157 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-common-2:18.0.0-7631.gbed76814.el9.x86_64 108/165 2023-12-02T00:27:10.196 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-common-2:18.0.0-7631.gbed76814.el9.x86_64 108/165 2023-12-02T00:27:10.764 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-common-2:18.0.0-7631.gbed76814.el9.x86_64 108/165 2023-12-02T00:27:10.799 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-base-2:18.0.0-7631.gbed76814.el9.x86_64 109/165 2023-12-02T00:27:10.886 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-base-2:18.0.0-7631.gbed76814.el9.x86_64 109/165 2023-12-02T00:27:10.886 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-12-02T00:27:10.886 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-12-02T00:27:10.887 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-02T00:27:10.899 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 95/165 2023-12-02T00:27:10.926 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-selinux-2:18.0.0-7631.gbed76814.el9.x86_64 110/165 2023-12-02T00:27:10.975 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 96/165 2023-12-02T00:27:11.006 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 97/165 2023-12-02T00:27:11.037 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : jq-1.6-16.el9.x86_64 98/165 2023-12-02T00:27:11.065 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2023-12-02T00:27:11.098 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 100/165 2023-12-02T00:27:11.416 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2023-12-02T00:27:11.445 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : librbd1-2:18.0.0-7631.gbed76814.el9.x86_64 102/165 2023-12-02T00:27:11.536 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librbd1-2:18.0.0-7631.gbed76814.el9.x86_64 102/165 2023-12-02T00:27:11.566 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rbd-2:18.0.0-7631.gbed76814.el9.x86_64 103/165 2023-12-02T00:27:11.602 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2023-12-02T00:27:12.060 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 105/165 2023-12-02T00:27:12.098 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librgw2-2:18.0.0-7631.gbed76814.el9.x86_64 106/165 2023-12-02T00:27:12.177 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librgw2-2:18.0.0-7631.gbed76814.el9.x86_64 106/165 2023-12-02T00:27:12.199 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rgw-2:18.0.0-7631.gbed76814.el9.x86_64 107/165 2023-12-02T00:27:14.399 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-common-2:18.0.0-7631.gbed76814.el9.x86_64 108/165 2023-12-02T00:27:14.429 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-common-2:18.0.0-7631.gbed76814.el9.x86_64 108/165 2023-12-02T00:27:14.993 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-common-2:18.0.0-7631.gbed76814.el9.x86_64 108/165 2023-12-02T00:27:15.032 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-base-2:18.0.0-7631.gbed76814.el9.x86_64 109/165 2023-12-02T00:27:15.105 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-base-2:18.0.0-7631.gbed76814.el9.x86_64 109/165 2023-12-02T00:27:15.105 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-12-02T00:27:15.105 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-12-02T00:27:15.105 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:27:15.135 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-selinux-2:18.0.0-7631.gbed76814.el9.x86_64 110/165 2023-12-02T00:27:22.198 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7631.gbed76814.el9.x86_64 110/165 2023-12-02T00:27:22.199 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /sys 2023-12-02T00:27:22.199 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /proc 2023-12-02T00:27:22.199 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /mnt 2023-12-02T00:27:22.199 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /var/tmp 2023-12-02T00:27:22.199 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /home 2023-12-02T00:27:22.199 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /root 2023-12-02T00:27:22.199 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /tmp 2023-12-02T00:27:22.199 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-02T00:27:22.365 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-volume-2:18.0.0-7631.gbed76814.el9.noarch 111/165 2023-12-02T00:27:22.381 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-volume-2:18.0.0-7631.gbed76814.el9.noarch 111/165 2023-12-02T00:27:22.381 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-02T00:27:22.381 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-12-02T00:27:22.381 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-02T00:27:23.960 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-osd-2:18.0.0-7631.gbed76814.el9.x86_64 112/165 2023-12-02T00:27:23.987 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-osd-2:18.0.0-7631.gbed76814.el9.x86_64 112/165 2023-12-02T00:27:23.987 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-02T00:27:23.987 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-12-02T00:27:23.987 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-12-02T00:27:23.987 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-12-02T00:27:23.987 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-02T00:27:24.186 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mds-2:18.0.0-7631.gbed76814.el9.x86_64 113/165 2023-12-02T00:27:24.209 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mds-2:18.0.0-7631.gbed76814.el9.x86_64 113/165 2023-12-02T00:27:24.209 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-02T00:27:24.209 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-12-02T00:27:24.209 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-12-02T00:27:24.209 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-12-02T00:27:24.209 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-02T00:27:24.662 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mon-2:18.0.0-7631.gbed76814.el9.x86_64 114/165 2023-12-02T00:27:24.685 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mon-2:18.0.0-7631.gbed76814.el9.x86_64 114/165 2023-12-02T00:27:24.685 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-02T00:27:24.685 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-12-02T00:27:24.685 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-12-02T00:27:24.685 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-12-02T00:27:24.685 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-02T00:27:24.781 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2023-12-02T00:27:25.243 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: cephadm-2:18.0.0-7631.gbed76814.el9.noarch 116/165 2023-12-02T00:27:25.295 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : cephadm-2:18.0.0-7631.gbed76814.el9.noarch 116/165 2023-12-02T00:27:25.325 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7631.gbed76814.e 117/165 2023-12-02T00:27:25.386 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7631.gbed76814. 118/165 2023-12-02T00:27:25.410 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7631.gbed76814.el9.noa 119/165 2023-12-02T00:27:25.637 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7631.gbed76814.el9.noa 119/165 2023-12-02T00:27:25.670 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7631.gbed76814.el9.n 120/165 2023-12-02T00:27:26.643 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7631.gbed76814.el9.n 120/165 2023-12-02T00:27:26.670 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7631.gbed 121/165 2023-12-02T00:27:26.771 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7631.gbed 121/165 2023-12-02T00:27:26.796 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7631.gbed76814.el9.x86_64 110/165 2023-12-02T00:27:26.796 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /sys 2023-12-02T00:27:26.796 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /proc 2023-12-02T00:27:26.796 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /mnt 2023-12-02T00:27:26.796 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /var/tmp 2023-12-02T00:27:26.796 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /home 2023-12-02T00:27:26.796 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /root 2023-12-02T00:27:26.796 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /tmp 2023-12-02T00:27:26.796 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:27:26.928 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7631.gbed76814.el 122/165 2023-12-02T00:27:26.964 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-volume-2:18.0.0-7631.gbed76814.el9.noarch 111/165 2023-12-02T00:27:26.972 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-2:18.0.0-7631.gbed76814.el9.x86_64 123/165 2023-12-02T00:27:26.981 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-volume-2:18.0.0-7631.gbed76814.el9.noarch 111/165 2023-12-02T00:27:26.981 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-02T00:27:26.981 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-12-02T00:27:26.981 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:27:26.996 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7631.gbed76814.el9.x86_64 123/165 2023-12-02T00:27:26.996 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-02T00:27:26.996 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-12-02T00:27:26.996 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-12-02T00:27:26.996 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-12-02T00:27:26.996 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-02T00:27:27.040 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-rook-2:18.0.0-7631.gbed76814.el9.noarch 124/165 2023-12-02T00:27:27.050 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7631.gbed76814.el9.noarch 124/165 2023-12-02T00:27:27.098 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-2:18.0.0-7631.gbed76814.el9.x86_64 125/165 2023-12-02T00:27:27.884 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libtool-2.4.6-45.el9.x86_64 126/165 2023-12-02T00:27:27.917 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-radosgw-2:18.0.0-7631.gbed76814.el9.x86_64 127/165 2023-12-02T00:27:27.940 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7631.gbed76814.el9.x86_64 127/165 2023-12-02T00:27:27.940 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-02T00:27:27.940 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-12-02T00:27:27.940 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-12-02T00:27:27.941 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-12-02T00:27:27.941 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-02T00:27:27.984 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7631.gbed76 128/165 2023-12-02T00:27:28.006 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7631.gbed76 128/165 2023-12-02T00:27:28.006 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-02T00:27:28.006 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-12-02T00:27:28.006 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-02T00:27:28.051 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : cephfs-mirror-2:18.0.0-7631.gbed76814.el9.x86_64 129/165 2023-12-02T00:27:28.075 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-7631.gbed76814.el9.x86_64 129/165 2023-12-02T00:27:28.075 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-02T00:27:28.075 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-12-02T00:27:28.075 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-12-02T00:27:28.075 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-12-02T00:27:28.075 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-02T00:27:28.369 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : rbd-mirror-2:18.0.0-7631.gbed76814.el9.x86_64 130/165 2023-12-02T00:27:28.393 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7631.gbed76814.el9.x86_64 130/165 2023-12-02T00:27:28.393 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-02T00:27:28.393 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-12-02T00:27:28.393 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-12-02T00:27:28.394 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-12-02T00:27:28.394 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-02T00:27:28.600 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-osd-2:18.0.0-7631.gbed76814.el9.x86_64 112/165 2023-12-02T00:27:28.628 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-osd-2:18.0.0-7631.gbed76814.el9.x86_64 112/165 2023-12-02T00:27:28.628 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-02T00:27:28.628 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-12-02T00:27:28.628 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-12-02T00:27:28.628 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-12-02T00:27:28.628 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:27:28.843 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mds-2:18.0.0-7631.gbed76814.el9.x86_64 113/165 2023-12-02T00:27:28.867 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mds-2:18.0.0-7631.gbed76814.el9.x86_64 113/165 2023-12-02T00:27:28.867 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-02T00:27:28.867 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-12-02T00:27:28.867 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-12-02T00:27:28.867 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-12-02T00:27:28.867 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:27:29.344 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mon-2:18.0.0-7631.gbed76814.el9.x86_64 114/165 2023-12-02T00:27:29.368 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mon-2:18.0.0-7631.gbed76814.el9.x86_64 114/165 2023-12-02T00:27:29.368 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-02T00:27:29.368 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-12-02T00:27:29.368 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-12-02T00:27:29.368 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-12-02T00:27:29.368 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:27:29.453 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2023-12-02T00:27:29.910 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: cephadm-2:18.0.0-7631.gbed76814.el9.noarch 116/165 2023-12-02T00:27:29.944 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : cephadm-2:18.0.0-7631.gbed76814.el9.noarch 116/165 2023-12-02T00:27:29.973 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7631.gbed76814.e 117/165 2023-12-02T00:27:30.037 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7631.gbed76814. 118/165 2023-12-02T00:27:30.066 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7631.gbed76814.el9.noa 119/165 2023-12-02T00:27:30.304 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7631.gbed76814.el9.noa 119/165 2023-12-02T00:27:30.342 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7631.gbed76814.el9.n 120/165 2023-12-02T00:27:31.359 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7631.gbed76814.el9.n 120/165 2023-12-02T00:27:31.401 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7631.gbed 121/165 2023-12-02T00:27:31.508 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7631.gbed 121/165 2023-12-02T00:27:31.674 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7631.gbed76814.el 122/165 2023-12-02T00:27:31.703 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-2:18.0.0-7631.gbed76814.el9.x86_64 123/165 2023-12-02T00:27:31.728 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7631.gbed76814.el9.x86_64 123/165 2023-12-02T00:27:31.728 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-02T00:27:31.728 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-12-02T00:27:31.728 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-12-02T00:27:31.728 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-12-02T00:27:31.728 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:27:31.779 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-rook-2:18.0.0-7631.gbed76814.el9.noarch 124/165 2023-12-02T00:27:31.790 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7631.gbed76814.el9.noarch 124/165 2023-12-02T00:27:31.851 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-2:18.0.0-7631.gbed76814.el9.x86_64 125/165 2023-12-02T00:27:32.256 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-test-2:18.0.0-7631.gbed76814.el9.x86_64 131/165 2023-12-02T00:27:32.297 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : rbd-fuse-2:18.0.0-7631.gbed76814.el9.x86_64 132/165 2023-12-02T00:27:32.347 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : rbd-nbd-2:18.0.0-7631.gbed76814.el9.x86_64 133/165 2023-12-02T00:27:32.387 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 134/165 2023-12-02T00:27:32.579 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : elfutils-libelf-devel-0.190-1.el9.x86_64 135/165 2023-12-02T00:27:32.673 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libtool-2.4.6-45.el9.x86_64 126/165 2023-12-02T00:27:32.686 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 136/165 2023-12-02T00:27:32.706 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-radosgw-2:18.0.0-7631.gbed76814.el9.x86_64 127/165 2023-12-02T00:27:32.719 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-fuse-2:18.0.0-7631.gbed76814.el9.x86_64 137/165 2023-12-02T00:27:32.730 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7631.gbed76814.el9.x86_64 127/165 2023-12-02T00:27:32.730 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-02T00:27:32.730 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-12-02T00:27:32.730 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-12-02T00:27:32.731 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-12-02T00:27:32.731 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:27:32.752 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libcephfs-devel-2:18.0.0-7631.gbed76814.el9.x86_ 138/165 2023-12-02T00:27:32.773 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7631.gbed76 128/165 2023-12-02T00:27:32.797 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7631.gbed76 128/165 2023-12-02T00:27:32.797 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-02T00:27:32.797 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-12-02T00:27:32.797 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:27:32.813 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : elfutils-debuginfod-client-0.190-1.el9.x86_64 139/165 2023-12-02T00:27:32.840 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : cephfs-mirror-2:18.0.0-7631.gbed76814.el9.x86_64 129/165 2023-12-02T00:27:32.864 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-7631.gbed76814.el9.x86_64 129/165 2023-12-02T00:27:32.864 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-02T00:27:32.864 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-12-02T00:27:32.864 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-12-02T00:27:32.864 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-12-02T00:27:32.864 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:27:32.874 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 140/165 2023-12-02T00:27:32.921 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 141/165 2023-12-02T00:27:32.950 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : NetworkManager-team-1:1.45.7-1.el9.x86_64 142/165 2023-12-02T00:27:32.996 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : NetworkManager-tui-1:1.45.7-1.el9.x86_64 143/165 2023-12-02T00:27:33.183 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : rbd-mirror-2:18.0.0-7631.gbed76814.el9.x86_64 130/165 2023-12-02T00:27:33.207 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7631.gbed76814.el9.x86_64 130/165 2023-12-02T00:27:33.207 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-02T00:27:33.207 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-12-02T00:27:33.207 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-12-02T00:27:33.207 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-12-02T00:27:33.207 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:27:33.567 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 144/165 2023-12-02T00:27:33.604 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 144/165 2023-12-02T00:27:33.626 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 144/165 2023-12-02T00:27:34.449 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : cephfs-top-2:18.0.0-7631.gbed76814.el9.noarch 145/165 2023-12-02T00:27:34.565 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 146/165 2023-12-02T00:27:34.634 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 147/165 2023-12-02T00:27:34.676 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/165 2023-12-02T00:27:34.774 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/165 2023-12-02T00:27:34.816 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/165 2023-12-02T00:27:34.840 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 151/165 2023-12-02T00:27:34.867 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 152/165 2023-12-02T00:27:34.892 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 153/165 2023-12-02T00:27:34.892 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 154/165 2023-12-02T00:27:34.977 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 154/165 2023-12-02T00:27:35.005 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 155/165 2023-12-02T00:27:35.045 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 156/165 2023-12-02T00:27:35.077 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 157/165 2023-12-02T00:27:35.077 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 158/165 2023-12-02T00:27:35.170 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 158/165 2023-12-02T00:27:35.195 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 159/165 2023-12-02T00:27:35.203 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 160/165 2023-12-02T00:27:35.203 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 160/165 2023-12-02T00:27:35.398 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 160/165 2023-12-02T00:27:35.398 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 161/165 2023-12-02T00:27:35.490 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 161/165 2023-12-02T00:27:35.514 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 162/165 2023-12-02T00:27:35.540 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 163/165 2023-12-02T00:27:35.566 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 164/165 2023-12-02T00:27:35.640 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 165/165 2023-12-02T00:27:37.245 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-test-2:18.0.0-7631.gbed76814.el9.x86_64 131/165 2023-12-02T00:27:37.295 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : rbd-fuse-2:18.0.0-7631.gbed76814.el9.x86_64 132/165 2023-12-02T00:27:37.336 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : rbd-nbd-2:18.0.0-7631.gbed76814.el9.x86_64 133/165 2023-12-02T00:27:37.367 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 134/165 2023-12-02T00:27:37.552 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : elfutils-libelf-devel-0.190-1.el9.x86_64 135/165 2023-12-02T00:27:37.670 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 136/165 2023-12-02T00:27:37.699 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-fuse-2:18.0.0-7631.gbed76814.el9.x86_64 137/165 2023-12-02T00:27:37.732 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libcephfs-devel-2:18.0.0-7631.gbed76814.el9.x86_ 138/165 2023-12-02T00:27:37.798 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : elfutils-debuginfod-client-0.190-1.el9.x86_64 139/165 2023-12-02T00:27:37.860 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 140/165 2023-12-02T00:27:37.891 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 141/165 2023-12-02T00:27:37.922 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : NetworkManager-team-1:1.45.7-1.el9.x86_64 142/165 2023-12-02T00:27:37.967 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : NetworkManager-tui-1:1.45.7-1.el9.x86_64 143/165 2023-12-02T00:27:38.563 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 144/165 2023-12-02T00:27:38.599 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 144/165 2023-12-02T00:27:38.620 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 144/165 2023-12-02T00:27:39.498 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : cephfs-top-2:18.0.0-7631.gbed76814.el9.noarch 145/165 2023-12-02T00:27:39.609 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 146/165 2023-12-02T00:27:39.653 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 147/165 2023-12-02T00:27:39.696 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/165 2023-12-02T00:27:39.785 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/165 2023-12-02T00:27:39.816 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/165 2023-12-02T00:27:39.841 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 151/165 2023-12-02T00:27:39.866 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 152/165 2023-12-02T00:27:39.893 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 153/165 2023-12-02T00:27:39.893 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 154/165 2023-12-02T00:27:39.977 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 154/165 2023-12-02T00:27:40.006 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 155/165 2023-12-02T00:27:40.036 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 156/165 2023-12-02T00:27:40.061 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 157/165 2023-12-02T00:27:40.061 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 158/165 2023-12-02T00:27:40.131 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 158/165 2023-12-02T00:27:40.156 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 159/165 2023-12-02T00:27:40.163 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 160/165 2023-12-02T00:27:40.163 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 160/165 2023-12-02T00:27:40.358 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 160/165 2023-12-02T00:27:40.359 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 161/165 2023-12-02T00:27:40.410 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: xfsdump-3.1.12-3.el9.x86_64 165/165 2023-12-02T00:27:40.411 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-2:18.0.0-7631.gbed76814.el9.x86_64 1/165 2023-12-02T00:27:40.411 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-base-2:18.0.0-7631.gbed76814.el9.x86_64 2/165 2023-12-02T00:27:40.411 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-common-2:18.0.0-7631.gbed76814.el9.x86_64 3/165 2023-12-02T00:27:40.411 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-fuse-2:18.0.0-7631.gbed76814.el9.x86_64 4/165 2023-12-02T00:27:40.411 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7631.gbed76 5/165 2023-12-02T00:27:40.411 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mds-2:18.0.0-7631.gbed76814.el9.x86_64 6/165 2023-12-02T00:27:40.411 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-2:18.0.0-7631.gbed76814.el9.x86_64 7/165 2023-12-02T00:27:40.411 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mon-2:18.0.0-7631.gbed76814.el9.x86_64 8/165 2023-12-02T00:27:40.411 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-osd-2:18.0.0-7631.gbed76814.el9.x86_64 9/165 2023-12-02T00:27:40.411 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-radosgw-2:18.0.0-7631.gbed76814.el9.x86_64 10/165 2023-12-02T00:27:40.411 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-selinux-2:18.0.0-7631.gbed76814.el9.x86_64 11/165 2023-12-02T00:27:40.411 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-test-2:18.0.0-7631.gbed76814.el9.x86_64 12/165 2023-12-02T00:27:40.411 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : cephfs-mirror-2:18.0.0-7631.gbed76814.el9.x86_64 13/165 2023-12-02T00:27:40.412 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libcephfs-devel-2:18.0.0-7631.gbed76814.el9.x86_ 14/165 2023-12-02T00:27:40.412 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libcephfs2-2:18.0.0-7631.gbed76814.el9.x86_64 15/165 2023-12-02T00:27:40.412 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libcephsqlite-2:18.0.0-7631.gbed76814.el9.x86_64 16/165 2023-12-02T00:27:40.412 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librados-devel-2:18.0.0-7631.gbed76814.el9.x86_6 17/165 2023-12-02T00:27:40.412 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libradosstriper1-2:18.0.0-7631.gbed76814.el9.x86 18/165 2023-12-02T00:27:40.412 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librgw2-2:18.0.0-7631.gbed76814.el9.x86_64 19/165 2023-12-02T00:27:40.412 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7631.gbed76814.el 20/165 2023-12-02T00:27:40.412 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-ceph-common-2:18.0.0-7631.gbed76814.el9. 21/165 2023-12-02T00:27:40.412 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cephfs-2:18.0.0-7631.gbed76814.el9.x86_6 22/165 2023-12-02T00:27:40.412 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-rados-2:18.0.0-7631.gbed76814.el9.x86_64 23/165 2023-12-02T00:27:40.412 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-rbd-2:18.0.0-7631.gbed76814.el9.x86_64 24/165 2023-12-02T00:27:40.412 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-rgw-2:18.0.0-7631.gbed76814.el9.x86_64 25/165 2023-12-02T00:27:40.412 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : rbd-fuse-2:18.0.0-7631.gbed76814.el9.x86_64 26/165 2023-12-02T00:27:40.412 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : rbd-mirror-2:18.0.0-7631.gbed76814.el9.x86_64 27/165 2023-12-02T00:27:40.413 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : rbd-nbd-2:18.0.0-7631.gbed76814.el9.x86_64 28/165 2023-12-02T00:27:40.413 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7631.gbed76814. 29/165 2023-12-02T00:27:40.413 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7631.gbed76814.el9.noa 30/165 2023-12-02T00:27:40.413 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7631.gbed76814.el9.n 31/165 2023-12-02T00:27:40.413 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7631.gbed 32/165 2023-12-02T00:27:40.413 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7631.gbed76814.el 33/165 2023-12-02T00:27:40.413 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7631.gbed76814.el9.noarch 34/165 2023-12-02T00:27:40.413 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7631.gbed76814.e 35/165 2023-12-02T00:27:40.413 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-volume-2:18.0.0-7631.gbed76814.el9.noarch 36/165 2023-12-02T00:27:40.413 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : cephadm-2:18.0.0-7631.gbed76814.el9.noarch 37/165 2023-12-02T00:27:40.413 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : cephfs-top-2:18.0.0-7631.gbed76814.el9.noarch 38/165 2023-12-02T00:27:40.414 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : automake-1.16.2-8.el9.noarch 39/165 2023-12-02T00:27:40.414 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : bison-3.7.4-5.el9.x86_64 40/165 2023-12-02T00:27:40.414 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 41/165 2023-12-02T00:27:40.414 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : elfutils-libelf-devel-0.190-1.el9.x86_64 42/165 2023-12-02T00:27:40.414 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/165 2023-12-02T00:27:40.414 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/165 2023-12-02T00:27:40.414 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/165 2023-12-02T00:27:40.414 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/165 2023-12-02T00:27:40.414 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : jq-1.6-16.el9.x86_64 47/165 2023-12-02T00:27:40.414 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/165 2023-12-02T00:27:40.414 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/165 2023-12-02T00:27:40.415 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/165 2023-12-02T00:27:40.415 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/165 2023-12-02T00:27:40.415 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/165 2023-12-02T00:27:40.416 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 53/165 2023-12-02T00:27:40.416 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/165 2023-12-02T00:27:40.416 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/165 2023-12-02T00:27:40.416 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : lua-5.4.4-4.el9.x86_64 56/165 2023-12-02T00:27:40.416 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 57/165 2023-12-02T00:27:40.416 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 58/165 2023-12-02T00:27:40.416 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 59/165 2023-12-02T00:27:40.416 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : postgresql-13.11-1.el9.x86_64 60/165 2023-12-02T00:27:40.416 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 61/165 2023-12-02T00:27:40.416 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 62/165 2023-12-02T00:27:40.416 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 63/165 2023-12-02T00:27:40.416 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/165 2023-12-02T00:27:40.416 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 65/165 2023-12-02T00:27:40.416 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 66/165 2023-12-02T00:27:40.416 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 67/165 2023-12-02T00:27:40.417 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 68/165 2023-12-02T00:27:40.417 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 69/165 2023-12-02T00:27:40.417 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 70/165 2023-12-02T00:27:40.417 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 71/165 2023-12-02T00:27:40.417 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 72/165 2023-12-02T00:27:40.417 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/165 2023-12-02T00:27:40.417 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 74/165 2023-12-02T00:27:40.417 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 75/165 2023-12-02T00:27:40.417 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 76/165 2023-12-02T00:27:40.417 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 77/165 2023-12-02T00:27:40.417 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 78/165 2023-12-02T00:27:40.417 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 79/165 2023-12-02T00:27:40.417 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 80/165 2023-12-02T00:27:40.417 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 81/165 2023-12-02T00:27:40.417 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 82/165 2023-12-02T00:27:40.418 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 83/165 2023-12-02T00:27:40.418 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 84/165 2023-12-02T00:27:40.418 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 85/165 2023-12-02T00:27:40.418 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 86/165 2023-12-02T00:27:40.418 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 87/165 2023-12-02T00:27:40.418 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 88/165 2023-12-02T00:27:40.418 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 89/165 2023-12-02T00:27:40.418 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-ply-3.11-14.el9.noarch 90/165 2023-12-02T00:27:40.418 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 91/165 2023-12-02T00:27:40.418 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 92/165 2023-12-02T00:27:40.418 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-urllib3-1.26.5-4.el9.noarch 93/165 2023-12-02T00:27:40.418 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 94/165 2023-12-02T00:27:40.418 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 95/165 2023-12-02T00:27:40.418 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 96/165 2023-12-02T00:27:40.418 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 97/165 2023-12-02T00:27:40.419 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 98/165 2023-12-02T00:27:40.419 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 99/165 2023-12-02T00:27:40.419 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 100/165 2023-12-02T00:27:40.419 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 101/165 2023-12-02T00:27:40.419 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 102/165 2023-12-02T00:27:40.419 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 103/165 2023-12-02T00:27:40.419 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 104/165 2023-12-02T00:27:40.419 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 105/165 2023-12-02T00:27:40.419 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 106/165 2023-12-02T00:27:40.419 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 107/165 2023-12-02T00:27:40.419 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 108/165 2023-12-02T00:27:40.419 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 109/165 2023-12-02T00:27:40.419 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-google-auth-1:2.23.4-1.el9.noarch 110/165 2023-12-02T00:27:40.419 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 111/165 2023-12-02T00:27:40.420 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 112/165 2023-12-02T00:27:40.420 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 113/165 2023-12-02T00:27:40.420 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 114/165 2023-12-02T00:27:40.420 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 115/165 2023-12-02T00:27:40.420 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 116/165 2023-12-02T00:27:40.420 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 117/165 2023-12-02T00:27:40.420 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 118/165 2023-12-02T00:27:40.420 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 119/165 2023-12-02T00:27:40.420 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 120/165 2023-12-02T00:27:40.420 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 121/165 2023-12-02T00:27:40.420 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 122/165 2023-12-02T00:27:40.420 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 123/165 2023-12-02T00:27:40.420 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 124/165 2023-12-02T00:27:40.420 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 125/165 2023-12-02T00:27:40.420 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 126/165 2023-12-02T00:27:40.421 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 127/165 2023-12-02T00:27:40.421 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 128/165 2023-12-02T00:27:40.421 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 129/165 2023-12-02T00:27:40.421 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 130/165 2023-12-02T00:27:40.421 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 131/165 2023-12-02T00:27:40.421 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 132/165 2023-12-02T00:27:40.421 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 133/165 2023-12-02T00:27:40.421 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : re2-1:20211101-3.el9.x86_64 134/165 2023-12-02T00:27:40.421 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 135/165 2023-12-02T00:27:40.421 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librados2-2:18.0.0-7631.gbed76814.el9.x86_64 136/165 2023-12-02T00:27:40.421 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 137/165 2023-12-02T00:27:40.421 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librbd1-2:18.0.0-7631.gbed76814.el9.x86_64 138/165 2023-12-02T00:27:40.421 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 139/165 2023-12-02T00:27:40.421 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 140/165 2023-12-02T00:27:40.421 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 141/165 2023-12-02T00:27:40.422 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : acl-2.3.1-4.el9.x86_64 142/165 2023-12-02T00:27:40.422 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : acl-2.3.1-3.el9.x86_64 143/165 2023-12-02T00:27:40.422 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : elfutils-debuginfod-client-0.190-1.el9.x86_64 144/165 2023-12-02T00:27:40.422 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 145/165 2023-12-02T00:27:40.422 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : elfutils-libelf-0.190-1.el9.x86_64 146/165 2023-12-02T00:27:40.422 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 147/165 2023-12-02T00:27:40.422 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : elfutils-libs-0.190-1.el9.x86_64 148/165 2023-12-02T00:27:40.422 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 149/165 2023-12-02T00:27:40.422 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 150/165 2023-12-02T00:27:40.422 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 151/165 2023-12-02T00:27:40.422 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 152/165 2023-12-02T00:27:40.422 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 153/165 2023-12-02T00:27:40.422 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : NetworkManager-1:1.45.7-1.el9.x86_64 154/165 2023-12-02T00:27:40.422 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 155/165 2023-12-02T00:27:40.422 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : NetworkManager-libnm-1:1.45.7-1.el9.x86_64 156/165 2023-12-02T00:27:40.423 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 157/165 2023-12-02T00:27:40.423 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : NetworkManager-team-1:1.45.7-1.el9.x86_64 158/165 2023-12-02T00:27:40.423 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 159/165 2023-12-02T00:27:40.423 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : NetworkManager-tui-1:1.45.7-1.el9.x86_64 160/165 2023-12-02T00:27:40.423 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 161/165 2023-12-02T00:27:40.423 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 162/165 2023-12-02T00:27:40.423 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 163/165 2023-12-02T00:27:40.423 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 164/165 2023-12-02T00:27:40.434 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 161/165 2023-12-02T00:27:40.458 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 162/165 2023-12-02T00:27:40.483 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 163/165 2023-12-02T00:27:40.509 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 164/165 2023-12-02T00:27:40.577 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 165/165 2023-12-02T00:27:42.991 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 165/165 2023-12-02T00:27:42.991 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-02T00:27:42.991 INFO:teuthology.orchestra.run.smithi162.stdout:Upgraded: 2023-12-02T00:27:42.991 INFO:teuthology.orchestra.run.smithi162.stdout: NetworkManager-1:1.45.7-1.el9.x86_64 2023-12-02T00:27:42.991 INFO:teuthology.orchestra.run.smithi162.stdout: NetworkManager-libnm-1:1.45.7-1.el9.x86_64 2023-12-02T00:27:42.991 INFO:teuthology.orchestra.run.smithi162.stdout: NetworkManager-team-1:1.45.7-1.el9.x86_64 2023-12-02T00:27:42.991 INFO:teuthology.orchestra.run.smithi162.stdout: NetworkManager-tui-1:1.45.7-1.el9.x86_64 2023-12-02T00:27:42.991 INFO:teuthology.orchestra.run.smithi162.stdout: acl-2.3.1-4.el9.x86_64 2023-12-02T00:27:42.991 INFO:teuthology.orchestra.run.smithi162.stdout: elfutils-debuginfod-client-0.190-1.el9.x86_64 2023-12-02T00:27:42.991 INFO:teuthology.orchestra.run.smithi162.stdout: elfutils-libelf-0.190-1.el9.x86_64 2023-12-02T00:27:42.991 INFO:teuthology.orchestra.run.smithi162.stdout: elfutils-libs-0.190-1.el9.x86_64 2023-12-02T00:27:42.992 INFO:teuthology.orchestra.run.smithi162.stdout: libacl-2.3.1-4.el9.x86_64 2023-12-02T00:27:42.992 INFO:teuthology.orchestra.run.smithi162.stdout: libacl-devel-2.3.1-4.el9.x86_64 2023-12-02T00:27:42.992 INFO:teuthology.orchestra.run.smithi162.stdout: librados2-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:42.992 INFO:teuthology.orchestra.run.smithi162.stdout: librbd1-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:42.992 INFO:teuthology.orchestra.run.smithi162.stdout: openssl-1:3.0.7-25.el9.x86_64 2023-12-02T00:27:42.992 INFO:teuthology.orchestra.run.smithi162.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2023-12-02T00:27:42.992 INFO:teuthology.orchestra.run.smithi162.stdout: xfsdump-3.1.12-4.el9.x86_64 2023-12-02T00:27:42.992 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2023-12-02T00:27:42.992 INFO:teuthology.orchestra.run.smithi162.stdout: automake-1.16.2-8.el9.noarch 2023-12-02T00:27:42.992 INFO:teuthology.orchestra.run.smithi162.stdout: bison-3.7.4-5.el9.x86_64 2023-12-02T00:27:42.992 INFO:teuthology.orchestra.run.smithi162.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-12-02T00:27:42.992 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:42.993 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-base-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:42.993 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-common-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:42.993 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-fuse-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:42.993 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-grafana-dashboards-2:18.0.0-7631.gbed76814.el9.noarch 2023-12-02T00:27:42.993 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-immutable-object-cache-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:42.993 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mds-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:42.993 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:42.993 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-cephadm-2:18.0.0-7631.gbed76814.el9.noarch 2023-12-02T00:27:42.993 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-dashboard-2:18.0.0-7631.gbed76814.el9.noarch 2023-12-02T00:27:42.993 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7631.gbed76814.el9.noarch 2023-12-02T00:27:42.993 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-modules-core-2:18.0.0-7631.gbed76814.el9.noarch 2023-12-02T00:27:42.993 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-rook-2:18.0.0-7631.gbed76814.el9.noarch 2023-12-02T00:27:42.993 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mon-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:42.994 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-osd-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:42.994 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-prometheus-alerts-2:18.0.0-7631.gbed76814.el9.noarch 2023-12-02T00:27:42.994 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-radosgw-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:42.994 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-selinux-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:42.994 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-test-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:42.994 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-volume-2:18.0.0-7631.gbed76814.el9.noarch 2023-12-02T00:27:42.994 INFO:teuthology.orchestra.run.smithi162.stdout: cephadm-2:18.0.0-7631.gbed76814.el9.noarch 2023-12-02T00:27:42.994 INFO:teuthology.orchestra.run.smithi162.stdout: cephfs-mirror-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:42.994 INFO:teuthology.orchestra.run.smithi162.stdout: cephfs-top-2:18.0.0-7631.gbed76814.el9.noarch 2023-12-02T00:27:42.994 INFO:teuthology.orchestra.run.smithi162.stdout: elfutils-libelf-devel-0.190-1.el9.x86_64 2023-12-02T00:27:42.994 INFO:teuthology.orchestra.run.smithi162.stdout: flex-2.6.4-9.el9.x86_64 2023-12-02T00:27:42.994 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-12-02T00:27:42.994 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-12-02T00:27:42.994 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-12-02T00:27:42.995 INFO:teuthology.orchestra.run.smithi162.stdout: fuse-2.9.9-15.el9.x86_64 2023-12-02T00:27:42.995 INFO:teuthology.orchestra.run.smithi162.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-12-02T00:27:42.995 INFO:teuthology.orchestra.run.smithi162.stdout: jq-1.6-16.el9.x86_64 2023-12-02T00:27:42.995 INFO:teuthology.orchestra.run.smithi162.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-12-02T00:27:42.995 INFO:teuthology.orchestra.run.smithi162.stdout: libarrow-9.0.0-9.el9.x86_64 2023-12-02T00:27:42.995 INFO:teuthology.orchestra.run.smithi162.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-12-02T00:27:42.995 INFO:teuthology.orchestra.run.smithi162.stdout: libcephfs-devel-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:42.995 INFO:teuthology.orchestra.run.smithi162.stdout: libcephfs2-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:42.995 INFO:teuthology.orchestra.run.smithi162.stdout: libcephsqlite-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:42.996 INFO:teuthology.orchestra.run.smithi162.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-12-02T00:27:42.996 INFO:teuthology.orchestra.run.smithi162.stdout: liboath-2.6.7-2.el9.x86_64 2023-12-02T00:27:42.996 INFO:teuthology.orchestra.run.smithi162.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-12-02T00:27:42.996 INFO:teuthology.orchestra.run.smithi162.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-12-02T00:27:42.996 INFO:teuthology.orchestra.run.smithi162.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-12-02T00:27:42.996 INFO:teuthology.orchestra.run.smithi162.stdout: librados-devel-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:42.996 INFO:teuthology.orchestra.run.smithi162.stdout: libradosstriper1-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:42.996 INFO:teuthology.orchestra.run.smithi162.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-12-02T00:27:42.996 INFO:teuthology.orchestra.run.smithi162.stdout: librgw2-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:42.996 INFO:teuthology.orchestra.run.smithi162.stdout: libtool-2.4.6-45.el9.x86_64 2023-12-02T00:27:42.996 INFO:teuthology.orchestra.run.smithi162.stdout: libunwind-1.6.2-1.el9.x86_64 2023-12-02T00:27:42.997 INFO:teuthology.orchestra.run.smithi162.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-12-02T00:27:42.997 INFO:teuthology.orchestra.run.smithi162.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-12-02T00:27:42.997 INFO:teuthology.orchestra.run.smithi162.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-12-02T00:27:42.997 INFO:teuthology.orchestra.run.smithi162.stdout: lua-5.4.4-4.el9.x86_64 2023-12-02T00:27:42.997 INFO:teuthology.orchestra.run.smithi162.stdout: lua-devel-5.4.4-4.el9.x86_64 2023-12-02T00:27:42.997 INFO:teuthology.orchestra.run.smithi162.stdout: luarocks-3.9.2-1.el9.noarch 2023-12-02T00:27:42.997 INFO:teuthology.orchestra.run.smithi162.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-12-02T00:27:42.997 INFO:teuthology.orchestra.run.smithi162.stdout: openblas-0.3.21-2.el9.x86_64 2023-12-02T00:27:42.997 INFO:teuthology.orchestra.run.smithi162.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-12-02T00:27:42.997 INFO:teuthology.orchestra.run.smithi162.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2023-12-02T00:27:42.998 INFO:teuthology.orchestra.run.smithi162.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-12-02T00:27:42.998 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql-13.11-1.el9.x86_64 2023-12-02T00:27:42.998 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-12-02T00:27:42.998 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-12-02T00:27:42.998 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql-server-13.11-1.el9.x86_64 2023-12-02T00:27:42.998 INFO:teuthology.orchestra.run.smithi162.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-12-02T00:27:42.998 INFO:teuthology.orchestra.run.smithi162.stdout: python3-babel-2.9.1-2.el9.noarch 2023-12-02T00:27:42.998 INFO:teuthology.orchestra.run.smithi162.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-12-02T00:27:42.998 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-12-02T00:27:42.998 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ceph-argparse-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:42.998 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ceph-common-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:42.998 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cephfs-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:42.998 INFO:teuthology.orchestra.run.smithi162.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-12-02T00:27:42.998 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-12-02T00:27:42.999 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-12-02T00:27:42.999 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-12-02T00:27:42.999 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-12-02T00:27:42.999 INFO:teuthology.orchestra.run.smithi162.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-12-02T00:27:42.999 INFO:teuthology.orchestra.run.smithi162.stdout: python3-google-auth-1:2.23.4-1.el9.noarch 2023-12-02T00:27:42.999 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-12-02T00:27:42.999 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-12-02T00:27:42.999 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-12-02T00:27:42.999 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-12-02T00:27:42.999 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-12-02T00:27:42.999 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-12-02T00:27:42.999 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-12-02T00:27:42.999 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-12-02T00:27:43.000 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-12-02T00:27:43.000 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-12-02T00:27:43.000 INFO:teuthology.orchestra.run.smithi162.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-12-02T00:27:43.000 INFO:teuthology.orchestra.run.smithi162.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-12-02T00:27:43.000 INFO:teuthology.orchestra.run.smithi162.stdout: python3-mako-1.1.4-6.el9.noarch 2023-12-02T00:27:43.000 INFO:teuthology.orchestra.run.smithi162.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-12-02T00:27:43.000 INFO:teuthology.orchestra.run.smithi162.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-12-02T00:27:43.000 INFO:teuthology.orchestra.run.smithi162.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-12-02T00:27:43.000 INFO:teuthology.orchestra.run.smithi162.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-12-02T00:27:43.000 INFO:teuthology.orchestra.run.smithi162.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-12-02T00:27:43.000 INFO:teuthology.orchestra.run.smithi162.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-12-02T00:27:43.000 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-12-02T00:27:43.000 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ply-3.11-14.el9.noarch 2023-12-02T00:27:43.000 INFO:teuthology.orchestra.run.smithi162.stdout: python3-portend-3.1.0-2.el9.noarch 2023-12-02T00:27:43.000 INFO:teuthology.orchestra.run.smithi162.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-12-02T00:27:43.001 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-12-02T00:27:43.001 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-12-02T00:27:43.001 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-12-02T00:27:43.001 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pycparser-2.20-6.el9.noarch 2023-12-02T00:27:43.001 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pytz-2021.1-5.el9.noarch 2023-12-02T00:27:43.001 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rados-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:43.001 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rbd-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:43.001 INFO:teuthology.orchestra.run.smithi162.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-12-02T00:27:43.001 INFO:teuthology.orchestra.run.smithi162.stdout: python3-requests-2.25.1-7.el9.noarch 2023-12-02T00:27:43.001 INFO:teuthology.orchestra.run.smithi162.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-12-02T00:27:43.001 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rgw-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:43.001 INFO:teuthology.orchestra.run.smithi162.stdout: python3-routes-2.5.1-5.el9.noarch 2023-12-02T00:27:43.001 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rsa-4.9-2.el9.noarch 2023-12-02T00:27:43.001 INFO:teuthology.orchestra.run.smithi162.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-12-02T00:27:43.002 INFO:teuthology.orchestra.run.smithi162.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-12-02T00:27:43.002 INFO:teuthology.orchestra.run.smithi162.stdout: python3-toml-0.10.2-6.el9.noarch 2023-12-02T00:27:43.002 INFO:teuthology.orchestra.run.smithi162.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-12-02T00:27:43.002 INFO:teuthology.orchestra.run.smithi162.stdout: python3-urllib3-1.26.5-4.el9.noarch 2023-12-02T00:27:43.002 INFO:teuthology.orchestra.run.smithi162.stdout: python3-webob-1.8.7-6.el9.noarch 2023-12-02T00:27:43.002 INFO:teuthology.orchestra.run.smithi162.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-12-02T00:27:43.002 INFO:teuthology.orchestra.run.smithi162.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-12-02T00:27:43.002 INFO:teuthology.orchestra.run.smithi162.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-12-02T00:27:43.002 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-fuse-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:43.002 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-mirror-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:43.002 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-nbd-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:43.002 INFO:teuthology.orchestra.run.smithi162.stdout: re2-1:20211101-3.el9.x86_64 2023-12-02T00:27:43.002 INFO:teuthology.orchestra.run.smithi162.stdout: socat-1.7.4.1-5.el9.x86_64 2023-12-02T00:27:43.003 INFO:teuthology.orchestra.run.smithi162.stdout: thrift-0.15.0-2.el9.x86_64 2023-12-02T00:27:43.003 INFO:teuthology.orchestra.run.smithi162.stdout: uuid-1.6.2-55.el9.x86_64 2023-12-02T00:27:43.003 INFO:teuthology.orchestra.run.smithi162.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-12-02T00:27:43.003 INFO:teuthology.orchestra.run.smithi162.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-12-02T00:27:43.003 INFO:teuthology.orchestra.run.smithi162.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-12-02T00:27:43.003 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-02T00:27:43.004 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2023-12-02T00:27:43.592 DEBUG:teuthology.parallel:result is None 2023-12-02T00:27:44.871 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: xfsdump-3.1.12-3.el9.x86_64 165/165 2023-12-02T00:27:44.872 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-2:18.0.0-7631.gbed76814.el9.x86_64 1/165 2023-12-02T00:27:44.872 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-base-2:18.0.0-7631.gbed76814.el9.x86_64 2/165 2023-12-02T00:27:44.872 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-common-2:18.0.0-7631.gbed76814.el9.x86_64 3/165 2023-12-02T00:27:44.872 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-fuse-2:18.0.0-7631.gbed76814.el9.x86_64 4/165 2023-12-02T00:27:44.872 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7631.gbed76 5/165 2023-12-02T00:27:44.872 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mds-2:18.0.0-7631.gbed76814.el9.x86_64 6/165 2023-12-02T00:27:44.872 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-2:18.0.0-7631.gbed76814.el9.x86_64 7/165 2023-12-02T00:27:44.872 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mon-2:18.0.0-7631.gbed76814.el9.x86_64 8/165 2023-12-02T00:27:44.872 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-osd-2:18.0.0-7631.gbed76814.el9.x86_64 9/165 2023-12-02T00:27:44.872 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-radosgw-2:18.0.0-7631.gbed76814.el9.x86_64 10/165 2023-12-02T00:27:44.872 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-selinux-2:18.0.0-7631.gbed76814.el9.x86_64 11/165 2023-12-02T00:27:44.872 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-test-2:18.0.0-7631.gbed76814.el9.x86_64 12/165 2023-12-02T00:27:44.872 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : cephfs-mirror-2:18.0.0-7631.gbed76814.el9.x86_64 13/165 2023-12-02T00:27:44.872 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libcephfs-devel-2:18.0.0-7631.gbed76814.el9.x86_ 14/165 2023-12-02T00:27:44.872 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libcephfs2-2:18.0.0-7631.gbed76814.el9.x86_64 15/165 2023-12-02T00:27:44.873 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libcephsqlite-2:18.0.0-7631.gbed76814.el9.x86_64 16/165 2023-12-02T00:27:44.873 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librados-devel-2:18.0.0-7631.gbed76814.el9.x86_6 17/165 2023-12-02T00:27:44.873 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libradosstriper1-2:18.0.0-7631.gbed76814.el9.x86 18/165 2023-12-02T00:27:44.873 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librgw2-2:18.0.0-7631.gbed76814.el9.x86_64 19/165 2023-12-02T00:27:44.873 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7631.gbed76814.el 20/165 2023-12-02T00:27:44.873 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-ceph-common-2:18.0.0-7631.gbed76814.el9. 21/165 2023-12-02T00:27:44.873 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cephfs-2:18.0.0-7631.gbed76814.el9.x86_6 22/165 2023-12-02T00:27:44.873 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rados-2:18.0.0-7631.gbed76814.el9.x86_64 23/165 2023-12-02T00:27:44.873 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rbd-2:18.0.0-7631.gbed76814.el9.x86_64 24/165 2023-12-02T00:27:44.873 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rgw-2:18.0.0-7631.gbed76814.el9.x86_64 25/165 2023-12-02T00:27:44.873 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : rbd-fuse-2:18.0.0-7631.gbed76814.el9.x86_64 26/165 2023-12-02T00:27:44.873 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : rbd-mirror-2:18.0.0-7631.gbed76814.el9.x86_64 27/165 2023-12-02T00:27:44.885 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : rbd-nbd-2:18.0.0-7631.gbed76814.el9.x86_64 28/165 2023-12-02T00:27:44.885 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7631.gbed76814. 29/165 2023-12-02T00:27:44.886 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7631.gbed76814.el9.noa 30/165 2023-12-02T00:27:44.886 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7631.gbed76814.el9.n 31/165 2023-12-02T00:27:44.886 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7631.gbed 32/165 2023-12-02T00:27:44.886 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7631.gbed76814.el 33/165 2023-12-02T00:27:44.886 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7631.gbed76814.el9.noarch 34/165 2023-12-02T00:27:44.886 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7631.gbed76814.e 35/165 2023-12-02T00:27:44.886 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-volume-2:18.0.0-7631.gbed76814.el9.noarch 36/165 2023-12-02T00:27:44.886 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : cephadm-2:18.0.0-7631.gbed76814.el9.noarch 37/165 2023-12-02T00:27:44.886 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : cephfs-top-2:18.0.0-7631.gbed76814.el9.noarch 38/165 2023-12-02T00:27:44.886 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : automake-1.16.2-8.el9.noarch 39/165 2023-12-02T00:27:44.886 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : bison-3.7.4-5.el9.x86_64 40/165 2023-12-02T00:27:44.886 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 41/165 2023-12-02T00:27:44.886 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : elfutils-libelf-devel-0.190-1.el9.x86_64 42/165 2023-12-02T00:27:44.886 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/165 2023-12-02T00:27:44.887 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/165 2023-12-02T00:27:44.887 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/165 2023-12-02T00:27:44.887 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/165 2023-12-02T00:27:44.887 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : jq-1.6-16.el9.x86_64 47/165 2023-12-02T00:27:44.887 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/165 2023-12-02T00:27:44.887 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/165 2023-12-02T00:27:44.887 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/165 2023-12-02T00:27:44.887 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/165 2023-12-02T00:27:44.887 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/165 2023-12-02T00:27:44.887 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 53/165 2023-12-02T00:27:44.887 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/165 2023-12-02T00:27:44.887 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/165 2023-12-02T00:27:44.888 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : lua-5.4.4-4.el9.x86_64 56/165 2023-12-02T00:27:44.888 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 57/165 2023-12-02T00:27:44.888 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 58/165 2023-12-02T00:27:44.888 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 59/165 2023-12-02T00:27:44.888 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : postgresql-13.11-1.el9.x86_64 60/165 2023-12-02T00:27:44.888 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 61/165 2023-12-02T00:27:44.888 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 62/165 2023-12-02T00:27:44.888 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 63/165 2023-12-02T00:27:44.888 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/165 2023-12-02T00:27:44.888 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 65/165 2023-12-02T00:27:44.888 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 66/165 2023-12-02T00:27:44.888 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 67/165 2023-12-02T00:27:44.888 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 68/165 2023-12-02T00:27:44.888 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 69/165 2023-12-02T00:27:44.888 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 70/165 2023-12-02T00:27:44.889 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 71/165 2023-12-02T00:27:44.889 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 72/165 2023-12-02T00:27:44.889 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/165 2023-12-02T00:27:44.889 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 74/165 2023-12-02T00:27:44.889 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 75/165 2023-12-02T00:27:44.889 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 76/165 2023-12-02T00:27:44.889 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 77/165 2023-12-02T00:27:44.889 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 78/165 2023-12-02T00:27:44.889 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 79/165 2023-12-02T00:27:44.889 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 80/165 2023-12-02T00:27:44.889 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 81/165 2023-12-02T00:27:44.889 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 82/165 2023-12-02T00:27:44.889 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 83/165 2023-12-02T00:27:44.889 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 84/165 2023-12-02T00:27:44.889 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 85/165 2023-12-02T00:27:44.889 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 86/165 2023-12-02T00:27:44.890 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 87/165 2023-12-02T00:27:44.890 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 88/165 2023-12-02T00:27:44.890 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 89/165 2023-12-02T00:27:44.890 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-ply-3.11-14.el9.noarch 90/165 2023-12-02T00:27:44.890 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 91/165 2023-12-02T00:27:44.890 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 92/165 2023-12-02T00:27:44.890 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-urllib3-1.26.5-4.el9.noarch 93/165 2023-12-02T00:27:44.890 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 94/165 2023-12-02T00:27:44.890 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 95/165 2023-12-02T00:27:44.890 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 96/165 2023-12-02T00:27:44.890 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 97/165 2023-12-02T00:27:44.890 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 98/165 2023-12-02T00:27:44.890 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 99/165 2023-12-02T00:27:44.890 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 100/165 2023-12-02T00:27:44.891 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 101/165 2023-12-02T00:27:44.891 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 102/165 2023-12-02T00:27:44.891 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 103/165 2023-12-02T00:27:44.891 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 104/165 2023-12-02T00:27:44.891 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 105/165 2023-12-02T00:27:44.891 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 106/165 2023-12-02T00:27:44.891 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 107/165 2023-12-02T00:27:44.891 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 108/165 2023-12-02T00:27:44.891 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 109/165 2023-12-02T00:27:44.891 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-google-auth-1:2.23.4-1.el9.noarch 110/165 2023-12-02T00:27:44.891 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 111/165 2023-12-02T00:27:44.891 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 112/165 2023-12-02T00:27:44.891 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 113/165 2023-12-02T00:27:44.891 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 114/165 2023-12-02T00:27:44.891 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 115/165 2023-12-02T00:27:44.891 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 116/165 2023-12-02T00:27:44.892 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 117/165 2023-12-02T00:27:44.892 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 118/165 2023-12-02T00:27:44.892 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 119/165 2023-12-02T00:27:44.892 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 120/165 2023-12-02T00:27:44.892 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 121/165 2023-12-02T00:27:44.892 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 122/165 2023-12-02T00:27:44.892 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 123/165 2023-12-02T00:27:44.892 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 124/165 2023-12-02T00:27:44.892 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 125/165 2023-12-02T00:27:44.892 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 126/165 2023-12-02T00:27:44.892 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 127/165 2023-12-02T00:27:44.892 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 128/165 2023-12-02T00:27:44.892 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 129/165 2023-12-02T00:27:44.892 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 130/165 2023-12-02T00:27:44.893 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 131/165 2023-12-02T00:27:44.893 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 132/165 2023-12-02T00:27:44.893 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 133/165 2023-12-02T00:27:44.893 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : re2-1:20211101-3.el9.x86_64 134/165 2023-12-02T00:27:44.893 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 135/165 2023-12-02T00:27:44.893 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librados2-2:18.0.0-7631.gbed76814.el9.x86_64 136/165 2023-12-02T00:27:44.893 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 137/165 2023-12-02T00:27:44.893 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librbd1-2:18.0.0-7631.gbed76814.el9.x86_64 138/165 2023-12-02T00:27:44.893 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 139/165 2023-12-02T00:27:44.893 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 140/165 2023-12-02T00:27:44.893 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 141/165 2023-12-02T00:27:44.893 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : acl-2.3.1-4.el9.x86_64 142/165 2023-12-02T00:27:44.893 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : acl-2.3.1-3.el9.x86_64 143/165 2023-12-02T00:27:44.893 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : elfutils-debuginfod-client-0.190-1.el9.x86_64 144/165 2023-12-02T00:27:44.893 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 145/165 2023-12-02T00:27:44.893 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : elfutils-libelf-0.190-1.el9.x86_64 146/165 2023-12-02T00:27:44.894 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 147/165 2023-12-02T00:27:44.894 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : elfutils-libs-0.190-1.el9.x86_64 148/165 2023-12-02T00:27:44.894 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 149/165 2023-12-02T00:27:44.894 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 150/165 2023-12-02T00:27:44.894 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 151/165 2023-12-02T00:27:44.894 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 152/165 2023-12-02T00:27:44.894 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 153/165 2023-12-02T00:27:44.894 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : NetworkManager-1:1.45.7-1.el9.x86_64 154/165 2023-12-02T00:27:44.894 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 155/165 2023-12-02T00:27:44.894 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : NetworkManager-libnm-1:1.45.7-1.el9.x86_64 156/165 2023-12-02T00:27:44.894 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 157/165 2023-12-02T00:27:44.894 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : NetworkManager-team-1:1.45.7-1.el9.x86_64 158/165 2023-12-02T00:27:44.894 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 159/165 2023-12-02T00:27:44.894 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : NetworkManager-tui-1:1.45.7-1.el9.x86_64 160/165 2023-12-02T00:27:44.894 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 161/165 2023-12-02T00:27:44.895 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 162/165 2023-12-02T00:27:44.895 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 163/165 2023-12-02T00:27:44.895 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 164/165 2023-12-02T00:27:47.476 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 165/165 2023-12-02T00:27:47.476 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:27:47.476 INFO:teuthology.orchestra.run.smithi046.stdout:Upgraded: 2023-12-02T00:27:47.476 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager-1:1.45.7-1.el9.x86_64 2023-12-02T00:27:47.476 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager-libnm-1:1.45.7-1.el9.x86_64 2023-12-02T00:27:47.476 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager-team-1:1.45.7-1.el9.x86_64 2023-12-02T00:27:47.477 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager-tui-1:1.45.7-1.el9.x86_64 2023-12-02T00:27:47.477 INFO:teuthology.orchestra.run.smithi046.stdout: acl-2.3.1-4.el9.x86_64 2023-12-02T00:27:47.477 INFO:teuthology.orchestra.run.smithi046.stdout: elfutils-debuginfod-client-0.190-1.el9.x86_64 2023-12-02T00:27:47.477 INFO:teuthology.orchestra.run.smithi046.stdout: elfutils-libelf-0.190-1.el9.x86_64 2023-12-02T00:27:47.477 INFO:teuthology.orchestra.run.smithi046.stdout: elfutils-libs-0.190-1.el9.x86_64 2023-12-02T00:27:47.477 INFO:teuthology.orchestra.run.smithi046.stdout: libacl-2.3.1-4.el9.x86_64 2023-12-02T00:27:47.478 INFO:teuthology.orchestra.run.smithi046.stdout: libacl-devel-2.3.1-4.el9.x86_64 2023-12-02T00:27:47.478 INFO:teuthology.orchestra.run.smithi046.stdout: librados2-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:47.478 INFO:teuthology.orchestra.run.smithi046.stdout: librbd1-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:47.478 INFO:teuthology.orchestra.run.smithi046.stdout: openssl-1:3.0.7-25.el9.x86_64 2023-12-02T00:27:47.478 INFO:teuthology.orchestra.run.smithi046.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2023-12-02T00:27:47.478 INFO:teuthology.orchestra.run.smithi046.stdout: xfsdump-3.1.12-4.el9.x86_64 2023-12-02T00:27:47.478 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2023-12-02T00:27:47.478 INFO:teuthology.orchestra.run.smithi046.stdout: automake-1.16.2-8.el9.noarch 2023-12-02T00:27:47.478 INFO:teuthology.orchestra.run.smithi046.stdout: bison-3.7.4-5.el9.x86_64 2023-12-02T00:27:47.479 INFO:teuthology.orchestra.run.smithi046.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-12-02T00:27:47.479 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:47.479 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-base-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:47.479 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-common-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:47.479 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-fuse-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:47.479 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-grafana-dashboards-2:18.0.0-7631.gbed76814.el9.noarch 2023-12-02T00:27:47.479 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-immutable-object-cache-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:47.479 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mds-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:47.479 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:47.480 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-cephadm-2:18.0.0-7631.gbed76814.el9.noarch 2023-12-02T00:27:47.480 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-dashboard-2:18.0.0-7631.gbed76814.el9.noarch 2023-12-02T00:27:47.480 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7631.gbed76814.el9.noarch 2023-12-02T00:27:47.480 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-modules-core-2:18.0.0-7631.gbed76814.el9.noarch 2023-12-02T00:27:47.480 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-rook-2:18.0.0-7631.gbed76814.el9.noarch 2023-12-02T00:27:47.480 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mon-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:47.480 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-osd-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:47.480 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-prometheus-alerts-2:18.0.0-7631.gbed76814.el9.noarch 2023-12-02T00:27:47.480 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-radosgw-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:47.480 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-selinux-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:47.481 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-test-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:47.481 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-volume-2:18.0.0-7631.gbed76814.el9.noarch 2023-12-02T00:27:47.481 INFO:teuthology.orchestra.run.smithi046.stdout: cephadm-2:18.0.0-7631.gbed76814.el9.noarch 2023-12-02T00:27:47.481 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-mirror-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:47.481 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-top-2:18.0.0-7631.gbed76814.el9.noarch 2023-12-02T00:27:47.481 INFO:teuthology.orchestra.run.smithi046.stdout: elfutils-libelf-devel-0.190-1.el9.x86_64 2023-12-02T00:27:47.481 INFO:teuthology.orchestra.run.smithi046.stdout: flex-2.6.4-9.el9.x86_64 2023-12-02T00:27:47.481 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-12-02T00:27:47.482 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-12-02T00:27:47.482 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-12-02T00:27:47.482 INFO:teuthology.orchestra.run.smithi046.stdout: fuse-2.9.9-15.el9.x86_64 2023-12-02T00:27:47.482 INFO:teuthology.orchestra.run.smithi046.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-12-02T00:27:47.482 INFO:teuthology.orchestra.run.smithi046.stdout: jq-1.6-16.el9.x86_64 2023-12-02T00:27:47.482 INFO:teuthology.orchestra.run.smithi046.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-12-02T00:27:47.483 INFO:teuthology.orchestra.run.smithi046.stdout: libarrow-9.0.0-9.el9.x86_64 2023-12-02T00:27:47.483 INFO:teuthology.orchestra.run.smithi046.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-12-02T00:27:47.483 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs-devel-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:47.483 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs2-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:47.483 INFO:teuthology.orchestra.run.smithi046.stdout: libcephsqlite-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:47.483 INFO:teuthology.orchestra.run.smithi046.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-12-02T00:27:47.483 INFO:teuthology.orchestra.run.smithi046.stdout: liboath-2.6.7-2.el9.x86_64 2023-12-02T00:27:47.483 INFO:teuthology.orchestra.run.smithi046.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-12-02T00:27:47.483 INFO:teuthology.orchestra.run.smithi046.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-12-02T00:27:47.483 INFO:teuthology.orchestra.run.smithi046.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-12-02T00:27:47.483 INFO:teuthology.orchestra.run.smithi046.stdout: librados-devel-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:47.483 INFO:teuthology.orchestra.run.smithi046.stdout: libradosstriper1-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:47.483 INFO:teuthology.orchestra.run.smithi046.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-12-02T00:27:47.483 INFO:teuthology.orchestra.run.smithi046.stdout: librgw2-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:47.483 INFO:teuthology.orchestra.run.smithi046.stdout: libtool-2.4.6-45.el9.x86_64 2023-12-02T00:27:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: libunwind-1.6.2-1.el9.x86_64 2023-12-02T00:27:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-12-02T00:27:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-12-02T00:27:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-12-02T00:27:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: lua-5.4.4-4.el9.x86_64 2023-12-02T00:27:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: lua-devel-5.4.4-4.el9.x86_64 2023-12-02T00:27:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: luarocks-3.9.2-1.el9.noarch 2023-12-02T00:27:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-12-02T00:27:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: openblas-0.3.21-2.el9.x86_64 2023-12-02T00:27:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-12-02T00:27:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2023-12-02T00:27:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-12-02T00:27:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-13.11-1.el9.x86_64 2023-12-02T00:27:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-12-02T00:27:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-12-02T00:27:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-server-13.11-1.el9.x86_64 2023-12-02T00:27:47.485 INFO:teuthology.orchestra.run.smithi046.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-12-02T00:27:47.485 INFO:teuthology.orchestra.run.smithi046.stdout: python3-babel-2.9.1-2.el9.noarch 2023-12-02T00:27:47.485 INFO:teuthology.orchestra.run.smithi046.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-12-02T00:27:47.485 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-12-02T00:27:47.485 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-argparse-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:47.485 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-common-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:47.485 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cephfs-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:47.486 INFO:teuthology.orchestra.run.smithi046.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-12-02T00:27:47.486 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-12-02T00:27:47.486 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-12-02T00:27:47.486 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-12-02T00:27:47.486 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-12-02T00:27:47.486 INFO:teuthology.orchestra.run.smithi046.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-12-02T00:27:47.486 INFO:teuthology.orchestra.run.smithi046.stdout: python3-google-auth-1:2.23.4-1.el9.noarch 2023-12-02T00:27:47.486 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-12-02T00:27:47.487 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-12-02T00:27:47.487 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-12-02T00:27:47.487 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-12-02T00:27:47.487 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-12-02T00:27:47.487 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-12-02T00:27:47.487 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-12-02T00:27:47.487 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-12-02T00:27:47.488 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-12-02T00:27:47.488 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-12-02T00:27:47.488 INFO:teuthology.orchestra.run.smithi046.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-12-02T00:27:47.488 INFO:teuthology.orchestra.run.smithi046.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-12-02T00:27:47.488 INFO:teuthology.orchestra.run.smithi046.stdout: python3-mako-1.1.4-6.el9.noarch 2023-12-02T00:27:47.488 INFO:teuthology.orchestra.run.smithi046.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-12-02T00:27:47.488 INFO:teuthology.orchestra.run.smithi046.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-12-02T00:27:47.489 INFO:teuthology.orchestra.run.smithi046.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-12-02T00:27:47.489 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-12-02T00:27:47.489 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-12-02T00:27:47.489 INFO:teuthology.orchestra.run.smithi046.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-12-02T00:27:47.489 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-12-02T00:27:47.489 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ply-3.11-14.el9.noarch 2023-12-02T00:27:47.489 INFO:teuthology.orchestra.run.smithi046.stdout: python3-portend-3.1.0-2.el9.noarch 2023-12-02T00:27:47.490 INFO:teuthology.orchestra.run.smithi046.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-12-02T00:27:47.490 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-12-02T00:27:47.490 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-12-02T00:27:47.490 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-12-02T00:27:47.490 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pycparser-2.20-6.el9.noarch 2023-12-02T00:27:47.490 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pytz-2021.1-5.el9.noarch 2023-12-02T00:27:47.491 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rados-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:47.491 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rbd-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:47.491 INFO:teuthology.orchestra.run.smithi046.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-12-02T00:27:47.491 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-2.25.1-7.el9.noarch 2023-12-02T00:27:47.491 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-12-02T00:27:47.491 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rgw-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:47.491 INFO:teuthology.orchestra.run.smithi046.stdout: python3-routes-2.5.1-5.el9.noarch 2023-12-02T00:27:47.492 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rsa-4.9-2.el9.noarch 2023-12-02T00:27:47.492 INFO:teuthology.orchestra.run.smithi046.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-12-02T00:27:47.492 INFO:teuthology.orchestra.run.smithi046.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-12-02T00:27:47.492 INFO:teuthology.orchestra.run.smithi046.stdout: python3-toml-0.10.2-6.el9.noarch 2023-12-02T00:27:47.492 INFO:teuthology.orchestra.run.smithi046.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-12-02T00:27:47.492 INFO:teuthology.orchestra.run.smithi046.stdout: python3-urllib3-1.26.5-4.el9.noarch 2023-12-02T00:27:47.493 INFO:teuthology.orchestra.run.smithi046.stdout: python3-webob-1.8.7-6.el9.noarch 2023-12-02T00:27:47.493 INFO:teuthology.orchestra.run.smithi046.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-12-02T00:27:47.493 INFO:teuthology.orchestra.run.smithi046.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-12-02T00:27:47.493 INFO:teuthology.orchestra.run.smithi046.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-12-02T00:27:47.493 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-fuse-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:47.493 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-mirror-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:47.493 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-nbd-2:18.0.0-7631.gbed76814.el9.x86_64 2023-12-02T00:27:47.493 INFO:teuthology.orchestra.run.smithi046.stdout: re2-1:20211101-3.el9.x86_64 2023-12-02T00:27:47.493 INFO:teuthology.orchestra.run.smithi046.stdout: socat-1.7.4.1-5.el9.x86_64 2023-12-02T00:27:47.493 INFO:teuthology.orchestra.run.smithi046.stdout: thrift-0.15.0-2.el9.x86_64 2023-12-02T00:27:47.493 INFO:teuthology.orchestra.run.smithi046.stdout: uuid-1.6.2-55.el9.x86_64 2023-12-02T00:27:47.493 INFO:teuthology.orchestra.run.smithi046.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-12-02T00:27:47.494 INFO:teuthology.orchestra.run.smithi046.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-12-02T00:27:47.494 INFO:teuthology.orchestra.run.smithi046.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-12-02T00:27:47.494 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:27:47.494 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2023-12-02T00:27:48.060 DEBUG:teuthology.parallel:result is None 2023-12-02T00:27:48.061 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bed7681480ccb0192b91a35e691658867e1e96b7 2023-12-02T00:27:48.418 DEBUG:teuthology.orchestra.run.smithi046:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-12-02T00:27:48.442 INFO:teuthology.orchestra.run.smithi046.stdout:18.0.0-7631.gbed76814.el9 2023-12-02T00:27:48.442 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7631.gbed76814.el9 2023-12-02T00:27:48.443 INFO:teuthology.task.install:The correct ceph version 18.0.0-7631.gbed76814 is installed. 2023-12-02T00:27:48.443 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bed7681480ccb0192b91a35e691658867e1e96b7 2023-12-02T00:27:48.762 DEBUG:teuthology.orchestra.run.smithi162:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-12-02T00:27:48.787 INFO:teuthology.orchestra.run.smithi162.stdout:18.0.0-7631.gbed76814.el9 2023-12-02T00:27:48.788 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7631.gbed76814.el9 2023-12-02T00:27:48.788 INFO:teuthology.task.install:The correct ceph version 18.0.0-7631.gbed76814 is installed. 2023-12-02T00:27:48.788 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-12-02T00:27:48.788 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-02T00:27:48.788 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-12-02T00:27:48.827 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-12-02T00:27:48.828 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-12-02T00:27:48.869 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-12-02T00:27:48.870 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-02T00:27:48.870 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/daemon-helper 2023-12-02T00:27:48.906 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-12-02T00:27:48.980 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-12-02T00:27:48.980 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/usr/bin/daemon-helper 2023-12-02T00:27:49.016 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-12-02T00:27:49.094 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-12-02T00:27:49.094 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-02T00:27:49.094 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/adjust-ulimits 2023-12-02T00:27:49.127 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-12-02T00:27:49.201 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-12-02T00:27:49.201 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/usr/bin/adjust-ulimits 2023-12-02T00:27:49.232 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-12-02T00:27:49.302 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-12-02T00:27:49.302 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-02T00:27:49.303 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/stdin-killer 2023-12-02T00:27:49.336 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-12-02T00:27:49.409 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-12-02T00:27:49.409 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/usr/bin/stdin-killer 2023-12-02T00:27:49.437 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-12-02T00:27:49.507 INFO:teuthology.run_tasks:Running task ceph... 2023-12-02T00:27:49.610 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-12-02T00:27:49.610 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 777 /var/log/ceph 2023-12-02T00:27:49.613 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 777 /var/log/ceph 2023-12-02T00:27:49.643 INFO:tasks.ceph:Disabling ceph logrotate... 2023-12-02T00:27:49.643 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /etc/logrotate.d/ceph 2023-12-02T00:27:49.686 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -f -- /etc/logrotate.d/ceph 2023-12-02T00:27:49.714 INFO:tasks.ceph:Creating extra log directories... 2023-12-02T00:27:49.714 DEBUG:teuthology.orchestra.run.smithi046:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-12-02T00:27:49.757 DEBUG:teuthology.orchestra.run.smithi162:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-12-02T00:27:49.792 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-12-02T00:27:49.792 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-12-02T00:27:49.792 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-12-02T00:27:49.792 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-02T00:27:49.792 DEBUG:teuthology.orchestra.run.smithi046:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-12-02T00:27:49.843 DEBUG:teuthology.orchestra.run.smithi046:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-12-02T00:27:49.974 DEBUG:teuthology.orchestra.run.smithi046:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-12-02T00:27:50.005 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-12-02T00:27:50.005 DEBUG:teuthology.orchestra.run.smithi162:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-12-02T00:27:50.020 DEBUG:teuthology.orchestra.run.smithi162:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-12-02T00:27:50.139 DEBUG:teuthology.orchestra.run.smithi162:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-12-02T00:27:50.171 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-12-02T00:27:50.172 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-12-02T00:27:50.172 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-12-01_17:57:53-fs-main-distro-default-smithi/7474581', 'branch': 'main', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/pool-perm}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7474581', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2023-12-01_17:57:53-fs-main-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': 'bed7681480ccb0192b91a35e691658867e1e96b7'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'bed7681480ccb0192b91a35e691658867e1e96b7'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': 'bed7681480ccb0192b91a35e691658867e1e96b7'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 75, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 1991, 'sha1': 'bed7681480ccb0192b91a35e691658867e1e96b7', 'sleep_before_teardown': 0, 'subset': '111/12000', 'suite': 'fs', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_bed7681480ccb0192b91a35e691658867e1e96b7/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'bed7681480ccb0192b91a35e691658867e1e96b7', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi046.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKZy/FA+EREo0z8VJi9Rp6/BsV8NFlDNytLV8JJkAdXZb9ZW5B/l6lq8qWNioT2ApsmOClzljsrv9/+hKUss4Ag=', 'smithi162.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD+84dNHyVg7BDh7SUQ8Op99UGvFszg9wn8IU0wSp3Htbhuzt5b5g5WRjXDiOjAh4yuQB30H0+XZrpmNFRD4KmQ='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bed7681480ccb0192b91a35e691658867e1e96b7'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': 'bed7681480ccb0192b91a35e691658867e1e96b7', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_pool_perm']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': 'f2de108beb0a6dde7f3e6c96039071828a1f6269', 'timestamp': '2023-12-01_17:57:53', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.232157'} 2023-12-02T00:27:50.172 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-12-02T00:27:50.175 DEBUG:teuthology.orchestra.run.smithi162:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-12-02T00:27:50.222 DEBUG:teuthology.orchestra.run.smithi046:> sudo install -d -m0777 -- /var/run/ceph 2023-12-02T00:27:50.233 DEBUG:teuthology.orchestra.run.smithi162:> sudo install -d -m0777 -- /var/run/ceph 2023-12-02T00:27:50.289 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-02T00:27:50.289 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/scratch_devs of=/dev/stdout 2023-12-02T00:27:50.316 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-12-02T00:27:50.317 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_1 2023-12-02T00:27:50.372 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-12-02T00:27:50.372 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-02T00:27:50.372 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 809 Links: 1 2023-12-02T00:27:50.372 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-02T00:27:50.372 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2023-12-02T00:27:50.372 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2023-12-02 00:27:40.920113148 +0000 2023-12-02T00:27:50.373 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2023-12-02 00:25:36.005850160 +0000 2023-12-02T00:27:50.373 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2023-12-02 00:25:36.005850160 +0000 2023-12-02T00:27:50.373 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2023-12-02 00:25:36.005850160 +0000 2023-12-02T00:27:50.373 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-12-02T00:27:50.442 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2023-12-02T00:27:50.442 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2023-12-02T00:27:50.442 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 9.999e-05 s, 5.1 MB/s 2023-12-02T00:27:50.443 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-12-02T00:27:50.500 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_2 2023-12-02T00:27:50.556 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-12-02T00:27:50.556 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-02T00:27:50.556 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 830 Links: 1 2023-12-02T00:27:50.556 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-02T00:27:50.556 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2023-12-02T00:27:50.556 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2023-12-02 00:27:40.920113148 +0000 2023-12-02T00:27:50.556 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2023-12-02 00:25:36.513844616 +0000 2023-12-02T00:27:50.556 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2023-12-02 00:25:36.513844616 +0000 2023-12-02T00:27:50.556 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2023-12-02 00:25:36.513844616 +0000 2023-12-02T00:27:50.557 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-12-02T00:27:50.624 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2023-12-02T00:27:50.625 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2023-12-02T00:27:50.625 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 8.2932e-05 s, 6.2 MB/s 2023-12-02T00:27:50.626 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-12-02T00:27:50.682 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_3 2023-12-02T00:27:50.736 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-12-02T00:27:50.737 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-02T00:27:50.737 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 847 Links: 1 2023-12-02T00:27:50.737 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-02T00:27:50.737 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2023-12-02T00:27:50.737 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2023-12-02 00:27:40.921113137 +0000 2023-12-02T00:27:50.737 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2023-12-02 00:25:37.020839082 +0000 2023-12-02T00:27:50.737 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2023-12-02 00:25:37.020839082 +0000 2023-12-02T00:27:50.737 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2023-12-02 00:25:37.020839082 +0000 2023-12-02T00:27:50.737 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-12-02T00:27:50.806 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2023-12-02T00:27:50.806 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2023-12-02T00:27:50.806 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 9.6367e-05 s, 5.3 MB/s 2023-12-02T00:27:50.807 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-12-02T00:27:50.864 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_4 2023-12-02T00:27:50.919 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-12-02T00:27:50.920 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-02T00:27:50.920 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 865 Links: 1 2023-12-02T00:27:50.920 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-02T00:27:50.920 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2023-12-02T00:27:50.920 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2023-12-02 00:27:40.921113137 +0000 2023-12-02T00:27:50.920 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2023-12-02 00:25:37.562833167 +0000 2023-12-02T00:27:50.920 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2023-12-02 00:25:37.562833167 +0000 2023-12-02T00:27:50.920 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2023-12-02 00:25:37.562833167 +0000 2023-12-02T00:27:50.920 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-12-02T00:27:50.989 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2023-12-02T00:27:50.989 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2023-12-02T00:27:50.989 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 9.6548e-05 s, 5.3 MB/s 2023-12-02T00:27:50.990 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-12-02T00:27:51.047 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-02T00:27:51.047 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-12-02T00:27:51.047 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/scratch_devs of=/dev/stdout 2023-12-02T00:27:51.064 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-12-02T00:27:51.065 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_1 2023-12-02T00:27:51.120 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-12-02T00:27:51.120 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-02T00:27:51.120 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 5h/5d Inode: 664 Links: 1 2023-12-02T00:27:51.120 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-02T00:27:51.120 INFO:teuthology.orchestra.run.smithi162.stdout:Context: system_u:object_r:device_t:s0 2023-12-02T00:27:51.120 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2023-12-02 00:27:35.976078764 +0000 2023-12-02T00:27:51.120 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2023-12-02 00:25:18.833710976 +0000 2023-12-02T00:27:51.120 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2023-12-02 00:25:18.833710976 +0000 2023-12-02T00:27:51.120 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: 2023-12-02 00:25:18.833710976 +0000 2023-12-02T00:27:51.120 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-12-02T00:27:51.186 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2023-12-02T00:27:51.186 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2023-12-02T00:27:51.187 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.000197366 s, 2.6 MB/s 2023-12-02T00:27:51.188 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-12-02T00:27:51.244 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_2 2023-12-02T00:27:51.300 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-12-02T00:27:51.301 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-02T00:27:51.301 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 5h/5d Inode: 684 Links: 1 2023-12-02T00:27:51.301 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-02T00:27:51.301 INFO:teuthology.orchestra.run.smithi162.stdout:Context: system_u:object_r:device_t:s0 2023-12-02T00:27:51.301 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2023-12-02 00:27:35.977078744 +0000 2023-12-02T00:27:51.301 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2023-12-02 00:25:19.390699993 +0000 2023-12-02T00:27:51.301 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2023-12-02 00:25:19.390699993 +0000 2023-12-02T00:27:51.301 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: 2023-12-02 00:25:19.390699993 +0000 2023-12-02T00:27:51.302 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-12-02T00:27:51.368 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2023-12-02T00:27:51.368 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2023-12-02T00:27:51.369 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.000193466 s, 2.6 MB/s 2023-12-02T00:27:51.370 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-12-02T00:27:51.427 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_3 2023-12-02T00:27:51.484 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-12-02T00:27:51.484 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-02T00:27:51.484 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 5h/5d Inode: 702 Links: 1 2023-12-02T00:27:51.484 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-02T00:27:51.484 INFO:teuthology.orchestra.run.smithi162.stdout:Context: system_u:object_r:device_t:s0 2023-12-02T00:27:51.484 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2023-12-02 00:27:35.977078744 +0000 2023-12-02T00:27:51.484 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2023-12-02 00:25:19.911689720 +0000 2023-12-02T00:27:51.484 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2023-12-02 00:25:19.911689720 +0000 2023-12-02T00:27:51.484 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: 2023-12-02 00:25:19.911689720 +0000 2023-12-02T00:27:51.485 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-12-02T00:27:51.550 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2023-12-02T00:27:51.550 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2023-12-02T00:27:51.550 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.000280529 s, 1.8 MB/s 2023-12-02T00:27:51.551 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-12-02T00:27:51.607 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_4 2023-12-02T00:27:51.662 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-12-02T00:27:51.662 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-02T00:27:51.663 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 5h/5d Inode: 721 Links: 1 2023-12-02T00:27:51.663 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-02T00:27:51.663 INFO:teuthology.orchestra.run.smithi162.stdout:Context: system_u:object_r:device_t:s0 2023-12-02T00:27:51.663 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2023-12-02 00:27:35.978078725 +0000 2023-12-02T00:27:51.663 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2023-12-02 00:25:20.449679112 +0000 2023-12-02T00:27:51.663 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2023-12-02 00:25:20.449679112 +0000 2023-12-02T00:27:51.663 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: 2023-12-02 00:25:20.449679112 +0000 2023-12-02T00:27:51.663 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-12-02T00:27:51.729 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2023-12-02T00:27:51.729 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2023-12-02T00:27:51.729 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.000208345 s, 2.5 MB/s 2023-12-02T00:27:51.730 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-12-02T00:27:51.786 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-02T00:27:51.786 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi046.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi162.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-12-02T00:27:51.787 INFO:tasks.ceph:Generating config... 2023-12-02T00:27:51.788 INFO:tasks.ceph:[client] client mount timeout = 600 2023-12-02T00:27:51.788 INFO:tasks.ceph:[client] debug client = 20 2023-12-02T00:27:51.788 INFO:tasks.ceph:[client] debug ms = 1 2023-12-02T00:27:51.788 INFO:tasks.ceph:[client] pid file = 2023-12-02T00:27:51.788 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-12-02T00:27:51.788 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-12-02T00:27:51.788 INFO:tasks.ceph:[mds] debug mds = 20 2023-12-02T00:27:51.788 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-12-02T00:27:51.788 INFO:tasks.ceph:[mds] debug ms = 1 2023-12-02T00:27:51.788 INFO:tasks.ceph:[mds] mds debug frag = True 2023-12-02T00:27:51.788 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-12-02T00:27:51.788 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-12-02T00:27:51.789 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-12-02T00:27:51.789 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-12-02T00:27:51.789 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-12-02T00:27:51.789 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-12-02T00:27:51.789 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-12-02T00:27:51.789 INFO:tasks.ceph:[mgr] debug ms = 1 2023-12-02T00:27:51.789 INFO:tasks.ceph:[mon] debug mon = 20 2023-12-02T00:27:51.789 INFO:tasks.ceph:[mon] debug ms = 1 2023-12-02T00:27:51.789 INFO:tasks.ceph:[mon] debug paxos = 20 2023-12-02T00:27:51.789 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-12-02T00:27:51.789 INFO:tasks.ceph:[osd] bdev async discard = True 2023-12-02T00:27:51.790 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-12-02T00:27:51.790 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-12-02T00:27:51.790 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-12-02T00:27:51.790 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-12-02T00:27:51.790 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-12-02T00:27:51.790 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-12-02T00:27:51.790 INFO:tasks.ceph:[osd] debug ms = 1 2023-12-02T00:27:51.790 INFO:tasks.ceph:[osd] debug osd = 20 2023-12-02T00:27:51.790 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-12-02T00:27:51.790 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-12-02T00:27:51.790 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-12-02T00:27:51.790 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-12-02T00:27:51.791 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-12-02T00:27:51.791 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-12-02T00:27:51.791 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-12-02T00:27:51.791 INFO:tasks.ceph:Setting up mon.a... 2023-12-02T00:27:51.791 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-12-02T00:27:51.849 INFO:teuthology.orchestra.run.smithi046.stdout:creating /etc/ceph/ceph.keyring 2023-12-02T00:27:51.852 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-12-02T00:27:51.902 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-12-02T00:27:51.933 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.46'), ('mon.b', '172.21.15.162'), ('mon.c', '[v2:172.21.15.162:3301,v1:172.21.15.162:6790]')] 2023-12-02T00:27:51.934 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.46,172.21.15.162,[v2:172.21.15.162:3301,v1:172.21.15.162:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-12-02T00:27:51.935 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-02T00:27:51.935 DEBUG:teuthology.orchestra.run.smithi046:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-12-02T00:27:51.990 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.46 --add b 172.21.15.162 --addv c '[v2:172.21.15.162:3301,v1:172.21.15.162:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-12-02T00:27:52.076 INFO:teuthology.orchestra.run.smithi046.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-12-02T00:27:52.076 INFO:teuthology.orchestra.run.smithi046.stdout:monmaptool: generated fsid 6d588070-5443-40f5-81f6-108a1e3f56c3 2023-12-02T00:27:52.076 INFO:teuthology.orchestra.run.smithi046.stdout:setting min_mon_release = pacific 2023-12-02T00:27:52.076 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 0 2023-12-02T00:27:52.076 INFO:teuthology.orchestra.run.smithi046.stdout:fsid 6d588070-5443-40f5-81f6-108a1e3f56c3 2023-12-02T00:27:52.076 INFO:teuthology.orchestra.run.smithi046.stdout:last_changed 2023-12-02T00:27:52.075676+0000 2023-12-02T00:27:52.076 INFO:teuthology.orchestra.run.smithi046.stdout:created 2023-12-02T00:27:52.075676+0000 2023-12-02T00:27:52.076 INFO:teuthology.orchestra.run.smithi046.stdout:min_mon_release 16 (pacific) 2023-12-02T00:27:52.077 INFO:teuthology.orchestra.run.smithi046.stdout:election_strategy: 1 2023-12-02T00:27:52.077 INFO:teuthology.orchestra.run.smithi046.stdout:0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2023-12-02T00:27:52.077 INFO:teuthology.orchestra.run.smithi046.stdout:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.b 2023-12-02T00:27:52.077 INFO:teuthology.orchestra.run.smithi046.stdout:2: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.c 2023-12-02T00:27:52.077 INFO:teuthology.orchestra.run.smithi046.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-12-02T00:27:52.079 DEBUG:teuthology.orchestra.run.smithi046:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-12-02T00:27:52.095 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 6d588070-5443-40f5-81f6-108a1e3f56c3... 2023-12-02T00:27:52.096 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-12-02T00:27:52.137 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-12-02T00:27:52.188 INFO:teuthology.orchestra.run.smithi162.stdout:[global] 2023-12-02T00:27:52.189 INFO:teuthology.orchestra.run.smithi162.stdout: chdir = "" 2023-12-02T00:27:52.189 INFO:teuthology.orchestra.run.smithi162.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-12-02T00:27:52.189 INFO:teuthology.orchestra.run.smithi162.stdout: auth supported = cephx 2023-12-02T00:27:52.189 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-02T00:27:52.189 INFO:teuthology.orchestra.run.smithi162.stdout: filestore xattr use omap = true 2023-12-02T00:27:52.189 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-02T00:27:52.189 INFO:teuthology.orchestra.run.smithi162.stdout: mon clock drift allowed = 1.000 2023-12-02T00:27:52.189 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-02T00:27:52.189 INFO:teuthology.orchestra.run.smithi162.stdout: osd crush chooseleaf type = 0 2023-12-02T00:27:52.189 INFO:teuthology.orchestra.run.smithi162.stdout: auth debug = true 2023-12-02T00:27:52.189 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-02T00:27:52.189 INFO:teuthology.orchestra.run.smithi162.stdout: ms die on old message = true 2023-12-02T00:27:52.189 INFO:teuthology.orchestra.run.smithi162.stdout: ms die on bug = true 2023-12-02T00:27:52.190 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-02T00:27:52.190 INFO:teuthology.orchestra.run.smithi162.stdout: mon max pg per osd = 10000 # >= luminous 2023-12-02T00:27:52.190 INFO:teuthology.orchestra.run.smithi162.stdout: mon pg warn max object skew = 0 2023-12-02T00:27:52.190 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-02T00:27:52.190 INFO:teuthology.orchestra.run.smithi162.stdout: # disable pg_autoscaler by default for new pools 2023-12-02T00:27:52.190 INFO:teuthology.orchestra.run.smithi162.stdout: osd_pool_default_pg_autoscale_mode = off 2023-12-02T00:27:52.190 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-02T00:27:52.190 INFO:teuthology.orchestra.run.smithi162.stdout: osd pool default size = 2 2023-12-02T00:27:52.190 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-02T00:27:52.190 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd allow primary affinity = true 2023-12-02T00:27:52.190 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd allow pg remap = true 2023-12-02T00:27:52.190 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on legacy crush tunables = false 2023-12-02T00:27:52.190 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on crush straw calc version zero = false 2023-12-02T00:27:52.190 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on no sortbitwise = false 2023-12-02T00:27:52.191 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on osd down out interval zero = false 2023-12-02T00:27:52.191 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on too few osds = false 2023-12-02T00:27:52.191 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-12-02T00:27:52.191 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_pool_no_redundancy = false 2023-12-02T00:27:52.191 INFO:teuthology.orchestra.run.smithi162.stdout: mon_allow_pool_size_one = true 2023-12-02T00:27:52.191 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-02T00:27:52.191 INFO:teuthology.orchestra.run.smithi162.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-12-02T00:27:52.191 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-02T00:27:52.191 INFO:teuthology.orchestra.run.smithi162.stdout: osd default data pool replay window = 5 2023-12-02T00:27:52.191 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-02T00:27:52.191 INFO:teuthology.orchestra.run.smithi162.stdout: mon allow pool delete = true 2023-12-02T00:27:52.191 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-02T00:27:52.191 INFO:teuthology.orchestra.run.smithi162.stdout: mon cluster log file level = debug 2023-12-02T00:27:52.191 INFO:teuthology.orchestra.run.smithi162.stdout: debug asserts on shutdown = true 2023-12-02T00:27:52.192 INFO:teuthology.orchestra.run.smithi162.stdout: mon health detail to clog = false 2023-12-02T00:27:52.192 INFO:teuthology.orchestra.run.smithi162.stdout: mon host = "172.21.15.46,172.21.15.162,[v2:172.21.15.162:3301,v1:172.21.15.162:6790]" 2023-12-02T00:27:52.192 INFO:teuthology.orchestra.run.smithi162.stdout: fsid = 6d588070-5443-40f5-81f6-108a1e3f56c3 2023-12-02T00:27:52.192 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-02T00:27:52.192 INFO:teuthology.orchestra.run.smithi162.stdout:[osd] 2023-12-02T00:27:52.192 INFO:teuthology.orchestra.run.smithi162.stdout: osd journal size = 100 2023-12-02T00:27:52.192 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-02T00:27:52.192 INFO:teuthology.orchestra.run.smithi162.stdout: osd scrub load threshold = 5.0 2023-12-02T00:27:52.192 INFO:teuthology.orchestra.run.smithi162.stdout: osd scrub max interval = 600 2023-12-02T00:27:52.192 INFO:teuthology.orchestra.run.smithi162.stdout: osd mclock profile = high_recovery_ops 2023-12-02T00:27:52.192 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-02T00:27:52.192 INFO:teuthology.orchestra.run.smithi162.stdout: osd recover clone overlap = true 2023-12-02T00:27:52.192 INFO:teuthology.orchestra.run.smithi162.stdout: osd recovery max chunk = 1048576 2023-12-02T00:27:52.192 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-02T00:27:52.192 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug shutdown = true 2023-12-02T00:27:52.193 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug op order = true 2023-12-02T00:27:52.193 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug verify stray on activate = true 2023-12-02T00:27:52.193 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug trim objects = true 2023-12-02T00:27:52.193 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-02T00:27:52.193 INFO:teuthology.orchestra.run.smithi162.stdout: osd open classes on start = true 2023-12-02T00:27:52.193 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug pg log writeout = true 2023-12-02T00:27:52.193 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-02T00:27:52.193 INFO:teuthology.orchestra.run.smithi162.stdout: osd deep scrub update digest min age = 30 2023-12-02T00:27:52.193 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-02T00:27:52.193 INFO:teuthology.orchestra.run.smithi162.stdout: osd map max advance = 10 2023-12-02T00:27:52.193 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-02T00:27:52.193 INFO:teuthology.orchestra.run.smithi162.stdout: journal zero on create = true 2023-12-02T00:27:52.193 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-02T00:27:52.194 INFO:teuthology.orchestra.run.smithi162.stdout: filestore ondisk finisher threads = 3 2023-12-02T00:27:52.194 INFO:teuthology.orchestra.run.smithi162.stdout: filestore apply finisher threads = 3 2023-12-02T00:27:52.194 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-02T00:27:52.194 INFO:teuthology.orchestra.run.smithi162.stdout: bdev debug aio = true 2023-12-02T00:27:52.195 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug misdirected ops = true 2023-12-02T00:27:52.195 INFO:teuthology.orchestra.run.smithi162.stdout: bdev async discard = True 2023-12-02T00:27:52.195 INFO:teuthology.orchestra.run.smithi162.stdout: bdev enable discard = True 2023-12-02T00:27:52.195 INFO:teuthology.orchestra.run.smithi162.stdout: bluestore allocator = bitmap 2023-12-02T00:27:52.195 INFO:teuthology.orchestra.run.smithi162.stdout: bluestore block size = 96636764160 2023-12-02T00:27:52.195 INFO:teuthology.orchestra.run.smithi162.stdout: bluestore fsck on mount = True 2023-12-02T00:27:52.195 INFO:teuthology.orchestra.run.smithi162.stdout: debug bluefs = 1/20 2023-12-02T00:27:52.195 INFO:teuthology.orchestra.run.smithi162.stdout: debug bluestore = 1/20 2023-12-02T00:27:52.195 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2023-12-02T00:27:52.195 INFO:teuthology.orchestra.run.smithi162.stdout: debug osd = 20 2023-12-02T00:27:52.195 INFO:teuthology.orchestra.run.smithi162.stdout: debug rocksdb = 4/10 2023-12-02T00:27:52.195 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd backfillfull_ratio = 0.85 2023-12-02T00:27:52.195 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd full ratio = 0.9 2023-12-02T00:27:52.196 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd nearfull ratio = 0.8 2023-12-02T00:27:52.196 INFO:teuthology.orchestra.run.smithi162.stdout: osd failsafe full ratio = 0.95 2023-12-02T00:27:52.196 INFO:teuthology.orchestra.run.smithi162.stdout: osd objectstore = bluestore 2023-12-02T00:27:52.196 INFO:teuthology.orchestra.run.smithi162.stdout: osd op complaint time = 180 2023-12-02T00:27:52.196 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-02T00:27:52.196 INFO:teuthology.orchestra.run.smithi162.stdout:[mgr] 2023-12-02T00:27:52.196 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2023-12-02T00:27:52.196 INFO:teuthology.orchestra.run.smithi162.stdout: debug mgr = 20 2023-12-02T00:27:52.196 INFO:teuthology.orchestra.run.smithi162.stdout: debug mon = 20 2023-12-02T00:27:52.197 INFO:teuthology.orchestra.run.smithi162.stdout: debug auth = 20 2023-12-02T00:27:52.197 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min pgs per osd = 4 2023-12-02T00:27:52.197 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min bytes per osd = 10 2023-12-02T00:27:52.197 INFO:teuthology.orchestra.run.smithi162.stdout: mgr/telemetry/nag = false 2023-12-02T00:27:52.197 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-02T00:27:52.197 INFO:teuthology.orchestra.run.smithi162.stdout:[mon] 2023-12-02T00:27:52.197 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2023-12-02T00:27:52.197 INFO:teuthology.orchestra.run.smithi162.stdout: debug mon = 20 2023-12-02T00:27:52.197 INFO:teuthology.orchestra.run.smithi162.stdout: debug paxos = 20 2023-12-02T00:27:52.197 INFO:teuthology.orchestra.run.smithi162.stdout: debug auth = 20 2023-12-02T00:27:52.197 INFO:teuthology.orchestra.run.smithi162.stdout: mon data avail warn = 5 2023-12-02T00:27:52.197 INFO:teuthology.orchestra.run.smithi162.stdout: mon mgr mkfs grace = 240 2023-12-02T00:27:52.197 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min pgs per osd = 4 2023-12-02T00:27:52.197 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd reporter subtree level = osd 2023-12-02T00:27:52.198 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd prime pg temp = true 2023-12-02T00:27:52.198 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min bytes per osd = 10 2023-12-02T00:27:52.198 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-02T00:27:52.198 INFO:teuthology.orchestra.run.smithi162.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-12-02T00:27:52.198 INFO:teuthology.orchestra.run.smithi162.stdout: auth mon ticket ttl = 660 # 11m 2023-12-02T00:27:52.198 INFO:teuthology.orchestra.run.smithi162.stdout: auth service ticket ttl = 240 # 4m 2023-12-02T00:27:52.198 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-02T00:27:52.198 INFO:teuthology.orchestra.run.smithi162.stdout: # don't complain about insecure global_id in the test suite 2023-12-02T00:27:52.198 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-12-02T00:27:52.198 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-12-02T00:27:52.198 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-02T00:27:52.198 INFO:teuthology.orchestra.run.smithi162.stdout: # 1m isn't quite enough 2023-12-02T00:27:52.198 INFO:teuthology.orchestra.run.smithi162.stdout: mon_down_mkfs_grace = 2m 2023-12-02T00:27:52.198 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-02T00:27:52.198 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_filestore_osds = false 2023-12-02T00:27:52.199 INFO:teuthology.orchestra.run.smithi162.stdout: mon op complaint time = 120 2023-12-02T00:27:52.199 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-02T00:27:52.199 INFO:teuthology.orchestra.run.smithi162.stdout:[client] 2023-12-02T00:27:52.199 INFO:teuthology.orchestra.run.smithi162.stdout: rgw cache enabled = true 2023-12-02T00:27:52.199 INFO:teuthology.orchestra.run.smithi162.stdout: rgw enable ops log = true 2023-12-02T00:27:52.199 INFO:teuthology.orchestra.run.smithi162.stdout: rgw enable usage log = true 2023-12-02T00:27:52.199 INFO:teuthology.orchestra.run.smithi162.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-12-02T00:27:52.199 INFO:teuthology.orchestra.run.smithi162.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-12-02T00:27:52.199 INFO:teuthology.orchestra.run.smithi162.stdout: client mount timeout = 600 2023-12-02T00:27:52.199 INFO:teuthology.orchestra.run.smithi162.stdout: debug client = 20 2023-12-02T00:27:52.199 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2023-12-02T00:27:52.199 INFO:teuthology.orchestra.run.smithi162.stdout: pid file = "" 2023-12-02T00:27:52.199 INFO:teuthology.orchestra.run.smithi162.stdout: rados mon op timeout = 900 2023-12-02T00:27:52.199 INFO:teuthology.orchestra.run.smithi162.stdout: rados osd op timeout = 900 2023-12-02T00:27:52.199 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.a] 2023-12-02T00:27:52.200 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.b] 2023-12-02T00:27:52.200 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.c] 2023-12-02T00:27:52.200 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.a] 2023-12-02T00:27:52.200 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.c] 2023-12-02T00:27:52.200 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.b] 2023-12-02T00:27:52.200 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.d] 2023-12-02T00:27:52.200 INFO:teuthology.orchestra.run.smithi162.stdout:[mds] 2023-12-02T00:27:52.200 INFO:teuthology.orchestra.run.smithi162.stdout: debug mds = 20 2023-12-02T00:27:52.200 INFO:teuthology.orchestra.run.smithi162.stdout: debug mds balancer = 20 2023-12-02T00:27:52.200 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2023-12-02T00:27:52.200 INFO:teuthology.orchestra.run.smithi162.stdout: mds debug frag = True 2023-12-02T00:27:52.200 INFO:teuthology.orchestra.run.smithi162.stdout: mds debug scatterstat = True 2023-12-02T00:27:52.200 INFO:teuthology.orchestra.run.smithi162.stdout: mds op complaint time = 180 2023-12-02T00:27:52.200 INFO:teuthology.orchestra.run.smithi162.stdout: mds verify scatter = True 2023-12-02T00:27:52.200 INFO:teuthology.orchestra.run.smithi162.stdout: osd op complaint time = 180 2023-12-02T00:27:52.201 INFO:teuthology.orchestra.run.smithi162.stdout: rados mon op timeout = 900 2023-12-02T00:27:52.201 INFO:teuthology.orchestra.run.smithi162.stdout: rados osd op timeout = 900 2023-12-02T00:27:52.201 INFO:teuthology.orchestra.run.smithi046.stdout:[global] 2023-12-02T00:27:52.201 INFO:teuthology.orchestra.run.smithi046.stdout: chdir = "" 2023-12-02T00:27:52.201 INFO:teuthology.orchestra.run.smithi046.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-12-02T00:27:52.201 INFO:teuthology.orchestra.run.smithi046.stdout: auth supported = cephx 2023-12-02T00:27:52.201 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:27:52.202 INFO:teuthology.orchestra.run.smithi046.stdout: filestore xattr use omap = true 2023-12-02T00:27:52.202 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:27:52.202 INFO:teuthology.orchestra.run.smithi046.stdout: mon clock drift allowed = 1.000 2023-12-02T00:27:52.202 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:27:52.202 INFO:teuthology.orchestra.run.smithi046.stdout: osd crush chooseleaf type = 0 2023-12-02T00:27:52.202 INFO:teuthology.orchestra.run.smithi046.stdout: auth debug = true 2023-12-02T00:27:52.202 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:27:52.202 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on old message = true 2023-12-02T00:27:52.202 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on bug = true 2023-12-02T00:27:52.202 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:27:52.202 INFO:teuthology.orchestra.run.smithi046.stdout: mon max pg per osd = 10000 # >= luminous 2023-12-02T00:27:52.202 INFO:teuthology.orchestra.run.smithi046.stdout: mon pg warn max object skew = 0 2023-12-02T00:27:52.202 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:27:52.203 INFO:teuthology.orchestra.run.smithi046.stdout: # disable pg_autoscaler by default for new pools 2023-12-02T00:27:52.203 INFO:teuthology.orchestra.run.smithi046.stdout: osd_pool_default_pg_autoscale_mode = off 2023-12-02T00:27:52.203 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:27:52.203 INFO:teuthology.orchestra.run.smithi046.stdout: osd pool default size = 2 2023-12-02T00:27:52.203 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:27:52.203 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd allow primary affinity = true 2023-12-02T00:27:52.203 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd allow pg remap = true 2023-12-02T00:27:52.203 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on legacy crush tunables = false 2023-12-02T00:27:52.203 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on crush straw calc version zero = false 2023-12-02T00:27:52.203 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on no sortbitwise = false 2023-12-02T00:27:52.203 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on osd down out interval zero = false 2023-12-02T00:27:52.203 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on too few osds = false 2023-12-02T00:27:52.203 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-12-02T00:27:52.203 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_pool_no_redundancy = false 2023-12-02T00:27:52.204 INFO:teuthology.orchestra.run.smithi046.stdout: mon_allow_pool_size_one = true 2023-12-02T00:27:52.204 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:27:52.204 INFO:teuthology.orchestra.run.smithi046.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-12-02T00:27:52.204 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:27:52.204 INFO:teuthology.orchestra.run.smithi046.stdout: osd default data pool replay window = 5 2023-12-02T00:27:52.204 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:27:52.204 INFO:teuthology.orchestra.run.smithi046.stdout: mon allow pool delete = true 2023-12-02T00:27:52.204 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:27:52.204 INFO:teuthology.orchestra.run.smithi046.stdout: mon cluster log file level = debug 2023-12-02T00:27:52.204 INFO:teuthology.orchestra.run.smithi046.stdout: debug asserts on shutdown = true 2023-12-02T00:27:52.204 INFO:teuthology.orchestra.run.smithi046.stdout: mon health detail to clog = false 2023-12-02T00:27:52.204 INFO:teuthology.orchestra.run.smithi046.stdout: mon host = "172.21.15.46,172.21.15.162,[v2:172.21.15.162:3301,v1:172.21.15.162:6790]" 2023-12-02T00:27:52.204 INFO:teuthology.orchestra.run.smithi046.stdout: fsid = 6d588070-5443-40f5-81f6-108a1e3f56c3 2023-12-02T00:27:52.204 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:27:52.204 INFO:teuthology.orchestra.run.smithi046.stdout:[osd] 2023-12-02T00:27:52.205 INFO:teuthology.orchestra.run.smithi046.stdout: osd journal size = 100 2023-12-02T00:27:52.205 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:27:52.205 INFO:teuthology.orchestra.run.smithi046.stdout: osd scrub load threshold = 5.0 2023-12-02T00:27:52.205 INFO:teuthology.orchestra.run.smithi046.stdout: osd scrub max interval = 600 2023-12-02T00:27:52.205 INFO:teuthology.orchestra.run.smithi046.stdout: osd mclock profile = high_recovery_ops 2023-12-02T00:27:52.205 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:27:52.205 INFO:teuthology.orchestra.run.smithi046.stdout: osd recover clone overlap = true 2023-12-02T00:27:52.205 INFO:teuthology.orchestra.run.smithi046.stdout: osd recovery max chunk = 1048576 2023-12-02T00:27:52.205 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:27:52.206 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug shutdown = true 2023-12-02T00:27:52.206 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug op order = true 2023-12-02T00:27:52.206 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug verify stray on activate = true 2023-12-02T00:27:52.206 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug trim objects = true 2023-12-02T00:27:52.206 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:27:52.206 INFO:teuthology.orchestra.run.smithi046.stdout: osd open classes on start = true 2023-12-02T00:27:52.206 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug pg log writeout = true 2023-12-02T00:27:52.206 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:27:52.206 INFO:teuthology.orchestra.run.smithi046.stdout: osd deep scrub update digest min age = 30 2023-12-02T00:27:52.206 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:27:52.206 INFO:teuthology.orchestra.run.smithi046.stdout: osd map max advance = 10 2023-12-02T00:27:52.206 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:27:52.206 INFO:teuthology.orchestra.run.smithi046.stdout: journal zero on create = true 2023-12-02T00:27:52.206 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:27:52.207 INFO:teuthology.orchestra.run.smithi046.stdout: filestore ondisk finisher threads = 3 2023-12-02T00:27:52.207 INFO:teuthology.orchestra.run.smithi046.stdout: filestore apply finisher threads = 3 2023-12-02T00:27:52.207 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:27:52.207 INFO:teuthology.orchestra.run.smithi046.stdout: bdev debug aio = true 2023-12-02T00:27:52.207 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug misdirected ops = true 2023-12-02T00:27:52.207 INFO:teuthology.orchestra.run.smithi046.stdout: bdev async discard = True 2023-12-02T00:27:52.207 INFO:teuthology.orchestra.run.smithi046.stdout: bdev enable discard = True 2023-12-02T00:27:52.207 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore allocator = bitmap 2023-12-02T00:27:52.207 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore block size = 96636764160 2023-12-02T00:27:52.207 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore fsck on mount = True 2023-12-02T00:27:52.208 INFO:teuthology.orchestra.run.smithi046.stdout: debug bluefs = 1/20 2023-12-02T00:27:52.208 INFO:teuthology.orchestra.run.smithi046.stdout: debug bluestore = 1/20 2023-12-02T00:27:52.208 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2023-12-02T00:27:52.208 INFO:teuthology.orchestra.run.smithi046.stdout: debug osd = 20 2023-12-02T00:27:52.208 INFO:teuthology.orchestra.run.smithi046.stdout: debug rocksdb = 4/10 2023-12-02T00:27:52.208 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd backfillfull_ratio = 0.85 2023-12-02T00:27:52.208 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd full ratio = 0.9 2023-12-02T00:27:52.208 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd nearfull ratio = 0.8 2023-12-02T00:27:52.208 INFO:teuthology.orchestra.run.smithi046.stdout: osd failsafe full ratio = 0.95 2023-12-02T00:27:52.208 INFO:teuthology.orchestra.run.smithi046.stdout: osd objectstore = bluestore 2023-12-02T00:27:52.208 INFO:teuthology.orchestra.run.smithi046.stdout: osd op complaint time = 180 2023-12-02T00:27:52.208 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:27:52.208 INFO:teuthology.orchestra.run.smithi046.stdout:[mgr] 2023-12-02T00:27:52.208 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2023-12-02T00:27:52.208 INFO:teuthology.orchestra.run.smithi046.stdout: debug mgr = 20 2023-12-02T00:27:52.209 INFO:teuthology.orchestra.run.smithi046.stdout: debug mon = 20 2023-12-02T00:27:52.209 INFO:teuthology.orchestra.run.smithi046.stdout: debug auth = 20 2023-12-02T00:27:52.209 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min pgs per osd = 4 2023-12-02T00:27:52.209 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min bytes per osd = 10 2023-12-02T00:27:52.209 INFO:teuthology.orchestra.run.smithi046.stdout: mgr/telemetry/nag = false 2023-12-02T00:27:52.209 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:27:52.209 INFO:teuthology.orchestra.run.smithi046.stdout:[mon] 2023-12-02T00:27:52.209 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2023-12-02T00:27:52.209 INFO:teuthology.orchestra.run.smithi046.stdout: debug mon = 20 2023-12-02T00:27:52.209 INFO:teuthology.orchestra.run.smithi046.stdout: debug paxos = 20 2023-12-02T00:27:52.209 INFO:teuthology.orchestra.run.smithi046.stdout: debug auth = 20 2023-12-02T00:27:52.209 INFO:teuthology.orchestra.run.smithi046.stdout: mon data avail warn = 5 2023-12-02T00:27:52.209 INFO:teuthology.orchestra.run.smithi046.stdout: mon mgr mkfs grace = 240 2023-12-02T00:27:52.209 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min pgs per osd = 4 2023-12-02T00:27:52.209 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd reporter subtree level = osd 2023-12-02T00:27:52.209 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd prime pg temp = true 2023-12-02T00:27:52.210 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min bytes per osd = 10 2023-12-02T00:27:52.210 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:27:52.210 INFO:teuthology.orchestra.run.smithi046.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-12-02T00:27:52.210 INFO:teuthology.orchestra.run.smithi046.stdout: auth mon ticket ttl = 660 # 11m 2023-12-02T00:27:52.210 INFO:teuthology.orchestra.run.smithi046.stdout: auth service ticket ttl = 240 # 4m 2023-12-02T00:27:52.210 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:27:52.210 INFO:teuthology.orchestra.run.smithi046.stdout: # don't complain about insecure global_id in the test suite 2023-12-02T00:27:52.210 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-12-02T00:27:52.210 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-12-02T00:27:52.210 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:27:52.210 INFO:teuthology.orchestra.run.smithi046.stdout: # 1m isn't quite enough 2023-12-02T00:27:52.210 INFO:teuthology.orchestra.run.smithi046.stdout: mon_down_mkfs_grace = 2m 2023-12-02T00:27:52.210 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:27:52.210 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_filestore_osds = false 2023-12-02T00:27:52.210 INFO:teuthology.orchestra.run.smithi046.stdout: mon op complaint time = 120 2023-12-02T00:27:52.211 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:27:52.211 INFO:teuthology.orchestra.run.smithi046.stdout:[client] 2023-12-02T00:27:52.211 INFO:teuthology.orchestra.run.smithi046.stdout: rgw cache enabled = true 2023-12-02T00:27:52.211 INFO:teuthology.orchestra.run.smithi046.stdout: rgw enable ops log = true 2023-12-02T00:27:52.211 INFO:teuthology.orchestra.run.smithi046.stdout: rgw enable usage log = true 2023-12-02T00:27:52.211 INFO:teuthology.orchestra.run.smithi046.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-12-02T00:27:52.211 INFO:teuthology.orchestra.run.smithi046.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-12-02T00:27:52.211 INFO:teuthology.orchestra.run.smithi046.stdout: client mount timeout = 600 2023-12-02T00:27:52.211 INFO:teuthology.orchestra.run.smithi046.stdout: debug client = 20 2023-12-02T00:27:52.211 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2023-12-02T00:27:52.211 INFO:teuthology.orchestra.run.smithi046.stdout: pid file = "" 2023-12-02T00:27:52.211 INFO:teuthology.orchestra.run.smithi046.stdout: rados mon op timeout = 900 2023-12-02T00:27:52.211 INFO:teuthology.orchestra.run.smithi046.stdout: rados osd op timeout = 900 2023-12-02T00:27:52.211 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.a] 2023-12-02T00:27:52.211 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.b] 2023-12-02T00:27:52.211 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.c] 2023-12-02T00:27:52.212 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.a] 2023-12-02T00:27:52.212 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.c] 2023-12-02T00:27:52.212 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.b] 2023-12-02T00:27:52.212 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.d] 2023-12-02T00:27:52.212 INFO:teuthology.orchestra.run.smithi046.stdout:[mds] 2023-12-02T00:27:52.212 INFO:teuthology.orchestra.run.smithi046.stdout: debug mds = 20 2023-12-02T00:27:52.212 INFO:teuthology.orchestra.run.smithi046.stdout: debug mds balancer = 20 2023-12-02T00:27:52.212 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2023-12-02T00:27:52.212 INFO:teuthology.orchestra.run.smithi046.stdout: mds debug frag = True 2023-12-02T00:27:52.212 INFO:teuthology.orchestra.run.smithi046.stdout: mds debug scatterstat = True 2023-12-02T00:27:52.212 INFO:teuthology.orchestra.run.smithi046.stdout: mds op complaint time = 180 2023-12-02T00:27:52.212 INFO:teuthology.orchestra.run.smithi046.stdout: mds verify scatter = True 2023-12-02T00:27:52.212 INFO:teuthology.orchestra.run.smithi046.stdout: osd op complaint time = 180 2023-12-02T00:27:52.212 INFO:teuthology.orchestra.run.smithi046.stdout: rados mon op timeout = 900 2023-12-02T00:27:52.212 INFO:teuthology.orchestra.run.smithi046.stdout: rados osd op timeout = 900 2023-12-02T00:27:52.214 INFO:tasks.ceph:Creating admin key on mon.a... 2023-12-02T00:27:52.214 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-12-02T00:27:52.313 INFO:tasks.ceph:Copying monmap to all nodes... 2023-12-02T00:27:52.313 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-02T00:27:52.313 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-12-02T00:27:52.328 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-02T00:27:52.328 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-12-02T00:27:52.384 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi046.front.sepia.ceph.com 2023-12-02T00:27:52.384 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-02T00:27:52.384 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/ceph/ceph.keyring 2023-12-02T00:27:52.384 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-12-02T00:27:52.471 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-02T00:27:52.471 DEBUG:teuthology.orchestra.run.smithi046:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-12-02T00:27:52.526 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi162.front.sepia.ceph.com 2023-12-02T00:27:52.526 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-12-02T00:27:52.526 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/ceph/ceph.keyring 2023-12-02T00:27:52.526 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-12-02T00:27:52.567 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-12-02T00:27:52.567 DEBUG:teuthology.orchestra.run.smithi162:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-12-02T00:27:52.622 INFO:tasks.ceph:Setting up mon nodes... 2023-12-02T00:27:52.623 INFO:tasks.ceph:Setting up mgr nodes... 2023-12-02T00:27:52.623 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-12-02T00:27:52.685 INFO:teuthology.orchestra.run.smithi046.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-12-02T00:27:52.688 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-12-02T00:27:52.754 INFO:teuthology.orchestra.run.smithi162.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-12-02T00:27:52.756 INFO:tasks.ceph:Setting up mds nodes... 2023-12-02T00:27:52.757 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-12-02T00:27:52.819 INFO:teuthology.orchestra.run.smithi046.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-12-02T00:27:52.822 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-12-02T00:27:52.852 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-12-02T00:27:52.954 INFO:teuthology.orchestra.run.smithi046.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-12-02T00:27:52.956 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-12-02T00:27:52.988 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-12-02T00:27:53.043 INFO:teuthology.orchestra.run.smithi162.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-12-02T00:27:53.045 DEBUG:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-12-02T00:27:53.072 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-12-02T00:27:53.166 INFO:teuthology.orchestra.run.smithi162.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-12-02T00:27:53.168 DEBUG:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-12-02T00:27:53.195 INFO:tasks.ceph_client:Setting up client nodes... 2023-12-02T00:27:53.195 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-12-02T00:27:53.242 INFO:teuthology.orchestra.run.smithi046.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-12-02T00:27:53.261 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-12-02T00:27:53.307 INFO:teuthology.orchestra.run.smithi046.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-12-02T00:27:53.325 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-12-02T00:27:53.368 INFO:teuthology.orchestra.run.smithi162.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-12-02T00:27:53.383 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-12-02T00:27:53.465 INFO:teuthology.orchestra.run.smithi162.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-12-02T00:27:53.480 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-12-02T00:27:53.481 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi046.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi162.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-12-02T00:27:53.481 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-12-02T00:27:53.512 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-02T00:27:53.512 INFO:tasks.ceph:role: osd.0 2023-12-02T00:27:53.512 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi046.front.sepia.ceph.com 2023-12-02T00:27:53.512 DEBUG:teuthology.orchestra.run.smithi046:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-12-02T00:27:53.582 INFO:teuthology.orchestra.run.smithi046.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-12-02T00:27:53.582 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-02T00:27:53.582 INFO:teuthology.orchestra.run.smithi046.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-02T00:27:53.582 INFO:teuthology.orchestra.run.smithi046.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-02T00:27:53.582 INFO:teuthology.orchestra.run.smithi046.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-02T00:27:53.582 INFO:teuthology.orchestra.run.smithi046.stdout: = sunit=0 swidth=0 blks 2023-12-02T00:27:53.583 INFO:teuthology.orchestra.run.smithi046.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-02T00:27:53.583 INFO:teuthology.orchestra.run.smithi046.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-02T00:27:53.583 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-02T00:27:53.583 INFO:teuthology.orchestra.run.smithi046.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-02T00:27:53.615 INFO:teuthology.orchestra.run.smithi046.stdout:Discarding blocks...Done. 2023-12-02T00:27:53.617 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi046.front.sepia.ceph.com -o noatime 2023-12-02T00:27:53.617 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-12-02T00:27:53.692 DEBUG:teuthology.orchestra.run.smithi046:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-12-02T00:27:53.767 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-12-02T00:27:53.838 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-02T00:27:53.838 INFO:tasks.ceph:role: osd.1 2023-12-02T00:27:53.839 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi046.front.sepia.ceph.com 2023-12-02T00:27:53.839 DEBUG:teuthology.orchestra.run.smithi046:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-12-02T00:27:53.909 INFO:teuthology.orchestra.run.smithi046.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-12-02T00:27:53.909 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-02T00:27:53.909 INFO:teuthology.orchestra.run.smithi046.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-02T00:27:53.909 INFO:teuthology.orchestra.run.smithi046.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-02T00:27:53.909 INFO:teuthology.orchestra.run.smithi046.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-02T00:27:53.909 INFO:teuthology.orchestra.run.smithi046.stdout: = sunit=0 swidth=0 blks 2023-12-02T00:27:53.909 INFO:teuthology.orchestra.run.smithi046.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-02T00:27:53.909 INFO:teuthology.orchestra.run.smithi046.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-02T00:27:53.909 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-02T00:27:53.909 INFO:teuthology.orchestra.run.smithi046.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-02T00:27:53.941 INFO:teuthology.orchestra.run.smithi046.stdout:Discarding blocks...Done. 2023-12-02T00:27:53.946 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi046.front.sepia.ceph.com -o noatime 2023-12-02T00:27:53.946 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-12-02T00:27:54.020 DEBUG:teuthology.orchestra.run.smithi046:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-12-02T00:27:54.094 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-12-02T00:27:54.165 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-02T00:27:54.165 INFO:tasks.ceph:role: osd.2 2023-12-02T00:27:54.166 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi046.front.sepia.ceph.com 2023-12-02T00:27:54.166 DEBUG:teuthology.orchestra.run.smithi046:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-12-02T00:27:54.237 INFO:teuthology.orchestra.run.smithi046.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-12-02T00:27:54.237 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-02T00:27:54.237 INFO:teuthology.orchestra.run.smithi046.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-02T00:27:54.237 INFO:teuthology.orchestra.run.smithi046.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-02T00:27:54.238 INFO:teuthology.orchestra.run.smithi046.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-02T00:27:54.238 INFO:teuthology.orchestra.run.smithi046.stdout: = sunit=0 swidth=0 blks 2023-12-02T00:27:54.238 INFO:teuthology.orchestra.run.smithi046.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-02T00:27:54.238 INFO:teuthology.orchestra.run.smithi046.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-02T00:27:54.238 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-02T00:27:54.238 INFO:teuthology.orchestra.run.smithi046.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-02T00:27:54.269 INFO:teuthology.orchestra.run.smithi046.stdout:Discarding blocks...Done. 2023-12-02T00:27:54.271 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi046.front.sepia.ceph.com -o noatime 2023-12-02T00:27:54.271 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-12-02T00:27:54.345 DEBUG:teuthology.orchestra.run.smithi046:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-12-02T00:27:54.419 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-12-02T00:27:54.490 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-02T00:27:54.490 INFO:tasks.ceph:role: osd.3 2023-12-02T00:27:54.490 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi046.front.sepia.ceph.com 2023-12-02T00:27:54.491 DEBUG:teuthology.orchestra.run.smithi046:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-12-02T00:27:54.561 INFO:teuthology.orchestra.run.smithi046.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-12-02T00:27:54.561 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-02T00:27:54.561 INFO:teuthology.orchestra.run.smithi046.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-02T00:27:54.561 INFO:teuthology.orchestra.run.smithi046.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-02T00:27:54.561 INFO:teuthology.orchestra.run.smithi046.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-02T00:27:54.561 INFO:teuthology.orchestra.run.smithi046.stdout: = sunit=0 swidth=0 blks 2023-12-02T00:27:54.561 INFO:teuthology.orchestra.run.smithi046.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-02T00:27:54.561 INFO:teuthology.orchestra.run.smithi046.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-02T00:27:54.561 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-02T00:27:54.561 INFO:teuthology.orchestra.run.smithi046.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-02T00:27:54.594 INFO:teuthology.orchestra.run.smithi046.stdout:Discarding blocks...Done. 2023-12-02T00:27:54.595 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi046.front.sepia.ceph.com -o noatime 2023-12-02T00:27:54.595 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-12-02T00:27:54.669 DEBUG:teuthology.orchestra.run.smithi046:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-12-02T00:27:54.744 DEBUG:teuthology.orchestra.run.smithi046:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-02T00:27:54.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:27:54.850+0000 7f1bdcaa5500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-12-02T00:27:54.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:27:54.850+0000 7f1bdcaa5500 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-12-02T00:27:54.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:27:54.850+0000 7f1bdcaa5500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-12-02T00:27:54.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:27:54.850+0000 7f1bdcaa5500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-12-02T00:27:54.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:27:54.850+0000 7f1bdcaa5500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-12-02T00:27:54.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:27:54.850+0000 7f1bdcaa5500 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-12-02T00:27:57.706 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-12-02T00:27:57.742 DEBUG:teuthology.orchestra.run.smithi046:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-02T00:27:57.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:27:57.835+0000 7fc3da8c9500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-12-02T00:27:57.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:27:57.835+0000 7fc3da8c9500 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-12-02T00:27:57.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:27:57.835+0000 7fc3da8c9500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-12-02T00:27:57.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:27:57.835+0000 7fc3da8c9500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-12-02T00:27:57.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:27:57.835+0000 7fc3da8c9500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-12-02T00:27:57.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:27:57.835+0000 7fc3da8c9500 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-12-02T00:28:00.688 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-12-02T00:28:00.725 DEBUG:teuthology.orchestra.run.smithi046:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-02T00:28:00.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:00.819+0000 7fbf4621b500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-12-02T00:28:00.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:00.819+0000 7fbf4621b500 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-12-02T00:28:00.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:00.819+0000 7fbf4621b500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-12-02T00:28:00.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:00.819+0000 7fbf4621b500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-12-02T00:28:00.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:00.819+0000 7fbf4621b500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-12-02T00:28:00.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:00.819+0000 7fbf4621b500 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-12-02T00:28:03.675 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-12-02T00:28:03.711 DEBUG:teuthology.orchestra.run.smithi046:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-02T00:28:03.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:03.804+0000 7fb167e14500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-12-02T00:28:03.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:03.804+0000 7fb167e14500 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-12-02T00:28:03.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:03.804+0000 7fb167e14500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-12-02T00:28:03.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:03.804+0000 7fb167e14500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-12-02T00:28:03.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:03.804+0000 7fb167e14500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-12-02T00:28:03.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:03.805+0000 7fb167e14500 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-12-02T00:28:06.662 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-12-02T00:28:06.696 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-12-02T00:28:06.728 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-02T00:28:06.728 INFO:tasks.ceph:role: osd.4 2023-12-02T00:28:06.728 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi162.front.sepia.ceph.com 2023-12-02T00:28:06.729 DEBUG:teuthology.orchestra.run.smithi162:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-12-02T00:28:06.806 INFO:teuthology.orchestra.run.smithi162.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-12-02T00:28:06.806 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-02T00:28:06.806 INFO:teuthology.orchestra.run.smithi162.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-02T00:28:06.807 INFO:teuthology.orchestra.run.smithi162.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-02T00:28:06.807 INFO:teuthology.orchestra.run.smithi162.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-02T00:28:06.807 INFO:teuthology.orchestra.run.smithi162.stdout: = sunit=0 swidth=0 blks 2023-12-02T00:28:06.807 INFO:teuthology.orchestra.run.smithi162.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-02T00:28:06.807 INFO:teuthology.orchestra.run.smithi162.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-02T00:28:06.807 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-02T00:28:06.807 INFO:teuthology.orchestra.run.smithi162.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-02T00:28:07.320 INFO:teuthology.orchestra.run.smithi162.stdout:Discarding blocks...Done. 2023-12-02T00:28:07.323 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi162.front.sepia.ceph.com -o noatime 2023-12-02T00:28:07.323 DEBUG:teuthology.orchestra.run.smithi162:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-12-02T00:28:07.481 DEBUG:teuthology.orchestra.run.smithi162:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-12-02T00:28:07.520 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-12-02T00:28:07.590 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-02T00:28:07.590 INFO:tasks.ceph:role: osd.5 2023-12-02T00:28:07.590 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi162.front.sepia.ceph.com 2023-12-02T00:28:07.590 DEBUG:teuthology.orchestra.run.smithi162:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-12-02T00:28:07.665 INFO:teuthology.orchestra.run.smithi162.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-12-02T00:28:07.665 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-02T00:28:07.665 INFO:teuthology.orchestra.run.smithi162.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-02T00:28:07.665 INFO:teuthology.orchestra.run.smithi162.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-02T00:28:07.665 INFO:teuthology.orchestra.run.smithi162.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-02T00:28:07.665 INFO:teuthology.orchestra.run.smithi162.stdout: = sunit=0 swidth=0 blks 2023-12-02T00:28:07.665 INFO:teuthology.orchestra.run.smithi162.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-02T00:28:07.666 INFO:teuthology.orchestra.run.smithi162.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-02T00:28:07.666 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-02T00:28:07.666 INFO:teuthology.orchestra.run.smithi162.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-02T00:28:08.108 INFO:teuthology.orchestra.run.smithi162.stdout:Discarding blocks...Done. 2023-12-02T00:28:08.111 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi162.front.sepia.ceph.com -o noatime 2023-12-02T00:28:08.111 DEBUG:teuthology.orchestra.run.smithi162:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-12-02T00:28:08.209 DEBUG:teuthology.orchestra.run.smithi162:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-12-02T00:28:08.243 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-12-02T00:28:08.313 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-02T00:28:08.313 INFO:tasks.ceph:role: osd.6 2023-12-02T00:28:08.313 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi162.front.sepia.ceph.com 2023-12-02T00:28:08.314 DEBUG:teuthology.orchestra.run.smithi162:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-12-02T00:28:08.390 INFO:teuthology.orchestra.run.smithi162.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-12-02T00:28:08.390 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-02T00:28:08.390 INFO:teuthology.orchestra.run.smithi162.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-02T00:28:08.390 INFO:teuthology.orchestra.run.smithi162.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-02T00:28:08.390 INFO:teuthology.orchestra.run.smithi162.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-02T00:28:08.391 INFO:teuthology.orchestra.run.smithi162.stdout: = sunit=0 swidth=0 blks 2023-12-02T00:28:08.391 INFO:teuthology.orchestra.run.smithi162.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-02T00:28:08.391 INFO:teuthology.orchestra.run.smithi162.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-02T00:28:08.391 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-02T00:28:08.391 INFO:teuthology.orchestra.run.smithi162.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-02T00:28:08.891 INFO:teuthology.orchestra.run.smithi162.stdout:Discarding blocks...Done. 2023-12-02T00:28:08.893 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi162.front.sepia.ceph.com -o noatime 2023-12-02T00:28:08.893 DEBUG:teuthology.orchestra.run.smithi162:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-12-02T00:28:09.035 DEBUG:teuthology.orchestra.run.smithi162:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-12-02T00:28:09.074 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-12-02T00:28:09.144 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-02T00:28:09.144 INFO:tasks.ceph:role: osd.7 2023-12-02T00:28:09.144 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi162.front.sepia.ceph.com 2023-12-02T00:28:09.144 DEBUG:teuthology.orchestra.run.smithi162:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-12-02T00:28:09.221 INFO:teuthology.orchestra.run.smithi162.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-12-02T00:28:09.221 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-02T00:28:09.221 INFO:teuthology.orchestra.run.smithi162.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-02T00:28:09.221 INFO:teuthology.orchestra.run.smithi162.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-02T00:28:09.222 INFO:teuthology.orchestra.run.smithi162.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-02T00:28:09.222 INFO:teuthology.orchestra.run.smithi162.stdout: = sunit=0 swidth=0 blks 2023-12-02T00:28:09.222 INFO:teuthology.orchestra.run.smithi162.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-02T00:28:09.222 INFO:teuthology.orchestra.run.smithi162.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-02T00:28:09.222 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-02T00:28:09.222 INFO:teuthology.orchestra.run.smithi162.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-02T00:28:09.720 INFO:teuthology.orchestra.run.smithi162.stdout:Discarding blocks...Done. 2023-12-02T00:28:09.722 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi162.front.sepia.ceph.com -o noatime 2023-12-02T00:28:09.722 DEBUG:teuthology.orchestra.run.smithi162:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-12-02T00:28:09.861 DEBUG:teuthology.orchestra.run.smithi162:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-12-02T00:28:09.898 DEBUG:teuthology.orchestra.run.smithi162:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-02T00:28:10.005 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:10.003+0000 7fc352594500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-12-02T00:28:10.005 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:10.004+0000 7fc352594500 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-12-02T00:28:10.005 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:10.004+0000 7fc352594500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-12-02T00:28:10.005 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:10.004+0000 7fc352594500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-12-02T00:28:10.005 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:10.004+0000 7fc352594500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-12-02T00:28:10.005 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:10.004+0000 7fc352594500 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-12-02T00:28:12.863 DEBUG:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-12-02T00:28:12.895 DEBUG:teuthology.orchestra.run.smithi162:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-02T00:28:12.987 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:12.985+0000 7f45ad4a9500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-12-02T00:28:12.987 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:12.986+0000 7f45ad4a9500 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-12-02T00:28:12.987 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:12.986+0000 7f45ad4a9500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-12-02T00:28:12.987 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:12.986+0000 7f45ad4a9500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-12-02T00:28:12.987 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:12.986+0000 7f45ad4a9500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-12-02T00:28:12.987 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:12.986+0000 7f45ad4a9500 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-12-02T00:28:15.836 DEBUG:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-12-02T00:28:15.867 DEBUG:teuthology.orchestra.run.smithi162:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-02T00:28:15.958 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:15.957+0000 7f53cf1bc500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-12-02T00:28:15.958 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:15.957+0000 7f53cf1bc500 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-12-02T00:28:15.958 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:15.957+0000 7f53cf1bc500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-12-02T00:28:15.958 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:15.957+0000 7f53cf1bc500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-12-02T00:28:15.959 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:15.957+0000 7f53cf1bc500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-12-02T00:28:15.959 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:15.958+0000 7f53cf1bc500 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-12-02T00:28:18.811 DEBUG:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-12-02T00:28:18.841 DEBUG:teuthology.orchestra.run.smithi162:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-02T00:28:18.933 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:18.930+0000 7f5a9bcb6500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-12-02T00:28:18.934 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:18.930+0000 7f5a9bcb6500 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-12-02T00:28:18.934 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:18.930+0000 7f5a9bcb6500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-12-02T00:28:18.935 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:18.930+0000 7f5a9bcb6500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-12-02T00:28:18.935 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:18.930+0000 7f5a9bcb6500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-12-02T00:28:18.935 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:18.931+0000 7f5a9bcb6500 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-12-02T00:28:20.174 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-02T00:28:20.177 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-02T00:28:21.797 DEBUG:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-12-02T00:28:21.829 INFO:tasks.ceph:Reading keys from all nodes... 2023-12-02T00:28:21.829 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-02T00:28:21.829 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-12-02T00:28:21.866 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-02T00:28:21.866 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-12-02T00:28:21.941 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-02T00:28:21.941 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-12-02T00:28:22.018 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-02T00:28:22.018 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-12-02T00:28:22.089 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-02T00:28:22.089 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-12-02T00:28:22.161 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-02T00:28:22.162 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-12-02T00:28:22.235 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-02T00:28:22.235 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-12-02T00:28:22.307 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-12-02T00:28:22.307 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-12-02T00:28:22.339 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-12-02T00:28:22.339 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-12-02T00:28:22.409 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-12-02T00:28:22.409 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-12-02T00:28:22.478 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-12-02T00:28:22.478 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-12-02T00:28:22.547 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-12-02T00:28:22.547 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-12-02T00:28:22.617 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-12-02T00:28:22.617 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-12-02T00:28:22.685 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-12-02T00:28:22.685 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-12-02T00:28:22.753 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-02T00:28:22.753 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-12-02T00:28:22.771 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-02T00:28:22.771 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-12-02T00:28:22.828 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-12-02T00:28:22.828 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-12-02T00:28:22.844 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-12-02T00:28:22.844 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-12-02T00:28:22.900 INFO:tasks.ceph:Adding keys to all mons... 2023-12-02T00:28:22.900 DEBUG:teuthology.orchestra.run.smithi046:> sudo tee -a /etc/ceph/ceph.keyring 2023-12-02T00:28:22.903 DEBUG:teuthology.orchestra.run.smithi162:> sudo tee -a /etc/ceph/ceph.keyring 2023-12-02T00:28:22.943 INFO:teuthology.orchestra.run.smithi046.stdout:[mgr.y] 2023-12-02T00:28:22.943 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAIemplE8PaKBAAwleK4eGvEOffn5EzY+drMQ== 2023-12-02T00:28:22.943 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.a] 2023-12-02T00:28:22.943 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAIemplYQrZMBAAqrkez6r//VUITan4SKdf4w== 2023-12-02T00:28:22.943 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.c] 2023-12-02T00:28:22.943 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAIempl69PfOBAAQOgkrli+yM4wpJabPb3QxQ== 2023-12-02T00:28:22.943 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.0] 2023-12-02T00:28:22.943 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAKempluFfMMhAAmbD1GpzugHbxobJBl3RtkQ== 2023-12-02T00:28:22.943 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.1] 2023-12-02T00:28:22.943 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQANempleNzmMRAALK5ExII/uG0A8pXBsc86pg== 2023-12-02T00:28:22.943 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.2] 2023-12-02T00:28:22.944 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAQemplKDzyMBAAfPwOg3WNbFSBUZUqpG1l7w== 2023-12-02T00:28:22.944 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.3] 2023-12-02T00:28:22.944 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQATemplIbgAMBAAbynDehkgSKMaPHXYgwK2cA== 2023-12-02T00:28:22.944 INFO:teuthology.orchestra.run.smithi046.stdout:[mgr.x] 2023-12-02T00:28:22.944 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAIemplbBH1LBAAxs6g6VawCdt6IxZJn2yrQA== 2023-12-02T00:28:22.944 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.b] 2023-12-02T00:28:22.944 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAJemplKoSRAhAAuQHpGMNCUSFEtCtWMvY8yA== 2023-12-02T00:28:22.944 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.d] 2023-12-02T00:28:22.944 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAJemplsTHqCRAAi7WJBcwYYDl0/qZkMy7GsQ== 2023-12-02T00:28:22.944 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.4] 2023-12-02T00:28:22.944 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAaempl/vxEABAAThzqmuy54tDx5dYyTilSzg== 2023-12-02T00:28:22.944 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.5] 2023-12-02T00:28:22.944 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAcemplWd7NOhAAO0eGoj+K+P0F4QvV4SMUxg== 2023-12-02T00:28:22.944 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.6] 2023-12-02T00:28:22.944 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAfemplz38cORAAmb09RTXtiWf2J4q0jbYEug== 2023-12-02T00:28:22.945 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.7] 2023-12-02T00:28:22.945 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAiempl+Md9NxAA5bPhqhkvQPRZ59k3OE9R9w== 2023-12-02T00:28:22.945 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2023-12-02T00:28:22.945 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAJempl5Lx1DhAAjpK6ajoQ5jQiUr/EMpFeLg== 2023-12-02T00:28:22.945 INFO:teuthology.orchestra.run.smithi046.stdout:[client.2] 2023-12-02T00:28:22.945 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAJempl6t9OEhAACGdJ4lIgFImVMyS+vbiyrw== 2023-12-02T00:28:22.945 INFO:teuthology.orchestra.run.smithi046.stdout:[client.1] 2023-12-02T00:28:22.945 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAJempl0gj4FRAAHZtOS7DxQQOSmYsu54uPZg== 2023-12-02T00:28:22.945 INFO:teuthology.orchestra.run.smithi046.stdout:[client.3] 2023-12-02T00:28:22.945 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAJemplapXEGxAAz74ppCDNM8zW2+ozYOiifQ== 2023-12-02T00:28:22.965 INFO:teuthology.orchestra.run.smithi162.stdout:[mgr.y] 2023-12-02T00:28:22.966 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQAIemplE8PaKBAAwleK4eGvEOffn5EzY+drMQ== 2023-12-02T00:28:22.966 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.a] 2023-12-02T00:28:22.966 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQAIemplYQrZMBAAqrkez6r//VUITan4SKdf4w== 2023-12-02T00:28:22.966 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.c] 2023-12-02T00:28:22.966 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQAIempl69PfOBAAQOgkrli+yM4wpJabPb3QxQ== 2023-12-02T00:28:22.966 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.0] 2023-12-02T00:28:22.966 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQAKempluFfMMhAAmbD1GpzugHbxobJBl3RtkQ== 2023-12-02T00:28:22.966 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.1] 2023-12-02T00:28:22.966 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQANempleNzmMRAALK5ExII/uG0A8pXBsc86pg== 2023-12-02T00:28:22.966 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.2] 2023-12-02T00:28:22.966 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQAQemplKDzyMBAAfPwOg3WNbFSBUZUqpG1l7w== 2023-12-02T00:28:22.967 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.3] 2023-12-02T00:28:22.967 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQATemplIbgAMBAAbynDehkgSKMaPHXYgwK2cA== 2023-12-02T00:28:22.967 INFO:teuthology.orchestra.run.smithi162.stdout:[mgr.x] 2023-12-02T00:28:22.967 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQAIemplbBH1LBAAxs6g6VawCdt6IxZJn2yrQA== 2023-12-02T00:28:22.967 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.b] 2023-12-02T00:28:22.967 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQAJemplKoSRAhAAuQHpGMNCUSFEtCtWMvY8yA== 2023-12-02T00:28:22.967 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.d] 2023-12-02T00:28:22.967 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQAJemplsTHqCRAAi7WJBcwYYDl0/qZkMy7GsQ== 2023-12-02T00:28:22.967 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.4] 2023-12-02T00:28:22.967 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQAaempl/vxEABAAThzqmuy54tDx5dYyTilSzg== 2023-12-02T00:28:22.967 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.5] 2023-12-02T00:28:22.967 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQAcemplWd7NOhAAO0eGoj+K+P0F4QvV4SMUxg== 2023-12-02T00:28:22.967 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.6] 2023-12-02T00:28:22.968 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQAfemplz38cORAAmb09RTXtiWf2J4q0jbYEug== 2023-12-02T00:28:22.968 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.7] 2023-12-02T00:28:22.968 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQAiempl+Md9NxAA5bPhqhkvQPRZ59k3OE9R9w== 2023-12-02T00:28:22.968 INFO:teuthology.orchestra.run.smithi162.stdout:[client.0] 2023-12-02T00:28:22.968 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQAJempl5Lx1DhAAjpK6ajoQ5jQiUr/EMpFeLg== 2023-12-02T00:28:22.968 INFO:teuthology.orchestra.run.smithi162.stdout:[client.2] 2023-12-02T00:28:22.968 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQAJempl6t9OEhAACGdJ4lIgFImVMyS+vbiyrw== 2023-12-02T00:28:22.968 INFO:teuthology.orchestra.run.smithi162.stdout:[client.1] 2023-12-02T00:28:22.968 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQAJempl0gj4FRAAHZtOS7DxQQOSmYsu54uPZg== 2023-12-02T00:28:22.968 INFO:teuthology.orchestra.run.smithi162.stdout:[client.3] 2023-12-02T00:28:22.968 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQAJemplapXEGxAAz74ppCDNM8zW2+ozYOiifQ== 2023-12-02T00:28:22.969 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-12-02T00:28:22.989 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-12-02T00:28:23.062 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-02T00:28:23.064 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-02T00:28:23.124 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-02T00:28:23.126 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-02T00:28:23.184 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-02T00:28:23.186 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-02T00:28:23.240 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-02T00:28:23.242 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-02T00:28:23.295 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-02T00:28:23.297 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-02T00:28:23.351 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-02T00:28:23.353 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-02T00:28:23.407 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-12-02T00:28:23.409 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-12-02T00:28:23.462 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-02T00:28:23.465 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-02T00:28:23.521 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-02T00:28:23.523 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-02T00:28:23.580 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-02T00:28:23.582 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-02T00:28:23.635 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-02T00:28:23.638 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-02T00:28:23.691 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-02T00:28:23.693 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-02T00:28:23.747 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-02T00:28:23.749 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-02T00:28:23.803 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-02T00:28:23.805 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-02T00:28:23.861 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-02T00:28:23.864 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-02T00:28:23.921 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-02T00:28:23.923 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-02T00:28:23.980 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-02T00:28:23.982 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-02T00:28:24.039 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-12-02T00:28:24.039 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-12-02T00:28:24.070 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-12-02T00:28:24.171 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-12-02T00:28:24.202 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-12-02T00:28:24.231 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-12-02T00:28:24.329 DEBUG:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-12-02T00:28:24.356 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-12-02T00:28:24.423 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-12-02T00:28:24.520 DEBUG:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-12-02T00:28:24.547 DEBUG:teuthology.orchestra.run.smithi046:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-12-02T00:28:24.549 DEBUG:teuthology.orchestra.run.smithi162:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-12-02T00:28:24.602 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-12-02T00:28:24.603 INFO:tasks.ceph.mon.a:Restarting daemon 2023-12-02T00:28:24.603 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-12-02T00:28:24.606 INFO:tasks.ceph.mon.a:Started 2023-12-02T00:28:24.606 INFO:tasks.ceph.mon.b:Restarting daemon 2023-12-02T00:28:24.607 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-12-02T00:28:24.645 INFO:tasks.ceph.mon.b:Started 2023-12-02T00:28:24.646 INFO:tasks.ceph.mon.c:Restarting daemon 2023-12-02T00:28:24.646 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-12-02T00:28:24.648 INFO:tasks.ceph.mon.c:Started 2023-12-02T00:28:24.648 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-12-02T00:28:24.648 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-12-02T00:28:24.648 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-12-02T00:28:24.651 INFO:tasks.ceph.mgr.y:Started 2023-12-02T00:28:24.651 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-12-02T00:28:24.651 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-12-02T00:28:24.653 INFO:tasks.ceph.mgr.x:Started 2023-12-02T00:28:24.653 INFO:tasks.ceph:Setting crush tunables to default 2023-12-02T00:28:24.654 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph osd crush tunables default 2023-12-02T00:28:24.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.760+0000 7ff217d90640 1 Processor -- start 2023-12-02T00:28:24.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.760+0000 7ff217d90640 1 -- start start 2023-12-02T00:28:24.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.761+0000 7ff217d90640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff210146bb0 0x7ff21015d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:24.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.761+0000 7ff217d90640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2100b34f0 con 0x7ff2100b3980 2023-12-02T00:28:24.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.761+0000 7ff217d90640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff210146f90 con 0x7ff2100b22a0 2023-12-02T00:28:24.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.761+0000 7ff217d90640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff21015e1d0 con 0x7ff210146bb0 2023-12-02T00:28:24.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.761+0000 7ff21632d640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff2100b3980 0x7ff210071760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54392/0 (socket says 172.21.15.46:54392) 2023-12-02T00:28:24.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.761+0000 7ff21632d640 1 -- 172.21.15.46:0/1633238518 learned_addr learned my addr 172.21.15.46:0/1633238518 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:28:24.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.762+0000 7ff215b2c640 1 --2- 172.21.15.46:0/1633238518 >> v2:172.21.15.162:3300/0 conn(0x7ff210146bb0 0x7ff21015d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:24.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.762+0000 7ff215b2c640 1 -- 172.21.15.46:0/1633238518 >> v2:172.21.15.162:3300/0 conn(0x7ff210146bb0 msgr2=0x7ff21015d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-12-02T00:28:24.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.762+0000 7ff215b2c640 1 -- 172.21.15.46:0/1633238518 >> v2:172.21.15.162:3300/0 conn(0x7ff210146bb0 msgr2=0x7ff21015d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-02T00:28:24.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.762+0000 7ff215b2c640 1 --2- 172.21.15.46:0/1633238518 >> v2:172.21.15.162:3300/0 conn(0x7ff210146bb0 0x7ff21015d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-02T00:28:24.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.762+0000 7ff215b2c640 1 --2- 172.21.15.46:0/1633238518 >> v2:172.21.15.162:3300/0 conn(0x7ff210146bb0 0x7ff21015d930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-02T00:28:24.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.957+0000 7ff214b2a640 1 -- 172.21.15.46:0/1633238518 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1910974122 0 0) 0x7ff2100b34f0 con 0x7ff2100b3980 2023-12-02T00:28:24.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.957+0000 7ff214b2a640 1 -- 172.21.15.46:0/1633238518 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1f8003520 con 0x7ff2100b3980 2023-12-02T00:28:24.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.958+0000 7ff214b2a640 1 -- 172.21.15.46:0/1633238518 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1180342692 0 0) 0x7ff1f8003520 con 0x7ff2100b3980 2023-12-02T00:28:24.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.958+0000 7ff214b2a640 1 -- 172.21.15.46:0/1633238518 >> v2:172.21.15.162:3300/0 conn(0x7ff210146bb0 msgr2=0x7ff21015d930 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-02T00:28:24.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.958+0000 7ff214b2a640 1 --2- 172.21.15.46:0/1633238518 >> v2:172.21.15.162:3300/0 conn(0x7ff210146bb0 0x7ff21015d930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:24.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.958+0000 7ff214b2a640 1 -- 172.21.15.46:0/1633238518 >> v1:172.21.15.162:6789/0 conn(0x7ff2100b22a0 legacy=0x7ff21015eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:24.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.958+0000 7ff214b2a640 1 -- 172.21.15.46:0/1633238518 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff21015e4b0 con 0x7ff2100b3980 2023-12-02T00:28:24.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.958+0000 7ff214b2a640 1 -- 172.21.15.46:0/1633238518 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7ff208001670 con 0x7ff2100b3980 2023-12-02T00:28:24.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.958+0000 7ff214b2a640 1 -- 172.21.15.46:0/1633238518 >> v1:172.21.15.46:6789/0 conn(0x7ff2100b3980 legacy=0x7ff210071760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:24.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.958+0000 7ff214b2a640 1 --2- 172.21.15.46:0/1633238518 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1f8003970 0x7ff1f8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:24.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.958+0000 7ff214b2a640 1 --2- 172.21.15.46:0/1633238518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1f80047c0 0x7ff1f8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:24.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.958+0000 7ff214b2a640 1 --2- 172.21.15.46:0/1633238518 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1f8005380 0x7ff1f800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:24.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.958+0000 7ff214b2a640 1 -- 172.21.15.46:0/1633238518 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff21015e1d0 con 0x7ff1f80047c0 2023-12-02T00:28:24.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.958+0000 7ff214b2a640 1 -- 172.21.15.46:0/1633238518 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f800c210 con 0x7ff1f8005380 2023-12-02T00:28:24.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.958+0000 7ff214b2a640 1 -- 172.21.15.46:0/1633238518 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f800c3d0 con 0x7ff1f8003970 2023-12-02T00:28:24.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.958+0000 7ff21532b640 1 --2- 172.21.15.46:0/1633238518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1f80047c0 0x7ff1f8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:24.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.958+0000 7ff21632d640 1 --2- 172.21.15.46:0/1633238518 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1f8005380 0x7ff1f800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:24.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.958+0000 7ff215b2c640 1 --2- 172.21.15.46:0/1633238518 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1f8003970 0x7ff1f8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:24.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.958+0000 7ff21532b640 1 -- 172.21.15.46:0/1633238518 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1f8003970 msgr2=0x7ff1f8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:24.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.958+0000 7ff21532b640 1 --2- 172.21.15.46:0/1633238518 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1f8003970 0x7ff1f8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:24.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.958+0000 7ff21532b640 1 -- 172.21.15.46:0/1633238518 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1f8005380 msgr2=0x7ff1f800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:24.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.958+0000 7ff21532b640 1 --2- 172.21.15.46:0/1633238518 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1f8005380 0x7ff1f800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:24.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.958+0000 7ff21532b640 1 -- 172.21.15.46:0/1633238518 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f8003520 con 0x7ff1f80047c0 2023-12-02T00:28:24.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.958+0000 7ff21632d640 1 -- 172.21.15.46:0/1633238518 reap_dead start 2023-12-02T00:28:24.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.959+0000 7ff21532b640 1 --2- 172.21.15.46:0/1633238518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1f80047c0 0x7ff1f8004c40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff204001720 tx=0x7ff1f80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b530d66c5de23c server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:24.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.959+0000 7ff214b2a640 1 -- 172.21.15.46:0/1633238518 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2040052d0 con 0x7ff1f80047c0 2023-12-02T00:28:24.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.959+0000 7ff214b2a640 1 -- 172.21.15.46:0/1633238518 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff20402e050 con 0x7ff1f80047c0 2023-12-02T00:28:24.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.959+0000 7ff214b2a640 1 -- 172.21.15.46:0/1633238518 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff204003830 con 0x7ff1f80047c0 2023-12-02T00:28:24.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.960+0000 7ff217d90640 1 -- 172.21.15.46:0/1633238518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1f80047c0 msgr2=0x7ff1f8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:24.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.960+0000 7ff217d90640 1 --2- 172.21.15.46:0/1633238518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1f80047c0 0x7ff1f8004c40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff204001720 tx=0x7ff1f80057f0 comp rx=0 tx=0).stop 2023-12-02T00:28:24.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.960+0000 7ff217d90640 1 -- 172.21.15.46:0/1633238518 shutdown_connections 2023-12-02T00:28:24.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.960+0000 7ff217d90640 1 --2- 172.21.15.46:0/1633238518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1f80047c0 0x7ff1f8004c40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:24.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.960+0000 7ff217d90640 1 -- 172.21.15.46:0/1633238518 >> 172.21.15.46:0/1633238518 conn(0x7ff210066270 msgr2=0x7ff210066650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:24.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.960+0000 7ff217d90640 1 -- 172.21.15.46:0/1633238518 shutdown_connections 2023-12-02T00:28:24.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.960+0000 7ff217d90640 1 -- 172.21.15.46:0/1633238518 wait complete. 2023-12-02T00:28:24.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.961+0000 7ff217d90640 1 Processor -- start 2023-12-02T00:28:24.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.962+0000 7ff217d90640 1 -- start start 2023-12-02T00:28:24.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.962+0000 7ff217d90640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1f80047c0 0x7ff2100b1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:24.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.962+0000 7ff217d90640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2100c50f0 0x7ff2100c5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:24.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.962+0000 7ff217d90640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff210137520 0x7ff2101379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:24.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.962+0000 7ff217d90640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff210160be0 con 0x7ff1f80047c0 2023-12-02T00:28:24.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.962+0000 7ff217d90640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2100c5c60 con 0x7ff210137520 2023-12-02T00:28:24.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.962+0000 7ff217d90640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2100c5f40 con 0x7ff2100c50f0 2023-12-02T00:28:24.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.962+0000 7ff215b2c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1f80047c0 0x7ff2100b1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:24.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.962+0000 7ff215b2c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1f80047c0 0x7ff2100b1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55904/0 (socket says 172.21.15.46:55904) 2023-12-02T00:28:24.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.962+0000 7ff215b2c640 1 -- 172.21.15.46:0/2750172241 learned_addr learned my addr 172.21.15.46:0/2750172241 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:24.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.962+0000 7ff21532b640 1 --2- 172.21.15.46:0/2750172241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2100c50f0 0x7ff2100c5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:24.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.963+0000 7ff21632d640 1 --2- 172.21.15.46:0/2750172241 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff210137520 0x7ff2101379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:24.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.963+0000 7ff215b2c640 1 -- 172.21.15.46:0/2750172241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2100c50f0 msgr2=0x7ff2100c5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:24.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.963+0000 7ff215b2c640 1 --2- 172.21.15.46:0/2750172241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2100c50f0 0x7ff2100c5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:24.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.963+0000 7ff215b2c640 1 -- 172.21.15.46:0/2750172241 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff210137520 msgr2=0x7ff2101379a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:24.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.963+0000 7ff215b2c640 1 --2- 172.21.15.46:0/2750172241 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff210137520 0x7ff2101379a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:24.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.963+0000 7ff215b2c640 1 -- 172.21.15.46:0/2750172241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2100c6220 con 0x7ff1f80047c0 2023-12-02T00:28:24.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.963+0000 7ff21532b640 1 --2- 172.21.15.46:0/2750172241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2100c50f0 0x7ff2100c5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:28:24.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.963+0000 7ff215b2c640 1 --2- 172.21.15.46:0/2750172241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1f80047c0 0x7ff2100b1070 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff200007fb0 tx=0x7ff20000f5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:24.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.964+0000 7ff20e7fc640 1 -- 172.21.15.46:0/2750172241 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff200015670 con 0x7ff1f80047c0 2023-12-02T00:28:24.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.964+0000 7ff20e7fc640 1 -- 172.21.15.46:0/2750172241 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2000157d0 con 0x7ff1f80047c0 2023-12-02T00:28:24.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.964+0000 7ff217d90640 1 -- 172.21.15.46:0/2750172241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2100c6500 con 0x7ff1f80047c0 2023-12-02T00:28:24.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.964+0000 7ff20e7fc640 1 -- 172.21.15.46:0/2750172241 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff200015b40 con 0x7ff1f80047c0 2023-12-02T00:28:24.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.964+0000 7ff217d90640 1 -- 172.21.15.46:0/2750172241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff210137ee0 con 0x7ff1f80047c0 2023-12-02T00:28:24.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.964+0000 7ff20e7fc640 1 -- 172.21.15.46:0/2750172241 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff200015ca0 con 0x7ff1f80047c0 2023-12-02T00:28:24.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.964+0000 7ff20e7fc640 1 -- 172.21.15.46:0/2750172241 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff2000044a0 con 0x7ff1f80047c0 2023-12-02T00:28:24.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.965+0000 7ff217d90640 1 -- 172.21.15.46:0/2750172241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff21015d580 con 0x7ff1f80047c0 2023-12-02T00:28:24.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:24.967+0000 7ff20e7fc640 1 -- 172.21.15.46:0/2750172241 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7ff200004660 con 0x7ff1f80047c0 2023-12-02T00:28:25.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:25.036+0000 7ff217d90640 1 -- 172.21.15.46:0/2750172241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7ff210067420 con 0x7ff1f80047c0 2023-12-02T00:28:25.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:25.038+0000 7ff20e7fc640 1 -- 172.21.15.46:0/2750172241 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7ff200004800 con 0x7ff1f80047c0 2023-12-02T00:28:25.039 INFO:teuthology.orchestra.run.smithi046.stderr:adjusted tunables profile to default 2023-12-02T00:28:25.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:25.039+0000 7ff217d90640 1 -- 172.21.15.46:0/2750172241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1f80047c0 msgr2=0x7ff2100b1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:25.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:25.039+0000 7ff217d90640 1 --2- 172.21.15.46:0/2750172241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1f80047c0 0x7ff2100b1070 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff200007fb0 tx=0x7ff20000f5e0 comp rx=0 tx=0).stop 2023-12-02T00:28:25.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:25.039+0000 7ff217d90640 1 -- 172.21.15.46:0/2750172241 shutdown_connections 2023-12-02T00:28:25.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:25.039+0000 7ff217d90640 1 --2- 172.21.15.46:0/2750172241 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff210137520 0x7ff2101379a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:25.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:25.039+0000 7ff217d90640 1 --2- 172.21.15.46:0/2750172241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2100c50f0 0x7ff2100c5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:25.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:25.039+0000 7ff217d90640 1 --2- 172.21.15.46:0/2750172241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1f80047c0 0x7ff2100b1070 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:25.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:25.039+0000 7ff217d90640 1 -- 172.21.15.46:0/2750172241 >> 172.21.15.46:0/2750172241 conn(0x7ff210066270 msgr2=0x7ff2100cf3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:25.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:25.039+0000 7ff217d90640 1 -- 172.21.15.46:0/2750172241 shutdown_connections 2023-12-02T00:28:25.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:25.039+0000 7ff217d90640 1 -- 172.21.15.46:0/2750172241 wait complete. 2023-12-02T00:28:25.051 INFO:tasks.ceph:check_enable_crimson: False 2023-12-02T00:28:25.051 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-12-02T00:28:25.052 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-02T00:28:25.052 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-12-02T00:28:25.083 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-02T00:28:25.083 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-12-02T00:28:25.153 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-02T00:28:25.153 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-12-02T00:28:25.224 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-02T00:28:25.225 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-12-02T00:28:25.296 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-12-02T00:28:25.296 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-12-02T00:28:25.323 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-12-02T00:28:25.323 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-12-02T00:28:25.392 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-12-02T00:28:25.392 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-12-02T00:28:25.437 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-12-02T00:28:25.436+0000 7f1c2ce2b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-02T00:28:25.460 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-12-02T00:28:25.460 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-12-02T00:28:25.461 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-12-02T00:28:25.459+0000 7fee736b6040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-02T00:28:25.492 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-12-02T00:28:25.491+0000 7f1c2ce2b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-02T00:28:25.514 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-12-02T00:28:25.513+0000 7fee736b6040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-02T00:28:25.519 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph osd new f48f92eb-acb2-46f4-8721-3cb619b01bce 0 2023-12-02T00:28:25.659 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.658+0000 7efd520ad640 1 Processor -- start 2023-12-02T00:28:25.659 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.658+0000 7efd520ad640 1 -- start start 2023-12-02T00:28:25.660 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.659+0000 7efd520ad640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd4c13d060 0x7efd4c06d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:25.660 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.659+0000 7efd520ad640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7efd4c15e3f0 0x7efd4c0ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:25.661 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.659+0000 7efd520ad640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd4c15e860 con 0x7efd4c06daa0 2023-12-02T00:28:25.661 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.659+0000 7efd520ad640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd4c06df20 con 0x7efd4c15e3f0 2023-12-02T00:28:25.661 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.659+0000 7efd520ad640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd4c13d440 con 0x7efd4c13d060 2023-12-02T00:28:25.661 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.659+0000 7efd4bfff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7efd4c15e3f0 0x7efd4c0ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:25.661 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.659+0000 7efd4b7fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd4c13d060 0x7efd4c06d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:25.661 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.659+0000 7efd42ffd640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7efd4c06daa0 0x7efd4c15dce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:58802/0 (socket says 172.21.15.162:58802) 2023-12-02T00:28:25.661 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.659+0000 7efd42ffd640 1 -- 172.21.15.162:0/1721063576 learned_addr learned my addr 172.21.15.162:0/1721063576 (peer_addr_for_me v1:172.21.15.162:0/0) 2023-12-02T00:28:25.661 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.660+0000 7efd4b7fe640 1 -- 172.21.15.162:0/1721063576 >> v2:172.21.15.162:3300/0 conn(0x7efd4c15e3f0 msgr2=0x7efd4c0ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:25.661 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.660+0000 7efd4b7fe640 1 --2- 172.21.15.162:0/1721063576 >> v2:172.21.15.162:3300/0 conn(0x7efd4c15e3f0 0x7efd4c0ceaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:25.661 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.660+0000 7efd4b7fe640 1 -- 172.21.15.162:0/1721063576 >> v1:172.21.15.162:6789/0 conn(0x7efd4c06daa0 legacy=0x7efd4c15dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:25.661 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.660+0000 7efd4affd640 1 -- 172.21.15.162:0/1721063576 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 653877246 0 0) 0x7efd34001110 con 0x7efd4c06daa0 2023-12-02T00:28:25.661 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.660+0000 7efd4b7fe640 1 -- 172.21.15.162:0/1721063576 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd4c15eaa0 con 0x7efd4c13d060 2023-12-02T00:28:25.661 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.660+0000 7efd4bfff640 1 --2- 172.21.15.162:0/1721063576 >> v2:172.21.15.162:3300/0 conn(0x7efd4c15e3f0 0x7efd4c0ceaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:28:25.663 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.660+0000 7efd4b7fe640 1 --2- 172.21.15.162:0/1721063576 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd4c13d060 0x7efd4c06d560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efd3c0097a0 tx=0x7efd3c02b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9086c96f400ca886 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:25.663 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.660+0000 7efd4affd640 1 -- 172.21.15.162:0/1721063576 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3c02d040 con 0x7efd4c13d060 2023-12-02T00:28:25.663 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.660+0000 7efd4affd640 1 -- 172.21.15.162:0/1721063576 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd3c031810 con 0x7efd4c13d060 2023-12-02T00:28:25.663 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.660+0000 7efd4affd640 1 -- 172.21.15.162:0/1721063576 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3c031ba0 con 0x7efd4c13d060 2023-12-02T00:28:25.663 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.661+0000 7efd520ad640 1 -- 172.21.15.162:0/1721063576 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd4c13d060 msgr2=0x7efd4c06d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:25.664 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.661+0000 7efd520ad640 1 --2- 172.21.15.162:0/1721063576 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd4c13d060 0x7efd4c06d560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efd3c0097a0 tx=0x7efd3c02b8b0 comp rx=0 tx=0).stop 2023-12-02T00:28:25.664 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.661+0000 7efd520ad640 1 -- 172.21.15.162:0/1721063576 shutdown_connections 2023-12-02T00:28:25.664 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.661+0000 7efd520ad640 1 --2- 172.21.15.162:0/1721063576 >> v2:172.21.15.162:3300/0 conn(0x7efd4c15e3f0 0x7efd4c0ceaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:25.664 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.661+0000 7efd520ad640 1 --2- 172.21.15.162:0/1721063576 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd4c13d060 0x7efd4c06d560 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:25.664 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.661+0000 7efd520ad640 1 -- 172.21.15.162:0/1721063576 >> 172.21.15.162:0/1721063576 conn(0x7efd4c066270 msgr2=0x7efd4c066650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:25.664 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.661+0000 7efd520ad640 1 -- 172.21.15.162:0/1721063576 shutdown_connections 2023-12-02T00:28:25.664 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.661+0000 7efd520ad640 1 -- 172.21.15.162:0/1721063576 wait complete. 2023-12-02T00:28:25.664 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.662+0000 7efd520ad640 1 Processor -- start 2023-12-02T00:28:25.664 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.662+0000 7efd520ad640 1 -- start start 2023-12-02T00:28:25.666 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.663+0000 7efd520ad640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd4c06daa0 0x7efd4c155190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:25.666 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.663+0000 7efd520ad640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd4c13d060 0x7efd4c138ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:25.666 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.663+0000 7efd520ad640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd4c15e3f0 0x7efd4c157410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:25.666 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.663+0000 7efd520ad640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd4c1564a0 con 0x7efd4c15e3f0 2023-12-02T00:28:25.666 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.663+0000 7efd520ad640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd4c139010 con 0x7efd4c06daa0 2023-12-02T00:28:25.667 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.663+0000 7efd520ad640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd4c155890 con 0x7efd4c13d060 2023-12-02T00:28:25.667 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.663+0000 7efd42ffd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd4c13d060 0x7efd4c138ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:25.667 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.663+0000 7efd4b7fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd4c06daa0 0x7efd4c155190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:25.667 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.663+0000 7efd42ffd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd4c13d060 0x7efd4c138ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.162:54406/0 (socket says 172.21.15.162:54406) 2023-12-02T00:28:25.667 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.663+0000 7efd42ffd640 1 -- 172.21.15.162:0/1181024775 learned_addr learned my addr 172.21.15.162:0/1181024775 (peer_addr_for_me v2:172.21.15.162:0/0) 2023-12-02T00:28:25.667 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.663+0000 7efd4bfff640 1 --2- 172.21.15.162:0/1181024775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd4c15e3f0 0x7efd4c157410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:25.667 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.663+0000 7efd42ffd640 1 -- 172.21.15.162:0/1181024775 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd4c06daa0 msgr2=0x7efd4c155190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:25.667 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.663+0000 7efd42ffd640 1 --2- 172.21.15.162:0/1181024775 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd4c06daa0 0x7efd4c155190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:25.667 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.663+0000 7efd42ffd640 1 -- 172.21.15.162:0/1181024775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd4c15e3f0 msgr2=0x7efd4c157410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:25.667 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.663+0000 7efd42ffd640 1 --2- 172.21.15.162:0/1181024775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd4c15e3f0 0x7efd4c157410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:25.667 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.663+0000 7efd42ffd640 1 -- 172.21.15.162:0/1181024775 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd44009a70 con 0x7efd4c13d060 2023-12-02T00:28:25.667 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.663+0000 7efd4b7fe640 1 --2- 172.21.15.162:0/1181024775 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd4c06daa0 0x7efd4c155190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:28:25.667 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.663+0000 7efd4bfff640 1 --2- 172.21.15.162:0/1181024775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd4c15e3f0 0x7efd4c157410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:28:25.667 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.664+0000 7efd42ffd640 1 --2- 172.21.15.162:0/1181024775 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd4c13d060 0x7efd4c138ad0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efd3400c780 tx=0x7efd3400cc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:25.667 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.664+0000 7efd48ff9640 1 -- 172.21.15.162:0/1181024775 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd34005ab0 con 0x7efd4c13d060 2023-12-02T00:28:25.667 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.664+0000 7efd48ff9640 1 -- 172.21.15.162:0/1181024775 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd34005c10 con 0x7efd4c13d060 2023-12-02T00:28:25.668 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.664+0000 7efd520ad640 1 -- 172.21.15.162:0/1181024775 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd3c009400 con 0x7efd4c13d060 2023-12-02T00:28:25.668 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.664+0000 7efd48ff9640 1 -- 172.21.15.162:0/1181024775 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd34007070 con 0x7efd4c13d060 2023-12-02T00:28:25.668 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.664+0000 7efd520ad640 1 -- 172.21.15.162:0/1181024775 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd4c155e50 con 0x7efd4c13d060 2023-12-02T00:28:25.668 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.664+0000 7efd48ff9640 1 -- 172.21.15.162:0/1181024775 <== mon.2 v2:172.21.15.162:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7efd340074d0 con 0x7efd4c13d060 2023-12-02T00:28:25.668 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.664+0000 7efd48ff9640 1 -- 172.21.15.162:0/1181024775 <== mon.2 v2:172.21.15.162:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7efd3401c780 con 0x7efd4c13d060 2023-12-02T00:28:25.668 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.665+0000 7efd520ad640 1 -- 172.21.15.162:0/1181024775 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd4c156650 con 0x7efd4c13d060 2023-12-02T00:28:25.668 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.667+0000 7efd48ff9640 1 -- 172.21.15.162:0/1181024775 <== mon.2 v2:172.21.15.162:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7efd34017070 con 0x7efd4c13d060 2023-12-02T00:28:25.737 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.736+0000 7efd520ad640 1 -- 172.21.15.162:0/1181024775 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f48f92eb-acb2-46f4-8721-3cb619b01bce", "id": 0} v 0) v1 -- 0x7efd4c067420 con 0x7efd4c13d060 2023-12-02T00:28:25.740 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.739+0000 7efd48ff9640 1 -- 172.21.15.162:0/1181024775 <== mon.2 v2:172.21.15.162:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f48f92eb-acb2-46f4-8721-3cb619b01bce", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7efd4c15e860 con 0x7efd4c13d060 2023-12-02T00:28:25.740 INFO:teuthology.orchestra.run.smithi162.stdout:0 2023-12-02T00:28:25.741 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.740+0000 7efd520ad640 1 -- 172.21.15.162:0/1181024775 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd4c13d060 msgr2=0x7efd4c138ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:25.741 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.740+0000 7efd520ad640 1 --2- 172.21.15.162:0/1181024775 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd4c13d060 0x7efd4c138ad0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efd3400c780 tx=0x7efd3400cc40 comp rx=0 tx=0).stop 2023-12-02T00:28:25.741 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.740+0000 7efd520ad640 1 -- 172.21.15.162:0/1181024775 shutdown_connections 2023-12-02T00:28:25.741 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.740+0000 7efd520ad640 1 --2- 172.21.15.162:0/1181024775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd4c15e3f0 0x7efd4c157410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:25.741 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.740+0000 7efd520ad640 1 --2- 172.21.15.162:0/1181024775 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd4c13d060 0x7efd4c138ad0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:25.741 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.740+0000 7efd520ad640 1 --2- 172.21.15.162:0/1181024775 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd4c06daa0 0x7efd4c155190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:25.742 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.740+0000 7efd520ad640 1 -- 172.21.15.162:0/1181024775 >> 172.21.15.162:0/1181024775 conn(0x7efd4c066270 msgr2=0x7efd4c159bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:25.742 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.740+0000 7efd520ad640 1 -- 172.21.15.162:0/1181024775 shutdown_connections 2023-12-02T00:28:25.742 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.740+0000 7efd520ad640 1 -- 172.21.15.162:0/1181024775 wait complete. 2023-12-02T00:28:25.752 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph osd new 52634887-f882-4f8d-b25b-b78368c750ea 1 2023-12-02T00:28:25.851 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.850+0000 7fe798c48640 1 Processor -- start 2023-12-02T00:28:25.851 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.850+0000 7fe798c48640 1 -- start start 2023-12-02T00:28:25.853 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.851+0000 7fe798c48640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7940b2e30 0x7fe79406de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:25.853 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.851+0000 7fe798c48640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe79406a570 con 0x7fe794161810 2023-12-02T00:28:25.853 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.851+0000 7fe798c48640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe79406cc50 con 0x7fe794160f00 2023-12-02T00:28:25.853 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.851+0000 7fe798c48640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe79415dfb0 con 0x7fe7940b2e30 2023-12-02T00:28:25.853 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.852+0000 7fe791d9b640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fe794160f00 0x7fe7940b2720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:58818/0 (socket says 172.21.15.162:58818) 2023-12-02T00:28:25.853 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.852+0000 7fe791d9b640 1 -- 172.21.15.162:0/4232035661 learned_addr learned my addr 172.21.15.162:0/4232035661 (peer_addr_for_me v1:172.21.15.162:0/0) 2023-12-02T00:28:25.853 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.852+0000 7fe792d9d640 1 --2- 172.21.15.162:0/4232035661 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7940b2e30 0x7fe79406de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:25.853 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.852+0000 7fe79159a640 1 -- 172.21.15.162:0/4232035661 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 970072888 0 0) 0x7fe79406cc50 con 0x7fe794160f00 2023-12-02T00:28:25.853 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.852+0000 7fe79159a640 1 -- 172.21.15.162:0/4232035661 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe77c003520 con 0x7fe794160f00 2023-12-02T00:28:25.853 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.852+0000 7fe792d9d640 1 -- 172.21.15.162:0/4232035661 >> v1:172.21.15.162:6789/0 conn(0x7fe794160f00 legacy=0x7fe7940b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:25.853 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.852+0000 7fe792d9d640 1 -- 172.21.15.162:0/4232035661 >> v1:172.21.15.46:6789/0 conn(0x7fe794161810 legacy=0x7fe79415d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:25.853 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.852+0000 7fe792d9d640 1 -- 172.21.15.162:0/4232035661 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe79415e1d0 con 0x7fe7940b2e30 2023-12-02T00:28:25.855 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.853+0000 7fe792d9d640 1 --2- 172.21.15.162:0/4232035661 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7940b2e30 0x7fe79406de80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe78c009dc0 tx=0x7fe78c02be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81fb55c72f8df165 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:25.855 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.853+0000 7fe79159a640 1 -- 172.21.15.162:0/4232035661 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78c02cc40 con 0x7fe7940b2e30 2023-12-02T00:28:25.855 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.853+0000 7fe79159a640 1 -- 172.21.15.162:0/4232035661 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe78c02cda0 con 0x7fe7940b2e30 2023-12-02T00:28:25.855 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.853+0000 7fe79159a640 1 -- 172.21.15.162:0/4232035661 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78c034660 con 0x7fe7940b2e30 2023-12-02T00:28:25.855 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.853+0000 7fe798c48640 1 -- 172.21.15.162:0/4232035661 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7940b2e30 msgr2=0x7fe79406de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:25.855 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.853+0000 7fe798c48640 1 --2- 172.21.15.162:0/4232035661 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7940b2e30 0x7fe79406de80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe78c009dc0 tx=0x7fe78c02be90 comp rx=0 tx=0).stop 2023-12-02T00:28:25.855 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.853+0000 7fe798c48640 1 -- 172.21.15.162:0/4232035661 shutdown_connections 2023-12-02T00:28:25.855 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.853+0000 7fe798c48640 1 --2- 172.21.15.162:0/4232035661 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7940b2e30 0x7fe79406de80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:25.855 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.853+0000 7fe798c48640 1 -- 172.21.15.162:0/4232035661 >> 172.21.15.162:0/4232035661 conn(0x7fe7940b1a50 msgr2=0x7fe7940cf670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:25.855 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.854+0000 7fe798c48640 1 -- 172.21.15.162:0/4232035661 shutdown_connections 2023-12-02T00:28:25.855 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.854+0000 7fe798c48640 1 -- 172.21.15.162:0/4232035661 wait complete. 2023-12-02T00:28:25.856 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.855+0000 7fe798c48640 1 Processor -- start 2023-12-02T00:28:25.856 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.855+0000 7fe798c48640 1 -- start start 2023-12-02T00:28:25.858 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.856+0000 7fe798c48640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7940b2e30 0x7fe7941e9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:25.858 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.856+0000 7fe798c48640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe794160f00 0x7fe79415fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:25.858 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.856+0000 7fe798c48640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe794161810 0x7fe794160430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:25.858 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.856+0000 7fe798c48640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7941e8280 con 0x7fe794161810 2023-12-02T00:28:25.858 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.856+0000 7fe798c48640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7941e8560 con 0x7fe794160f00 2023-12-02T00:28:25.858 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.856+0000 7fe798c48640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7941e8840 con 0x7fe7940b2e30 2023-12-02T00:28:25.858 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.856+0000 7fe791d9b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe794160f00 0x7fe79415fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:25.858 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.856+0000 7fe79259c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7940b2e30 0x7fe7941e9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:25.858 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.856+0000 7fe79259c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7940b2e30 0x7fe7941e9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.162:54420/0 (socket says 172.21.15.162:54420) 2023-12-02T00:28:25.858 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.856+0000 7fe79259c640 1 -- 172.21.15.162:0/1796790094 learned_addr learned my addr 172.21.15.162:0/1796790094 (peer_addr_for_me v2:172.21.15.162:0/0) 2023-12-02T00:28:25.859 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.856+0000 7fe792d9d640 1 --2- 172.21.15.162:0/1796790094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe794161810 0x7fe794160430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:25.859 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.856+0000 7fe79259c640 1 -- 172.21.15.162:0/1796790094 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe794160f00 msgr2=0x7fe79415fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:25.859 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.856+0000 7fe79259c640 1 --2- 172.21.15.162:0/1796790094 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe794160f00 0x7fe79415fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:25.859 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.856+0000 7fe79259c640 1 -- 172.21.15.162:0/1796790094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe794161810 msgr2=0x7fe794160430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:25.859 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.856+0000 7fe79259c640 1 --2- 172.21.15.162:0/1796790094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe794161810 0x7fe794160430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:25.859 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.856+0000 7fe79259c640 1 -- 172.21.15.162:0/1796790094 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe78c009a70 con 0x7fe7940b2e30 2023-12-02T00:28:25.859 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.856+0000 7fe79259c640 1 --2- 172.21.15.162:0/1796790094 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7940b2e30 0x7fe7941e9040 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe78400b3f0 tx=0x7fe78400b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:25.859 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.856+0000 7fe782ffd640 1 -- 172.21.15.162:0/1796790094 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe784011830 con 0x7fe7940b2e30 2023-12-02T00:28:25.859 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.856+0000 7fe782ffd640 1 -- 172.21.15.162:0/1796790094 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe784011990 con 0x7fe7940b2e30 2023-12-02T00:28:25.859 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.856+0000 7fe782ffd640 1 -- 172.21.15.162:0/1796790094 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe784011d40 con 0x7fe7940b2e30 2023-12-02T00:28:25.859 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.856+0000 7fe798c48640 1 -- 172.21.15.162:0/1796790094 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7941e8b20 con 0x7fe7940b2e30 2023-12-02T00:28:25.859 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.856+0000 7fe798c48640 1 -- 172.21.15.162:0/1796790094 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe79413c230 con 0x7fe7940b2e30 2023-12-02T00:28:25.859 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.857+0000 7fe782ffd640 1 -- 172.21.15.162:0/1796790094 <== mon.2 v2:172.21.15.162:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe784004320 con 0x7fe7940b2e30 2023-12-02T00:28:25.860 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.857+0000 7fe782ffd640 1 -- 172.21.15.162:0/1796790094 <== mon.2 v2:172.21.15.162:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fe78401a4d0 con 0x7fe7940b2e30 2023-12-02T00:28:25.860 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.857+0000 7fe798c48640 1 -- 172.21.15.162:0/1796790094 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7941e8d10 con 0x7fe7940b2e30 2023-12-02T00:28:25.860 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.859+0000 7fe782ffd640 1 -- 172.21.15.162:0/1796790094 <== mon.2 v2:172.21.15.162:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fe784038ce0 con 0x7fe7940b2e30 2023-12-02T00:28:25.932 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.931+0000 7fe798c48640 1 -- 172.21.15.162:0/1796790094 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "osd new", "uuid": "52634887-f882-4f8d-b25b-b78368c750ea", "id": 1} v 0) v1 -- 0x7fe794067420 con 0x7fe7940b2e30 2023-12-02T00:28:25.935 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.934+0000 7fe782ffd640 1 -- 172.21.15.162:0/1796790094 <== mon.2 v2:172.21.15.162:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "52634887-f882-4f8d-b25b-b78368c750ea", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe784027070 con 0x7fe7940b2e30 2023-12-02T00:28:25.935 INFO:teuthology.orchestra.run.smithi162.stdout:1 2023-12-02T00:28:25.936 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.935+0000 7fe798c48640 1 -- 172.21.15.162:0/1796790094 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7940b2e30 msgr2=0x7fe7941e9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:25.936 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.935+0000 7fe798c48640 1 --2- 172.21.15.162:0/1796790094 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7940b2e30 0x7fe7941e9040 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe78400b3f0 tx=0x7fe78400b8b0 comp rx=0 tx=0).stop 2023-12-02T00:28:25.936 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.935+0000 7fe798c48640 1 -- 172.21.15.162:0/1796790094 shutdown_connections 2023-12-02T00:28:25.936 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.935+0000 7fe798c48640 1 --2- 172.21.15.162:0/1796790094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe794161810 0x7fe794160430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:25.936 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.935+0000 7fe798c48640 1 --2- 172.21.15.162:0/1796790094 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe794160f00 0x7fe79415fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:25.936 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.935+0000 7fe798c48640 1 --2- 172.21.15.162:0/1796790094 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7940b2e30 0x7fe7941e9040 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:25.936 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.935+0000 7fe798c48640 1 -- 172.21.15.162:0/1796790094 >> 172.21.15.162:0/1796790094 conn(0x7fe7940b1a50 msgr2=0x7fe79406d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:25.936 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.935+0000 7fe798c48640 1 -- 172.21.15.162:0/1796790094 shutdown_connections 2023-12-02T00:28:25.937 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:25.935+0000 7fe798c48640 1 -- 172.21.15.162:0/1796790094 wait complete. 2023-12-02T00:28:25.947 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph osd new f8d40b7c-3802-453a-ad40-9322d0e1a3f8 2 2023-12-02T00:28:26.047 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.047+0000 7f537a5b6640 1 Processor -- start 2023-12-02T00:28:26.048 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.047+0000 7f537a5b6640 1 -- start start 2023-12-02T00:28:26.049 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.047+0000 7f537a5b6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f537415b130 0x7f537415b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:26.049 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.047+0000 7f537a5b6640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f53740cd970 0x7f53740cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:26.049 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.047+0000 7f537a5b6640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53740cd110 con 0x7f53740cb770 2023-12-02T00:28:26.049 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.047+0000 7f537a5b6640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53740d0730 con 0x7f53740cd970 2023-12-02T00:28:26.049 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.047+0000 7f537a5b6640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53740cd3b0 con 0x7f537415b130 2023-12-02T00:28:26.049 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.047+0000 7f5373fff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f537415b130 0x7f537415b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:26.049 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.047+0000 7f53737fe640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f53740cb770 0x7f537415f6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:58832/0 (socket says 172.21.15.162:58832) 2023-12-02T00:28:26.049 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.048+0000 7f53737fe640 1 -- 172.21.15.162:0/1361113386 learned_addr learned my addr 172.21.15.162:0/1361113386 (peer_addr_for_me v1:172.21.15.162:0/0) 2023-12-02T00:28:26.049 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.048+0000 7f5378b53640 1 --2- 172.21.15.162:0/1361113386 >> v2:172.21.15.46:3300/0 conn(0x7f53740cd970 0x7f53740cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:26.050 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.048+0000 7f5372ffd640 1 -- 172.21.15.162:0/1361113386 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4171843090 0 0) 0x7f53740cd110 con 0x7f53740cb770 2023-12-02T00:28:26.050 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.048+0000 7f5372ffd640 1 -- 172.21.15.162:0/1361113386 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f535c003520 con 0x7f53740cb770 2023-12-02T00:28:26.050 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.048+0000 7f5373fff640 1 -- 172.21.15.162:0/1361113386 >> v2:172.21.15.46:3300/0 conn(0x7f53740cd970 msgr2=0x7f53740cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:26.050 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.048+0000 7f5373fff640 1 --2- 172.21.15.162:0/1361113386 >> v2:172.21.15.46:3300/0 conn(0x7f53740cd970 0x7f53740cdde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:26.050 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.048+0000 7f5373fff640 1 -- 172.21.15.162:0/1361113386 >> v1:172.21.15.162:6789/0 conn(0x7f53740cb770 legacy=0x7f537415f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:26.050 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.048+0000 7f5373fff640 1 -- 172.21.15.162:0/1361113386 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53740cd690 con 0x7f537415b130 2023-12-02T00:28:26.051 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.049+0000 7f5373fff640 1 --2- 172.21.15.162:0/1361113386 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f537415b130 0x7f537415b510 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f53640097a0 tx=0x7f536402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f43f56870399e4b2 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:26.051 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.049+0000 7f5372ffd640 1 -- 172.21.15.162:0/1361113386 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5364031830 con 0x7f537415b130 2023-12-02T00:28:26.051 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.049+0000 7f5372ffd640 1 -- 172.21.15.162:0/1361113386 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5364031990 con 0x7f537415b130 2023-12-02T00:28:26.051 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.049+0000 7f5372ffd640 1 -- 172.21.15.162:0/1361113386 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5364031d20 con 0x7f537415b130 2023-12-02T00:28:26.051 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.050+0000 7f537a5b6640 1 -- 172.21.15.162:0/1361113386 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f537415b130 msgr2=0x7f537415b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:26.051 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.050+0000 7f537a5b6640 1 --2- 172.21.15.162:0/1361113386 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f537415b130 0x7f537415b510 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f53640097a0 tx=0x7f536402b8d0 comp rx=0 tx=0).stop 2023-12-02T00:28:26.052 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.050+0000 7f537a5b6640 1 -- 172.21.15.162:0/1361113386 shutdown_connections 2023-12-02T00:28:26.052 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.050+0000 7f537a5b6640 1 --2- 172.21.15.162:0/1361113386 >> v2:172.21.15.46:3300/0 conn(0x7f53740cd970 0x7f53740cdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:26.052 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.050+0000 7f537a5b6640 1 --2- 172.21.15.162:0/1361113386 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f537415b130 0x7f537415b510 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:26.052 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.050+0000 7f537a5b6640 1 -- 172.21.15.162:0/1361113386 >> 172.21.15.162:0/1361113386 conn(0x7f53740b2c10 msgr2=0x7f53740b3020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:26.052 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.050+0000 7f537a5b6640 1 -- 172.21.15.162:0/1361113386 shutdown_connections 2023-12-02T00:28:26.052 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.050+0000 7f537a5b6640 1 -- 172.21.15.162:0/1361113386 wait complete. 2023-12-02T00:28:26.054 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.051+0000 7f537a5b6640 1 Processor -- start 2023-12-02T00:28:26.054 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.051+0000 7f537a5b6640 1 -- start start 2023-12-02T00:28:26.054 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.052+0000 7f537a5b6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53740cb770 0x7f53741f2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:26.054 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.052+0000 7f537a5b6640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f53740cd970 0x7f53741f5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:26.055 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.052+0000 7f537a5b6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f537415b130 0x7f53740c7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:26.055 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.052+0000 7f537a5b6640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53740c6010 con 0x7f537415b130 2023-12-02T00:28:26.055 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.052+0000 7f537a5b6640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53740c62f0 con 0x7f53740cd970 2023-12-02T00:28:26.055 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.052+0000 7f537a5b6640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53740c65d0 con 0x7f53740cb770 2023-12-02T00:28:26.055 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.052+0000 7f53737fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f53740cd970 0x7f53741f5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:26.055 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.052+0000 7f5373fff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53740cb770 0x7f53741f2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:26.055 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.052+0000 7f53737fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f53740cd970 0x7f53741f5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:59888/0 (socket says 172.21.15.162:59888) 2023-12-02T00:28:26.055 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.052+0000 7f5373fff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53740cb770 0x7f53741f2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.162:54450/0 (socket says 172.21.15.162:54450) 2023-12-02T00:28:26.055 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.052+0000 7f53737fe640 1 -- 172.21.15.162:0/608307765 learned_addr learned my addr 172.21.15.162:0/608307765 (peer_addr_for_me v2:172.21.15.162:0/0) 2023-12-02T00:28:26.055 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.052+0000 7f5378b53640 1 --2- 172.21.15.162:0/608307765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f537415b130 0x7f53740c7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:26.055 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.052+0000 7f5373fff640 1 -- 172.21.15.162:0/608307765 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f53740cd970 msgr2=0x7f53741f5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:26.055 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.052+0000 7f5373fff640 1 --2- 172.21.15.162:0/608307765 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f53740cd970 0x7f53741f5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:26.055 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.052+0000 7f5373fff640 1 -- 172.21.15.162:0/608307765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f537415b130 msgr2=0x7f53740c7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:26.055 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.052+0000 7f5373fff640 1 --2- 172.21.15.162:0/608307765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f537415b130 0x7f53740c7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:26.055 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.052+0000 7f5373fff640 1 -- 172.21.15.162:0/608307765 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5364009400 con 0x7f53740cb770 2023-12-02T00:28:26.056 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.052+0000 7f53737fe640 1 --2- 172.21.15.162:0/608307765 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f53740cd970 0x7f53741f5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:28:26.056 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.053+0000 7f5373fff640 1 --2- 172.21.15.162:0/608307765 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53740cb770 0x7f53741f2ff0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5364033000 tx=0x7f53640046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:26.056 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.053+0000 7f5370ff9640 1 -- 172.21.15.162:0/608307765 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536403a830 con 0x7f53740cb770 2023-12-02T00:28:26.056 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.053+0000 7f5370ff9640 1 -- 172.21.15.162:0/608307765 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f536403a9d0 con 0x7f53740cb770 2023-12-02T00:28:26.056 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.053+0000 7f537a5b6640 1 -- 172.21.15.162:0/608307765 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53740c68b0 con 0x7f53740cb770 2023-12-02T00:28:26.056 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.053+0000 7f5370ff9640 1 -- 172.21.15.162:0/608307765 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536403ad80 con 0x7f53740cb770 2023-12-02T00:28:26.056 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.053+0000 7f537a5b6640 1 -- 172.21.15.162:0/608307765 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53740c5bc0 con 0x7f53740cb770 2023-12-02T00:28:26.056 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.053+0000 7f5370ff9640 1 -- 172.21.15.162:0/608307765 <== mon.2 v2:172.21.15.162:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f536404d700 con 0x7f53740cb770 2023-12-02T00:28:26.056 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.053+0000 7f5370ff9640 1 -- 172.21.15.162:0/608307765 <== mon.2 v2:172.21.15.162:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f53640449f0 con 0x7f53740cb770 2023-12-02T00:28:26.056 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.054+0000 7f537a5b6640 1 -- 172.21.15.162:0/608307765 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5374067fc0 con 0x7f53740cb770 2023-12-02T00:28:26.057 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-12-02T00:28:26.055+0000 7f1c2ce2b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-02T00:28:26.057 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.056+0000 7f5370ff9640 1 -- 172.21.15.162:0/608307765 <== mon.2 v2:172.21.15.162:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f5364035020 con 0x7f53740cb770 2023-12-02T00:28:26.063 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-12-02T00:28:26.062+0000 7fee736b6040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-02T00:28:26.110 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-12-02T00:28:26.110+0000 7f1c2ce2b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-02T00:28:26.200 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-12-02T00:28:26.116+0000 7fee736b6040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-02T00:28:26.200 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-12-02T00:28:26.163+0000 7fee736b6040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-02T00:28:26.200 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.128+0000 7f537a5b6640 1 -- 172.21.15.162:0/608307765 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f8d40b7c-3802-453a-ad40-9322d0e1a3f8", "id": 2} v 0) v1 -- 0x7f5374067420 con 0x7f53740cb770 2023-12-02T00:28:26.200 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.131+0000 7f5370ff9640 1 -- 172.21.15.162:0/608307765 <== mon.2 v2:172.21.15.162:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f8d40b7c-3802-453a-ad40-9322d0e1a3f8", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f53640445d0 con 0x7f53740cb770 2023-12-02T00:28:26.201 INFO:teuthology.orchestra.run.smithi162.stdout:2 2023-12-02T00:28:26.202 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.132+0000 7f537a5b6640 1 -- 172.21.15.162:0/608307765 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53740cb770 msgr2=0x7f53741f2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:26.202 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.132+0000 7f537a5b6640 1 --2- 172.21.15.162:0/608307765 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53740cb770 0x7f53741f2ff0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5364033000 tx=0x7f53640046c0 comp rx=0 tx=0).stop 2023-12-02T00:28:26.202 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.132+0000 7f537a5b6640 1 -- 172.21.15.162:0/608307765 shutdown_connections 2023-12-02T00:28:26.202 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.132+0000 7f537a5b6640 1 --2- 172.21.15.162:0/608307765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f537415b130 0x7f53740c7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:26.202 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.132+0000 7f537a5b6640 1 --2- 172.21.15.162:0/608307765 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f53740cd970 0x7f53741f5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:26.202 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.132+0000 7f537a5b6640 1 --2- 172.21.15.162:0/608307765 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53740cb770 0x7f53741f2ff0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:26.202 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.132+0000 7f537a5b6640 1 -- 172.21.15.162:0/608307765 >> 172.21.15.162:0/608307765 conn(0x7f53740b2c10 msgr2=0x7f53740b2370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:26.202 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.132+0000 7f537a5b6640 1 -- 172.21.15.162:0/608307765 shutdown_connections 2023-12-02T00:28:26.203 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.132+0000 7f537a5b6640 1 -- 172.21.15.162:0/608307765 wait complete. 2023-12-02T00:28:26.203 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph osd new c99837cf-79d2-4b7e-9829-62697b43ff06 3 2023-12-02T00:28:26.203 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-12-02T00:28:26.158+0000 7f1c2ce2b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-02T00:28:26.213 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-12-02T00:28:26.212+0000 7f1c2ce2b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-02T00:28:26.220 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-12-02T00:28:26.219+0000 7fee736b6040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-02T00:28:26.259 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-12-02T00:28:26.258+0000 7f1c2ce2b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-02T00:28:26.266 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-12-02T00:28:26.263+0000 7fee736b6040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-02T00:28:26.322 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.303+0000 7f6db1fd0640 1 Processor -- start 2023-12-02T00:28:26.323 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.303+0000 7f6db1fd0640 1 -- start start 2023-12-02T00:28:26.323 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.304+0000 7f6db1fd0640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6dac161810 0x7f6dac15d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:26.323 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.304+0000 7f6db1fd0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6dac160f00 0x7f6dac0b2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:26.323 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.304+0000 7f6db1fd0640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dac06a570 con 0x7f6dac0b2c60 2023-12-02T00:28:26.323 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.304+0000 7f6db1fd0640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dac0b30d0 con 0x7f6dac161810 2023-12-02T00:28:26.323 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.304+0000 7f6db1fd0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dac06cc50 con 0x7f6dac160f00 2023-12-02T00:28:26.323 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.304+0000 7f6daaffd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6dac160f00 0x7f6dac0b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:26.323 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.304+0000 7f6daaffd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6dac160f00 0x7f6dac0b2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.162:54460/0 (socket says 172.21.15.162:54460) 2023-12-02T00:28:26.323 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.304+0000 7f6dabfff640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f6dac0b2c60 0x7f6dac06de80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:58846/0 (socket says 172.21.15.162:58846) 2023-12-02T00:28:26.323 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.304+0000 7f6daaffd640 1 -- 172.21.15.162:0/283683353 learned_addr learned my addr 172.21.15.162:0/283683353 (peer_addr_for_me v2:172.21.15.162:0/0) 2023-12-02T00:28:26.323 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.304+0000 7f6dab7fe640 1 --2- 172.21.15.162:0/283683353 >> v2:172.21.15.46:3300/0 conn(0x7f6dac161810 0x7f6dac15d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:26.323 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.304+0000 7f6daa7fc640 1 -- 172.21.15.162:0/283683353 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3545330351 0 0) 0x7f6dac06a570 con 0x7f6dac0b2c60 2023-12-02T00:28:26.323 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.304+0000 7f6daa7fc640 1 -- 172.21.15.162:0/283683353 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d94003520 con 0x7f6dac0b2c60 2023-12-02T00:28:26.324 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.304+0000 7f6daaffd640 1 -- 172.21.15.162:0/283683353 >> v2:172.21.15.46:3300/0 conn(0x7f6dac161810 msgr2=0x7f6dac15d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:26.324 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.304+0000 7f6daaffd640 1 --2- 172.21.15.162:0/283683353 >> v2:172.21.15.46:3300/0 conn(0x7f6dac161810 0x7f6dac15d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:26.324 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.304+0000 7f6daaffd640 1 -- 172.21.15.162:0/283683353 >> v1:172.21.15.162:6789/0 conn(0x7f6dac0b2c60 legacy=0x7f6dac06de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:26.324 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.304+0000 7f6daaffd640 1 -- 172.21.15.162:0/283683353 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dac15e1d0 con 0x7f6dac160f00 2023-12-02T00:28:26.324 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.305+0000 7f6daaffd640 1 --2- 172.21.15.162:0/283683353 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6dac160f00 0x7f6dac0b2720 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6da0009870 tx=0x7f6da002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec9df137b81ae563 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:26.324 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.305+0000 7f6daa7fc640 1 -- 172.21.15.162:0/283683353 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da0030840 con 0x7f6dac160f00 2023-12-02T00:28:26.324 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.305+0000 7f6daa7fc640 1 -- 172.21.15.162:0/283683353 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da00309a0 con 0x7f6dac160f00 2023-12-02T00:28:26.324 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.305+0000 7f6daa7fc640 1 -- 172.21.15.162:0/283683353 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da0030d30 con 0x7f6dac160f00 2023-12-02T00:28:26.324 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.306+0000 7f6db1fd0640 1 -- 172.21.15.162:0/283683353 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6dac160f00 msgr2=0x7f6dac0b2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:26.324 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.306+0000 7f6db1fd0640 1 --2- 172.21.15.162:0/283683353 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6dac160f00 0x7f6dac0b2720 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6da0009870 tx=0x7f6da002b980 comp rx=0 tx=0).stop 2023-12-02T00:28:26.324 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.306+0000 7f6db1fd0640 1 -- 172.21.15.162:0/283683353 shutdown_connections 2023-12-02T00:28:26.324 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.306+0000 7f6db1fd0640 1 --2- 172.21.15.162:0/283683353 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6dac160f00 0x7f6dac0b2720 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:26.324 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.306+0000 7f6db1fd0640 1 --2- 172.21.15.162:0/283683353 >> v2:172.21.15.46:3300/0 conn(0x7f6dac161810 0x7f6dac15d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:26.324 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.306+0000 7f6db1fd0640 1 -- 172.21.15.162:0/283683353 >> 172.21.15.162:0/283683353 conn(0x7f6dac0b1a50 msgr2=0x7f6dac0cf670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:26.325 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.306+0000 7f6db1fd0640 1 -- 172.21.15.162:0/283683353 shutdown_connections 2023-12-02T00:28:26.325 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.306+0000 7f6db1fd0640 1 -- 172.21.15.162:0/283683353 wait complete. 2023-12-02T00:28:26.325 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.307+0000 7f6db1fd0640 1 Processor -- start 2023-12-02T00:28:26.325 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.307+0000 7f6db1fd0640 1 -- start start 2023-12-02T00:28:26.325 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.308+0000 7f6db1fd0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6dac0b2c60 0x7f6dac156640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:26.325 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.308+0000 7f6db1fd0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dac160f00 0x7f6dac158970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:26.325 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.308+0000 7f6db1fd0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6dac161810 0x7f6dac159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:26.325 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.308+0000 7f6db1fd0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dac1593c0 con 0x7f6dac160f00 2023-12-02T00:28:26.325 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.308+0000 7f6db1fd0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dac1596a0 con 0x7f6dac161810 2023-12-02T00:28:26.325 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.308+0000 7f6db1fd0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dac15e9b0 con 0x7f6dac0b2c60 2023-12-02T00:28:26.325 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.308+0000 7f6dab7fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6dac0b2c60 0x7f6dac156640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:26.325 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.308+0000 7f6dabfff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6dac161810 0x7f6dac159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:26.325 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.308+0000 7f6dab7fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6dac0b2c60 0x7f6dac156640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.162:54464/0 (socket says 172.21.15.162:54464) 2023-12-02T00:28:26.325 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.308+0000 7f6dabfff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6dac161810 0x7f6dac159dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:59896/0 (socket says 172.21.15.162:59896) 2023-12-02T00:28:26.326 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.308+0000 7f6dab7fe640 1 -- 172.21.15.162:0/1742254376 learned_addr learned my addr 172.21.15.162:0/1742254376 (peer_addr_for_me v2:172.21.15.162:0/0) 2023-12-02T00:28:26.326 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.308+0000 7f6daaffd640 1 --2- 172.21.15.162:0/1742254376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dac160f00 0x7f6dac158970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:26.326 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.308+0000 7f6dab7fe640 1 -- 172.21.15.162:0/1742254376 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6dac161810 msgr2=0x7f6dac159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:26.326 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.308+0000 7f6dab7fe640 1 --2- 172.21.15.162:0/1742254376 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6dac161810 0x7f6dac159dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:26.326 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.308+0000 7f6dab7fe640 1 -- 172.21.15.162:0/1742254376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dac160f00 msgr2=0x7f6dac158970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:26.326 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.308+0000 7f6dab7fe640 1 --2- 172.21.15.162:0/1742254376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dac160f00 0x7f6dac158970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:26.326 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.308+0000 7f6dab7fe640 1 -- 172.21.15.162:0/1742254376 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da00094d0 con 0x7f6dac0b2c60 2023-12-02T00:28:26.326 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.308+0000 7f6dabfff640 1 --2- 172.21.15.162:0/1742254376 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6dac161810 0x7f6dac159dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:28:26.326 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.309+0000 7f6dab7fe640 1 --2- 172.21.15.162:0/1742254376 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6dac0b2c60 0x7f6dac156640 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c00efc0 tx=0x7f6d9c00c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:26.326 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.309+0000 7f6d93fff640 1 -- 172.21.15.162:0/1742254376 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c009b80 con 0x7f6dac0b2c60 2023-12-02T00:28:26.326 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.309+0000 7f6d93fff640 1 -- 172.21.15.162:0/1742254376 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d9c00f040 con 0x7f6dac0b2c60 2023-12-02T00:28:26.326 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.309+0000 7f6d93fff640 1 -- 172.21.15.162:0/1742254376 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c0146a0 con 0x7f6dac0b2c60 2023-12-02T00:28:26.326 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.309+0000 7f6db1fd0640 1 -- 172.21.15.162:0/1742254376 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dac15ec90 con 0x7f6dac0b2c60 2023-12-02T00:28:26.326 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.309+0000 7f6db1fd0640 1 -- 172.21.15.162:0/1742254376 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dac13f4a0 con 0x7f6dac0b2c60 2023-12-02T00:28:26.326 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.309+0000 7f6d93fff640 1 -- 172.21.15.162:0/1742254376 <== mon.2 v2:172.21.15.162:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f6d9c014b00 con 0x7f6dac0b2c60 2023-12-02T00:28:26.327 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.309+0000 7f6d93fff640 1 -- 172.21.15.162:0/1742254376 <== mon.2 v2:172.21.15.162:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f6d9c013870 con 0x7f6dac0b2c60 2023-12-02T00:28:26.327 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.310+0000 7f6db1fd0640 1 -- 172.21.15.162:0/1742254376 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dac13f690 con 0x7f6dac0b2c60 2023-12-02T00:28:26.327 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.312+0000 7f6d93fff640 1 -- 172.21.15.162:0/1742254376 <== mon.2 v2:172.21.15.162:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f6d9c018020 con 0x7f6dac0b2c60 2023-12-02T00:28:26.365 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-12-02T00:28:26.365+0000 7f1c2ce2b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-02T00:28:26.370 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-12-02T00:28:26.369+0000 7fee736b6040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-02T00:28:26.382 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.382+0000 7f6db1fd0640 1 -- 172.21.15.162:0/1742254376 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c99837cf-79d2-4b7e-9829-62697b43ff06", "id": 3} v 0) v1 -- 0x7f6dac067420 con 0x7f6dac0b2c60 2023-12-02T00:28:26.386 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.385+0000 7f6d93fff640 1 -- 172.21.15.162:0/1742254376 <== mon.2 v2:172.21.15.162:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c99837cf-79d2-4b7e-9829-62697b43ff06", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6d9c013510 con 0x7f6dac0b2c60 2023-12-02T00:28:26.386 INFO:teuthology.orchestra.run.smithi162.stdout:3 2023-12-02T00:28:26.387 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.386+0000 7f6db1fd0640 1 -- 172.21.15.162:0/1742254376 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6dac0b2c60 msgr2=0x7f6dac156640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:26.387 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.386+0000 7f6db1fd0640 1 --2- 172.21.15.162:0/1742254376 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6dac0b2c60 0x7f6dac156640 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c00efc0 tx=0x7f6d9c00c490 comp rx=0 tx=0).stop 2023-12-02T00:28:26.387 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.386+0000 7f6db1fd0640 1 -- 172.21.15.162:0/1742254376 shutdown_connections 2023-12-02T00:28:26.387 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.386+0000 7f6db1fd0640 1 --2- 172.21.15.162:0/1742254376 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6dac161810 0x7f6dac159dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:26.387 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.386+0000 7f6db1fd0640 1 --2- 172.21.15.162:0/1742254376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dac160f00 0x7f6dac158970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:26.387 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.386+0000 7f6db1fd0640 1 --2- 172.21.15.162:0/1742254376 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6dac0b2c60 0x7f6dac156640 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:26.387 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.386+0000 7f6db1fd0640 1 -- 172.21.15.162:0/1742254376 >> 172.21.15.162:0/1742254376 conn(0x7f6dac0b1a50 msgr2=0x7f6dac0af390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:26.387 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.386+0000 7f6db1fd0640 1 -- 172.21.15.162:0/1742254376 shutdown_connections 2023-12-02T00:28:26.387 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.386+0000 7f6db1fd0640 1 -- 172.21.15.162:0/1742254376 wait complete. 2023-12-02T00:28:26.397 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph osd new b745b29b-2d14-4f6f-adc7-54ce56d0d8ef 4 2023-12-02T00:28:26.439 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-12-02T00:28:26.438+0000 7f1c2ce2b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-02T00:28:26.443 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-12-02T00:28:26.442+0000 7fee736b6040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-02T00:28:26.488 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-12-02T00:28:26.487+0000 7f1c2ce2b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-02T00:28:26.491 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-12-02T00:28:26.490+0000 7fee736b6040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-02T00:28:26.539 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.538+0000 7fdac5f83640 1 Processor -- start 2023-12-02T00:28:26.540 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.539+0000 7fdac5f83640 1 -- start start 2023-12-02T00:28:26.541 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.539+0000 7fdac5f83640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdac0157960 0x7fdac015e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:26.541 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.539+0000 7fdac5f83640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdac00b3330 0x7fdac00b3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:26.541 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.539+0000 7fdac5f83640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdac00b2890 con 0x7fdac006b4e0 2023-12-02T00:28:26.541 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.539+0000 7fdac5f83640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdac00b2f10 con 0x7fdac00b3330 2023-12-02T00:28:26.541 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.539+0000 7fdac5f83640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdac00b4180 con 0x7fdac0157960 2023-12-02T00:28:26.541 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.539+0000 7fdabf7fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdac0157960 0x7fdac015e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:26.541 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.539+0000 7fdabf7fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdac0157960 0x7fdac015e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.162:54476/0 (socket says 172.21.15.162:54476) 2023-12-02T00:28:26.541 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.540+0000 7fdabf7fe640 1 -- 172.21.15.162:0/3992656991 learned_addr learned my addr 172.21.15.162:0/3992656991 (peer_addr_for_me v2:172.21.15.162:0/0) 2023-12-02T00:28:26.541 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.540+0000 7fdab7fff640 1 --2- 172.21.15.162:0/3992656991 >> v2:172.21.15.46:3300/0 conn(0x7fdac00b3330 0x7fdac00b3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:26.541 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.540+0000 7fdabf7fe640 1 -- 172.21.15.162:0/3992656991 >> v2:172.21.15.46:3300/0 conn(0x7fdac00b3330 msgr2=0x7fdac00b3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:26.542 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.540+0000 7fdabf7fe640 1 --2- 172.21.15.162:0/3992656991 >> v2:172.21.15.46:3300/0 conn(0x7fdac00b3330 0x7fdac00b3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:26.542 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.540+0000 7fdabf7fe640 1 -- 172.21.15.162:0/3992656991 >> v1:172.21.15.46:6789/0 conn(0x7fdac006b4e0 legacy=0x7fdac006cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:26.542 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.540+0000 7fdabf7fe640 1 -- 172.21.15.162:0/3992656991 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac00b4450 con 0x7fdac0157960 2023-12-02T00:28:26.542 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.540+0000 7fdab7fff640 1 --2- 172.21.15.162:0/3992656991 >> v2:172.21.15.46:3300/0 conn(0x7fdac00b3330 0x7fdac00b3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:28:26.543 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.540+0000 7fdabf7fe640 1 --2- 172.21.15.162:0/3992656991 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdac0157960 0x7fdac015e690 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdab0009870 tx=0x7fdab002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=baef23cb68744843 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:26.544 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.541+0000 7fdabeffd640 1 -- 172.21.15.162:0/3992656991 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab0030840 con 0x7fdac0157960 2023-12-02T00:28:26.544 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.541+0000 7fdabeffd640 1 -- 172.21.15.162:0/3992656991 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdab00309a0 con 0x7fdac0157960 2023-12-02T00:28:26.544 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.541+0000 7fdabeffd640 1 -- 172.21.15.162:0/3992656991 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab0030d30 con 0x7fdac0157960 2023-12-02T00:28:26.544 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.541+0000 7fdac5f83640 1 -- 172.21.15.162:0/3992656991 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdac0157960 msgr2=0x7fdac015e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:26.544 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.541+0000 7fdac5f83640 1 --2- 172.21.15.162:0/3992656991 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdac0157960 0x7fdac015e690 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdab0009870 tx=0x7fdab002b980 comp rx=0 tx=0).stop 2023-12-02T00:28:26.544 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.541+0000 7fdac5f83640 1 -- 172.21.15.162:0/3992656991 shutdown_connections 2023-12-02T00:28:26.544 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.541+0000 7fdac5f83640 1 --2- 172.21.15.162:0/3992656991 >> v2:172.21.15.46:3300/0 conn(0x7fdac00b3330 0x7fdac00b3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:26.544 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.541+0000 7fdac5f83640 1 --2- 172.21.15.162:0/3992656991 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdac0157960 0x7fdac015e690 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:26.544 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.541+0000 7fdac5f83640 1 -- 172.21.15.162:0/3992656991 >> 172.21.15.162:0/3992656991 conn(0x7fdac00b1bc0 msgr2=0x7fdac0070ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:26.544 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.541+0000 7fdac5f83640 1 -- 172.21.15.162:0/3992656991 shutdown_connections 2023-12-02T00:28:26.544 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.541+0000 7fdac5f83640 1 -- 172.21.15.162:0/3992656991 wait complete. 2023-12-02T00:28:26.544 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.543+0000 7fdac5f83640 1 Processor -- start 2023-12-02T00:28:26.544 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.543+0000 7fdac5f83640 1 -- start start 2023-12-02T00:28:26.546 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.543+0000 7fdac5f83640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdac006b4e0 0x7fdac00cc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:26.546 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.543+0000 7fdac5f83640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdac00b3330 0x7fdac00ce890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:26.546 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.543+0000 7fdac5f83640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdac0157960 0x7fdac0147190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:26.546 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.543+0000 7fdac5f83640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac00cd380 con 0x7fdac0157960 2023-12-02T00:28:26.546 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.543+0000 7fdac5f83640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac0159300 con 0x7fdac006b4e0 2023-12-02T00:28:26.546 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.543+0000 7fdac5f83640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdac00cedd0 con 0x7fdac00b3330 2023-12-02T00:28:26.546 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.543+0000 7fdab7fff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdac00b3330 0x7fdac00ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:26.546 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.543+0000 7fdabf7fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdac006b4e0 0x7fdac00cc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:26.546 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.543+0000 7fdab7fff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdac00b3330 0x7fdac00ce890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.162:54490/0 (socket says 172.21.15.162:54490) 2023-12-02T00:28:26.546 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.543+0000 7fdabf7fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdac006b4e0 0x7fdac00cc0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:59908/0 (socket says 172.21.15.162:59908) 2023-12-02T00:28:26.546 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.543+0000 7fdab7fff640 1 -- 172.21.15.162:0/1642238835 learned_addr learned my addr 172.21.15.162:0/1642238835 (peer_addr_for_me v2:172.21.15.162:0/0) 2023-12-02T00:28:26.546 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.543+0000 7fdabffff640 1 --2- 172.21.15.162:0/1642238835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdac0157960 0x7fdac0147190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:26.547 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.544+0000 7fdab7fff640 1 -- 172.21.15.162:0/1642238835 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdac006b4e0 msgr2=0x7fdac00cc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:26.547 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.544+0000 7fdab7fff640 1 --2- 172.21.15.162:0/1642238835 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdac006b4e0 0x7fdac00cc0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:26.547 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.544+0000 7fdab7fff640 1 -- 172.21.15.162:0/1642238835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdac0157960 msgr2=0x7fdac0147190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:26.547 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.544+0000 7fdab7fff640 1 --2- 172.21.15.162:0/1642238835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdac0157960 0x7fdac0147190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:26.547 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.544+0000 7fdab7fff640 1 -- 172.21.15.162:0/1642238835 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab00094d0 con 0x7fdac00b3330 2023-12-02T00:28:26.547 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.544+0000 7fdab7fff640 1 --2- 172.21.15.162:0/1642238835 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdac00b3330 0x7fdac00ce890 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdaa800d800 tx=0x7fdaa800dcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:26.547 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-12-02T00:28:26.544+0000 7f1c2ce2b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-02T00:28:26.548 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.544+0000 7fdabcff9640 1 -- 172.21.15.162:0/1642238835 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa8004030 con 0x7fdac00b3330 2023-12-02T00:28:26.549 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.544+0000 7fdabcff9640 1 -- 172.21.15.162:0/1642238835 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaa8004190 con 0x7fdac00b3330 2023-12-02T00:28:26.549 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.544+0000 7fdac5f83640 1 -- 172.21.15.162:0/1642238835 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdac01476d0 con 0x7fdac00b3330 2023-12-02T00:28:26.549 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.544+0000 7fdabcff9640 1 -- 172.21.15.162:0/1642238835 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa8004520 con 0x7fdac00b3330 2023-12-02T00:28:26.549 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.544+0000 7fdac5f83640 1 -- 172.21.15.162:0/1642238835 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdac00cccc0 con 0x7fdac00b3330 2023-12-02T00:28:26.549 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.544+0000 7fdabcff9640 1 -- 172.21.15.162:0/1642238835 <== mon.2 v2:172.21.15.162:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fdaa80099a0 con 0x7fdac00b3330 2023-12-02T00:28:26.549 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.544+0000 7fdabcff9640 1 -- 172.21.15.162:0/1642238835 <== mon.2 v2:172.21.15.162:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7fdaa8003cf0 con 0x7fdac00b3330 2023-12-02T00:28:26.549 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.545+0000 7fdac5f83640 1 -- 172.21.15.162:0/1642238835 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdac00cd5f0 con 0x7fdac00b3330 2023-12-02T00:28:26.549 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.547+0000 7fdabcff9640 1 -- 172.21.15.162:0/1642238835 <== mon.2 v2:172.21.15.162:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fdaa8009c50 con 0x7fdac00b3330 2023-12-02T00:28:26.549 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-12-02T00:28:26.544+0000 7fee736b6040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-02T00:28:26.594 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-12-02T00:28:26.593+0000 7f1c2ce2b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-02T00:28:26.595 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-12-02T00:28:26.593+0000 7fee736b6040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-02T00:28:26.620 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.619+0000 7fdac5f83640 1 -- 172.21.15.162:0/1642238835 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "osd new", "uuid": "b745b29b-2d14-4f6f-adc7-54ce56d0d8ef", "id": 4} v 0) v1 -- 0x7fdac0067420 con 0x7fdac00b3330 2023-12-02T00:28:26.625 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.624+0000 7fdabcff9640 1 -- 172.21.15.162:0/1642238835 <== mon.2 v2:172.21.15.162:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b745b29b-2d14-4f6f-adc7-54ce56d0d8ef", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fdaa8009e10 con 0x7fdac00b3330 2023-12-02T00:28:26.625 INFO:teuthology.orchestra.run.smithi162.stdout:4 2023-12-02T00:28:26.627 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.625+0000 7fdac5f83640 1 -- 172.21.15.162:0/1642238835 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdac00b3330 msgr2=0x7fdac00ce890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:26.627 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.625+0000 7fdac5f83640 1 --2- 172.21.15.162:0/1642238835 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdac00b3330 0x7fdac00ce890 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdaa800d800 tx=0x7fdaa800dcc0 comp rx=0 tx=0).stop 2023-12-02T00:28:26.627 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.625+0000 7fdac5f83640 1 -- 172.21.15.162:0/1642238835 shutdown_connections 2023-12-02T00:28:26.627 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.625+0000 7fdac5f83640 1 --2- 172.21.15.162:0/1642238835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdac0157960 0x7fdac0147190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:26.627 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.625+0000 7fdac5f83640 1 --2- 172.21.15.162:0/1642238835 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdac00b3330 0x7fdac00ce890 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:26.627 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.625+0000 7fdac5f83640 1 --2- 172.21.15.162:0/1642238835 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdac006b4e0 0x7fdac00cc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:26.627 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.625+0000 7fdac5f83640 1 -- 172.21.15.162:0/1642238835 >> 172.21.15.162:0/1642238835 conn(0x7fdac00b1bc0 msgr2=0x7fdac0155c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:26.627 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.625+0000 7fdac5f83640 1 -- 172.21.15.162:0/1642238835 shutdown_connections 2023-12-02T00:28:26.627 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.625+0000 7fdac5f83640 1 -- 172.21.15.162:0/1642238835 wait complete. 2023-12-02T00:28:26.637 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph osd new b90c10b2-c5ff-4fe7-964b-419d1bb28bee 5 2023-12-02T00:28:26.737 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.736+0000 7ff59de6e640 1 Processor -- start 2023-12-02T00:28:26.737 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.736+0000 7ff59de6e640 1 -- start start 2023-12-02T00:28:26.746 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.737+0000 7ff59de6e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff598161810 0x7ff59815d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:26.746 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.737+0000 7ff59de6e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff5980b2e30 0x7ff59806de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:26.746 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.737+0000 7ff59de6e640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff59806a570 con 0x7ff598160f00 2023-12-02T00:28:26.746 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.737+0000 7ff59de6e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff59806cc50 con 0x7ff5980b2e30 2023-12-02T00:28:26.746 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.737+0000 7ff59de6e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff59815e1d0 con 0x7ff598161810 2023-12-02T00:28:26.746 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.737+0000 7ff5977fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff598161810 0x7ff59815d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:26.746 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.737+0000 7ff597fff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff5980b2e30 0x7ff59806de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:26.746 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.737+0000 7ff5977fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff598161810 0x7ff59815d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.162:54498/0 (socket says 172.21.15.162:54498) 2023-12-02T00:28:26.746 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.737+0000 7ff5977fe640 1 -- 172.21.15.162:0/4278628241 learned_addr learned my addr 172.21.15.162:0/4278628241 (peer_addr_for_me v2:172.21.15.162:0/0) 2023-12-02T00:28:26.746 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.737+0000 7ff5977fe640 1 -- 172.21.15.162:0/4278628241 >> v2:172.21.15.162:3300/0 conn(0x7ff5980b2e30 msgr2=0x7ff59806de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:26.746 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.737+0000 7ff5977fe640 1 --2- 172.21.15.162:0/4278628241 >> v2:172.21.15.162:3300/0 conn(0x7ff5980b2e30 0x7ff59806de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:26.746 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.737+0000 7ff5977fe640 1 -- 172.21.15.162:0/4278628241 >> v1:172.21.15.46:6789/0 conn(0x7ff598160f00 legacy=0x7ff5980b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:26.747 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.737+0000 7ff5977fe640 1 -- 172.21.15.162:0/4278628241 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff59815e4b0 con 0x7ff598161810 2023-12-02T00:28:26.747 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.737+0000 7ff597fff640 1 --2- 172.21.15.162:0/4278628241 >> v2:172.21.15.162:3300/0 conn(0x7ff5980b2e30 0x7ff59806de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:28:26.747 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.738+0000 7ff5977fe640 1 --2- 172.21.15.162:0/4278628241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff598161810 0x7ff59815d8a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff58c009870 tx=0x7ff58c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51b96f1d03774fcf server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:26.747 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.738+0000 7ff5967fc640 1 -- 172.21.15.162:0/4278628241 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c030840 con 0x7ff598161810 2023-12-02T00:28:26.747 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.738+0000 7ff5967fc640 1 -- 172.21.15.162:0/4278628241 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff58c0309a0 con 0x7ff598161810 2023-12-02T00:28:26.747 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.738+0000 7ff5967fc640 1 -- 172.21.15.162:0/4278628241 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c030d30 con 0x7ff598161810 2023-12-02T00:28:26.747 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.739+0000 7ff59de6e640 1 -- 172.21.15.162:0/4278628241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff598161810 msgr2=0x7ff59815d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:26.747 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.739+0000 7ff59de6e640 1 --2- 172.21.15.162:0/4278628241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff598161810 0x7ff59815d8a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff58c009870 tx=0x7ff58c02b980 comp rx=0 tx=0).stop 2023-12-02T00:28:26.747 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.739+0000 7ff59de6e640 1 -- 172.21.15.162:0/4278628241 shutdown_connections 2023-12-02T00:28:26.747 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.739+0000 7ff59de6e640 1 --2- 172.21.15.162:0/4278628241 >> v2:172.21.15.162:3300/0 conn(0x7ff5980b2e30 0x7ff59806de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:26.747 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.739+0000 7ff59de6e640 1 --2- 172.21.15.162:0/4278628241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff598161810 0x7ff59815d8a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:26.747 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.739+0000 7ff59de6e640 1 -- 172.21.15.162:0/4278628241 >> 172.21.15.162:0/4278628241 conn(0x7ff5980b1a50 msgr2=0x7ff5980cf670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:26.748 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.739+0000 7ff59de6e640 1 -- 172.21.15.162:0/4278628241 shutdown_connections 2023-12-02T00:28:26.748 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.739+0000 7ff59de6e640 1 -- 172.21.15.162:0/4278628241 wait complete. 2023-12-02T00:28:26.748 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.740+0000 7ff59de6e640 1 Processor -- start 2023-12-02T00:28:26.748 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.740+0000 7ff59de6e640 1 -- start start 2023-12-02T00:28:26.748 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.741+0000 7ff59de6e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5980b2e30 0x7ff598135530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:26.748 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.741+0000 7ff59de6e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff598160f00 0x7ff5980cd610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:26.748 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.741+0000 7ff59de6e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff598161810 0x7ff5980cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:26.748 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.741+0000 7ff59de6e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5981e8610 con 0x7ff598160f00 2023-12-02T00:28:26.748 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.741+0000 7ff59de6e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff598133410 con 0x7ff598161810 2023-12-02T00:28:26.748 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.741+0000 7ff59de6e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5981570b0 con 0x7ff5980b2e30 2023-12-02T00:28:26.748 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.741+0000 7ff5977fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5980b2e30 0x7ff598135530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:26.748 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.741+0000 7ff5977fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5980b2e30 0x7ff598135530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.162:54504/0 (socket says 172.21.15.162:54504) 2023-12-02T00:28:26.749 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.741+0000 7ff5977fe640 1 -- 172.21.15.162:0/1082091501 learned_addr learned my addr 172.21.15.162:0/1082091501 (peer_addr_for_me v2:172.21.15.162:0/0) 2023-12-02T00:28:26.749 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.741+0000 7ff597fff640 1 --2- 172.21.15.162:0/1082091501 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff598161810 0x7ff5980cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:26.749 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.741+0000 7ff596ffd640 1 --2- 172.21.15.162:0/1082091501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff598160f00 0x7ff5980cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:26.749 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.741+0000 7ff5977fe640 1 -- 172.21.15.162:0/1082091501 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff598161810 msgr2=0x7ff5980cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:26.749 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.741+0000 7ff5977fe640 1 --2- 172.21.15.162:0/1082091501 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff598161810 0x7ff5980cdb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:26.749 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.741+0000 7ff5977fe640 1 -- 172.21.15.162:0/1082091501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff598160f00 msgr2=0x7ff5980cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:26.749 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.741+0000 7ff5977fe640 1 --2- 172.21.15.162:0/1082091501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff598160f00 0x7ff5980cd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:26.749 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.741+0000 7ff5977fe640 1 -- 172.21.15.162:0/1082091501 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff590009a70 con 0x7ff5980b2e30 2023-12-02T00:28:26.749 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.741+0000 7ff5977fe640 1 --2- 172.21.15.162:0/1082091501 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5980b2e30 0x7ff598135530 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff58c02b9c0 tx=0x7ff58c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:26.749 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.742+0000 7ff57ffff640 1 -- 172.21.15.162:0/1082091501 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c03e070 con 0x7ff5980b2e30 2023-12-02T00:28:26.749 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.742+0000 7ff57ffff640 1 -- 172.21.15.162:0/1082091501 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff58c039a10 con 0x7ff5980b2e30 2023-12-02T00:28:26.749 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.742+0000 7ff57ffff640 1 -- 172.21.15.162:0/1082091501 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c039dc0 con 0x7ff5980b2e30 2023-12-02T00:28:26.749 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.742+0000 7ff59de6e640 1 -- 172.21.15.162:0/1082091501 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff58c0094d0 con 0x7ff5980b2e30 2023-12-02T00:28:26.750 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.742+0000 7ff59de6e640 1 -- 172.21.15.162:0/1082091501 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff598132d50 con 0x7ff5980b2e30 2023-12-02T00:28:26.750 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.742+0000 7ff57ffff640 1 -- 172.21.15.162:0/1082091501 <== mon.2 v2:172.21.15.162:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff58c04b700 con 0x7ff5980b2e30 2023-12-02T00:28:26.750 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.742+0000 7ff57ffff640 1 -- 172.21.15.162:0/1082091501 <== mon.2 v2:172.21.15.162:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7ff58c042b50 con 0x7ff5980b2e30 2023-12-02T00:28:26.750 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.743+0000 7ff59de6e640 1 -- 172.21.15.162:0/1082091501 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff598067420 con 0x7ff5980b2e30 2023-12-02T00:28:26.750 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.746+0000 7ff57ffff640 1 -- 172.21.15.162:0/1082091501 <== mon.2 v2:172.21.15.162:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7ff58c034070 con 0x7ff5980b2e30 2023-12-02T00:28:26.819 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.818+0000 7ff59de6e640 1 -- 172.21.15.162:0/1082091501 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "osd new", "uuid": "b90c10b2-c5ff-4fe7-964b-419d1bb28bee", "id": 5} v 0) v1 -- 0x7ff5980d0170 con 0x7ff5980b2e30 2023-12-02T00:28:26.822 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.822+0000 7ff57ffff640 1 -- 172.21.15.162:0/1082091501 <== mon.2 v2:172.21.15.162:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b90c10b2-c5ff-4fe7-964b-419d1bb28bee", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff58c042db0 con 0x7ff5980b2e30 2023-12-02T00:28:26.823 INFO:teuthology.orchestra.run.smithi162.stdout:5 2023-12-02T00:28:26.824 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.823+0000 7ff59de6e640 1 -- 172.21.15.162:0/1082091501 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5980b2e30 msgr2=0x7ff598135530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:26.824 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.823+0000 7ff59de6e640 1 --2- 172.21.15.162:0/1082091501 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5980b2e30 0x7ff598135530 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff58c02b9c0 tx=0x7ff58c0046c0 comp rx=0 tx=0).stop 2023-12-02T00:28:26.824 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.823+0000 7ff59de6e640 1 -- 172.21.15.162:0/1082091501 shutdown_connections 2023-12-02T00:28:26.824 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.823+0000 7ff59de6e640 1 --2- 172.21.15.162:0/1082091501 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff598161810 0x7ff5980cdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:26.824 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.823+0000 7ff59de6e640 1 --2- 172.21.15.162:0/1082091501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff598160f00 0x7ff5980cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:26.824 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.823+0000 7ff59de6e640 1 --2- 172.21.15.162:0/1082091501 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5980b2e30 0x7ff598135530 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:26.824 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.823+0000 7ff59de6e640 1 -- 172.21.15.162:0/1082091501 >> 172.21.15.162:0/1082091501 conn(0x7ff5980b1a50 msgr2=0x7ff59806e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:26.824 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.823+0000 7ff59de6e640 1 -- 172.21.15.162:0/1082091501 shutdown_connections 2023-12-02T00:28:26.824 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.823+0000 7ff59de6e640 1 -- 172.21.15.162:0/1082091501 wait complete. 2023-12-02T00:28:26.834 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph osd new 2294203e-5c5a-4bec-9dcd-c5a576ff3cc9 6 2023-12-02T00:28:26.936 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.935+0000 7efcec87c640 1 Processor -- start 2023-12-02T00:28:26.936 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.935+0000 7efcec87c640 1 -- start start 2023-12-02T00:28:26.938 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.936+0000 7efcec87c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7efce813d060 0x7efce806d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:26.938 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.936+0000 7efcec87c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7efce815e3f0 0x7efce80ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:26.938 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.936+0000 7efcec87c640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efce815e810 con 0x7efce806daa0 2023-12-02T00:28:26.938 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.936+0000 7efcec87c640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efce806df50 con 0x7efce815e3f0 2023-12-02T00:28:26.938 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.936+0000 7efcec87c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efce813d440 con 0x7efce813d060 2023-12-02T00:28:26.938 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.936+0000 7efce659c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7efce813d060 0x7efce806d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:26.938 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.936+0000 7efce5d9b640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7efce806daa0 0x7efce815dce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:58856/0 (socket says 172.21.15.162:58856) 2023-12-02T00:28:26.938 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.936+0000 7efce5d9b640 1 -- 172.21.15.162:0/402485423 learned_addr learned my addr 172.21.15.162:0/402485423 (peer_addr_for_me v1:172.21.15.162:0/0) 2023-12-02T00:28:26.938 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.936+0000 7efce6d9d640 1 --2- 172.21.15.162:0/402485423 >> v2:172.21.15.46:3300/0 conn(0x7efce815e3f0 0x7efce80ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:26.938 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.936+0000 7efce559a640 1 -- 172.21.15.162:0/402485423 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2322449905 0 0) 0x7efce815e810 con 0x7efce806daa0 2023-12-02T00:28:26.938 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.937+0000 7efce559a640 1 -- 172.21.15.162:0/402485423 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcd4003520 con 0x7efce806daa0 2023-12-02T00:28:26.938 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.937+0000 7efce659c640 1 -- 172.21.15.162:0/402485423 >> v2:172.21.15.46:3300/0 conn(0x7efce815e3f0 msgr2=0x7efce80ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:26.938 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.937+0000 7efce659c640 1 --2- 172.21.15.162:0/402485423 >> v2:172.21.15.46:3300/0 conn(0x7efce815e3f0 0x7efce80ceaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:26.939 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.937+0000 7efce659c640 1 -- 172.21.15.162:0/402485423 >> v1:172.21.15.162:6789/0 conn(0x7efce806daa0 legacy=0x7efce815dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:26.939 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.937+0000 7efce659c640 1 -- 172.21.15.162:0/402485423 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce815eaa0 con 0x7efce813d060 2023-12-02T00:28:26.939 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.937+0000 7efce659c640 1 --2- 172.21.15.162:0/402485423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efce813d060 0x7efce806d560 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7efce8072760 tx=0x7efcd802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=249534dd6d2f07b4 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:26.940 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.937+0000 7efce559a640 1 -- 172.21.15.162:0/402485423 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd8031830 con 0x7efce813d060 2023-12-02T00:28:26.940 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.937+0000 7efce559a640 1 -- 172.21.15.162:0/402485423 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd8031990 con 0x7efce813d060 2023-12-02T00:28:26.940 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.937+0000 7efce559a640 1 -- 172.21.15.162:0/402485423 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd8031d20 con 0x7efce813d060 2023-12-02T00:28:26.940 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.938+0000 7efcec87c640 1 -- 172.21.15.162:0/402485423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efce813d060 msgr2=0x7efce806d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:26.940 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.938+0000 7efcec87c640 1 --2- 172.21.15.162:0/402485423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efce813d060 0x7efce806d560 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7efce8072760 tx=0x7efcd802b8d0 comp rx=0 tx=0).stop 2023-12-02T00:28:26.940 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.938+0000 7efcec87c640 1 -- 172.21.15.162:0/402485423 shutdown_connections 2023-12-02T00:28:26.940 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.938+0000 7efcec87c640 1 --2- 172.21.15.162:0/402485423 >> v2:172.21.15.46:3300/0 conn(0x7efce815e3f0 0x7efce80ceaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:26.940 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.938+0000 7efcec87c640 1 --2- 172.21.15.162:0/402485423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efce813d060 0x7efce806d560 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:26.940 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.938+0000 7efcec87c640 1 -- 172.21.15.162:0/402485423 >> 172.21.15.162:0/402485423 conn(0x7efce8066270 msgr2=0x7efce8066650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:26.940 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.938+0000 7efcec87c640 1 -- 172.21.15.162:0/402485423 shutdown_connections 2023-12-02T00:28:26.940 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.938+0000 7efcec87c640 1 -- 172.21.15.162:0/402485423 wait complete. 2023-12-02T00:28:26.945 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.940+0000 7efcec87c640 1 Processor -- start 2023-12-02T00:28:26.945 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.940+0000 7efcec87c640 1 -- start start 2023-12-02T00:28:26.945 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.940+0000 7efcec87c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efce806daa0 0x7efce81561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:26.945 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.940+0000 7efcec87c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efce813d060 0x7efce813b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:26.946 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.940+0000 7efcec87c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efce815e3f0 0x7efce81f3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:26.946 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.940+0000 7efcec87c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce81574f0 con 0x7efce806daa0 2023-12-02T00:28:26.946 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.940+0000 7efcec87c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce813bee0 con 0x7efce813d060 2023-12-02T00:28:26.946 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.940+0000 7efcec87c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efce8156870 con 0x7efce815e3f0 2023-12-02T00:28:26.946 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.940+0000 7efce5d9b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efce813d060 0x7efce813b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:26.946 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.940+0000 7efce6d9d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efce815e3f0 0x7efce81f3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:26.946 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.940+0000 7efce5d9b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efce813d060 0x7efce813b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:59942/0 (socket says 172.21.15.162:59942) 2023-12-02T00:28:26.946 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.941+0000 7efce5d9b640 1 -- 172.21.15.162:0/1984857592 learned_addr learned my addr 172.21.15.162:0/1984857592 (peer_addr_for_me v2:172.21.15.162:0/0) 2023-12-02T00:28:26.946 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.941+0000 7efce6d9d640 1 --2- 172.21.15.162:0/1984857592 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efce815e3f0 0x7efce81f3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.162:54510/0 (socket says 172.21.15.162:54510) 2023-12-02T00:28:26.946 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.941+0000 7efce659c640 1 --2- 172.21.15.162:0/1984857592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efce806daa0 0x7efce81561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:26.946 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.941+0000 7efce5d9b640 1 -- 172.21.15.162:0/1984857592 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efce815e3f0 msgr2=0x7efce81f3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:26.946 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.941+0000 7efce5d9b640 1 --2- 172.21.15.162:0/1984857592 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efce815e3f0 0x7efce81f3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:26.947 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.941+0000 7efce5d9b640 1 -- 172.21.15.162:0/1984857592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efce806daa0 msgr2=0x7efce81561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:26.947 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.941+0000 7efce5d9b640 1 --2- 172.21.15.162:0/1984857592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efce806daa0 0x7efce81561b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:26.947 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.941+0000 7efce5d9b640 1 -- 172.21.15.162:0/1984857592 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd8009400 con 0x7efce813d060 2023-12-02T00:28:26.947 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.941+0000 7efce6d9d640 1 --2- 172.21.15.162:0/1984857592 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efce815e3f0 0x7efce81f3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:28:26.947 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.941+0000 7efce5d9b640 1 --2- 172.21.15.162:0/1984857592 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efce813d060 0x7efce813b9a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7efcd000c500 tx=0x7efcd000c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:26.947 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.941+0000 7efcdeffd640 1 -- 172.21.15.162:0/1984857592 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd0005ce0 con 0x7efce813d060 2023-12-02T00:28:26.947 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.941+0000 7efcdeffd640 1 -- 172.21.15.162:0/1984857592 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd0005e40 con 0x7efce813d060 2023-12-02T00:28:26.947 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.941+0000 7efcec87c640 1 -- 172.21.15.162:0/1984857592 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efce8156b50 con 0x7efce813d060 2023-12-02T00:28:26.947 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.941+0000 7efcdeffd640 1 -- 172.21.15.162:0/1984857592 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd0007330 con 0x7efce813d060 2023-12-02T00:28:26.947 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.941+0000 7efcec87c640 1 -- 172.21.15.162:0/1984857592 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efce8157110 con 0x7efce813d060 2023-12-02T00:28:26.947 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.941+0000 7efcdeffd640 1 -- 172.21.15.162:0/1984857592 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7efcd001c440 con 0x7efce813d060 2023-12-02T00:28:26.947 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.941+0000 7efcdeffd640 1 -- 172.21.15.162:0/1984857592 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7efcd0012a90 con 0x7efce813d060 2023-12-02T00:28:26.947 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.942+0000 7efcec87c640 1 -- 172.21.15.162:0/1984857592 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efce8157630 con 0x7efce813d060 2023-12-02T00:28:26.947 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:26.944+0000 7efcdeffd640 1 -- 172.21.15.162:0/1984857592 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7efce8157630 con 0x7efce813d060 2023-12-02T00:28:26.997 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-12-02T00:28:26.996+0000 7fee736b6040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-02T00:28:27.005 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-12-02T00:28:27.004+0000 7f1c2ce2b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-02T00:28:27.016 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.015+0000 7efcec87c640 1 -- 172.21.15.162:0/1984857592 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2294203e-5c5a-4bec-9dcd-c5a576ff3cc9", "id": 6} v 0) v1 -- 0x7efce8067420 con 0x7efce813d060 2023-12-02T00:28:27.019 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.018+0000 7efcdeffd640 1 -- 172.21.15.162:0/1984857592 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2294203e-5c5a-4bec-9dcd-c5a576ff3cc9", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7efcd0017070 con 0x7efce813d060 2023-12-02T00:28:27.019 INFO:teuthology.orchestra.run.smithi162.stdout:6 2023-12-02T00:28:27.020 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.019+0000 7efcec87c640 1 -- 172.21.15.162:0/1984857592 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efce813d060 msgr2=0x7efce813b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:27.020 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.019+0000 7efcec87c640 1 --2- 172.21.15.162:0/1984857592 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efce813d060 0x7efce813b9a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7efcd000c500 tx=0x7efcd000c9c0 comp rx=0 tx=0).stop 2023-12-02T00:28:27.020 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.019+0000 7efcec87c640 1 -- 172.21.15.162:0/1984857592 shutdown_connections 2023-12-02T00:28:27.020 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.019+0000 7efcec87c640 1 --2- 172.21.15.162:0/1984857592 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efce815e3f0 0x7efce81f3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:27.020 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.019+0000 7efcec87c640 1 --2- 172.21.15.162:0/1984857592 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efce813d060 0x7efce813b9a0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:27.020 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.019+0000 7efcec87c640 1 --2- 172.21.15.162:0/1984857592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efce806daa0 0x7efce81561b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:27.020 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.019+0000 7efcec87c640 1 -- 172.21.15.162:0/1984857592 >> 172.21.15.162:0/1984857592 conn(0x7efce8066270 msgr2=0x7efce80b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:27.021 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.019+0000 7efcec87c640 1 -- 172.21.15.162:0/1984857592 shutdown_connections 2023-12-02T00:28:27.021 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.019+0000 7efcec87c640 1 -- 172.21.15.162:0/1984857592 wait complete. 2023-12-02T00:28:27.030 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph osd new ca4e6592-a26f-4360-981b-3bcad8251164 7 2023-12-02T00:28:27.131 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.130+0000 7ff4a9ea6640 1 Processor -- start 2023-12-02T00:28:27.131 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.130+0000 7ff4a9ea6640 1 -- start start 2023-12-02T00:28:27.133 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.131+0000 7ff4a9ea6640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff4a406eee0 0x7ff4a415b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:27.133 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.131+0000 7ff4a9ea6640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4a413e470 con 0x7ff4a413e090 2023-12-02T00:28:27.133 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.131+0000 7ff4a9ea6640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4a415c2f0 con 0x7ff4a415f830 2023-12-02T00:28:27.133 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.131+0000 7ff4a9ea6640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4a415c5d0 con 0x7ff4a406eee0 2023-12-02T00:28:27.133 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.131+0000 7ff4a3fff640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7ff4a415f830 0x7ff4a415bb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:58870/0 (socket says 172.21.15.162:58870) 2023-12-02T00:28:27.133 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.131+0000 7ff4a3fff640 1 -- 172.21.15.162:0/672063924 learned_addr learned my addr 172.21.15.162:0/672063924 (peer_addr_for_me v1:172.21.15.162:0/0) 2023-12-02T00:28:27.133 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.131+0000 7ff4a2ffd640 1 --2- 172.21.15.162:0/672063924 >> v2:172.21.15.46:3300/0 conn(0x7ff4a406eee0 0x7ff4a415b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:27.133 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.131+0000 7ff4a27fc640 1 -- 172.21.15.162:0/672063924 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2587506607 0 0) 0x7ff4a415c2f0 con 0x7ff4a415f830 2023-12-02T00:28:27.133 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.131+0000 7ff4a27fc640 1 -- 172.21.15.162:0/672063924 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff48c003520 con 0x7ff4a415f830 2023-12-02T00:28:27.133 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.131+0000 7ff4a27fc640 1 -- 172.21.15.162:0/672063924 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 119534694 0 0) 0x7ff4a413e470 con 0x7ff4a413e090 2023-12-02T00:28:27.134 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.131+0000 7ff4a27fc640 1 -- 172.21.15.162:0/672063924 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4a415c2f0 con 0x7ff4a413e090 2023-12-02T00:28:27.134 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.131+0000 7ff4a2ffd640 1 -- 172.21.15.162:0/672063924 >> v1:172.21.15.162:6789/0 conn(0x7ff4a415f830 legacy=0x7ff4a415bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:27.134 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.131+0000 7ff4a27fc640 1 -- 172.21.15.162:0/672063924 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4065405233 0 0) 0x7ff48c003520 con 0x7ff4a415f830 2023-12-02T00:28:27.134 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.131+0000 7ff4a2ffd640 1 -- 172.21.15.162:0/672063924 >> v1:172.21.15.46:6789/0 conn(0x7ff4a413e090 legacy=0x7ff4a406e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:27.134 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.131+0000 7ff4a2ffd640 1 -- 172.21.15.162:0/672063924 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a415ed50 con 0x7ff4a406eee0 2023-12-02T00:28:27.134 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.132+0000 7ff4a2ffd640 1 --2- 172.21.15.162:0/672063924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a406eee0 0x7ff4a415b5c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff498009720 tx=0x7ff49802b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eca69055a407cea server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:27.134 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.132+0000 7ff4a27fc640 1 -- 172.21.15.162:0/672063924 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c0019f0 con 0x7ff4a406eee0 2023-12-02T00:28:27.134 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.132+0000 7ff4a27fc640 1 -- 172.21.15.162:0/672063924 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff49802e050 con 0x7ff4a406eee0 2023-12-02T00:28:27.134 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.132+0000 7ff4a27fc640 1 -- 172.21.15.162:0/672063924 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff498030730 con 0x7ff4a406eee0 2023-12-02T00:28:27.135 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.133+0000 7ff4a9ea6640 1 -- 172.21.15.162:0/672063924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a406eee0 msgr2=0x7ff4a415b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:27.135 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.133+0000 7ff4a9ea6640 1 --2- 172.21.15.162:0/672063924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a406eee0 0x7ff4a415b5c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff498009720 tx=0x7ff49802b730 comp rx=0 tx=0).stop 2023-12-02T00:28:27.135 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.133+0000 7ff4a9ea6640 1 -- 172.21.15.162:0/672063924 shutdown_connections 2023-12-02T00:28:27.135 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.133+0000 7ff4a9ea6640 1 --2- 172.21.15.162:0/672063924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a406eee0 0x7ff4a415b5c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:27.135 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.133+0000 7ff4a9ea6640 1 -- 172.21.15.162:0/672063924 >> 172.21.15.162:0/672063924 conn(0x7ff4a406a8c0 msgr2=0x7ff4a406acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:27.135 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.133+0000 7ff4a9ea6640 1 -- 172.21.15.162:0/672063924 shutdown_connections 2023-12-02T00:28:27.136 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.133+0000 7ff4a9ea6640 1 -- 172.21.15.162:0/672063924 wait complete. 2023-12-02T00:28:27.136 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.134+0000 7ff4a9ea6640 1 Processor -- start 2023-12-02T00:28:27.136 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.134+0000 7ff4a9ea6640 1 -- start start 2023-12-02T00:28:27.138 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.135+0000 7ff4a9ea6640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4a406eee0 0x7ff4a4153600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:27.138 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.135+0000 7ff4a9ea6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a413e090 0x7ff4a4139980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:27.138 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.135+0000 7ff4a9ea6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4a415f830 0x7ff4a413c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:27.138 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.135+0000 7ff4a9ea6640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a4154910 con 0x7ff4a413e090 2023-12-02T00:28:27.138 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.135+0000 7ff4a9ea6640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a4140430 con 0x7ff4a406eee0 2023-12-02T00:28:27.139 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.135+0000 7ff4a9ea6640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a4153d00 con 0x7ff4a415f830 2023-12-02T00:28:27.139 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.135+0000 7ff4a3fff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4a415f830 0x7ff4a413c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:27.139 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.135+0000 7ff4a37fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4a406eee0 0x7ff4a4153600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:27.139 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.135+0000 7ff4a3fff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4a415f830 0x7ff4a413c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.162:54516/0 (socket says 172.21.15.162:54516) 2023-12-02T00:28:27.139 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.135+0000 7ff4a37fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4a406eee0 0x7ff4a4153600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:59948/0 (socket says 172.21.15.162:59948) 2023-12-02T00:28:27.139 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.135+0000 7ff4a3fff640 1 -- 172.21.15.162:0/587605434 learned_addr learned my addr 172.21.15.162:0/587605434 (peer_addr_for_me v2:172.21.15.162:0/0) 2023-12-02T00:28:27.139 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.135+0000 7ff4a2ffd640 1 --2- 172.21.15.162:0/587605434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a413e090 0x7ff4a4139980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:27.139 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.135+0000 7ff4a3fff640 1 -- 172.21.15.162:0/587605434 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4a406eee0 msgr2=0x7ff4a4153600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:27.139 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.135+0000 7ff4a3fff640 1 --2- 172.21.15.162:0/587605434 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4a406eee0 0x7ff4a4153600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:27.139 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.135+0000 7ff4a3fff640 1 -- 172.21.15.162:0/587605434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a413e090 msgr2=0x7ff4a4139980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:27.139 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.135+0000 7ff4a3fff640 1 --2- 172.21.15.162:0/587605434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a413e090 0x7ff4a4139980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:27.139 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.135+0000 7ff4a3fff640 1 -- 172.21.15.162:0/587605434 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff498009400 con 0x7ff4a415f830 2023-12-02T00:28:27.139 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.135+0000 7ff4a2ffd640 1 --2- 172.21.15.162:0/587605434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a413e090 0x7ff4a4139980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:28:27.139 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.136+0000 7ff4a3fff640 1 --2- 172.21.15.162:0/587605434 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4a415f830 0x7ff4a413c6d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff49c00caf0 tx=0x7ff49c0021c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:27.140 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.136+0000 7ff48bfff640 1 -- 172.21.15.162:0/587605434 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c00a6d0 con 0x7ff4a415f830 2023-12-02T00:28:27.140 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.136+0000 7ff48bfff640 1 -- 172.21.15.162:0/587605434 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff49c00a850 con 0x7ff4a415f830 2023-12-02T00:28:27.140 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.136+0000 7ff4a9ea6640 1 -- 172.21.15.162:0/587605434 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4a4153fe0 con 0x7ff4a415f830 2023-12-02T00:28:27.140 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.136+0000 7ff48bfff640 1 -- 172.21.15.162:0/587605434 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c00ac40 con 0x7ff4a415f830 2023-12-02T00:28:27.140 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.136+0000 7ff4a9ea6640 1 -- 172.21.15.162:0/587605434 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4a41545a0 con 0x7ff4a415f830 2023-12-02T00:28:27.140 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.136+0000 7ff48bfff640 1 -- 172.21.15.162:0/587605434 <== mon.2 v2:172.21.15.162:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff49c0073d0 con 0x7ff4a415f830 2023-12-02T00:28:27.140 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.136+0000 7ff48bfff640 1 -- 172.21.15.162:0/587605434 <== mon.2 v2:172.21.15.162:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7ff49c012c40 con 0x7ff4a415f830 2023-12-02T00:28:27.140 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.137+0000 7ff4a9ea6640 1 -- 172.21.15.162:0/587605434 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4a415f2d0 con 0x7ff4a415f830 2023-12-02T00:28:27.140 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-12-02T00:28:27.137+0000 7fee736b6040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-02T00:28:27.141 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.141+0000 7ff48bfff640 1 -- 172.21.15.162:0/587605434 <== mon.2 v2:172.21.15.162:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7ff49c018070 con 0x7ff4a415f830 2023-12-02T00:28:27.151 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-12-02T00:28:27.150+0000 7f1c2ce2b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-02T00:28:27.212 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.211+0000 7ff4a9ea6640 1 -- 172.21.15.162:0/587605434 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ca4e6592-a26f-4360-981b-3bcad8251164", "id": 7} v 0) v1 -- 0x7ff4a4154a80 con 0x7ff4a415f830 2023-12-02T00:28:27.218 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.218+0000 7ff48bfff640 1 -- 172.21.15.162:0/587605434 <== mon.2 v2:172.21.15.162:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ca4e6592-a26f-4360-981b-3bcad8251164", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff49c00ada0 con 0x7ff4a415f830 2023-12-02T00:28:27.219 INFO:teuthology.orchestra.run.smithi162.stdout:7 2023-12-02T00:28:27.220 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.219+0000 7ff4a9ea6640 1 -- 172.21.15.162:0/587605434 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4a415f830 msgr2=0x7ff4a413c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:27.220 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.219+0000 7ff4a9ea6640 1 --2- 172.21.15.162:0/587605434 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4a415f830 0x7ff4a413c6d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff49c00caf0 tx=0x7ff49c0021c0 comp rx=0 tx=0).stop 2023-12-02T00:28:27.220 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.219+0000 7ff4a9ea6640 1 -- 172.21.15.162:0/587605434 shutdown_connections 2023-12-02T00:28:27.220 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.219+0000 7ff4a9ea6640 1 --2- 172.21.15.162:0/587605434 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4a415f830 0x7ff4a413c6d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:27.221 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.219+0000 7ff4a9ea6640 1 --2- 172.21.15.162:0/587605434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a413e090 0x7ff4a4139980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:27.221 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.219+0000 7ff4a9ea6640 1 --2- 172.21.15.162:0/587605434 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4a406eee0 0x7ff4a4153600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:27.221 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.219+0000 7ff4a9ea6640 1 -- 172.21.15.162:0/587605434 >> 172.21.15.162:0/587605434 conn(0x7ff4a406a8c0 msgr2=0x7ff4a40b1160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:27.221 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.219+0000 7ff4a9ea6640 1 -- 172.21.15.162:0/587605434 shutdown_connections 2023-12-02T00:28:27.221 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:28:27.219+0000 7ff4a9ea6640 1 -- 172.21.15.162:0/587605434 wait complete. 2023-12-02T00:28:27.230 INFO:tasks.ceph.osd.0:Restarting daemon 2023-12-02T00:28:27.230 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-12-02T00:28:27.233 INFO:tasks.ceph.osd.0:Started 2023-12-02T00:28:27.233 INFO:tasks.ceph.osd.1:Restarting daemon 2023-12-02T00:28:27.233 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-12-02T00:28:27.235 INFO:tasks.ceph.osd.1:Started 2023-12-02T00:28:27.235 INFO:tasks.ceph.osd.2:Restarting daemon 2023-12-02T00:28:27.235 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-12-02T00:28:27.238 INFO:tasks.ceph.osd.2:Started 2023-12-02T00:28:27.238 INFO:tasks.ceph.osd.3:Restarting daemon 2023-12-02T00:28:27.238 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-12-02T00:28:27.240 INFO:tasks.ceph.osd.3:Started 2023-12-02T00:28:27.241 INFO:tasks.ceph.osd.4:Restarting daemon 2023-12-02T00:28:27.241 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-12-02T00:28:27.243 INFO:tasks.ceph.osd.4:Started 2023-12-02T00:28:27.243 INFO:tasks.ceph.osd.5:Restarting daemon 2023-12-02T00:28:27.243 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-12-02T00:28:27.245 INFO:tasks.ceph.osd.5:Started 2023-12-02T00:28:27.245 INFO:tasks.ceph.osd.6:Restarting daemon 2023-12-02T00:28:27.245 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-12-02T00:28:27.247 INFO:tasks.ceph.osd.6:Started 2023-12-02T00:28:27.247 INFO:tasks.ceph.osd.7:Restarting daemon 2023-12-02T00:28:27.247 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-12-02T00:28:27.249 INFO:tasks.ceph.osd.7:Started 2023-12-02T00:28:27.249 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:28:27.259 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-12-02T00:28:27.258+0000 7fee736b6040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-02T00:28:27.279 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-12-02T00:28:27.278+0000 7f1c2ce2b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-02T00:28:27.333 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-12-02T00:28:27.332+0000 7fee736b6040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-02T00:28:27.349 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-12-02T00:28:27.348+0000 7f1c2ce2b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-02T00:28:27.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.382+0000 7fd89f426640 1 Processor -- start 2023-12-02T00:28:27.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.382+0000 7fd89f426640 1 -- start start 2023-12-02T00:28:27.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.383+0000 7fd89f426640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd8980b4b60 0x7fd898160fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:27.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.383+0000 7fd89f426640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd898071550 0x7fd8980b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:27.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.383+0000 7fd89f426640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8980b3520 con 0x7fd89806a4d0 2023-12-02T00:28:27.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.383+0000 7fd89f426640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8980b3220 con 0x7fd898071550 2023-12-02T00:28:27.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.383+0000 7fd89f426640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd898161810 con 0x7fd8980b4b60 2023-12-02T00:28:27.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.383+0000 7fd89c9c1640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd898071550 0x7fd8980b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:27.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.383+0000 7fd89c9c1640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd898071550 0x7fd8980b39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55966/0 (socket says 172.21.15.46:55966) 2023-12-02T00:28:27.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.383+0000 7fd89d9c3640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd89806a4d0 0x7fd89815d980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54422/0 (socket says 172.21.15.46:54422) 2023-12-02T00:28:27.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.383+0000 7fd89d9c3640 1 -- 172.21.15.46:0/52303759 learned_addr learned my addr 172.21.15.46:0/52303759 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:28:27.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.383+0000 7fd89d1c2640 1 --2- 172.21.15.46:0/52303759 >> v2:172.21.15.162:3300/0 conn(0x7fd8980b4b60 0x7fd898160fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:27.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.383+0000 7fd897fff640 1 -- 172.21.15.46:0/52303759 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2759054364 0 0) 0x7fd8980b3520 con 0x7fd89806a4d0 2023-12-02T00:28:27.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.383+0000 7fd897fff640 1 -- 172.21.15.46:0/52303759 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd880003520 con 0x7fd89806a4d0 2023-12-02T00:28:27.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.383+0000 7fd89c9c1640 1 -- 172.21.15.46:0/52303759 >> v2:172.21.15.162:3300/0 conn(0x7fd8980b4b60 msgr2=0x7fd898160fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:27.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.383+0000 7fd89c9c1640 1 --2- 172.21.15.46:0/52303759 >> v2:172.21.15.162:3300/0 conn(0x7fd8980b4b60 0x7fd898160fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:27.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.383+0000 7fd89c9c1640 1 -- 172.21.15.46:0/52303759 >> v1:172.21.15.46:6789/0 conn(0x7fd89806a4d0 legacy=0x7fd89815d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:27.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.383+0000 7fd89c9c1640 1 -- 172.21.15.46:0/52303759 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd898161af0 con 0x7fd898071550 2023-12-02T00:28:27.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.384+0000 7fd89c9c1640 1 --2- 172.21.15.46:0/52303759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd898071550 0x7fd8980b39b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd88c0097a0 tx=0x7fd88c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6c40f8d88a49fa7 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:27.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.384+0000 7fd897fff640 1 -- 172.21.15.46:0/52303759 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88c031830 con 0x7fd898071550 2023-12-02T00:28:27.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.384+0000 7fd897fff640 1 -- 172.21.15.46:0/52303759 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd88c031990 con 0x7fd898071550 2023-12-02T00:28:27.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.384+0000 7fd897fff640 1 -- 172.21.15.46:0/52303759 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88c031d20 con 0x7fd898071550 2023-12-02T00:28:27.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.385+0000 7fd89f426640 1 -- 172.21.15.46:0/52303759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd898071550 msgr2=0x7fd8980b39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:27.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.385+0000 7fd89f426640 1 --2- 172.21.15.46:0/52303759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd898071550 0x7fd8980b39b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd88c0097a0 tx=0x7fd88c02b8d0 comp rx=0 tx=0).stop 2023-12-02T00:28:27.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.385+0000 7fd89f426640 1 -- 172.21.15.46:0/52303759 shutdown_connections 2023-12-02T00:28:27.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.385+0000 7fd89f426640 1 --2- 172.21.15.46:0/52303759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd898071550 0x7fd8980b39b0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:27.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.385+0000 7fd89f426640 1 --2- 172.21.15.46:0/52303759 >> v2:172.21.15.162:3300/0 conn(0x7fd8980b4b60 0x7fd898160fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:27.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.385+0000 7fd89f426640 1 -- 172.21.15.46:0/52303759 >> 172.21.15.46:0/52303759 conn(0x7fd898066270 msgr2=0x7fd898066650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:27.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.385+0000 7fd89f426640 1 -- 172.21.15.46:0/52303759 shutdown_connections 2023-12-02T00:28:27.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.385+0000 7fd89f426640 1 -- 172.21.15.46:0/52303759 wait complete. 2023-12-02T00:28:27.389 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.386+0000 7fd89f426640 1 Processor -- start 2023-12-02T00:28:27.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.386+0000 7fd89f426640 1 -- start start 2023-12-02T00:28:27.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.387+0000 7fd89f426640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd89806a4d0 0x7fd8981f2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:27.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.387+0000 7fd89f426640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd898071550 0x7fd8981f5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:27.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.387+0000 7fd89f426640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8980b4b60 0x7fd89815d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:27.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.387+0000 7fd89f426640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd898139d80 con 0x7fd898071550 2023-12-02T00:28:27.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.387+0000 7fd89f426640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8981f2ff0 con 0x7fd89806a4d0 2023-12-02T00:28:27.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.387+0000 7fd89f426640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd898139100 con 0x7fd8980b4b60 2023-12-02T00:28:27.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.387+0000 7fd89c9c1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd898071550 0x7fd8981f5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:27.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.387+0000 7fd89c9c1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd898071550 0x7fd8981f5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55976/0 (socket says 172.21.15.46:55976) 2023-12-02T00:28:27.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.387+0000 7fd89c9c1640 1 -- 172.21.15.46:0/2491071852 learned_addr learned my addr 172.21.15.46:0/2491071852 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:27.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.387+0000 7fd89d1c2640 1 --2- 172.21.15.46:0/2491071852 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd89806a4d0 0x7fd8981f2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:27.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.387+0000 7fd89d9c3640 1 --2- 172.21.15.46:0/2491071852 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8980b4b60 0x7fd89815d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:27.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.387+0000 7fd89c9c1640 1 -- 172.21.15.46:0/2491071852 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8980b4b60 msgr2=0x7fd89815d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:27.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.387+0000 7fd89c9c1640 1 --2- 172.21.15.46:0/2491071852 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8980b4b60 0x7fd89815d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:27.391 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.387+0000 7fd89c9c1640 1 -- 172.21.15.46:0/2491071852 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd89806a4d0 msgr2=0x7fd8981f2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:27.391 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.387+0000 7fd89c9c1640 1 --2- 172.21.15.46:0/2491071852 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd89806a4d0 0x7fd8981f2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:27.391 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.387+0000 7fd89c9c1640 1 -- 172.21.15.46:0/2491071852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd88c009400 con 0x7fd898071550 2023-12-02T00:28:27.391 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.388+0000 7fd89c9c1640 1 --2- 172.21.15.46:0/2491071852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd898071550 0x7fd8981f5820 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd88c02bee0 tx=0x7fd88c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:27.391 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.388+0000 7fd895ffb640 1 -- 172.21.15.46:0/2491071852 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88c03a630 con 0x7fd898071550 2023-12-02T00:28:27.391 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.388+0000 7fd895ffb640 1 -- 172.21.15.46:0/2491071852 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd88c03a7b0 con 0x7fd898071550 2023-12-02T00:28:27.391 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.388+0000 7fd895ffb640 1 -- 172.21.15.46:0/2491071852 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88c03ab60 con 0x7fd898071550 2023-12-02T00:28:27.391 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.388+0000 7fd89f426640 1 -- 172.21.15.46:0/2491071852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8981393e0 con 0x7fd898071550 2023-12-02T00:28:27.391 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.388+0000 7fd89f426640 1 -- 172.21.15.46:0/2491071852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8981399a0 con 0x7fd898071550 2023-12-02T00:28:27.391 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.388+0000 7fd895ffb640 1 -- 172.21.15.46:0/2491071852 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd88c043410 con 0x7fd898071550 2023-12-02T00:28:27.391 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.388+0000 7fd895ffb640 1 -- 172.21.15.46:0/2491071852 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fd88c030480 con 0x7fd898071550 2023-12-02T00:28:27.391 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.389+0000 7fd89f426640 1 -- 172.21.15.46:0/2491071852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd898139ec0 con 0x7fd898071550 2023-12-02T00:28:27.392 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.391+0000 7fd895ffb640 1 -- 172.21.15.46:0/2491071852 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fd898139ec0 con 0x7fd898071550 2023-12-02T00:28:27.441 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-12-02T00:28:27.440+0000 7fee736b6040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-02T00:28:27.449 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-12-02T00:28:27.448+0000 7f1c2ce2b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-02T00:28:27.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.453+0000 7fd89f426640 1 -- 172.21.15.46:0/2491071852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd8980b56e0 con 0x7fd898071550 2023-12-02T00:28:27.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.454+0000 7fd895ffb640 1 -- 172.21.15.46:0/2491071852 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fd88c048070 con 0x7fd898071550 2023-12-02T00:28:27.455 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:28:27.455 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":10,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:28:27.213351+0000","last_up_change":"0.000000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:28:27.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.455+0000 7fd89f426640 1 -- 172.21.15.46:0/2491071852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd898071550 msgr2=0x7fd8981f5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:27.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.455+0000 7fd89f426640 1 --2- 172.21.15.46:0/2491071852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd898071550 0x7fd8981f5820 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd88c02bee0 tx=0x7fd88c0046c0 comp rx=0 tx=0).stop 2023-12-02T00:28:27.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.455+0000 7fd89f426640 1 -- 172.21.15.46:0/2491071852 shutdown_connections 2023-12-02T00:28:27.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.455+0000 7fd89f426640 1 --2- 172.21.15.46:0/2491071852 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8980b4b60 0x7fd89815d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:27.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.455+0000 7fd89f426640 1 --2- 172.21.15.46:0/2491071852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd898071550 0x7fd8981f5820 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:27.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.455+0000 7fd89f426640 1 --2- 172.21.15.46:0/2491071852 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd89806a4d0 0x7fd8981f2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:27.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.455+0000 7fd89f426640 1 -- 172.21.15.46:0/2491071852 >> 172.21.15.46:0/2491071852 conn(0x7fd898066270 msgr2=0x7fd8980733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:27.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.455+0000 7fd89f426640 1 -- 172.21.15.46:0/2491071852 shutdown_connections 2023-12-02T00:28:27.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:27.455+0000 7fd89f426640 1 -- 172.21.15.46:0/2491071852 wait complete. 2023-12-02T00:28:27.470 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-12-02T00:28:27.470 INFO:tasks.ceph:Waiting for OSDs to come up 2023-12-02T00:28:27.544 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-12-02T00:28:27.543+0000 7f1c2ce2b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-02T00:28:27.546 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-12-02T00:28:27.545+0000 7fee736b6040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-02T00:28:27.687 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-12-02T00:28:27.686+0000 7f1c2ce2b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-02T00:28:27.692 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-12-02T00:28:27.691+0000 7fee736b6040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-02T00:28:27.729 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-12-02T00:28:27.728+0000 7f1c2ce2b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-02T00:28:27.736 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-12-02T00:28:27.735+0000 7fee736b6040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-02T00:28:27.844 INFO:tasks.ceph.osd.1.smithi046.stderr:2023-12-02T00:28:27.843+0000 7f8dd6e16500 -1 Falling back to public interface 2023-12-02T00:28:27.845 INFO:tasks.ceph.osd.6.smithi162.stderr:2023-12-02T00:28:27.844+0000 7f548daaf500 -1 Falling back to public interface 2023-12-02T00:28:27.846 INFO:tasks.ceph.osd.4.smithi162.stderr:2023-12-02T00:28:27.845+0000 7f4d42c26500 -1 Falling back to public interface 2023-12-02T00:28:27.852 INFO:tasks.ceph.osd.5.smithi162.stderr:2023-12-02T00:28:27.851+0000 7f1159c0e500 -1 Falling back to public interface 2023-12-02T00:28:27.869 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-12-02T00:28:27.869+0000 7f1c2ce2b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-02T00:28:27.871 INFO:tasks.ceph.osd.0.smithi046.stderr:2023-12-02T00:28:27.870+0000 7f018db94500 -1 Falling back to public interface 2023-12-02T00:28:27.874 INFO:tasks.ceph.osd.3.smithi046.stderr:2023-12-02T00:28:27.873+0000 7fd522e0c500 -1 Falling back to public interface 2023-12-02T00:28:27.875 INFO:tasks.ceph.osd.7.smithi162.stderr:2023-12-02T00:28:27.875+0000 7f98d6a8c500 -1 Falling back to public interface 2023-12-02T00:28:27.878 INFO:tasks.ceph.osd.2.smithi046.stderr:2023-12-02T00:28:27.877+0000 7f9700c0f500 -1 Falling back to public interface 2023-12-02T00:28:27.889 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-12-02T00:28:27.888+0000 7fee736b6040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-02T00:28:28.277 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-12-02T00:28:28.370 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.369+0000 7f8a2b090640 1 Processor -- start 2023-12-02T00:28:28.370 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.369+0000 7f8a2b090640 1 -- start start 2023-12-02T00:28:28.371 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.369+0000 7f8a2b090640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8a24146be0 0x7f8a240b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:28.371 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.369+0000 7f8a2b090640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8a241617f0 0x7f8a2406b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:28.372 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.369+0000 7f8a2b090640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a24161c20 con 0x7f8a240b31c0 2023-12-02T00:28:28.372 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.370+0000 7f8a2b090640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a240b3650 con 0x7f8a24146be0 2023-12-02T00:28:28.372 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.370+0000 7f8a2b090640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a24161e50 con 0x7f8a241617f0 2023-12-02T00:28:28.372 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.370+0000 7f8a2962d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8a241617f0 0x7f8a2406b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:28.372 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.370+0000 7f8a28e2c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8a24146be0 0x7f8a240b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:28.372 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.370+0000 7f8a2962d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8a241617f0 0x7f8a2406b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.46:36262/0 (socket says 172.21.15.46:36262) 2023-12-02T00:28:28.372 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.370+0000 7f8a28e2c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8a24146be0 0x7f8a240b2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.46:39898/0 (socket says 172.21.15.46:39898) 2023-12-02T00:28:28.372 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.370+0000 7f8a2962d640 1 -- 172.21.15.46:0/520478642 learned_addr learned my addr 172.21.15.46:0/520478642 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:28.372 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.370+0000 7f8a23fff640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f8a240b31c0 0x7f8a241610e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.46:48846/0 (socket says 172.21.15.46:48846) 2023-12-02T00:28:28.372 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.370+0000 7f8a2962d640 1 -- 172.21.15.46:0/520478642 >> v2:172.21.15.162:3300/0 conn(0x7f8a24146be0 msgr2=0x7f8a240b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:28.372 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.370+0000 7f8a2962d640 1 --2- 172.21.15.46:0/520478642 >> v2:172.21.15.162:3300/0 conn(0x7f8a24146be0 0x7f8a240b2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:28.372 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.370+0000 7f8a2962d640 1 -- 172.21.15.46:0/520478642 >> v1:172.21.15.162:6789/0 conn(0x7f8a240b31c0 legacy=0x7f8a241610e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:28.372 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.370+0000 7f8a237fe640 1 -- 172.21.15.46:0/520478642 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1018339636 0 0) 0x7f8a24161c20 con 0x7f8a240b31c0 2023-12-02T00:28:28.372 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.370+0000 7f8a2962d640 1 -- 172.21.15.46:0/520478642 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a2415e470 con 0x7f8a241617f0 2023-12-02T00:28:28.372 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.370+0000 7f8a28e2c640 1 --2- 172.21.15.46:0/520478642 >> v2:172.21.15.162:3300/0 conn(0x7f8a24146be0 0x7f8a240b2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:28:28.374 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.371+0000 7f8a2962d640 1 --2- 172.21.15.46:0/520478642 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8a241617f0 0x7f8a2406b7f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c009e50 tx=0x7f8a1c02bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70e89f6c2cd41547 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:28.374 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.371+0000 7f8a237fe640 1 -- 172.21.15.46:0/520478642 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c02ca70 con 0x7f8a241617f0 2023-12-02T00:28:28.374 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.371+0000 7f8a237fe640 1 -- 172.21.15.46:0/520478642 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a1c02cbd0 con 0x7f8a241617f0 2023-12-02T00:28:28.374 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.371+0000 7f8a237fe640 1 -- 172.21.15.46:0/520478642 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c0343f0 con 0x7f8a241617f0 2023-12-02T00:28:28.374 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.372+0000 7f8a2b090640 1 -- 172.21.15.46:0/520478642 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8a241617f0 msgr2=0x7f8a2406b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:28.374 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.372+0000 7f8a2b090640 1 --2- 172.21.15.46:0/520478642 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8a241617f0 0x7f8a2406b7f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c009e50 tx=0x7f8a1c02bed0 comp rx=0 tx=0).stop 2023-12-02T00:28:28.374 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.372+0000 7f8a2b090640 1 -- 172.21.15.46:0/520478642 shutdown_connections 2023-12-02T00:28:28.374 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.372+0000 7f8a2b090640 1 --2- 172.21.15.46:0/520478642 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8a241617f0 0x7f8a2406b7f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:28.374 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.372+0000 7f8a2b090640 1 --2- 172.21.15.46:0/520478642 >> v2:172.21.15.162:3300/0 conn(0x7f8a24146be0 0x7f8a240b2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:28.374 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.372+0000 7f8a2b090640 1 -- 172.21.15.46:0/520478642 >> 172.21.15.46:0/520478642 conn(0x7f8a240b6510 msgr2=0x7f8a240b68f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:28.374 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.372+0000 7f8a2b090640 1 -- 172.21.15.46:0/520478642 shutdown_connections 2023-12-02T00:28:28.374 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.372+0000 7f8a2b090640 1 -- 172.21.15.46:0/520478642 wait complete. 2023-12-02T00:28:28.377 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.373+0000 7f8a2b090640 1 Processor -- start 2023-12-02T00:28:28.377 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.373+0000 7f8a2b090640 1 -- start start 2023-12-02T00:28:28.377 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.374+0000 7f8a2b090640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a240b31c0 0x7f8a2413f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:28.378 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.374+0000 7f8a2b090640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a24146be0 0x7f8a2413c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:28.378 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.374+0000 7f8a2b090640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8a241617f0 0x7f8a2406ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:28.378 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.374+0000 7f8a2b090640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2413c6e0 con 0x7f8a240b31c0 2023-12-02T00:28:28.378 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.374+0000 7f8a2b090640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2413f720 con 0x7f8a24146be0 2023-12-02T00:28:28.378 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.374+0000 7f8a2b090640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2413fa00 con 0x7f8a241617f0 2023-12-02T00:28:28.380 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.374+0000 7f8a28e2c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a240b31c0 0x7f8a2413f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:28.380 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.374+0000 7f8a28e2c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a240b31c0 0x7f8a2413f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55980/0 (socket says 172.21.15.46:55980) 2023-12-02T00:28:28.380 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.374+0000 7f8a28e2c640 1 -- 172.21.15.46:0/346118885 learned_addr learned my addr 172.21.15.46:0/346118885 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:28.380 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.374+0000 7f8a2962d640 1 --2- 172.21.15.46:0/346118885 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8a241617f0 0x7f8a2406ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:28.380 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.374+0000 7f8a23fff640 1 --2- 172.21.15.46:0/346118885 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a24146be0 0x7f8a2413c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:28.380 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.375+0000 7f8a2962d640 1 -- 172.21.15.46:0/346118885 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a24146be0 msgr2=0x7f8a2413c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:28.381 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.375+0000 7f8a2962d640 1 --2- 172.21.15.46:0/346118885 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a24146be0 0x7f8a2413c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:28.381 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.375+0000 7f8a2962d640 1 -- 172.21.15.46:0/346118885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a240b31c0 msgr2=0x7f8a2413f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:28.381 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.375+0000 7f8a2962d640 1 --2- 172.21.15.46:0/346118885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a240b31c0 0x7f8a2413f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:28.381 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.375+0000 7f8a2962d640 1 -- 172.21.15.46:0/346118885 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a14009400 con 0x7f8a241617f0 2023-12-02T00:28:28.381 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.375+0000 7f8a28e2c640 1 --2- 172.21.15.46:0/346118885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a240b31c0 0x7f8a2413f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:28:28.381 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.375+0000 7f8a2962d640 1 --2- 172.21.15.46:0/346118885 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8a241617f0 0x7f8a2406ea00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c032000 tx=0x7f8a1c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:28.381 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.375+0000 7f8a217fa640 1 -- 172.21.15.46:0/346118885 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c009510 con 0x7f8a241617f0 2023-12-02T00:28:28.381 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.375+0000 7f8a217fa640 1 -- 172.21.15.46:0/346118885 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a1c03e450 con 0x7f8a241617f0 2023-12-02T00:28:28.381 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.375+0000 7f8a2b090640 1 -- 172.21.15.46:0/346118885 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a1c009b30 con 0x7f8a241617f0 2023-12-02T00:28:28.381 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.375+0000 7f8a217fa640 1 -- 172.21.15.46:0/346118885 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c03e820 con 0x7f8a241617f0 2023-12-02T00:28:28.381 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.375+0000 7f8a2b090640 1 -- 172.21.15.46:0/346118885 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a2413ffc0 con 0x7f8a241617f0 2023-12-02T00:28:28.381 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.375+0000 7f8a217fa640 1 -- 172.21.15.46:0/346118885 <== mon.2 v2:172.21.15.162:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f8a1c03ec80 con 0x7f8a241617f0 2023-12-02T00:28:28.381 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.375+0000 7f8a217fa640 1 -- 172.21.15.46:0/346118885 <== mon.2 v2:172.21.15.162:3301/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f8a1c048400 con 0x7f8a241617f0 2023-12-02T00:28:28.382 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.376+0000 7f8a2b090640 1 -- 172.21.15.46:0/346118885 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a24067940 con 0x7f8a241617f0 2023-12-02T00:28:28.382 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.378+0000 7f8a217fa640 1 -- 172.21.15.46:0/346118885 <== mon.2 v2:172.21.15.162:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f8a1c043020 con 0x7f8a241617f0 2023-12-02T00:28:28.442 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.441+0000 7f8a2b090640 1 -- 172.21.15.46:0/346118885 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8a2413c8c0 con 0x7f8a241617f0 2023-12-02T00:28:28.442 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.441+0000 7f8a217fa640 1 -- 172.21.15.46:0/346118885 <== mon.2 v2:172.21.15.162:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f8a2413c8c0 con 0x7f8a241617f0 2023-12-02T00:28:28.442 INFO:teuthology.misc.health.smithi046.stdout: 2023-12-02T00:28:28.443 INFO:teuthology.misc.health.smithi046.stdout:{"epoch":10,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:28:27.213351+0000","last_up_change":"0.000000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:28:28.443 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.442+0000 7f8a2b090640 1 -- 172.21.15.46:0/346118885 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8a241617f0 msgr2=0x7f8a2406ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:28.444 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.442+0000 7f8a2b090640 1 --2- 172.21.15.46:0/346118885 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8a241617f0 0x7f8a2406ea00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c032000 tx=0x7f8a1c004a40 comp rx=0 tx=0).stop 2023-12-02T00:28:28.444 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.442+0000 7f8a2b090640 1 -- 172.21.15.46:0/346118885 shutdown_connections 2023-12-02T00:28:28.444 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.442+0000 7f8a2b090640 1 --2- 172.21.15.46:0/346118885 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8a241617f0 0x7f8a2406ea00 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:28.444 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.442+0000 7f8a2b090640 1 --2- 172.21.15.46:0/346118885 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a24146be0 0x7f8a2413c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:28.444 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.442+0000 7f8a2b090640 1 --2- 172.21.15.46:0/346118885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a240b31c0 0x7f8a2413f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:28.444 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.442+0000 7f8a2b090640 1 -- 172.21.15.46:0/346118885 >> 172.21.15.46:0/346118885 conn(0x7f8a240b6510 msgr2=0x7f8a2415eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:28.444 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.442+0000 7f8a2b090640 1 -- 172.21.15.46:0/346118885 shutdown_connections 2023-12-02T00:28:28.444 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:28.443+0000 7f8a2b090640 1 -- 172.21.15.46:0/346118885 wait complete. 2023-12-02T00:28:28.453 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-12-02T00:28:28.704 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-12-02T00:28:28.703+0000 7f1c2ce2b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-02T00:28:28.729 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-12-02T00:28:28.728+0000 7fee736b6040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-02T00:28:28.977 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-12-02T00:28:28.976+0000 7f1c2ce2b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-02T00:28:29.014 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-12-02T00:28:29.013+0000 7fee736b6040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-02T00:28:29.029 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-12-02T00:28:29.028+0000 7f1c2ce2b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-02T00:28:29.068 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-12-02T00:28:29.067+0000 7fee736b6040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-02T00:28:29.131 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-12-02T00:28:29.130+0000 7f1c2ce2b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-02T00:28:29.173 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-12-02T00:28:29.172+0000 7fee736b6040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-02T00:28:30.159 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-12-02T00:28:30.157+0000 7f1bcda7d640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-12-02T00:28:30.159 INFO:tasks.ceph.mgr.x.smithi162.stderr:2023-12-02T00:28:30.158+0000 7f1bcda7d640 -1 mgr.server handle_report got status from non-daemon mon.c 2023-12-02T00:28:31.314 INFO:tasks.ceph.osd.5.smithi162.stderr:2023-12-02T00:28:31.312+0000 7f1159c0e500 -1 osd.5 0 log_to_monitors true 2023-12-02T00:28:31.321 INFO:tasks.ceph.osd.1.smithi046.stderr:2023-12-02T00:28:31.315+0000 7f8dd6e16500 -1 osd.1 0 log_to_monitors true 2023-12-02T00:28:31.363 INFO:tasks.ceph.osd.0.smithi046.stderr:2023-12-02T00:28:31.361+0000 7f018db94500 -1 osd.0 0 log_to_monitors true 2023-12-02T00:28:31.378 INFO:tasks.ceph.osd.4.smithi162.stderr:2023-12-02T00:28:31.375+0000 7f4d42c26500 -1 osd.4 0 log_to_monitors true 2023-12-02T00:28:31.378 INFO:tasks.ceph.osd.6.smithi162.stderr:2023-12-02T00:28:31.376+0000 7f548daaf500 -1 osd.6 0 log_to_monitors true 2023-12-02T00:28:31.379 INFO:tasks.ceph.osd.7.smithi162.stderr:2023-12-02T00:28:31.376+0000 7f98d6a8c500 -1 osd.7 0 log_to_monitors true 2023-12-02T00:28:31.386 INFO:tasks.ceph.osd.3.smithi046.stderr:2023-12-02T00:28:31.381+0000 7fd522e0c500 -1 osd.3 0 log_to_monitors true 2023-12-02T00:28:31.395 INFO:tasks.ceph.osd.2.smithi046.stderr:2023-12-02T00:28:31.392+0000 7f9700c0f500 -1 osd.2 0 log_to_monitors true 2023-12-02T00:28:34.672 INFO:tasks.ceph.osd.3.smithi046.stderr:2023-12-02T00:28:34.670+0000 7fd51e026640 -1 osd.3 0 waiting for initial osdmap 2023-12-02T00:28:34.673 INFO:tasks.ceph.osd.1.smithi046.stderr:2023-12-02T00:28:34.673+0000 7f8dd1ec3640 -1 osd.1 0 waiting for initial osdmap 2023-12-02T00:28:34.682 INFO:tasks.ceph.osd.3.smithi046.stderr:2023-12-02T00:28:34.681+0000 7fd51665e640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-02T00:28:34.686 INFO:tasks.ceph.osd.1.smithi046.stderr:2023-12-02T00:28:34.685+0000 7f8dca4fb640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-02T00:28:34.705 INFO:tasks.ceph.osd.0.smithi046.stderr:2023-12-02T00:28:34.704+0000 7f018ab37640 -1 osd.0 0 waiting for initial osdmap 2023-12-02T00:28:34.706 INFO:tasks.ceph.osd.2.smithi046.stderr:2023-12-02T00:28:34.705+0000 7f96fbcab640 -1 osd.2 0 waiting for initial osdmap 2023-12-02T00:28:34.721 INFO:tasks.ceph.osd.2.smithi046.stderr:2023-12-02T00:28:34.720+0000 7f96f42e3640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-02T00:28:34.721 INFO:tasks.ceph.osd.0.smithi046.stderr:2023-12-02T00:28:34.720+0000 7f0181937640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-02T00:28:35.260 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-12-02T00:28:35.282 INFO:tasks.ceph.osd.6.smithi162.stderr:2023-12-02T00:28:35.280+0000 7f548921a640 -1 osd.6 0 waiting for initial osdmap 2023-12-02T00:28:35.294 INFO:tasks.ceph.osd.5.smithi162.stderr:2023-12-02T00:28:35.293+0000 7f11564e3640 -1 osd.5 0 waiting for initial osdmap 2023-12-02T00:28:35.295 INFO:tasks.ceph.osd.6.smithi162.stderr:2023-12-02T00:28:35.294+0000 7f5481852640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-02T00:28:35.305 INFO:tasks.ceph.osd.7.smithi162.stderr:2023-12-02T00:28:35.303+0000 7f98d21f9640 -1 osd.7 0 waiting for initial osdmap 2023-12-02T00:28:35.305 INFO:tasks.ceph.osd.4.smithi162.stderr:2023-12-02T00:28:35.303+0000 7f4d3dd45640 -1 osd.4 0 waiting for initial osdmap 2023-12-02T00:28:35.311 INFO:tasks.ceph.osd.5.smithi162.stderr:2023-12-02T00:28:35.310+0000 7f114d2e3640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-02T00:28:35.313 INFO:tasks.ceph.osd.4.smithi162.stderr:2023-12-02T00:28:35.312+0000 7f4d3637d640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-02T00:28:35.314 INFO:tasks.ceph.osd.7.smithi162.stderr:2023-12-02T00:28:35.312+0000 7f98ca831640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-02T00:28:35.359 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.358+0000 7f223556e640 1 Processor -- start 2023-12-02T00:28:35.360 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.358+0000 7f223556e640 1 -- start start 2023-12-02T00:28:35.361 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.359+0000 7f223556e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f22300ccf10 0x7f223006c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:35.361 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.359+0000 7f223556e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f223009a6e0 0x7f2230161100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:35.361 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.359+0000 7f223556e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2230161640 0x7f223015cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:35.361 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.359+0000 7f223556e640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2230161ab0 con 0x7f223009a6e0 2023-12-02T00:28:35.361 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.359+0000 7f223556e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22300cf390 con 0x7f22300ccf10 2023-12-02T00:28:35.361 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.359+0000 7f223556e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22300cf670 con 0x7f2230161640 2023-12-02T00:28:35.362 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.359+0000 7f222e7fc640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f223009a6e0 0x7f2230161100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:35.362 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.359+0000 7f222e7fc640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f223009a6e0 0x7f2230161100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35444/0 (socket says 172.21.15.46:35444) 2023-12-02T00:28:35.362 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.359+0000 7f222e7fc640 1 -- 172.21.15.46:0/778156030 learned_addr learned my addr 172.21.15.46:0/778156030 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:35.362 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.359+0000 7f222f7fe640 1 --2- 172.21.15.46:0/778156030 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2230161640 0x7f223015cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:35.362 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.359+0000 7f222effd640 1 --2- 172.21.15.46:0/778156030 >> v2:172.21.15.162:3300/0 conn(0x7f22300ccf10 0x7f223006c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:35.362 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.360+0000 7f222e7fc640 1 -- 172.21.15.46:0/778156030 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2230161640 msgr2=0x7f223015cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:35.362 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.360+0000 7f222e7fc640 1 --2- 172.21.15.46:0/778156030 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2230161640 0x7f223015cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:35.362 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.360+0000 7f222e7fc640 1 -- 172.21.15.46:0/778156030 >> v2:172.21.15.162:3300/0 conn(0x7f22300ccf10 msgr2=0x7f223006c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:35.362 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.360+0000 7f222e7fc640 1 --2- 172.21.15.46:0/778156030 >> v2:172.21.15.162:3300/0 conn(0x7f22300ccf10 0x7f223006c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:35.362 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.360+0000 7f222e7fc640 1 -- 172.21.15.46:0/778156030 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22300cf950 con 0x7f223009a6e0 2023-12-02T00:28:35.362 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.361+0000 7f222e7fc640 1 --2- 172.21.15.46:0/778156030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f223009a6e0 0x7f2230161100 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2230071580 tx=0x7f221802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaec7133e649118c server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:35.363 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.361+0000 7f222dffb640 1 -- 172.21.15.46:0/778156030 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2218031870 con 0x7f223009a6e0 2023-12-02T00:28:35.364 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.361+0000 7f222dffb640 1 -- 172.21.15.46:0/778156030 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22180319d0 con 0x7f223009a6e0 2023-12-02T00:28:35.364 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.361+0000 7f222dffb640 1 -- 172.21.15.46:0/778156030 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2218031d60 con 0x7f223009a6e0 2023-12-02T00:28:35.364 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.361+0000 7f223556e640 1 -- 172.21.15.46:0/778156030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f223009a6e0 msgr2=0x7f2230161100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:35.364 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.361+0000 7f223556e640 1 --2- 172.21.15.46:0/778156030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f223009a6e0 0x7f2230161100 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2230071580 tx=0x7f221802b910 comp rx=0 tx=0).stop 2023-12-02T00:28:35.364 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.361+0000 7f223556e640 1 -- 172.21.15.46:0/778156030 shutdown_connections 2023-12-02T00:28:35.364 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.361+0000 7f223556e640 1 --2- 172.21.15.46:0/778156030 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2230161640 0x7f223015cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:35.364 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.361+0000 7f223556e640 1 --2- 172.21.15.46:0/778156030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f223009a6e0 0x7f2230161100 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:35.364 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.361+0000 7f223556e640 1 --2- 172.21.15.46:0/778156030 >> v2:172.21.15.162:3300/0 conn(0x7f22300ccf10 0x7f223006c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:35.364 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.361+0000 7f223556e640 1 -- 172.21.15.46:0/778156030 >> 172.21.15.46:0/778156030 conn(0x7f2230099a70 msgr2=0x7f2230099e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:35.364 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.361+0000 7f223556e640 1 -- 172.21.15.46:0/778156030 shutdown_connections 2023-12-02T00:28:35.364 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.361+0000 7f223556e640 1 -- 172.21.15.46:0/778156030 wait complete. 2023-12-02T00:28:35.364 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.363+0000 7f223556e640 1 Processor -- start 2023-12-02T00:28:35.364 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.363+0000 7f223556e640 1 -- start start 2023-12-02T00:28:35.366 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.363+0000 7f223556e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f223009a6e0 0x7f22301f2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:35.366 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.363+0000 7f223556e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22300ccf10 0x7f22301f3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:35.366 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.363+0000 7f223556e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2230161640 0x7f223015ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:35.367 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.363+0000 7f223556e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22301f5180 con 0x7f22300ccf10 2023-12-02T00:28:35.367 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.363+0000 7f223556e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22301f4040 con 0x7f223009a6e0 2023-12-02T00:28:35.367 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.363+0000 7f223556e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22301f4570 con 0x7f2230161640 2023-12-02T00:28:35.367 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.363+0000 7f222e7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22300ccf10 0x7f22301f3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:35.367 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.364+0000 7f222e7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22300ccf10 0x7f22301f3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35452/0 (socket says 172.21.15.46:35452) 2023-12-02T00:28:35.367 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.364+0000 7f222e7fc640 1 -- 172.21.15.46:0/2550530756 learned_addr learned my addr 172.21.15.46:0/2550530756 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:35.367 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.364+0000 7f222f7fe640 1 --2- 172.21.15.46:0/2550530756 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2230161640 0x7f223015ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:35.367 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.364+0000 7f222effd640 1 --2- 172.21.15.46:0/2550530756 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f223009a6e0 0x7f22301f2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:35.367 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.364+0000 7f222e7fc640 1 -- 172.21.15.46:0/2550530756 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2230161640 msgr2=0x7f223015ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:35.367 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.364+0000 7f222e7fc640 1 --2- 172.21.15.46:0/2550530756 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2230161640 0x7f223015ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:35.367 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.364+0000 7f222e7fc640 1 -- 172.21.15.46:0/2550530756 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f223009a6e0 msgr2=0x7f22301f2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:35.367 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.364+0000 7f222e7fc640 1 --2- 172.21.15.46:0/2550530756 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f223009a6e0 0x7f22301f2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:35.367 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.364+0000 7f222e7fc640 1 -- 172.21.15.46:0/2550530756 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22180094d0 con 0x7f22300ccf10 2023-12-02T00:28:35.367 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.364+0000 7f222effd640 1 --2- 172.21.15.46:0/2550530756 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f223009a6e0 0x7f22301f2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:28:35.367 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.364+0000 7f222e7fc640 1 --2- 172.21.15.46:0/2550530756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22300ccf10 0x7f22301f3b00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f221802b950 tx=0x7f22180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:35.368 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.364+0000 7f22277fe640 1 -- 172.21.15.46:0/2550530756 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221803ac60 con 0x7f22300ccf10 2023-12-02T00:28:35.368 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.364+0000 7f22277fe640 1 -- 172.21.15.46:0/2550530756 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f221803adc0 con 0x7f22300ccf10 2023-12-02T00:28:35.368 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.364+0000 7f223556e640 1 -- 172.21.15.46:0/2550530756 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22300d05b0 con 0x7f22300ccf10 2023-12-02T00:28:35.368 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.364+0000 7f22277fe640 1 -- 172.21.15.46:0/2550530756 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2218044630 con 0x7f22300ccf10 2023-12-02T00:28:35.368 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.364+0000 7f223556e640 1 -- 172.21.15.46:0/2550530756 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22301f4b30 con 0x7f22300ccf10 2023-12-02T00:28:35.368 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.365+0000 7f223556e640 1 -- 172.21.15.46:0/2550530756 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2230067420 con 0x7f22300ccf10 2023-12-02T00:28:35.368 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.366+0000 7f22277fe640 1 -- 172.21.15.46:0/2550530756 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f221804d450 con 0x7f22300ccf10 2023-12-02T00:28:35.369 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.366+0000 7f22277fe640 1 --2- 172.21.15.46:0/2550530756 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f221006adb0 0x7f221006d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:35.369 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.366+0000 7f22277fe640 1 -- 172.21.15.46:0/2550530756 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f221802d040 con 0x7f22300ccf10 2023-12-02T00:28:35.369 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.366+0000 7f222effd640 1 --2- 172.21.15.46:0/2550530756 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f221006adb0 0x7f221006d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:35.369 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.367+0000 7f222effd640 1 --2- 172.21.15.46:0/2550530756 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f221006adb0 0x7f221006d270 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2228009b10 tx=0x7f2228008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:35.371 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.371+0000 7f22277fe640 1 -- 172.21.15.46:0/2550530756 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2218076ae0 con 0x7f22300ccf10 2023-12-02T00:28:35.548 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.547+0000 7f223556e640 1 -- 172.21.15.46:0/2550530756 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f223015e070 con 0x7f22300ccf10 2023-12-02T00:28:35.548 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.548+0000 7f22277fe640 1 -- 172.21.15.46:0/2550530756 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f2218035070 con 0x7f22300ccf10 2023-12-02T00:28:35.549 INFO:teuthology.misc.health.smithi046.stdout: 2023-12-02T00:28:35.549 INFO:teuthology.misc.health.smithi046.stdout:{"epoch":12,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:28:33.159755+0000","last_up_change":"0.000000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:28:35.551 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.549+0000 7f223556e640 1 -- 172.21.15.46:0/2550530756 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f221006adb0 msgr2=0x7f221006d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:35.551 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.549+0000 7f223556e640 1 --2- 172.21.15.46:0/2550530756 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f221006adb0 0x7f221006d270 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2228009b10 tx=0x7f2228008040 comp rx=0 tx=0).stop 2023-12-02T00:28:35.551 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.549+0000 7f223556e640 1 -- 172.21.15.46:0/2550530756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22300ccf10 msgr2=0x7f22301f3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:35.551 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.549+0000 7f223556e640 1 --2- 172.21.15.46:0/2550530756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22300ccf10 0x7f22301f3b00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f221802b950 tx=0x7f22180046c0 comp rx=0 tx=0).stop 2023-12-02T00:28:35.551 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.549+0000 7f223556e640 1 -- 172.21.15.46:0/2550530756 shutdown_connections 2023-12-02T00:28:35.551 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.550+0000 7f223556e640 1 --2- 172.21.15.46:0/2550530756 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f221006adb0 0x7f221006d270 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:35.551 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.550+0000 7f223556e640 1 --2- 172.21.15.46:0/2550530756 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2230161640 0x7f223015ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:35.551 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.550+0000 7f223556e640 1 --2- 172.21.15.46:0/2550530756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22300ccf10 0x7f22301f3b00 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:35.551 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.550+0000 7f223556e640 1 --2- 172.21.15.46:0/2550530756 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f223009a6e0 0x7f22301f2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:35.551 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.550+0000 7f223556e640 1 -- 172.21.15.46:0/2550530756 >> 172.21.15.46:0/2550530756 conn(0x7f2230099a70 msgr2=0x7f22300cb5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:35.551 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.550+0000 7f223556e640 1 -- 172.21.15.46:0/2550530756 shutdown_connections 2023-12-02T00:28:35.551 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:35.550+0000 7f223556e640 1 -- 172.21.15.46:0/2550530756 wait complete. 2023-12-02T00:28:35.560 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-12-02T00:28:42.368 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-12-02T00:28:42.471 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.470+0000 7f66e4456640 1 Processor -- start 2023-12-02T00:28:42.471 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.470+0000 7f66e4456640 1 -- start start 2023-12-02T00:28:42.475 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.471+0000 7f66e4456640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f66dc144dc0 0x7f66dc0d0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:42.475 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.471+0000 7f66e4456640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66dc0d0ab0 0x7f66dc0b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:42.475 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.471+0000 7f66e4456640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66dc0b4630 con 0x7f66dc0b56d0 2023-12-02T00:28:42.475 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.471+0000 7f66e4456640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66dc0b4910 con 0x7f66dc144dc0 2023-12-02T00:28:42.475 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.471+0000 7f66e4456640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66dc0b4bf0 con 0x7f66dc0d0ab0 2023-12-02T00:28:42.475 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.471+0000 7f66e29f3640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f66dc0b56d0 0x7f66dc15b750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41342/0 (socket says 172.21.15.46:41342) 2023-12-02T00:28:42.475 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.471+0000 7f66e29f3640 1 -- 172.21.15.46:0/4051541054 learned_addr learned my addr 172.21.15.46:0/4051541054 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:28:42.475 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.471+0000 7f66e21f2640 1 --2- 172.21.15.46:0/4051541054 >> v2:172.21.15.162:3300/0 conn(0x7f66dc144dc0 0x7f66dc0d0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:42.475 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.471+0000 7f66e19f1640 1 --2- 172.21.15.46:0/4051541054 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66dc0d0ab0 0x7f66dc0b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:42.475 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.471+0000 7f66e11f0640 1 -- 172.21.15.46:0/4051541054 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3481365998 0 0) 0x7f66dc0b4630 con 0x7f66dc0b56d0 2023-12-02T00:28:42.476 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.471+0000 7f66e11f0640 1 -- 172.21.15.46:0/4051541054 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66c8003520 con 0x7f66dc0b56d0 2023-12-02T00:28:42.476 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.472+0000 7f66e11f0640 1 -- 172.21.15.46:0/4051541054 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1336088096 0 0) 0x7f66c8003520 con 0x7f66dc0b56d0 2023-12-02T00:28:42.476 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.472+0000 7f66e11f0640 1 -- 172.21.15.46:0/4051541054 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66dc0d0ab0 msgr2=0x7f66dc0b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:42.476 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.472+0000 7f66e11f0640 1 --2- 172.21.15.46:0/4051541054 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66dc0d0ab0 0x7f66dc0b2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:42.476 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.472+0000 7f66e11f0640 1 -- 172.21.15.46:0/4051541054 >> v2:172.21.15.162:3300/0 conn(0x7f66dc144dc0 msgr2=0x7f66dc0d0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:42.476 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.472+0000 7f66e11f0640 1 --2- 172.21.15.46:0/4051541054 >> v2:172.21.15.162:3300/0 conn(0x7f66dc144dc0 0x7f66dc0d0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:42.476 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.472+0000 7f66e11f0640 1 -- 172.21.15.46:0/4051541054 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66dc0b4ed0 con 0x7f66dc0b56d0 2023-12-02T00:28:42.476 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.472+0000 7f66e11f0640 1 -- 172.21.15.46:0/4051541054 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7f66d40019f0 con 0x7f66dc0b56d0 2023-12-02T00:28:42.476 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.472+0000 7f66e11f0640 1 -- 172.21.15.46:0/4051541054 >> v1:172.21.15.46:6789/0 conn(0x7f66dc0b56d0 legacy=0x7f66dc15b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:42.476 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.472+0000 7f66e11f0640 1 --2- 172.21.15.46:0/4051541054 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66c8003970 0x7f66c8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:42.476 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.472+0000 7f66e11f0640 1 --2- 172.21.15.46:0/4051541054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66c80047c0 0x7f66c8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:42.476 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.472+0000 7f66e11f0640 1 --2- 172.21.15.46:0/4051541054 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66dc0d0ab0 0x7f66c8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:42.476 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.472+0000 7f66e11f0640 1 -- 172.21.15.46:0/4051541054 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66dc0b4910 con 0x7f66c80047c0 2023-12-02T00:28:42.476 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.472+0000 7f66e11f0640 1 -- 172.21.15.46:0/4051541054 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66dc0b4bf0 con 0x7f66c8003970 2023-12-02T00:28:42.477 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.472+0000 7f66e11f0640 1 -- 172.21.15.46:0/4051541054 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66c800b980 con 0x7f66dc0d0ab0 2023-12-02T00:28:42.477 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.472+0000 7f66e21f2640 1 --2- 172.21.15.46:0/4051541054 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66c8003970 0x7f66c8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:42.477 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.472+0000 7f66e19f1640 1 --2- 172.21.15.46:0/4051541054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66c80047c0 0x7f66c8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:42.477 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.472+0000 7f66e29f3640 1 --2- 172.21.15.46:0/4051541054 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66dc0d0ab0 0x7f66c8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:42.477 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.472+0000 7f66e21f2640 1 -- 172.21.15.46:0/4051541054 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66dc0d0ab0 msgr2=0x7f66c8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:42.477 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.472+0000 7f66e21f2640 1 --2- 172.21.15.46:0/4051541054 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66dc0d0ab0 0x7f66c8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:42.477 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.472+0000 7f66e21f2640 1 -- 172.21.15.46:0/4051541054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66c80047c0 msgr2=0x7f66c8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:42.477 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.472+0000 7f66e21f2640 1 --2- 172.21.15.46:0/4051541054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66c80047c0 0x7f66c8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:42.477 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.472+0000 7f66e21f2640 1 -- 172.21.15.46:0/4051541054 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c8003520 con 0x7f66c8003970 2023-12-02T00:28:42.477 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.472+0000 7f66e19f1640 1 --2- 172.21.15.46:0/4051541054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66c80047c0 0x7f66c8004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:28:42.477 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.473+0000 7f66e21f2640 1 --2- 172.21.15.46:0/4051541054 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66c8003970 0x7f66c8003d90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f66d8009ac0 tx=0x7f66c80057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a360281a6e539a49 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:42.477 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.473+0000 7f66e11f0640 1 -- 172.21.15.46:0/4051541054 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d802c710 con 0x7f66c8003970 2023-12-02T00:28:42.478 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.473+0000 7f66e11f0640 1 -- 172.21.15.46:0/4051541054 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66d802c870 con 0x7f66c8003970 2023-12-02T00:28:42.478 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.473+0000 7f66e11f0640 1 -- 172.21.15.46:0/4051541054 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d802cc00 con 0x7f66c8003970 2023-12-02T00:28:42.478 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.474+0000 7f66e4456640 1 -- 172.21.15.46:0/4051541054 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66c8003970 msgr2=0x7f66c8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:42.478 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.474+0000 7f66e4456640 1 --2- 172.21.15.46:0/4051541054 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66c8003970 0x7f66c8003d90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f66d8009ac0 tx=0x7f66c80057f0 comp rx=0 tx=0).stop 2023-12-02T00:28:42.478 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.474+0000 7f66e29f3640 1 -- 172.21.15.46:0/4051541054 reap_dead start 2023-12-02T00:28:42.478 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.474+0000 7f66e4456640 1 -- 172.21.15.46:0/4051541054 shutdown_connections 2023-12-02T00:28:42.478 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.474+0000 7f66e4456640 1 -- 172.21.15.46:0/4051541054 >> 172.21.15.46:0/4051541054 conn(0x7f66dc0b6510 msgr2=0x7f66dc0b68f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:42.478 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.474+0000 7f66e4456640 1 -- 172.21.15.46:0/4051541054 shutdown_connections 2023-12-02T00:28:42.478 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.474+0000 7f66e4456640 1 -- 172.21.15.46:0/4051541054 wait complete. 2023-12-02T00:28:42.481 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.475+0000 7f66e4456640 1 Processor -- start 2023-12-02T00:28:42.481 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.475+0000 7f66e4456640 1 -- start start 2023-12-02T00:28:42.481 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.476+0000 7f66e4456640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66c8003970 0x7f66dc0b0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:42.481 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.476+0000 7f66e4456640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66dc06c510 0x7f66dc1f7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:42.481 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.476+0000 7f66e4456640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66dc1404a0 0x7f66dc1589c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:42.482 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.476+0000 7f66e4456640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66dc06cc70 con 0x7f66dc06c510 2023-12-02T00:28:42.482 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.476+0000 7f66e4456640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66dc15d040 con 0x7f66dc1404a0 2023-12-02T00:28:42.482 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.476+0000 7f66e4456640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66dc06d210 con 0x7f66c8003970 2023-12-02T00:28:42.482 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.476+0000 7f66e19f1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66dc06c510 0x7f66dc1f7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:42.482 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.476+0000 7f66e19f1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66dc06c510 0x7f66dc1f7cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35472/0 (socket says 172.21.15.46:35472) 2023-12-02T00:28:42.482 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.476+0000 7f66e19f1640 1 -- 172.21.15.46:0/4020513859 learned_addr learned my addr 172.21.15.46:0/4020513859 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:42.482 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.476+0000 7f66e21f2640 1 --2- 172.21.15.46:0/4020513859 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66c8003970 0x7f66dc0b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:42.482 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.476+0000 7f66e29f3640 1 --2- 172.21.15.46:0/4020513859 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66dc1404a0 0x7f66dc1589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:42.482 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.476+0000 7f66e19f1640 1 -- 172.21.15.46:0/4020513859 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66c8003970 msgr2=0x7f66dc0b0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:42.482 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.476+0000 7f66e19f1640 1 --2- 172.21.15.46:0/4020513859 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66c8003970 0x7f66dc0b0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:42.482 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.476+0000 7f66e19f1640 1 -- 172.21.15.46:0/4020513859 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66dc1404a0 msgr2=0x7f66dc1589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:42.482 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.476+0000 7f66e19f1640 1 --2- 172.21.15.46:0/4020513859 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66dc1404a0 0x7f66dc1589c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:42.482 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.476+0000 7f66e19f1640 1 -- 172.21.15.46:0/4020513859 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66dc1f8200 con 0x7f66dc06c510 2023-12-02T00:28:42.483 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.477+0000 7f66e19f1640 1 --2- 172.21.15.46:0/4020513859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66dc06c510 0x7f66dc1f7cc0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f66cc011ca0 tx=0x7f66cc00f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:42.483 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.477+0000 7f66d2ffd640 1 -- 172.21.15.46:0/4020513859 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66cc011cd0 con 0x7f66dc06c510 2023-12-02T00:28:42.483 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.477+0000 7f66d2ffd640 1 -- 172.21.15.46:0/4020513859 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66cc011e30 con 0x7f66dc06c510 2023-12-02T00:28:42.483 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.477+0000 7f66d2ffd640 1 -- 172.21.15.46:0/4020513859 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66cc014680 con 0x7f66dc06c510 2023-12-02T00:28:42.483 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.477+0000 7f66e4456640 1 -- 172.21.15.46:0/4020513859 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66dc06c990 con 0x7f66dc06c510 2023-12-02T00:28:42.483 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.477+0000 7f66e4456640 1 -- 172.21.15.46:0/4020513859 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66dc15cc60 con 0x7f66dc06c510 2023-12-02T00:28:42.483 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.478+0000 7f66e4456640 1 -- 172.21.15.46:0/4020513859 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66dc067940 con 0x7f66dc06c510 2023-12-02T00:28:42.483 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.478+0000 7f66d2ffd640 1 -- 172.21.15.46:0/4020513859 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f66cc0147e0 con 0x7f66dc06c510 2023-12-02T00:28:42.483 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.479+0000 7f66d2ffd640 1 --2- 172.21.15.46:0/4020513859 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f66c006ae40 0x7f66c006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:42.483 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.479+0000 7f66d2ffd640 1 -- 172.21.15.46:0/4020513859 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f66cc089ab0 con 0x7f66dc06c510 2023-12-02T00:28:42.484 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.479+0000 7f66e21f2640 1 --2- 172.21.15.46:0/4020513859 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f66c006ae40 0x7f66c006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:42.484 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.480+0000 7f66e21f2640 1 --2- 172.21.15.46:0/4020513859 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f66c006ae40 0x7f66c006d300 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f66d8009ac0 tx=0x7f66d8030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:42.485 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.483+0000 7f66d2ffd640 1 -- 172.21.15.46:0/4020513859 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f66cc054f40 con 0x7f66dc06c510 2023-12-02T00:28:42.661 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.660+0000 7f66e4456640 1 -- 172.21.15.46:0/4020513859 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f66dc15c7e0 con 0x7f66dc06c510 2023-12-02T00:28:42.662 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.661+0000 7f66d2ffd640 1 -- 172.21.15.46:0/4020513859 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12826 (secure 0 0 0) 0x7f66dc15c7e0 con 0x7f66dc06c510 2023-12-02T00:28:42.662 INFO:teuthology.misc.health.smithi046.stdout: 2023-12-02T00:28:42.662 INFO:teuthology.misc.health.smithi046.stdout:{"epoch":17,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:28:40.294369+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:28:42.664 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.663+0000 7f66e4456640 1 -- 172.21.15.46:0/4020513859 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f66c006ae40 msgr2=0x7f66c006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:42.664 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.663+0000 7f66e4456640 1 --2- 172.21.15.46:0/4020513859 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f66c006ae40 0x7f66c006d300 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f66d8009ac0 tx=0x7f66d8030000 comp rx=0 tx=0).stop 2023-12-02T00:28:42.664 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.663+0000 7f66e4456640 1 -- 172.21.15.46:0/4020513859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66dc06c510 msgr2=0x7f66dc1f7cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:42.664 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.663+0000 7f66e4456640 1 --2- 172.21.15.46:0/4020513859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66dc06c510 0x7f66dc1f7cc0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f66cc011ca0 tx=0x7f66cc00f610 comp rx=0 tx=0).stop 2023-12-02T00:28:42.665 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.663+0000 7f66e4456640 1 -- 172.21.15.46:0/4020513859 shutdown_connections 2023-12-02T00:28:42.665 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.663+0000 7f66e4456640 1 --2- 172.21.15.46:0/4020513859 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f66c006ae40 0x7f66c006d300 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:42.665 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.663+0000 7f66e4456640 1 --2- 172.21.15.46:0/4020513859 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66dc1404a0 0x7f66dc1589c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:42.665 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.663+0000 7f66e4456640 1 --2- 172.21.15.46:0/4020513859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66dc06c510 0x7f66dc1f7cc0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:42.665 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.663+0000 7f66e4456640 1 --2- 172.21.15.46:0/4020513859 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66c8003970 0x7f66dc0b0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:42.665 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.663+0000 7f66e4456640 1 -- 172.21.15.46:0/4020513859 >> 172.21.15.46:0/4020513859 conn(0x7f66dc0b6510 msgr2=0x7f66dc0ce2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:42.665 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.663+0000 7f66e4456640 1 -- 172.21.15.46:0/4020513859 shutdown_connections 2023-12-02T00:28:42.665 INFO:teuthology.misc.health.smithi046.stderr:2023-12-02T00:28:42.663+0000 7f66e4456640 1 -- 172.21.15.46:0/4020513859 wait complete. 2023-12-02T00:28:42.674 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-12-02T00:28:42.674 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-12-02T00:28:42.675 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-02T00:28:42.675 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-02T00:28:42.677 INFO:tasks.ceph.mds.a:Started 2023-12-02T00:28:42.678 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-02T00:28:42.678 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-02T00:28:42.681 INFO:tasks.ceph.mds.c:Started 2023-12-02T00:28:42.681 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-02T00:28:42.682 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-02T00:28:42.685 INFO:tasks.ceph.mds.b:Started 2023-12-02T00:28:42.685 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-02T00:28:42.685 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-02T00:28:42.687 INFO:tasks.ceph.mds.d:Started 2023-12-02T00:28:42.687 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-12-02T00:28:42.688 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:28:42.760 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2023-12-02T00:28:42.760 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2023-12-02T00:28:42.765 INFO:tasks.ceph.mds.d.smithi162.stdout:starting mds.d at 2023-12-02T00:28:42.765 INFO:tasks.ceph.mds.b.smithi162.stdout:starting mds.b at 2023-12-02T00:28:42.766 INFO:tasks.ceph.mds.d.smithi162.stderr:2023-12-02T00:28:42.763+0000 7f7391f97640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-12-02T00:28:42.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.799+0000 7fbbade10640 1 Processor -- start 2023-12-02T00:28:42.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.799+0000 7fbbade10640 1 -- start start 2023-12-02T00:28:42.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.800+0000 7fbbade10640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbba8071bd0 0x7fbba815a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:42.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.800+0000 7fbbade10640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbba8157ee0 con 0x7fbba8157b00 2023-12-02T00:28:42.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.800+0000 7fbbade10640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbba806eb70 con 0x7fbba815ad60 2023-12-02T00:28:42.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.800+0000 7fbbade10640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbba806f510 con 0x7fbba8071bd0 2023-12-02T00:28:42.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.800+0000 7fbba77fe640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fbba8157b00 0x7fbba80714c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58008/0 (socket says 172.21.15.46:58008) 2023-12-02T00:28:42.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.800+0000 7fbba77fe640 1 -- 172.21.15.46:0/1801738057 learned_addr learned my addr 172.21.15.46:0/1801738057 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:28:42.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.800+0000 7fbba6ffd640 1 --2- 172.21.15.46:0/1801738057 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbba8071bd0 0x7fbba815a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:42.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.800+0000 7fbba67fc640 1 -- 172.21.15.46:0/1801738057 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 462587046 0 0) 0x7fbba8157ee0 con 0x7fbba8157b00 2023-12-02T00:28:42.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.800+0000 7fbba67fc640 1 -- 172.21.15.46:0/1801738057 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb90003520 con 0x7fbba8157b00 2023-12-02T00:28:42.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.800+0000 7fbba6ffd640 1 -- 172.21.15.46:0/1801738057 >> v1:172.21.15.162:6789/0 conn(0x7fbba815ad60 legacy=0x7fbba80b36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:42.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.800+0000 7fbba67fc640 1 -- 172.21.15.46:0/1801738057 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4285907455 0 0) 0x7fbba806eb70 con 0x7fbba815ad60 2023-12-02T00:28:42.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.800+0000 7fbba6ffd640 1 -- 172.21.15.46:0/1801738057 >> v1:172.21.15.46:6789/0 conn(0x7fbba8157b00 legacy=0x7fbba80714c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:42.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.800+0000 7fbba6ffd640 1 -- 172.21.15.46:0/1801738057 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba806ee50 con 0x7fbba8071bd0 2023-12-02T00:28:42.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.801+0000 7fbba6ffd640 1 --2- 172.21.15.46:0/1801738057 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbba8071bd0 0x7fbba815a820 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c0097d0 tx=0x7fbb9c02b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebd9b98c96e9600 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:42.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.801+0000 7fbba67fc640 1 -- 172.21.15.46:0/1801738057 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c031800 con 0x7fbba8071bd0 2023-12-02T00:28:42.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.801+0000 7fbba67fc640 1 -- 172.21.15.46:0/1801738057 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb9c031960 con 0x7fbba8071bd0 2023-12-02T00:28:42.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.801+0000 7fbba67fc640 1 -- 172.21.15.46:0/1801738057 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c031cf0 con 0x7fbba8071bd0 2023-12-02T00:28:42.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.802+0000 7fbbade10640 1 -- 172.21.15.46:0/1801738057 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbba8071bd0 msgr2=0x7fbba815a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:42.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.802+0000 7fbbade10640 1 --2- 172.21.15.46:0/1801738057 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbba8071bd0 0x7fbba815a820 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c0097d0 tx=0x7fbb9c02b8a0 comp rx=0 tx=0).stop 2023-12-02T00:28:42.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.802+0000 7fbbade10640 1 -- 172.21.15.46:0/1801738057 shutdown_connections 2023-12-02T00:28:42.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.802+0000 7fbbade10640 1 --2- 172.21.15.46:0/1801738057 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbba8071bd0 0x7fbba815a820 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:42.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.802+0000 7fbbade10640 1 -- 172.21.15.46:0/1801738057 >> 172.21.15.46:0/1801738057 conn(0x7fbba80b1de0 msgr2=0x7fbba80b21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:42.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.802+0000 7fbbade10640 1 -- 172.21.15.46:0/1801738057 shutdown_connections 2023-12-02T00:28:42.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.802+0000 7fbbade10640 1 -- 172.21.15.46:0/1801738057 wait complete. 2023-12-02T00:28:42.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.803+0000 7fbbade10640 1 Processor -- start 2023-12-02T00:28:42.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.803+0000 7fbbade10640 1 -- start start 2023-12-02T00:28:42.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.804+0000 7fbbade10640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbba8071bd0 0x7fbba81407a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:42.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.804+0000 7fbbade10640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbba8157b00 0x7fbba813d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:42.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.804+0000 7fbbade10640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbba815ad60 0x7fbba80ae980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:42.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.804+0000 7fbbade10640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba8140ce0 con 0x7fbba8071bd0 2023-12-02T00:28:42.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.804+0000 7fbbade10640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba813fcf0 con 0x7fbba8157b00 2023-12-02T00:28:42.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.804+0000 7fbbade10640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbba80af440 con 0x7fbba815ad60 2023-12-02T00:28:42.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.804+0000 7fbba77fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbba8071bd0 0x7fbba81407a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:42.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.804+0000 7fbba77fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbba8071bd0 0x7fbba81407a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36860/0 (socket says 172.21.15.46:36860) 2023-12-02T00:28:42.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.804+0000 7fbba77fe640 1 -- 172.21.15.46:0/3348630307 learned_addr learned my addr 172.21.15.46:0/3348630307 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:42.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.804+0000 7fbba6ffd640 1 --2- 172.21.15.46:0/3348630307 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbba8157b00 0x7fbba813d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:42.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.804+0000 7fbba7fff640 1 --2- 172.21.15.46:0/3348630307 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbba815ad60 0x7fbba80ae980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:42.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.804+0000 7fbba77fe640 1 -- 172.21.15.46:0/3348630307 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbba815ad60 msgr2=0x7fbba80ae980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:42.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.804+0000 7fbba77fe640 1 --2- 172.21.15.46:0/3348630307 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbba815ad60 0x7fbba80ae980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:42.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.804+0000 7fbba77fe640 1 -- 172.21.15.46:0/3348630307 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbba8157b00 msgr2=0x7fbba813d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:42.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.804+0000 7fbba77fe640 1 --2- 172.21.15.46:0/3348630307 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbba8157b00 0x7fbba813d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:42.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.804+0000 7fbba77fe640 1 -- 172.21.15.46:0/3348630307 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb9c009400 con 0x7fbba8071bd0 2023-12-02T00:28:42.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.804+0000 7fbba77fe640 1 --2- 172.21.15.46:0/3348630307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbba8071bd0 0x7fbba81407a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbb9800c430 tx=0x7fbb9800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:42.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.805+0000 7fbb8ffff640 1 -- 172.21.15.46:0/3348630307 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb98005ce0 con 0x7fbba8071bd0 2023-12-02T00:28:42.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.805+0000 7fbb8ffff640 1 -- 172.21.15.46:0/3348630307 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb98005e40 con 0x7fbba8071bd0 2023-12-02T00:28:42.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.805+0000 7fbbade10640 1 -- 172.21.15.46:0/3348630307 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbba80af720 con 0x7fbba8071bd0 2023-12-02T00:28:42.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.805+0000 7fbb8ffff640 1 -- 172.21.15.46:0/3348630307 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb980072f0 con 0x7fbba8071bd0 2023-12-02T00:28:42.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.805+0000 7fbbade10640 1 -- 172.21.15.46:0/3348630307 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbba8140280 con 0x7fbba8071bd0 2023-12-02T00:28:42.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.806+0000 7fbbade10640 1 -- 172.21.15.46:0/3348630307 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbba8067420 con 0x7fbba8071bd0 2023-12-02T00:28:42.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.806+0000 7fbb8ffff640 1 -- 172.21.15.46:0/3348630307 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbb98007450 con 0x7fbba8071bd0 2023-12-02T00:28:42.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.806+0000 7fbb8ffff640 1 --2- 172.21.15.46:0/3348630307 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fbb9406ae40 0x7fbb9406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:42.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.807+0000 7fbb8ffff640 1 -- 172.21.15.46:0/3348630307 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbb98088930 con 0x7fbba8071bd0 2023-12-02T00:28:42.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.807+0000 7fbba6ffd640 1 --2- 172.21.15.46:0/3348630307 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fbb9406ae40 0x7fbb9406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:42.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.807+0000 7fbba6ffd640 1 --2- 172.21.15.46:0/3348630307 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fbb9406ae40 0x7fbb9406d300 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c036000 tx=0x7fbb9c03d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:42.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.811+0000 7fbb8ffff640 1 -- 172.21.15.46:0/3348630307 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbb98053d50 con 0x7fbba8071bd0 2023-12-02T00:28:42.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.989+0000 7fbbade10640 1 -- 172.21.15.46:0/3348630307 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbba8072460 con 0x7fbba8071bd0 2023-12-02T00:28:42.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.990+0000 7fbb8ffff640 1 -- 172.21.15.46:0/3348630307 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12826 (secure 0 0 0) 0x7fbb98017070 con 0x7fbba8071bd0 2023-12-02T00:28:42.991 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:28:42.991 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":17,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:28:40.294369+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:28:42.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.991+0000 7fbbade10640 1 -- 172.21.15.46:0/3348630307 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fbb9406ae40 msgr2=0x7fbb9406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:42.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.991+0000 7fbbade10640 1 --2- 172.21.15.46:0/3348630307 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fbb9406ae40 0x7fbb9406d300 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c036000 tx=0x7fbb9c03d040 comp rx=0 tx=0).stop 2023-12-02T00:28:42.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.991+0000 7fbbade10640 1 -- 172.21.15.46:0/3348630307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbba8071bd0 msgr2=0x7fbba81407a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:42.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.991+0000 7fbbade10640 1 --2- 172.21.15.46:0/3348630307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbba8071bd0 0x7fbba81407a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbb9800c430 tx=0x7fbb9800c8f0 comp rx=0 tx=0).stop 2023-12-02T00:28:42.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.991+0000 7fbbade10640 1 -- 172.21.15.46:0/3348630307 shutdown_connections 2023-12-02T00:28:42.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.991+0000 7fbbade10640 1 --2- 172.21.15.46:0/3348630307 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fbb9406ae40 0x7fbb9406d300 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:42.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.991+0000 7fbbade10640 1 --2- 172.21.15.46:0/3348630307 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbba815ad60 0x7fbba80ae980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:42.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.991+0000 7fbbade10640 1 --2- 172.21.15.46:0/3348630307 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbba8157b00 0x7fbba813d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:42.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.991+0000 7fbbade10640 1 --2- 172.21.15.46:0/3348630307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbba8071bd0 0x7fbba81407a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:42.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.991+0000 7fbbade10640 1 -- 172.21.15.46:0/3348630307 >> 172.21.15.46:0/3348630307 conn(0x7fbba80b1de0 msgr2=0x7fbba80b4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:42.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.992+0000 7fbbade10640 1 -- 172.21.15.46:0/3348630307 shutdown_connections 2023-12-02T00:28:42.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:42.992+0000 7fbbade10640 1 -- 172.21.15.46:0/3348630307 wait complete. 2023-12-02T00:28:43.004 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-02T00:28:37.213645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-02T00:28:43.004 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-02T00:28:43.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.113+0000 7f533b026640 1 Processor -- start 2023-12-02T00:28:43.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.113+0000 7f533b026640 1 -- start start 2023-12-02T00:28:43.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.113+0000 7f533b026640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f533415b130 0x7f53340d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:43.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.113+0000 7f533b026640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f53340cf320 0x7f53340cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:43.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.113+0000 7f533b026640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53340b3f10 con 0x7f533415f760 2023-12-02T00:28:43.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.113+0000 7f533b026640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f533415fd20 con 0x7f53340cf320 2023-12-02T00:28:43.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.113+0000 7f533b026640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53340d0730 con 0x7f533415b130 2023-12-02T00:28:43.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.114+0000 7f53395c3640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f53340cf320 0x7f53340cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:43.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.114+0000 7f5333fff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f533415f760 0x7f53340cec10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58012/0 (socket says 172.21.15.46:58012) 2023-12-02T00:28:43.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.114+0000 7f5333fff640 1 -- 172.21.15.46:0/15950693 learned_addr learned my addr 172.21.15.46:0/15950693 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:28:43.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.114+0000 7f5338dc2640 1 --2- 172.21.15.46:0/15950693 >> v2:172.21.15.162:3300/0 conn(0x7f533415b130 0x7f53340d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:43.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.114+0000 7f53395c3640 1 -- 172.21.15.46:0/15950693 >> v2:172.21.15.162:3300/0 conn(0x7f533415b130 msgr2=0x7f53340d01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:43.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.114+0000 7f53395c3640 1 --2- 172.21.15.46:0/15950693 >> v2:172.21.15.162:3300/0 conn(0x7f533415b130 0x7f53340d01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:43.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.114+0000 7f53395c3640 1 -- 172.21.15.46:0/15950693 >> v1:172.21.15.46:6789/0 conn(0x7f533415f760 legacy=0x7f53340cec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:43.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.114+0000 7f53395c3640 1 -- 172.21.15.46:0/15950693 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53340d0a50 con 0x7f53340cf320 2023-12-02T00:28:43.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.115+0000 7f53395c3640 1 --2- 172.21.15.46:0/15950693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53340cf320 0x7f53340cd310 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f533406ede0 tx=0x7f531c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=652cde5b34c3cfa server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:43.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.115+0000 7f53337fe640 1 -- 172.21.15.46:0/15950693 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531c02e070 con 0x7f53340cf320 2023-12-02T00:28:43.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.115+0000 7f53337fe640 1 -- 172.21.15.46:0/15950693 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f531c0315a0 con 0x7f53340cf320 2023-12-02T00:28:43.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.115+0000 7f53337fe640 1 -- 172.21.15.46:0/15950693 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531c031930 con 0x7f53340cf320 2023-12-02T00:28:43.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.115+0000 7f533b026640 1 -- 172.21.15.46:0/15950693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53340cf320 msgr2=0x7f53340cd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:43.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.115+0000 7f533b026640 1 --2- 172.21.15.46:0/15950693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53340cf320 0x7f53340cd310 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f533406ede0 tx=0x7f531c02b840 comp rx=0 tx=0).stop 2023-12-02T00:28:43.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.115+0000 7f533b026640 1 -- 172.21.15.46:0/15950693 shutdown_connections 2023-12-02T00:28:43.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.115+0000 7f533b026640 1 --2- 172.21.15.46:0/15950693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53340cf320 0x7f53340cd310 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:43.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.115+0000 7f533b026640 1 --2- 172.21.15.46:0/15950693 >> v2:172.21.15.162:3300/0 conn(0x7f533415b130 0x7f53340d01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:43.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.115+0000 7f533b026640 1 -- 172.21.15.46:0/15950693 >> 172.21.15.46:0/15950693 conn(0x7f53340b2c10 msgr2=0x7f53340b2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:43.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.115+0000 7f533b026640 1 -- 172.21.15.46:0/15950693 shutdown_connections 2023-12-02T00:28:43.118 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.115+0000 7f533b026640 1 -- 172.21.15.46:0/15950693 wait complete. 2023-12-02T00:28:43.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.116+0000 7f533b026640 1 Processor -- start 2023-12-02T00:28:43.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.117+0000 7f533b026640 1 -- start start 2023-12-02T00:28:43.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.117+0000 7f533b026640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53340cf320 0x7f5334155b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:43.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.117+0000 7f533b026640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f533415b130 0x7f53340c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:43.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.117+0000 7f533b026640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f533415f760 0x7f53341590b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:43.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.117+0000 7f533b026640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53340acc60 con 0x7f53340cf320 2023-12-02T00:28:43.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.117+0000 7f533b026640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53340ca780 con 0x7f533415b130 2023-12-02T00:28:43.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.117+0000 7f533b026640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53340caa60 con 0x7f533415f760 2023-12-02T00:28:43.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.117+0000 7f5338dc2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53340cf320 0x7f5334155b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:43.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.117+0000 7f5338dc2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53340cf320 0x7f5334155b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36874/0 (socket says 172.21.15.46:36874) 2023-12-02T00:28:43.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.118+0000 7f5338dc2640 1 -- 172.21.15.46:0/3652074857 learned_addr learned my addr 172.21.15.46:0/3652074857 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:43.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.118+0000 7f5333fff640 1 --2- 172.21.15.46:0/3652074857 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f533415b130 0x7f53340c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:43.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.118+0000 7f53395c3640 1 --2- 172.21.15.46:0/3652074857 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f533415f760 0x7f53341590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:43.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.118+0000 7f5338dc2640 1 -- 172.21.15.46:0/3652074857 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f533415f760 msgr2=0x7f53341590b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:43.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.118+0000 7f5338dc2640 1 --2- 172.21.15.46:0/3652074857 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f533415f760 0x7f53341590b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:43.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.118+0000 7f5338dc2640 1 -- 172.21.15.46:0/3652074857 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f533415b130 msgr2=0x7f53340c9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:43.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.118+0000 7f5338dc2640 1 --2- 172.21.15.46:0/3652074857 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f533415b130 0x7f53340c9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:43.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.118+0000 7f5338dc2640 1 -- 172.21.15.46:0/3652074857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f531c009400 con 0x7f53340cf320 2023-12-02T00:28:43.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.118+0000 7f53395c3640 1 --2- 172.21.15.46:0/3652074857 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f533415f760 0x7f53341590b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:28:43.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.118+0000 7f5338dc2640 1 --2- 172.21.15.46:0/3652074857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53340cf320 0x7f5334155b70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f532c00efc0 tx=0x7f532c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:43.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.118+0000 7f53317fa640 1 -- 172.21.15.46:0/3652074857 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532c014480 con 0x7f53340cf320 2023-12-02T00:28:43.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.118+0000 7f53317fa640 1 -- 172.21.15.46:0/3652074857 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f532c00f040 con 0x7f53340cf320 2023-12-02T00:28:43.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.118+0000 7f533b026640 1 -- 172.21.15.46:0/3652074857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53340cad40 con 0x7f53340cf320 2023-12-02T00:28:43.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.118+0000 7f53317fa640 1 -- 172.21.15.46:0/3652074857 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532c014800 con 0x7f53340cf320 2023-12-02T00:28:43.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.118+0000 7f533b026640 1 -- 172.21.15.46:0/3652074857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53341560b0 con 0x7f53340cf320 2023-12-02T00:28:43.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.119+0000 7f533b026640 1 -- 172.21.15.46:0/3652074857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5334067fc0 con 0x7f53340cf320 2023-12-02T00:28:43.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.120+0000 7f53317fa640 1 -- 172.21.15.46:0/3652074857 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f532c014960 con 0x7f53340cf320 2023-12-02T00:28:43.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.120+0000 7f53317fa640 1 --2- 172.21.15.46:0/3652074857 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f531406ae40 0x7f531406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:43.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.120+0000 7f53317fa640 1 -- 172.21.15.46:0/3652074857 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f532c08a040 con 0x7f53340cf320 2023-12-02T00:28:43.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.120+0000 7f5333fff640 1 --2- 172.21.15.46:0/3652074857 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f531406ae40 0x7f531406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:43.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.121+0000 7f5333fff640 1 --2- 172.21.15.46:0/3652074857 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f531406ae40 0x7f531406d300 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f53340b29b0 tx=0x7f5324009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:43.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.124+0000 7f53317fa640 1 -- 172.21.15.46:0/3652074857 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f532c0554d0 con 0x7f53340cf320 2023-12-02T00:28:43.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.298+0000 7f533b026640 1 -- 172.21.15.46:0/3652074857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5334067420 con 0x7f53340cf320 2023-12-02T00:28:43.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.300+0000 7f53317fa640 1 -- 172.21.15.46:0/3652074857 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f532c058af0 con 0x7f53340cf320 2023-12-02T00:28:43.301 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-12-02T00:28:43.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.302+0000 7f533b026640 1 -- 172.21.15.46:0/3652074857 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f531406ae40 msgr2=0x7f531406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:43.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.302+0000 7f533b026640 1 --2- 172.21.15.46:0/3652074857 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f531406ae40 0x7f531406d300 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f53340b29b0 tx=0x7f5324009040 comp rx=0 tx=0).stop 2023-12-02T00:28:43.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.302+0000 7f533b026640 1 -- 172.21.15.46:0/3652074857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53340cf320 msgr2=0x7f5334155b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:43.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.302+0000 7f533b026640 1 --2- 172.21.15.46:0/3652074857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53340cf320 0x7f5334155b70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f532c00efc0 tx=0x7f532c00c490 comp rx=0 tx=0).stop 2023-12-02T00:28:43.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.302+0000 7f533b026640 1 -- 172.21.15.46:0/3652074857 shutdown_connections 2023-12-02T00:28:43.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.302+0000 7f533b026640 1 --2- 172.21.15.46:0/3652074857 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f531406ae40 0x7f531406d300 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:43.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.302+0000 7f533b026640 1 --2- 172.21.15.46:0/3652074857 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f533415f760 0x7f53341590b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:43.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.302+0000 7f533b026640 1 --2- 172.21.15.46:0/3652074857 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f533415b130 0x7f53340c9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:43.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.302+0000 7f533b026640 1 --2- 172.21.15.46:0/3652074857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53340cf320 0x7f5334155b70 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:43.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.302+0000 7f533b026640 1 -- 172.21.15.46:0/3652074857 >> 172.21.15.46:0/3652074857 conn(0x7f53340b2c10 msgr2=0x7f53340b0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:43.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.302+0000 7f533b026640 1 -- 172.21.15.46:0/3652074857 shutdown_connections 2023-12-02T00:28:43.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.302+0000 7f533b026640 1 -- 172.21.15.46:0/3652074857 wait complete. 2023-12-02T00:28:43.314 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-02T00:28:43.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.418+0000 7fd102ebb640 1 Processor -- start 2023-12-02T00:28:43.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.418+0000 7fd102ebb640 1 -- start start 2023-12-02T00:28:43.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.419+0000 7fd102ebb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0fc15d6c0 0x7fd0fc06b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:43.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.419+0000 7fd102ebb640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd0fc0d1d50 0x7fd0fc06ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:43.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.419+0000 7fd102ebb640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0fc06ce00 con 0x7fd0fc15d2e0 2023-12-02T00:28:43.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.419+0000 7fd102ebb640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0fc06bf40 con 0x7fd0fc0d1d50 2023-12-02T00:28:43.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.419+0000 7fd102ebb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0fc06c1f0 con 0x7fd0fc15d6c0 2023-12-02T00:28:43.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.419+0000 7fd101458640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd0fc0d1d50 0x7fd0fc06ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:43.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.419+0000 7fd101458640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd0fc0d1d50 0x7fd0fc06ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36882/0 (socket says 172.21.15.46:36882) 2023-12-02T00:28:43.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.419+0000 7fd101458640 1 -- 172.21.15.46:0/105550218 learned_addr learned my addr 172.21.15.46:0/105550218 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:43.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.420+0000 7fd0fbfff640 1 --2- 172.21.15.46:0/105550218 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0fc15d6c0 0x7fd0fc06b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:43.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.420+0000 7fd101458640 1 -- 172.21.15.46:0/105550218 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0fc15d6c0 msgr2=0x7fd0fc06b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:43.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.420+0000 7fd101458640 1 --2- 172.21.15.46:0/105550218 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0fc15d6c0 0x7fd0fc06b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:43.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.420+0000 7fd101458640 1 -- 172.21.15.46:0/105550218 >> v1:172.21.15.162:6789/0 conn(0x7fd0fc15d2e0 legacy=0x7fd0fc0d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:43.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.420+0000 7fd101458640 1 -- 172.21.15.46:0/105550218 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0fc06c4d0 con 0x7fd0fc0d1d50 2023-12-02T00:28:43.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.420+0000 7fd0fbfff640 1 --2- 172.21.15.46:0/105550218 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0fc15d6c0 0x7fd0fc06b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:28:43.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.420+0000 7fd101458640 1 --2- 172.21.15.46:0/105550218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0fc0d1d50 0x7fd0fc06ba00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec009870 tx=0x7fd0ec02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f66156c4078d2f84 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:43.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.421+0000 7fd0fb7fe640 1 -- 172.21.15.46:0/105550218 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ec031640 con 0x7fd0fc0d1d50 2023-12-02T00:28:43.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.421+0000 7fd0fb7fe640 1 -- 172.21.15.46:0/105550218 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0ec0317a0 con 0x7fd0fc0d1d50 2023-12-02T00:28:43.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.421+0000 7fd0fb7fe640 1 -- 172.21.15.46:0/105550218 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ec031b30 con 0x7fd0fc0d1d50 2023-12-02T00:28:43.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.421+0000 7fd102ebb640 1 -- 172.21.15.46:0/105550218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0fc0d1d50 msgr2=0x7fd0fc06ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:43.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.421+0000 7fd102ebb640 1 --2- 172.21.15.46:0/105550218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0fc0d1d50 0x7fd0fc06ba00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec009870 tx=0x7fd0ec02b8f0 comp rx=0 tx=0).stop 2023-12-02T00:28:43.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.421+0000 7fd102ebb640 1 -- 172.21.15.46:0/105550218 shutdown_connections 2023-12-02T00:28:43.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.421+0000 7fd102ebb640 1 --2- 172.21.15.46:0/105550218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0fc0d1d50 0x7fd0fc06ba00 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:43.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.421+0000 7fd102ebb640 1 --2- 172.21.15.46:0/105550218 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0fc15d6c0 0x7fd0fc06b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:43.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.421+0000 7fd102ebb640 1 -- 172.21.15.46:0/105550218 >> 172.21.15.46:0/105550218 conn(0x7fd0fc09c580 msgr2=0x7fd0fc0d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:43.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.421+0000 7fd102ebb640 1 -- 172.21.15.46:0/105550218 shutdown_connections 2023-12-02T00:28:43.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.421+0000 7fd102ebb640 1 -- 172.21.15.46:0/105550218 wait complete. 2023-12-02T00:28:43.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.422+0000 7fd102ebb640 1 Processor -- start 2023-12-02T00:28:43.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.422+0000 7fd102ebb640 1 -- start start 2023-12-02T00:28:43.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.423+0000 7fd102ebb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0fc0d1d50 0x7fd0fc0cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:43.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.423+0000 7fd102ebb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0fc15d2e0 0x7fd0fc0b3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:43.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.423+0000 7fd102ebb640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd0fc15d6c0 0x7fd0fc0b3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:43.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.423+0000 7fd102ebb640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0fc15ed30 con 0x7fd0fc0d1d50 2023-12-02T00:28:43.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.423+0000 7fd102ebb640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0fc15f010 con 0x7fd0fc15d6c0 2023-12-02T00:28:43.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.423+0000 7fd102ebb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0fc15f2f0 con 0x7fd0fc15d2e0 2023-12-02T00:28:43.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.423+0000 7fd100c57640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0fc0d1d50 0x7fd0fc0cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:43.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.423+0000 7fd100c57640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0fc0d1d50 0x7fd0fc0cb4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36898/0 (socket says 172.21.15.46:36898) 2023-12-02T00:28:43.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.423+0000 7fd100c57640 1 -- 172.21.15.46:0/3606050463 learned_addr learned my addr 172.21.15.46:0/3606050463 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:43.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.424+0000 7fd0fbfff640 1 --2- 172.21.15.46:0/3606050463 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0fc15d2e0 0x7fd0fc0b3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:43.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.424+0000 7fd101458640 1 --2- 172.21.15.46:0/3606050463 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd0fc15d6c0 0x7fd0fc0b3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:43.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.424+0000 7fd100c57640 1 -- 172.21.15.46:0/3606050463 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0fc15d2e0 msgr2=0x7fd0fc0b3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:43.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.424+0000 7fd100c57640 1 --2- 172.21.15.46:0/3606050463 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0fc15d2e0 0x7fd0fc0b3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:43.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.424+0000 7fd100c57640 1 -- 172.21.15.46:0/3606050463 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd0fc15d6c0 msgr2=0x7fd0fc0b3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:43.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.424+0000 7fd100c57640 1 --2- 172.21.15.46:0/3606050463 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd0fc15d6c0 0x7fd0fc0b3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:43.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.424+0000 7fd100c57640 1 -- 172.21.15.46:0/3606050463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0ec0094d0 con 0x7fd0fc0d1d50 2023-12-02T00:28:43.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.424+0000 7fd100c57640 1 --2- 172.21.15.46:0/3606050463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0fc0d1d50 0x7fd0fc0cb4c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd0f400c870 tx=0x7fd0f400cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:43.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.424+0000 7fd0f97fa640 1 -- 172.21.15.46:0/3606050463 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4003b70 con 0x7fd0fc0d1d50 2023-12-02T00:28:43.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.424+0000 7fd0f97fa640 1 -- 172.21.15.46:0/3606050463 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f4003cf0 con 0x7fd0fc0d1d50 2023-12-02T00:28:43.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.424+0000 7fd102ebb640 1 -- 172.21.15.46:0/3606050463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0fc15f5d0 con 0x7fd0fc0d1d50 2023-12-02T00:28:43.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.424+0000 7fd0f97fa640 1 -- 172.21.15.46:0/3606050463 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f400aa90 con 0x7fd0fc0d1d50 2023-12-02T00:28:43.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.424+0000 7fd102ebb640 1 -- 172.21.15.46:0/3606050463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0fc15fdd0 con 0x7fd0fc0d1d50 2023-12-02T00:28:43.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.425+0000 7fd102ebb640 1 -- 172.21.15.46:0/3606050463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0fc151fb0 con 0x7fd0fc0d1d50 2023-12-02T00:28:43.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.425+0000 7fd0f97fa640 1 -- 172.21.15.46:0/3606050463 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd0f400abf0 con 0x7fd0fc0d1d50 2023-12-02T00:28:43.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.426+0000 7fd0f97fa640 1 --2- 172.21.15.46:0/3606050463 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd0e806ae40 0x7fd0e806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:43.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.426+0000 7fd0f97fa640 1 -- 172.21.15.46:0/3606050463 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd0f4089040 con 0x7fd0fc0d1d50 2023-12-02T00:28:43.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.426+0000 7fd0fbfff640 1 --2- 172.21.15.46:0/3606050463 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd0e806ae40 0x7fd0e806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:43.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.427+0000 7fd0fbfff640 1 --2- 172.21.15.46:0/3606050463 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd0e806ae40 0x7fd0e806d300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc0ca370 tx=0x7fd0f0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:43.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.430+0000 7fd0f97fa640 1 -- 172.21.15.46:0/3606050463 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd0f4054500 con 0x7fd0fc0d1d50 2023-12-02T00:28:43.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.644+0000 7fd102ebb640 1 -- 172.21.15.46:0/3606050463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd0fc067420 con 0x7fd0fc0d1d50 2023-12-02T00:28:43.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.644+0000 7fd0f97fa640 1 -- 172.21.15.46:0/3606050463 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1321 (secure 0 0 0) 0x7fd0f4003e60 con 0x7fd0fc0d1d50 2023-12-02T00:28:43.645 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:28:43.645 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/987532381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":987532381},{"type":"v1","addr":"172.21.15.46:6835","nonce":987532381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-12-02T00:28:43.646 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 2 2023-12-02T00:28:43.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.646+0000 7fd102ebb640 1 -- 172.21.15.46:0/3606050463 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd0e806ae40 msgr2=0x7fd0e806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:43.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.646+0000 7fd102ebb640 1 --2- 172.21.15.46:0/3606050463 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd0e806ae40 0x7fd0e806d300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc0ca370 tx=0x7fd0f0008040 comp rx=0 tx=0).stop 2023-12-02T00:28:43.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.646+0000 7fd102ebb640 1 -- 172.21.15.46:0/3606050463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0fc0d1d50 msgr2=0x7fd0fc0cb4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:43.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.646+0000 7fd102ebb640 1 --2- 172.21.15.46:0/3606050463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0fc0d1d50 0x7fd0fc0cb4c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd0f400c870 tx=0x7fd0f400cd30 comp rx=0 tx=0).stop 2023-12-02T00:28:43.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.646+0000 7fd102ebb640 1 -- 172.21.15.46:0/3606050463 shutdown_connections 2023-12-02T00:28:43.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.646+0000 7fd102ebb640 1 --2- 172.21.15.46:0/3606050463 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd0e806ae40 0x7fd0e806d300 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:43.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.646+0000 7fd102ebb640 1 --2- 172.21.15.46:0/3606050463 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd0fc15d6c0 0x7fd0fc0b3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:43.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.646+0000 7fd102ebb640 1 --2- 172.21.15.46:0/3606050463 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0fc15d2e0 0x7fd0fc0b3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:43.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.646+0000 7fd102ebb640 1 --2- 172.21.15.46:0/3606050463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0fc0d1d50 0x7fd0fc0cb4c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:43.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.646+0000 7fd102ebb640 1 -- 172.21.15.46:0/3606050463 >> 172.21.15.46:0/3606050463 conn(0x7fd0fc09c580 msgr2=0x7fd0fc0d3140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:43.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.646+0000 7fd102ebb640 1 -- 172.21.15.46:0/3606050463 shutdown_connections 2023-12-02T00:28:43.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:43.646+0000 7fd102ebb640 1 -- 172.21.15.46:0/3606050463 wait complete. 2023-12-02T00:28:45.658 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-02T00:28:45.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.776+0000 7febbcdb0640 1 Processor -- start 2023-12-02T00:28:45.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.776+0000 7febbcdb0640 1 -- start start 2023-12-02T00:28:45.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.777+0000 7febbcdb0640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7febb80d2910 0x7febb80d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:45.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.777+0000 7febbcdb0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7febb80d3730 0x7febb80b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:45.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.777+0000 7febbcdb0640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febb80b2b10 con 0x7febb815d330 2023-12-02T00:28:45.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.777+0000 7febbcdb0640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febb80d1590 con 0x7febb80d2910 2023-12-02T00:28:45.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.777+0000 7febbcdb0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febb80d1870 con 0x7febb80d3730 2023-12-02T00:28:45.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.777+0000 7febadd9b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7febb80d2910 0x7febb80d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:45.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.777+0000 7febadd9b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7febb80d2910 0x7febb80d31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36900/0 (socket says 172.21.15.46:36900) 2023-12-02T00:28:45.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.777+0000 7febadd9b640 1 -- 172.21.15.46:0/2687188078 learned_addr learned my addr 172.21.15.46:0/2687188078 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:45.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.778+0000 7febb6d9d640 1 --2- 172.21.15.46:0/2687188078 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7febb80d3730 0x7febb80b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:45.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.778+0000 7febadd9b640 1 -- 172.21.15.46:0/2687188078 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7febb80d3730 msgr2=0x7febb80b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:45.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.778+0000 7febadd9b640 1 --2- 172.21.15.46:0/2687188078 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7febb80d3730 0x7febb80b6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:45.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.778+0000 7febadd9b640 1 -- 172.21.15.46:0/2687188078 >> v1:172.21.15.162:6789/0 conn(0x7febb815d330 legacy=0x7febb80b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:45.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.778+0000 7febadd9b640 1 -- 172.21.15.46:0/2687188078 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb80d3b90 con 0x7febb80d2910 2023-12-02T00:28:45.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.779+0000 7febadd9b640 1 --2- 172.21.15.46:0/2687188078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febb80d2910 0x7febb80d31f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7feba0009870 tx=0x7feba002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2082626c1e0fe5d server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:45.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.779+0000 7febb5d9b640 1 -- 172.21.15.46:0/2687188078 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba0031640 con 0x7febb80d2910 2023-12-02T00:28:45.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.779+0000 7febb5d9b640 1 -- 172.21.15.46:0/2687188078 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba00317a0 con 0x7febb80d2910 2023-12-02T00:28:45.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.779+0000 7febb5d9b640 1 -- 172.21.15.46:0/2687188078 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba0031b30 con 0x7febb80d2910 2023-12-02T00:28:45.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.779+0000 7febbcdb0640 1 -- 172.21.15.46:0/2687188078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febb80d2910 msgr2=0x7febb80d31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:45.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.779+0000 7febbcdb0640 1 --2- 172.21.15.46:0/2687188078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febb80d2910 0x7febb80d31f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7feba0009870 tx=0x7feba002b8f0 comp rx=0 tx=0).stop 2023-12-02T00:28:45.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.779+0000 7febbcdb0640 1 -- 172.21.15.46:0/2687188078 shutdown_connections 2023-12-02T00:28:45.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.779+0000 7febbcdb0640 1 --2- 172.21.15.46:0/2687188078 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7febb80d3730 0x7febb80b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:45.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.779+0000 7febbcdb0640 1 --2- 172.21.15.46:0/2687188078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febb80d2910 0x7febb80d31f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:45.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.779+0000 7febbcdb0640 1 -- 172.21.15.46:0/2687188078 >> 172.21.15.46:0/2687188078 conn(0x7febb8072020 msgr2=0x7febb8072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:45.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.779+0000 7febbcdb0640 1 -- 172.21.15.46:0/2687188078 shutdown_connections 2023-12-02T00:28:45.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.779+0000 7febbcdb0640 1 -- 172.21.15.46:0/2687188078 wait complete. 2023-12-02T00:28:45.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.781+0000 7febbcdb0640 1 Processor -- start 2023-12-02T00:28:45.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.781+0000 7febbcdb0640 1 -- start start 2023-12-02T00:28:45.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.782+0000 7febbcdb0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febb80d2910 0x7febb81601c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:45.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.782+0000 7febbcdb0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7febb80d3730 0x7febb81f2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:45.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.782+0000 7febbcdb0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7febb815d330 0x7febb81f3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:45.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.782+0000 7febbcdb0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb81f4070 con 0x7febb80d2910 2023-12-02T00:28:45.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.782+0000 7febbcdb0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb81607c0 con 0x7febb80d3730 2023-12-02T00:28:45.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.782+0000 7febbcdb0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febb81f42c0 con 0x7febb815d330 2023-12-02T00:28:45.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.782+0000 7febb659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febb80d2910 0x7febb81601c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:45.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.782+0000 7febb659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febb80d2910 0x7febb81601c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36906/0 (socket says 172.21.15.46:36906) 2023-12-02T00:28:45.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.782+0000 7febb659c640 1 -- 172.21.15.46:0/1852284299 learned_addr learned my addr 172.21.15.46:0/1852284299 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:45.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.782+0000 7febadd9b640 1 --2- 172.21.15.46:0/1852284299 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7febb80d3730 0x7febb81f2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:45.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.782+0000 7febb6d9d640 1 --2- 172.21.15.46:0/1852284299 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7febb815d330 0x7febb81f3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:45.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.782+0000 7febb659c640 1 -- 172.21.15.46:0/1852284299 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7febb815d330 msgr2=0x7febb81f3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:45.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.782+0000 7febb659c640 1 --2- 172.21.15.46:0/1852284299 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7febb815d330 0x7febb81f3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:45.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.782+0000 7febb659c640 1 -- 172.21.15.46:0/1852284299 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7febb80d3730 msgr2=0x7febb81f2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:45.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.782+0000 7febb659c640 1 --2- 172.21.15.46:0/1852284299 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7febb80d3730 0x7febb81f2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:45.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.782+0000 7febb659c640 1 -- 172.21.15.46:0/1852284299 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba00094d0 con 0x7febb80d2910 2023-12-02T00:28:45.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.782+0000 7febb6d9d640 1 --2- 172.21.15.46:0/1852284299 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7febb815d330 0x7febb81f3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:28:45.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.782+0000 7febadd9b640 1 --2- 172.21.15.46:0/1852284299 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7febb80d3730 0x7febb81f2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:28:45.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.782+0000 7febb659c640 1 --2- 172.21.15.46:0/1852284299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febb80d2910 0x7febb81601c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7feba800c400 tx=0x7feba800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:45.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.783+0000 7febaf7fe640 1 -- 172.21.15.46:0/1852284299 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba80052d0 con 0x7febb80d2910 2023-12-02T00:28:45.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.783+0000 7febaf7fe640 1 -- 172.21.15.46:0/1852284299 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba8005430 con 0x7febb80d2910 2023-12-02T00:28:45.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.783+0000 7febaf7fe640 1 -- 172.21.15.46:0/1852284299 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba8003830 con 0x7febb80d2910 2023-12-02T00:28:45.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.783+0000 7febbcdb0640 1 -- 172.21.15.46:0/1852284299 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febb81f45a0 con 0x7febb80d2910 2023-12-02T00:28:45.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.783+0000 7febbcdb0640 1 -- 172.21.15.46:0/1852284299 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febb80ccae0 con 0x7febb80d2910 2023-12-02T00:28:45.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.784+0000 7febbcdb0640 1 -- 172.21.15.46:0/1852284299 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febb8069400 con 0x7febb80d2910 2023-12-02T00:28:45.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.784+0000 7febaf7fe640 1 -- 172.21.15.46:0/1852284299 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7feba8003990 con 0x7febb80d2910 2023-12-02T00:28:45.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.784+0000 7febaf7fe640 1 --2- 172.21.15.46:0/1852284299 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7feb9806ae40 0x7feb9806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:45.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.785+0000 7febaf7fe640 1 -- 172.21.15.46:0/1852284299 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7feba80887b0 con 0x7febb80d2910 2023-12-02T00:28:45.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.785+0000 7febadd9b640 1 --2- 172.21.15.46:0/1852284299 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7feb9806ae40 0x7feb9806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:45.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.785+0000 7febadd9b640 1 --2- 172.21.15.46:0/1852284299 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7feb9806ae40 0x7feb9806d300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7feba0036000 tx=0x7feba003c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:45.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:45.789+0000 7febaf7fe640 1 -- 172.21.15.46:0/1852284299 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7feba8053c40 con 0x7febb80d2910 2023-12-02T00:28:46.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.001+0000 7febbcdb0640 1 -- 172.21.15.46:0/1852284299 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7febb806bf60 con 0x7febb80d2910 2023-12-02T00:28:46.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.001+0000 7febaf7fe640 1 -- 172.21.15.46:0/1852284299 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3694 (secure 0 0 0) 0x7feba8017e80 con 0x7febb80d2910 2023-12-02T00:28:46.003 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:28:46.003 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/2059113401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2059113401},{"type":"v1","addr":"172.21.15.46:6834","nonce":2059113401}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/987532381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":987532381},{"type":"v1","addr":"172.21.15.46:6835","nonce":987532381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/449974862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":449974862},{"type":"v1","addr":"172.21.15.162:6836","nonce":449974862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4253,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/1231844478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":1231844478},{"type":"v1","addr":"172.21.15.162:6837","nonce":1231844478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-12-02T00:28:46.003 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 3 2023-12-02T00:28:46.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.003+0000 7febbcdb0640 1 -- 172.21.15.46:0/1852284299 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7feb9806ae40 msgr2=0x7feb9806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:46.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.003+0000 7febbcdb0640 1 --2- 172.21.15.46:0/1852284299 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7feb9806ae40 0x7feb9806d300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7feba0036000 tx=0x7feba003c040 comp rx=0 tx=0).stop 2023-12-02T00:28:46.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.003+0000 7febbcdb0640 1 -- 172.21.15.46:0/1852284299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febb80d2910 msgr2=0x7febb81601c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:46.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.003+0000 7febbcdb0640 1 --2- 172.21.15.46:0/1852284299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febb80d2910 0x7febb81601c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7feba800c400 tx=0x7feba800c8c0 comp rx=0 tx=0).stop 2023-12-02T00:28:46.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.003+0000 7febbcdb0640 1 -- 172.21.15.46:0/1852284299 shutdown_connections 2023-12-02T00:28:46.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.003+0000 7febbcdb0640 1 --2- 172.21.15.46:0/1852284299 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7feb9806ae40 0x7feb9806d300 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:46.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.003+0000 7febbcdb0640 1 --2- 172.21.15.46:0/1852284299 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7febb815d330 0x7febb81f3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:46.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.003+0000 7febbcdb0640 1 --2- 172.21.15.46:0/1852284299 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7febb80d3730 0x7febb81f2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:46.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.003+0000 7febbcdb0640 1 --2- 172.21.15.46:0/1852284299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febb80d2910 0x7febb81601c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:46.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.003+0000 7febbcdb0640 1 -- 172.21.15.46:0/1852284299 >> 172.21.15.46:0/1852284299 conn(0x7febb8072020 msgr2=0x7febb8072400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:46.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.003+0000 7febbcdb0640 1 -- 172.21.15.46:0/1852284299 shutdown_connections 2023-12-02T00:28:46.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.004+0000 7febbcdb0640 1 -- 172.21.15.46:0/1852284299 wait complete. 2023-12-02T00:28:46.015 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:28:46.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.121+0000 7fd64310c640 1 Processor -- start 2023-12-02T00:28:46.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.121+0000 7fd64310c640 1 -- start start 2023-12-02T00:28:46.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.121+0000 7fd64310c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd63c15d330 0x7fd63c0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:46.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.121+0000 7fd64310c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd63c0d2d90 0x7fd63c0b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:46.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.121+0000 7fd64310c640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd63c0b2b10 con 0x7fd63c0d2910 2023-12-02T00:28:46.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.121+0000 7fd64310c640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd63c0d1590 con 0x7fd63c0d2d90 2023-12-02T00:28:46.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.121+0000 7fd64310c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c0d1870 con 0x7fd63c15d330 2023-12-02T00:28:46.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.122+0000 7fd6416a9640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd63c0d2d90 0x7fd63c0b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:46.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.122+0000 7fd63bfff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd63c0d2910 0x7fd63c0d31f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58020/0 (socket says 172.21.15.46:58020) 2023-12-02T00:28:46.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.122+0000 7fd63bfff640 1 -- 172.21.15.46:0/924232634 learned_addr learned my addr 172.21.15.46:0/924232634 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:28:46.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.122+0000 7fd640ea8640 1 --2- 172.21.15.46:0/924232634 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd63c15d330 0x7fd63c0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:46.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.122+0000 7fd6416a9640 1 -- 172.21.15.46:0/924232634 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd63c15d330 msgr2=0x7fd63c0b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:46.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.122+0000 7fd6416a9640 1 --2- 172.21.15.46:0/924232634 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd63c15d330 0x7fd63c0b8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:46.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.122+0000 7fd6416a9640 1 -- 172.21.15.46:0/924232634 >> v1:172.21.15.46:6789/0 conn(0x7fd63c0d2910 legacy=0x7fd63c0d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:46.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.122+0000 7fd6416a9640 1 -- 172.21.15.46:0/924232634 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd63c0d1b50 con 0x7fd63c0d2d90 2023-12-02T00:28:46.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.122+0000 7fd63b7fe640 1 -- 172.21.15.46:0/924232634 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 549135384 0 0) 0x7fd63c0b2b10 con 0x7fd63c0d2910 2023-12-02T00:28:46.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.123+0000 7fd6416a9640 1 --2- 172.21.15.46:0/924232634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd63c0d2d90 0x7fd63c0b6390 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd634009d90 tx=0x7fd63402be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a212e97786f2567b server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:46.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.123+0000 7fd63b7fe640 1 -- 172.21.15.46:0/924232634 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63402cbc0 con 0x7fd63c0d2d90 2023-12-02T00:28:46.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.123+0000 7fd63b7fe640 1 -- 172.21.15.46:0/924232634 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd63402cd20 con 0x7fd63c0d2d90 2023-12-02T00:28:46.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.123+0000 7fd63b7fe640 1 -- 172.21.15.46:0/924232634 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd634034660 con 0x7fd63c0d2d90 2023-12-02T00:28:46.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.123+0000 7fd64310c640 1 -- 172.21.15.46:0/924232634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd63c0d2d90 msgr2=0x7fd63c0b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:46.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.123+0000 7fd64310c640 1 --2- 172.21.15.46:0/924232634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd63c0d2d90 0x7fd63c0b6390 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd634009d90 tx=0x7fd63402be10 comp rx=0 tx=0).stop 2023-12-02T00:28:46.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.123+0000 7fd64310c640 1 -- 172.21.15.46:0/924232634 shutdown_connections 2023-12-02T00:28:46.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.123+0000 7fd64310c640 1 --2- 172.21.15.46:0/924232634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd63c0d2d90 0x7fd63c0b6390 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:46.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.123+0000 7fd64310c640 1 --2- 172.21.15.46:0/924232634 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd63c15d330 0x7fd63c0b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:46.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.123+0000 7fd64310c640 1 -- 172.21.15.46:0/924232634 >> 172.21.15.46:0/924232634 conn(0x7fd63c072020 msgr2=0x7fd63c072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:46.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.123+0000 7fd64310c640 1 -- 172.21.15.46:0/924232634 shutdown_connections 2023-12-02T00:28:46.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.123+0000 7fd64310c640 1 -- 172.21.15.46:0/924232634 wait complete. 2023-12-02T00:28:46.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.124+0000 7fd64310c640 1 Processor -- start 2023-12-02T00:28:46.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.124+0000 7fd64310c640 1 -- start start 2023-12-02T00:28:46.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.125+0000 7fd64310c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd63c0d2910 0x7fd63c0cc470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:46.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.125+0000 7fd64310c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd63c0d2d90 0x7fd63c15e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:46.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.125+0000 7fd64310c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd63c15d330 0x7fd63c159310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:46.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.125+0000 7fd64310c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c160730 con 0x7fd63c15d330 2023-12-02T00:28:46.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.125+0000 7fd64310c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c15e8d0 con 0x7fd63c0d2d90 2023-12-02T00:28:46.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.125+0000 7fd64310c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c15ebb0 con 0x7fd63c0d2910 2023-12-02T00:28:46.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.125+0000 7fd6416a9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd63c15d330 0x7fd63c159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:46.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.125+0000 7fd6416a9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd63c15d330 0x7fd63c159310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36920/0 (socket says 172.21.15.46:36920) 2023-12-02T00:28:46.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.125+0000 7fd6416a9640 1 -- 172.21.15.46:0/1375851983 learned_addr learned my addr 172.21.15.46:0/1375851983 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:46.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.126+0000 7fd640ea8640 1 --2- 172.21.15.46:0/1375851983 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd63c0d2910 0x7fd63c0cc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:46.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.126+0000 7fd63bfff640 1 --2- 172.21.15.46:0/1375851983 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd63c0d2d90 0x7fd63c15e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:46.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.126+0000 7fd6416a9640 1 -- 172.21.15.46:0/1375851983 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd63c0d2910 msgr2=0x7fd63c0cc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:46.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.126+0000 7fd6416a9640 1 --2- 172.21.15.46:0/1375851983 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd63c0d2910 0x7fd63c0cc470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:46.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.126+0000 7fd6416a9640 1 -- 172.21.15.46:0/1375851983 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd63c0d2d90 msgr2=0x7fd63c15e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:46.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.126+0000 7fd6416a9640 1 --2- 172.21.15.46:0/1375851983 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd63c0d2d90 0x7fd63c15e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:46.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.126+0000 7fd6416a9640 1 -- 172.21.15.46:0/1375851983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd634009a70 con 0x7fd63c15d330 2023-12-02T00:28:46.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.126+0000 7fd6416a9640 1 --2- 172.21.15.46:0/1375851983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd63c15d330 0x7fd63c159310 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd634009d90 tx=0x7fd634004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:46.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.126+0000 7fd6397fa640 1 -- 172.21.15.46:0/1375851983 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd634040650 con 0x7fd63c15d330 2023-12-02T00:28:46.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.126+0000 7fd6397fa640 1 -- 172.21.15.46:0/1375851983 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd63402ce90 con 0x7fd63c15d330 2023-12-02T00:28:46.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.126+0000 7fd64310c640 1 -- 172.21.15.46:0/1375851983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd63c15ee90 con 0x7fd63c15d330 2023-12-02T00:28:46.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.126+0000 7fd6397fa640 1 -- 172.21.15.46:0/1375851983 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd634040a10 con 0x7fd63c15d330 2023-12-02T00:28:46.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.126+0000 7fd64310c640 1 -- 172.21.15.46:0/1375851983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd63c15f450 con 0x7fd63c15d330 2023-12-02T00:28:46.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.127+0000 7fd64310c640 1 -- 172.21.15.46:0/1375851983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd63c069400 con 0x7fd63c15d330 2023-12-02T00:28:46.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.127+0000 7fd6397fa640 1 -- 172.21.15.46:0/1375851983 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd634040b70 con 0x7fd63c15d330 2023-12-02T00:28:46.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.128+0000 7fd6397fa640 1 --2- 172.21.15.46:0/1375851983 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd62006ae40 0x7fd62006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:46.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.128+0000 7fd6397fa640 1 -- 172.21.15.46:0/1375851983 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd6340abf30 con 0x7fd63c15d330 2023-12-02T00:28:46.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.128+0000 7fd640ea8640 1 --2- 172.21.15.46:0/1375851983 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd62006ae40 0x7fd62006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:46.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.128+0000 7fd640ea8640 1 --2- 172.21.15.46:0/1375851983 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd62006ae40 0x7fd62006d300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd63000aac0 tx=0x7fd630008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:46.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.132+0000 7fd6397fa640 1 -- 172.21.15.46:0/1375851983 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd6340773e0 con 0x7fd63c15d330 2023-12-02T00:28:46.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.310+0000 7fd64310c640 1 -- 172.21.15.46:0/1375851983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd63c0cde90 con 0x7fd63c15d330 2023-12-02T00:28:46.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.310+0000 7fd6397fa640 1 -- 172.21.15.46:0/1375851983 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12826 (secure 0 0 0) 0x7fd63407aa00 con 0x7fd63c15d330 2023-12-02T00:28:46.311 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:28:46.312 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":17,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:28:40.294369+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:28:46.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.313+0000 7fd64310c640 1 -- 172.21.15.46:0/1375851983 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd62006ae40 msgr2=0x7fd62006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:46.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.313+0000 7fd64310c640 1 --2- 172.21.15.46:0/1375851983 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd62006ae40 0x7fd62006d300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd63000aac0 tx=0x7fd630008040 comp rx=0 tx=0).stop 2023-12-02T00:28:46.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.313+0000 7fd64310c640 1 -- 172.21.15.46:0/1375851983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd63c15d330 msgr2=0x7fd63c159310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:46.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.313+0000 7fd64310c640 1 --2- 172.21.15.46:0/1375851983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd63c15d330 0x7fd63c159310 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd634009d90 tx=0x7fd634004a40 comp rx=0 tx=0).stop 2023-12-02T00:28:46.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.313+0000 7fd64310c640 1 -- 172.21.15.46:0/1375851983 shutdown_connections 2023-12-02T00:28:46.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.313+0000 7fd64310c640 1 --2- 172.21.15.46:0/1375851983 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd62006ae40 0x7fd62006d300 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:46.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.313+0000 7fd64310c640 1 --2- 172.21.15.46:0/1375851983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd63c15d330 0x7fd63c159310 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:46.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.313+0000 7fd64310c640 1 --2- 172.21.15.46:0/1375851983 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd63c0d2d90 0x7fd63c15e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:46.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.313+0000 7fd64310c640 1 --2- 172.21.15.46:0/1375851983 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd63c0d2910 0x7fd63c0cc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:46.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.313+0000 7fd64310c640 1 -- 172.21.15.46:0/1375851983 >> 172.21.15.46:0/1375851983 conn(0x7fd63c072020 msgr2=0x7fd63c0b6350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:46.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.313+0000 7fd64310c640 1 -- 172.21.15.46:0/1375851983 shutdown_connections 2023-12-02T00:28:46.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.313+0000 7fd64310c640 1 -- 172.21.15.46:0/1375851983 wait complete. 2023-12-02T00:28:46.325 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-02T00:28:37.213645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-02T00:28:46.325 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-02T00:28:46.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.434+0000 7f9331a87640 1 Processor -- start 2023-12-02T00:28:46.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.434+0000 7f9331a87640 1 -- start start 2023-12-02T00:28:46.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.435+0000 7f9331a87640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f932c06f210 0x7f932c140800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:46.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.435+0000 7f9331a87640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f932c140d40 0x7f932c13f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:46.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.435+0000 7f9331a87640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f932c15ba40 con 0x7f932c15c2f0 2023-12-02T00:28:46.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.435+0000 7f9331a87640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f932c15c6d0 con 0x7f932c140d40 2023-12-02T00:28:46.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.435+0000 7f9331a87640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f932c13e530 con 0x7f932c06f210 2023-12-02T00:28:46.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.435+0000 7f932bfff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f932c140d40 0x7f932c13f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:46.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.435+0000 7f932bfff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f932c140d40 0x7f932c13f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36934/0 (socket says 172.21.15.46:36934) 2023-12-02T00:28:46.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.435+0000 7f932bfff640 1 -- 172.21.15.46:0/1289102865 learned_addr learned my addr 172.21.15.46:0/1289102865 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:46.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.435+0000 7f932affd640 1 --2- 172.21.15.46:0/1289102865 >> v2:172.21.15.162:3300/0 conn(0x7f932c06f210 0x7f932c140800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:46.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.435+0000 7f932bfff640 1 -- 172.21.15.46:0/1289102865 >> v2:172.21.15.162:3300/0 conn(0x7f932c06f210 msgr2=0x7f932c140800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:46.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.435+0000 7f932bfff640 1 --2- 172.21.15.46:0/1289102865 >> v2:172.21.15.162:3300/0 conn(0x7f932c06f210 0x7f932c140800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:46.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.435+0000 7f932bfff640 1 -- 172.21.15.46:0/1289102865 >> v1:172.21.15.162:6789/0 conn(0x7f932c15c2f0 legacy=0x7f932c06eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:46.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.435+0000 7f932bfff640 1 -- 172.21.15.46:0/1289102865 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f932c141190 con 0x7f932c140d40 2023-12-02T00:28:46.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.436+0000 7f932bfff640 1 --2- 172.21.15.46:0/1289102865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f932c140d40 0x7f932c13f330 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f932c0d20f0 tx=0x7f932402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b43b4d2271a6a61 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:46.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.436+0000 7f932a7fc640 1 -- 172.21.15.46:0/1289102865 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932402ca80 con 0x7f932c140d40 2023-12-02T00:28:46.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.436+0000 7f932a7fc640 1 -- 172.21.15.46:0/1289102865 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f932402cbe0 con 0x7f932c140d40 2023-12-02T00:28:46.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.436+0000 7f932a7fc640 1 -- 172.21.15.46:0/1289102865 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93240343f0 con 0x7f932c140d40 2023-12-02T00:28:46.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.436+0000 7f9331a87640 1 -- 172.21.15.46:0/1289102865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f932c140d40 msgr2=0x7f932c13f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:46.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.436+0000 7f9331a87640 1 --2- 172.21.15.46:0/1289102865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f932c140d40 0x7f932c13f330 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f932c0d20f0 tx=0x7f932402bee0 comp rx=0 tx=0).stop 2023-12-02T00:28:46.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.436+0000 7f9331a87640 1 -- 172.21.15.46:0/1289102865 shutdown_connections 2023-12-02T00:28:46.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.436+0000 7f9331a87640 1 --2- 172.21.15.46:0/1289102865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f932c140d40 0x7f932c13f330 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:46.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.436+0000 7f9331a87640 1 --2- 172.21.15.46:0/1289102865 >> v2:172.21.15.162:3300/0 conn(0x7f932c06f210 0x7f932c140800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:46.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.436+0000 7f9331a87640 1 -- 172.21.15.46:0/1289102865 >> 172.21.15.46:0/1289102865 conn(0x7f932c06a6b0 msgr2=0x7f932c06aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:46.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.437+0000 7f9331a87640 1 -- 172.21.15.46:0/1289102865 shutdown_connections 2023-12-02T00:28:46.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.437+0000 7f9331a87640 1 -- 172.21.15.46:0/1289102865 wait complete. 2023-12-02T00:28:46.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.438+0000 7f9331a87640 1 Processor -- start 2023-12-02T00:28:46.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.438+0000 7f9331a87640 1 -- start start 2023-12-02T00:28:46.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.439+0000 7f9331a87640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f932c06f210 0x7f932c153120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:46.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.439+0000 7f9331a87640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f932c140d40 0x7f932c06c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:46.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.439+0000 7f9331a87640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f932c15c2f0 0x7f932c0b5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:46.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.439+0000 7f9331a87640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932c153700 con 0x7f932c140d40 2023-12-02T00:28:46.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.439+0000 7f9331a87640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932c1539e0 con 0x7f932c15c2f0 2023-12-02T00:28:46.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.439+0000 7f9331a87640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f932c153cc0 con 0x7f932c06f210 2023-12-02T00:28:46.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.439+0000 7f932affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f932c140d40 0x7f932c06c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:46.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.439+0000 7f932affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f932c140d40 0x7f932c06c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36940/0 (socket says 172.21.15.46:36940) 2023-12-02T00:28:46.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.439+0000 7f932affd640 1 -- 172.21.15.46:0/2875924885 learned_addr learned my addr 172.21.15.46:0/2875924885 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:46.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.439+0000 7f932bfff640 1 --2- 172.21.15.46:0/2875924885 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f932c15c2f0 0x7f932c0b5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:46.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.439+0000 7f932b7fe640 1 --2- 172.21.15.46:0/2875924885 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f932c06f210 0x7f932c153120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:46.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.439+0000 7f932affd640 1 -- 172.21.15.46:0/2875924885 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f932c06f210 msgr2=0x7f932c153120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:46.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.439+0000 7f932affd640 1 --2- 172.21.15.46:0/2875924885 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f932c06f210 0x7f932c153120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:46.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.439+0000 7f932affd640 1 -- 172.21.15.46:0/2875924885 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f932c15c2f0 msgr2=0x7f932c0b5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:46.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.439+0000 7f932affd640 1 --2- 172.21.15.46:0/2875924885 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f932c15c2f0 0x7f932c0b5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:46.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.439+0000 7f932affd640 1 -- 172.21.15.46:0/2875924885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9324009b30 con 0x7f932c140d40 2023-12-02T00:28:46.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.439+0000 7f932bfff640 1 --2- 172.21.15.46:0/2875924885 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f932c15c2f0 0x7f932c0b5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:28:46.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.439+0000 7f932affd640 1 --2- 172.21.15.46:0/2875924885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f932c140d40 0x7f932c06c4c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f931c00d800 tx=0x7f931c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:46.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.439+0000 7f9313fff640 1 -- 172.21.15.46:0/2875924885 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c00b6c0 con 0x7f932c140d40 2023-12-02T00:28:46.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.439+0000 7f9313fff640 1 -- 172.21.15.46:0/2875924885 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f931c00b820 con 0x7f932c140d40 2023-12-02T00:28:46.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.439+0000 7f9313fff640 1 -- 172.21.15.46:0/2875924885 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c00bbb0 con 0x7f932c140d40 2023-12-02T00:28:46.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.439+0000 7f9331a87640 1 -- 172.21.15.46:0/2875924885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f932c153fa0 con 0x7f932c140d40 2023-12-02T00:28:46.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.439+0000 7f9331a87640 1 -- 172.21.15.46:0/2875924885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f932c06ca00 con 0x7f932c140d40 2023-12-02T00:28:46.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.440+0000 7f9331a87640 1 -- 172.21.15.46:0/2875924885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f932c13e810 con 0x7f932c140d40 2023-12-02T00:28:46.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.441+0000 7f9313fff640 1 -- 172.21.15.46:0/2875924885 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f931c00bd10 con 0x7f932c140d40 2023-12-02T00:28:46.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.441+0000 7f9313fff640 1 --2- 172.21.15.46:0/2875924885 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f931806ae40 0x7f931806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:46.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.441+0000 7f9313fff640 1 -- 172.21.15.46:0/2875924885 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f931c089860 con 0x7f932c140d40 2023-12-02T00:28:46.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.441+0000 7f932b7fe640 1 --2- 172.21.15.46:0/2875924885 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f931806ae40 0x7f931806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:46.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.442+0000 7f932b7fe640 1 --2- 172.21.15.46:0/2875924885 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f931806ae40 0x7f931806d300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9320006fd0 tx=0x7f9320008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:46.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.445+0000 7f9313fff640 1 -- 172.21.15.46:0/2875924885 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f931c054c60 con 0x7f932c140d40 2023-12-02T00:28:46.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.619+0000 7f9331a87640 1 -- 172.21.15.46:0/2875924885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f932c0cef60 con 0x7f932c140d40 2023-12-02T00:28:46.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.620+0000 7f9313fff640 1 -- 172.21.15.46:0/2875924885 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f931c058280 con 0x7f932c140d40 2023-12-02T00:28:46.621 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-12-02T00:28:46.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.621+0000 7f9331a87640 1 -- 172.21.15.46:0/2875924885 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f931806ae40 msgr2=0x7f931806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:46.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.621+0000 7f9331a87640 1 --2- 172.21.15.46:0/2875924885 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f931806ae40 0x7f931806d300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9320006fd0 tx=0x7f9320008040 comp rx=0 tx=0).stop 2023-12-02T00:28:46.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.621+0000 7f9331a87640 1 -- 172.21.15.46:0/2875924885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f932c140d40 msgr2=0x7f932c06c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:46.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.621+0000 7f9331a87640 1 --2- 172.21.15.46:0/2875924885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f932c140d40 0x7f932c06c4c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f931c00d800 tx=0x7f931c00dcc0 comp rx=0 tx=0).stop 2023-12-02T00:28:46.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.622+0000 7f9331a87640 1 -- 172.21.15.46:0/2875924885 shutdown_connections 2023-12-02T00:28:46.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.622+0000 7f9331a87640 1 --2- 172.21.15.46:0/2875924885 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f931806ae40 0x7f931806d300 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:46.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.622+0000 7f9331a87640 1 --2- 172.21.15.46:0/2875924885 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f932c15c2f0 0x7f932c0b5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:46.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.622+0000 7f9331a87640 1 --2- 172.21.15.46:0/2875924885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f932c140d40 0x7f932c06c4c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:46.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.622+0000 7f9331a87640 1 --2- 172.21.15.46:0/2875924885 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f932c06f210 0x7f932c153120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:46.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.622+0000 7f9331a87640 1 -- 172.21.15.46:0/2875924885 >> 172.21.15.46:0/2875924885 conn(0x7f932c06a6b0 msgr2=0x7f932c06aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:46.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.622+0000 7f9331a87640 1 -- 172.21.15.46:0/2875924885 shutdown_connections 2023-12-02T00:28:46.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.622+0000 7f9331a87640 1 -- 172.21.15.46:0/2875924885 wait complete. 2023-12-02T00:28:46.634 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-02T00:28:46.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.739+0000 7ff57c49b640 1 Processor -- start 2023-12-02T00:28:46.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.739+0000 7ff57c49b640 1 -- start start 2023-12-02T00:28:46.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.740+0000 7ff57c49b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff57415d2d0 0x7ff5740b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:46.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.740+0000 7ff57c49b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff5740ce150 0x7ff5740b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:46.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.740+0000 7ff57c49b640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5740d1bb0 con 0x7ff5740d2040 2023-12-02T00:28:46.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.740+0000 7ff57c49b640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff57415d6b0 con 0x7ff5740ce150 2023-12-02T00:28:46.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.740+0000 7ff57c49b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5740ce700 con 0x7ff57415d2d0 2023-12-02T00:28:46.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.740+0000 7ff57aa38640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff5740ce150 0x7ff5740b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:46.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.740+0000 7ff57aa38640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff5740ce150 0x7ff5740b51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36944/0 (socket says 172.21.15.46:36944) 2023-12-02T00:28:46.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.740+0000 7ff57aa38640 1 -- 172.21.15.46:0/2145884860 learned_addr learned my addr 172.21.15.46:0/2145884860 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:46.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.740+0000 7ff579a36640 1 --1- 172.21.15.46:0/2145884860 >> v1:172.21.15.46:6789/0 conn(0x7ff5740d2040 0x7ff5740cda40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58032/0 (socket says 172.21.15.46:58032) 2023-12-02T00:28:46.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.740+0000 7ff57a237640 1 --2- 172.21.15.46:0/2145884860 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff57415d2d0 0x7ff5740b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:46.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.740+0000 7ff57aa38640 1 -- 172.21.15.46:0/2145884860 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff57415d2d0 msgr2=0x7ff5740b4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:46.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.740+0000 7ff57aa38640 1 --2- 172.21.15.46:0/2145884860 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff57415d2d0 0x7ff5740b4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:46.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.740+0000 7ff57aa38640 1 -- 172.21.15.46:0/2145884860 >> v1:172.21.15.46:6789/0 conn(0x7ff5740d2040 legacy=0x7ff5740cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:46.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.740+0000 7ff57aa38640 1 -- 172.21.15.46:0/2145884860 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5740d1280 con 0x7ff5740ce150 2023-12-02T00:28:46.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.740+0000 7ff57a237640 1 --2- 172.21.15.46:0/2145884860 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff57415d2d0 0x7ff5740b4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:28:46.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.740+0000 7ff579235640 1 -- 172.21.15.46:0/2145884860 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1826301111 0 0) 0x7ff5740d1bb0 con 0x7ff5740d2040 2023-12-02T00:28:46.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.741+0000 7ff57aa38640 1 --2- 172.21.15.46:0/2145884860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5740ce150 0x7ff5740b51a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff56c009e50 tx=0x7ff56c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9094becde8d9836 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:46.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.741+0000 7ff579235640 1 -- 172.21.15.46:0/2145884860 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56c02cb00 con 0x7ff5740ce150 2023-12-02T00:28:46.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.741+0000 7ff579235640 1 -- 172.21.15.46:0/2145884860 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff56c02cc60 con 0x7ff5740ce150 2023-12-02T00:28:46.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.741+0000 7ff579235640 1 -- 172.21.15.46:0/2145884860 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56c0343f0 con 0x7ff5740ce150 2023-12-02T00:28:46.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.742+0000 7ff57c49b640 1 -- 172.21.15.46:0/2145884860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5740ce150 msgr2=0x7ff5740b51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:46.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.742+0000 7ff57c49b640 1 --2- 172.21.15.46:0/2145884860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5740ce150 0x7ff5740b51a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff56c009e50 tx=0x7ff56c02bf60 comp rx=0 tx=0).stop 2023-12-02T00:28:46.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.742+0000 7ff57c49b640 1 -- 172.21.15.46:0/2145884860 shutdown_connections 2023-12-02T00:28:46.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.742+0000 7ff57c49b640 1 --2- 172.21.15.46:0/2145884860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5740ce150 0x7ff5740b51a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:46.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.742+0000 7ff57c49b640 1 --2- 172.21.15.46:0/2145884860 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff57415d2d0 0x7ff5740b4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:46.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.742+0000 7ff57c49b640 1 -- 172.21.15.46:0/2145884860 >> 172.21.15.46:0/2145884860 conn(0x7ff574072050 msgr2=0x7ff574072460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:46.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.742+0000 7ff57c49b640 1 -- 172.21.15.46:0/2145884860 shutdown_connections 2023-12-02T00:28:46.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.742+0000 7ff57c49b640 1 -- 172.21.15.46:0/2145884860 wait complete. 2023-12-02T00:28:46.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.743+0000 7ff57c49b640 1 Processor -- start 2023-12-02T00:28:46.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.743+0000 7ff57c49b640 1 -- start start 2023-12-02T00:28:46.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.744+0000 7ff57c49b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5740ce150 0x7ff574147dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:46.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.744+0000 7ff57c49b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5740d2040 0x7ff574148640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:46.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.744+0000 7ff57c49b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff57415d2d0 0x7ff57415ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:46.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.744+0000 7ff57c49b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff574149090 con 0x7ff5740ce150 2023-12-02T00:28:46.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.744+0000 7ff57c49b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff574149370 con 0x7ff57415d2d0 2023-12-02T00:28:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.744+0000 7ff57c49b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff574149650 con 0x7ff5740d2040 2023-12-02T00:28:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.744+0000 7ff57a237640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5740ce150 0x7ff574147dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.744+0000 7ff57a237640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5740ce150 0x7ff574147dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36952/0 (socket says 172.21.15.46:36952) 2023-12-02T00:28:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.744+0000 7ff57a237640 1 -- 172.21.15.46:0/3650000512 learned_addr learned my addr 172.21.15.46:0/3650000512 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.744+0000 7ff57a237640 1 -- 172.21.15.46:0/3650000512 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5740d2040 msgr2=0x7ff574148640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.744+0000 7ff57a237640 1 --2- 172.21.15.46:0/3650000512 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5740d2040 0x7ff574148640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.744+0000 7ff57aa38640 1 --2- 172.21.15.46:0/3650000512 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff57415d2d0 0x7ff57415ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.744+0000 7ff57a237640 1 -- 172.21.15.46:0/3650000512 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff57415d2d0 msgr2=0x7ff57415ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.744+0000 7ff57a237640 1 --2- 172.21.15.46:0/3650000512 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff57415d2d0 0x7ff57415ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.744+0000 7ff57a237640 1 -- 172.21.15.46:0/3650000512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff56c009b30 con 0x7ff5740ce150 2023-12-02T00:28:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.744+0000 7ff57aa38640 1 --2- 172.21.15.46:0/3650000512 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff57415d2d0 0x7ff57415ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:28:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.744+0000 7ff57a237640 1 --2- 172.21.15.46:0/3650000512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5740ce150 0x7ff574147dc0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff57000c7f0 tx=0x7ff57000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.744+0000 7ff56affd640 1 -- 172.21.15.46:0/3650000512 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff570007820 con 0x7ff5740ce150 2023-12-02T00:28:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.744+0000 7ff56affd640 1 -- 172.21.15.46:0/3650000512 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff570007980 con 0x7ff5740ce150 2023-12-02T00:28:46.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.744+0000 7ff57c49b640 1 -- 172.21.15.46:0/3650000512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff574149930 con 0x7ff5740ce150 2023-12-02T00:28:46.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.744+0000 7ff56affd640 1 -- 172.21.15.46:0/3650000512 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff570007cf0 con 0x7ff5740ce150 2023-12-02T00:28:46.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.744+0000 7ff57c49b640 1 -- 172.21.15.46:0/3650000512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff574148300 con 0x7ff5740ce150 2023-12-02T00:28:46.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.745+0000 7ff57c49b640 1 -- 172.21.15.46:0/3650000512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff574069400 con 0x7ff5740ce150 2023-12-02T00:28:46.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.746+0000 7ff56affd640 1 -- 172.21.15.46:0/3650000512 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff5700113f0 con 0x7ff5740ce150 2023-12-02T00:28:46.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.746+0000 7ff56affd640 1 --2- 172.21.15.46:0/3650000512 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff55806ae40 0x7ff55806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:46.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.746+0000 7ff56affd640 1 -- 172.21.15.46:0/3650000512 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff570089600 con 0x7ff5740ce150 2023-12-02T00:28:46.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.746+0000 7ff579a36640 1 --2- 172.21.15.46:0/3650000512 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff55806ae40 0x7ff55806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:46.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.747+0000 7ff579a36640 1 --2- 172.21.15.46:0/3650000512 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff55806ae40 0x7ff55806d300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff564007fd0 tx=0x7ff564007920 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:46.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.750+0000 7ff56affd640 1 -- 172.21.15.46:0/3650000512 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff570054a00 con 0x7ff5740ce150 2023-12-02T00:28:46.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.931+0000 7ff57c49b640 1 -- 172.21.15.46:0/3650000512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff5740cda40 con 0x7ff5740ce150 2023-12-02T00:28:46.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.931+0000 7ff56affd640 1 -- 172.21.15.46:0/3650000512 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+65 (secure 0 0 0) 0x7ff57001f060 con 0x7ff5740ce150 2023-12-02T00:28:46.932 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:28:46.932 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2023-12-02T00:28:46.933 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-02T00:28:46.933 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2023-12-02T00:28:46.933 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2023-12-02T00:28:46.933 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:28:46.933 INFO:teuthology.orchestra.run.smithi046.stdout:] 2023-12-02T00:28:46.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.933+0000 7ff57c49b640 1 -- 172.21.15.46:0/3650000512 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff55806ae40 msgr2=0x7ff55806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:46.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.933+0000 7ff57c49b640 1 --2- 172.21.15.46:0/3650000512 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff55806ae40 0x7ff55806d300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff564007fd0 tx=0x7ff564007920 comp rx=0 tx=0).stop 2023-12-02T00:28:46.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.933+0000 7ff57c49b640 1 -- 172.21.15.46:0/3650000512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5740ce150 msgr2=0x7ff574147dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:46.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.933+0000 7ff57c49b640 1 --2- 172.21.15.46:0/3650000512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5740ce150 0x7ff574147dc0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff57000c7f0 tx=0x7ff57000ccb0 comp rx=0 tx=0).stop 2023-12-02T00:28:46.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.933+0000 7ff57c49b640 1 -- 172.21.15.46:0/3650000512 shutdown_connections 2023-12-02T00:28:46.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.933+0000 7ff57c49b640 1 --2- 172.21.15.46:0/3650000512 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff55806ae40 0x7ff55806d300 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:46.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.933+0000 7ff57c49b640 1 --2- 172.21.15.46:0/3650000512 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff57415d2d0 0x7ff57415ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:46.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.933+0000 7ff57c49b640 1 --2- 172.21.15.46:0/3650000512 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5740d2040 0x7ff574148640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:46.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.933+0000 7ff57c49b640 1 --2- 172.21.15.46:0/3650000512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5740ce150 0x7ff574147dc0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:46.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.933+0000 7ff57c49b640 1 -- 172.21.15.46:0/3650000512 >> 172.21.15.46:0/3650000512 conn(0x7ff574072050 msgr2=0x7ff5740b29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:46.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.933+0000 7ff57c49b640 1 -- 172.21.15.46:0/3650000512 shutdown_connections 2023-12-02T00:28:46.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:46.933+0000 7ff57c49b640 1 -- 172.21.15.46:0/3650000512 wait complete. 2023-12-02T00:28:46.945 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-02T00:28:46.945 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-02T00:28:47.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.054+0000 7fe2489d0640 1 Processor -- start 2023-12-02T00:28:47.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.054+0000 7fe2489d0640 1 -- start start 2023-12-02T00:28:47.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.055+0000 7fe2489d0640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe24415b5d0 0x7fe24415ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:47.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.055+0000 7fe2489d0640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe24415bf40 con 0x7fe2441448f0 2023-12-02T00:28:47.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.055+0000 7fe2489d0640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe24415f3a0 con 0x7fe24406a4d0 2023-12-02T00:28:47.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.055+0000 7fe2489d0640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2440714c0 con 0x7fe24415b5d0 2023-12-02T00:28:47.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.055+0000 7fe24259c640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe2441448f0 0x7fe24415ebc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58040/0 (socket says 172.21.15.46:58040) 2023-12-02T00:28:47.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.055+0000 7fe24259c640 1 -- 172.21.15.46:0/3958031090 learned_addr learned my addr 172.21.15.46:0/3958031090 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:28:47.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.056+0000 7fe24159a640 1 -- 172.21.15.46:0/3958031090 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1868702599 0 0) 0x7fe24415bf40 con 0x7fe2441448f0 2023-12-02T00:28:47.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.056+0000 7fe24159a640 1 -- 172.21.15.46:0/3958031090 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe230003520 con 0x7fe2441448f0 2023-12-02T00:28:47.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.056+0000 7fe242d9d640 1 --2- 172.21.15.46:0/3958031090 >> v2:172.21.15.162:3300/0 conn(0x7fe24415b5d0 0x7fe24415ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:47.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.056+0000 7fe24159a640 1 -- 172.21.15.46:0/3958031090 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 290859068 0 0) 0x7fe230003520 con 0x7fe2441448f0 2023-12-02T00:28:47.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.056+0000 7fe24159a640 1 -- 172.21.15.46:0/3958031090 >> v2:172.21.15.162:3300/0 conn(0x7fe24415b5d0 msgr2=0x7fe24415ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:47.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.056+0000 7fe24159a640 1 --2- 172.21.15.46:0/3958031090 >> v2:172.21.15.162:3300/0 conn(0x7fe24415b5d0 0x7fe24415ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:47.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.056+0000 7fe24159a640 1 -- 172.21.15.46:0/3958031090 >> v1:172.21.15.162:6789/0 conn(0x7fe24406a4d0 legacy=0x7fe244071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:47.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.056+0000 7fe24159a640 1 -- 172.21.15.46:0/3958031090 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe24415f680 con 0x7fe2441448f0 2023-12-02T00:28:47.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.056+0000 7fe24159a640 1 -- 172.21.15.46:0/3958031090 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7fe23c001670 con 0x7fe2441448f0 2023-12-02T00:28:47.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.056+0000 7fe24159a640 1 -- 172.21.15.46:0/3958031090 >> v1:172.21.15.46:6789/0 conn(0x7fe2441448f0 legacy=0x7fe24415ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:47.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.056+0000 7fe24159a640 1 --2- 172.21.15.46:0/3958031090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe230003970 0x7fe230003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:47.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.056+0000 7fe24159a640 1 --2- 172.21.15.46:0/3958031090 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2300047c0 0x7fe230004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:47.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.056+0000 7fe24159a640 1 --2- 172.21.15.46:0/3958031090 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe230005180 0x7fe23000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:47.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.056+0000 7fe24159a640 1 -- 172.21.15.46:0/3958031090 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2440714c0 con 0x7fe230003970 2023-12-02T00:28:47.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.056+0000 7fe24159a640 1 -- 172.21.15.46:0/3958031090 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23000c020 con 0x7fe230005180 2023-12-02T00:28:47.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.056+0000 7fe24159a640 1 -- 172.21.15.46:0/3958031090 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe23000c1e0 con 0x7fe2300047c0 2023-12-02T00:28:47.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.056+0000 7fe24259c640 1 --2- 172.21.15.46:0/3958031090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe230003970 0x7fe230003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:47.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.056+0000 7fe242d9d640 1 --2- 172.21.15.46:0/3958031090 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe230005180 0x7fe23000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:47.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.056+0000 7fe241d9b640 1 --2- 172.21.15.46:0/3958031090 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2300047c0 0x7fe230004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:47.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.056+0000 7fe24259c640 1 -- 172.21.15.46:0/3958031090 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2300047c0 msgr2=0x7fe230004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:47.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.056+0000 7fe24259c640 1 --2- 172.21.15.46:0/3958031090 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2300047c0 0x7fe230004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:47.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.056+0000 7fe24259c640 1 -- 172.21.15.46:0/3958031090 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe230005180 msgr2=0x7fe23000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:47.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.056+0000 7fe24259c640 1 --2- 172.21.15.46:0/3958031090 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe230005180 0x7fe23000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:47.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.056+0000 7fe24259c640 1 -- 172.21.15.46:0/3958031090 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe230003520 con 0x7fe230003970 2023-12-02T00:28:47.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.056+0000 7fe242d9d640 1 -- 172.21.15.46:0/3958031090 reap_dead start 2023-12-02T00:28:47.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.057+0000 7fe24259c640 1 --2- 172.21.15.46:0/3958031090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe230003970 0x7fe230003d90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe23c00c240 tx=0x7fe2300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c44fdc989cfb2770 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:47.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.057+0000 7fe24159a640 1 -- 172.21.15.46:0/3958031090 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23c005ad0 con 0x7fe230003970 2023-12-02T00:28:47.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.057+0000 7fe24159a640 1 -- 172.21.15.46:0/3958031090 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe23c005c30 con 0x7fe230003970 2023-12-02T00:28:47.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.057+0000 7fe24159a640 1 -- 172.21.15.46:0/3958031090 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23c02ca90 con 0x7fe230003970 2023-12-02T00:28:47.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.058+0000 7fe2489d0640 1 -- 172.21.15.46:0/3958031090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe230003970 msgr2=0x7fe230003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:47.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.058+0000 7fe2489d0640 1 --2- 172.21.15.46:0/3958031090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe230003970 0x7fe230003d90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe23c00c240 tx=0x7fe2300057f0 comp rx=0 tx=0).stop 2023-12-02T00:28:47.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.058+0000 7fe2489d0640 1 -- 172.21.15.46:0/3958031090 shutdown_connections 2023-12-02T00:28:47.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.058+0000 7fe2489d0640 1 --2- 172.21.15.46:0/3958031090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe230003970 0x7fe230003d90 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:47.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.058+0000 7fe2489d0640 1 -- 172.21.15.46:0/3958031090 >> 172.21.15.46:0/3958031090 conn(0x7fe2440b1de0 msgr2=0x7fe2440b21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:47.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.058+0000 7fe2489d0640 1 -- 172.21.15.46:0/3958031090 shutdown_connections 2023-12-02T00:28:47.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.058+0000 7fe2489d0640 1 -- 172.21.15.46:0/3958031090 wait complete. 2023-12-02T00:28:47.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.059+0000 7fe2489d0640 1 Processor -- start 2023-12-02T00:28:47.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.059+0000 7fe2489d0640 1 -- start start 2023-12-02T00:28:47.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.060+0000 7fe2489d0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe230003970 0x7fe24413e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:47.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.060+0000 7fe2489d0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe24415a6b0 0x7fe2440cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:47.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.060+0000 7fe2489d0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2440cb6f0 0x7fe2441425a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:47.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.060+0000 7fe2489d0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24413de50 con 0x7fe2440cb6f0 2023-12-02T00:28:47.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.060+0000 7fe2489d0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24413f3f0 con 0x7fe230003970 2023-12-02T00:28:47.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.060+0000 7fe2489d0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe24413d250 con 0x7fe24415a6b0 2023-12-02T00:28:47.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.060+0000 7fe242d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2440cb6f0 0x7fe2441425a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:47.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.060+0000 7fe242d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2440cb6f0 0x7fe2441425a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36972/0 (socket says 172.21.15.46:36972) 2023-12-02T00:28:47.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.060+0000 7fe242d9d640 1 -- 172.21.15.46:0/1515586626 learned_addr learned my addr 172.21.15.46:0/1515586626 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:47.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.060+0000 7fe24259c640 1 --2- 172.21.15.46:0/1515586626 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe230003970 0x7fe24413e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:47.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.060+0000 7fe241d9b640 1 --2- 172.21.15.46:0/1515586626 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe24415a6b0 0x7fe2440cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:47.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.060+0000 7fe242d9d640 1 -- 172.21.15.46:0/1515586626 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe24415a6b0 msgr2=0x7fe2440cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:47.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.060+0000 7fe242d9d640 1 --2- 172.21.15.46:0/1515586626 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe24415a6b0 0x7fe2440cb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:47.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.060+0000 7fe242d9d640 1 -- 172.21.15.46:0/1515586626 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe230003970 msgr2=0x7fe24413e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:47.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.060+0000 7fe242d9d640 1 --2- 172.21.15.46:0/1515586626 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe230003970 0x7fe24413e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:47.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.060+0000 7fe242d9d640 1 -- 172.21.15.46:0/1515586626 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe24413d4f0 con 0x7fe2440cb6f0 2023-12-02T00:28:47.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.060+0000 7fe24259c640 1 --2- 172.21.15.46:0/1515586626 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe230003970 0x7fe24413e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:28:47.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.060+0000 7fe241d9b640 1 --2- 172.21.15.46:0/1515586626 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe24415a6b0 0x7fe2440cb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:28:47.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.061+0000 7fe242d9d640 1 --2- 172.21.15.46:0/1515586626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2440cb6f0 0x7fe2441425a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe24406cf10 tx=0x7fe23400f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:47.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.061+0000 7fe23affd640 1 -- 172.21.15.46:0/1515586626 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23400e040 con 0x7fe2440cb6f0 2023-12-02T00:28:47.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.061+0000 7fe23affd640 1 -- 172.21.15.46:0/1515586626 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe23400f040 con 0x7fe2440cb6f0 2023-12-02T00:28:47.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.061+0000 7fe2489d0640 1 -- 172.21.15.46:0/1515586626 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2440cbd50 con 0x7fe2440cb6f0 2023-12-02T00:28:47.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.061+0000 7fe23affd640 1 -- 172.21.15.46:0/1515586626 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe234015430 con 0x7fe2440cb6f0 2023-12-02T00:28:47.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.061+0000 7fe2489d0640 1 -- 172.21.15.46:0/1515586626 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe24413da70 con 0x7fe2440cb6f0 2023-12-02T00:28:47.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.062+0000 7fe2489d0640 1 -- 172.21.15.46:0/1515586626 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe24413df90 con 0x7fe2440cb6f0 2023-12-02T00:28:47.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.062+0000 7fe23affd640 1 -- 172.21.15.46:0/1515586626 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe234015590 con 0x7fe2440cb6f0 2023-12-02T00:28:47.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.062+0000 7fe23affd640 1 --2- 172.21.15.46:0/1515586626 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe22406ae40 0x7fe22406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:47.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.063+0000 7fe23affd640 1 -- 172.21.15.46:0/1515586626 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe234089ce0 con 0x7fe2440cb6f0 2023-12-02T00:28:47.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.063+0000 7fe24259c640 1 --2- 172.21.15.46:0/1515586626 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe22406ae40 0x7fe22406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:47.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.063+0000 7fe24259c640 1 --2- 172.21.15.46:0/1515586626 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe22406ae40 0x7fe22406d300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe23c034000 tx=0x7fe23c009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:47.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.067+0000 7fe23affd640 1 -- 172.21.15.46:0/1515586626 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe24413df90 con 0x7fe2440cb6f0 2023-12-02T00:28:47.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.242+0000 7fe2489d0640 1 -- 172.21.15.46:0/1515586626 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe24413df90 con 0x7fe2440cb6f0 2023-12-02T00:28:47.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.708+0000 7fe23affd640 1 -- 172.21.15.46:0/1515586626 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe24413df90 con 0x7fe2440cb6f0 2023-12-02T00:28:47.709 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2023-12-02T00:28:47.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.711+0000 7fe2489d0640 1 -- 172.21.15.46:0/1515586626 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe22406ae40 msgr2=0x7fe22406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:47.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.711+0000 7fe2489d0640 1 --2- 172.21.15.46:0/1515586626 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe22406ae40 0x7fe22406d300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe23c034000 tx=0x7fe23c009030 comp rx=0 tx=0).stop 2023-12-02T00:28:47.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.711+0000 7fe2489d0640 1 -- 172.21.15.46:0/1515586626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2440cb6f0 msgr2=0x7fe2441425a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:47.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.712+0000 7fe2489d0640 1 --2- 172.21.15.46:0/1515586626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2440cb6f0 0x7fe2441425a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe24406cf10 tx=0x7fe23400f1b0 comp rx=0 tx=0).stop 2023-12-02T00:28:47.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.712+0000 7fe2489d0640 1 -- 172.21.15.46:0/1515586626 shutdown_connections 2023-12-02T00:28:47.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.712+0000 7fe2489d0640 1 --2- 172.21.15.46:0/1515586626 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe22406ae40 0x7fe22406d300 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:47.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.712+0000 7fe2489d0640 1 --2- 172.21.15.46:0/1515586626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2440cb6f0 0x7fe2441425a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:47.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.712+0000 7fe2489d0640 1 --2- 172.21.15.46:0/1515586626 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe24415a6b0 0x7fe2440cb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:47.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.712+0000 7fe2489d0640 1 --2- 172.21.15.46:0/1515586626 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe230003970 0x7fe24413e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:47.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.712+0000 7fe2489d0640 1 -- 172.21.15.46:0/1515586626 >> 172.21.15.46:0/1515586626 conn(0x7fe2440b1de0 msgr2=0x7fe2440b21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:47.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.712+0000 7fe2489d0640 1 -- 172.21.15.46:0/1515586626 shutdown_connections 2023-12-02T00:28:47.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.713+0000 7fe2489d0640 1 -- 172.21.15.46:0/1515586626 wait complete. 2023-12-02T00:28:47.728 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-02T00:28:47.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.841+0000 7fcbe0753640 1 Processor -- start 2023-12-02T00:28:47.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.841+0000 7fcbe0753640 1 -- start start 2023-12-02T00:28:47.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.842+0000 7fcbe0753640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcbd80b6c20 0x7fcbd80b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:47.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.842+0000 7fcbe0753640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fcbd80cd4a0 0x7fcbd8099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:47.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.842+0000 7fcbe0753640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbd80b7000 con 0x7fcbd8099fb0 2023-12-02T00:28:47.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.842+0000 7fcbe0753640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbd80b29f0 con 0x7fcbd80cd4a0 2023-12-02T00:28:47.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.842+0000 7fcbe0753640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd80cd920 con 0x7fcbd80b6c20 2023-12-02T00:28:47.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.843+0000 7fcbddcee640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fcbd80cd4a0 0x7fcbd8099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:47.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.843+0000 7fcbde4ef640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcbd80b6c20 0x7fcbd80b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:47.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.843+0000 7fcbddcee640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fcbd80cd4a0 0x7fcbd8099a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.46:33028/0 (socket says 172.21.15.46:33028) 2023-12-02T00:28:47.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.843+0000 7fcbdecf0640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fcbd8099fb0 0x7fcbd815f0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.46:59134/0 (socket says 172.21.15.46:59134) 2023-12-02T00:28:47.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.843+0000 7fcbddcee640 1 -- 172.21.15.46:0/904059178 learned_addr learned my addr 172.21.15.46:0/904059178 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:47.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.843+0000 7fcbddcee640 1 -- 172.21.15.46:0/904059178 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcbd80b6c20 msgr2=0x7fcbd80b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:47.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.843+0000 7fcbddcee640 1 --2- 172.21.15.46:0/904059178 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcbd80b6c20 0x7fcbd80b1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:47.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.843+0000 7fcbddcee640 1 -- 172.21.15.46:0/904059178 >> v1:172.21.15.162:6789/0 conn(0x7fcbd8099fb0 legacy=0x7fcbd815f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:47.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.843+0000 7fcbddcee640 1 -- 172.21.15.46:0/904059178 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd80d1db0 con 0x7fcbd80cd4a0 2023-12-02T00:28:47.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.843+0000 7fcbde4ef640 1 --2- 172.21.15.46:0/904059178 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcbd80b6c20 0x7fcbd80b1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:28:47.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.844+0000 7fcbddcee640 1 --2- 172.21.15.46:0/904059178 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcbd80cd4a0 0x7fcbd8099a70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc009870 tx=0x7fcbcc02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58a37cfbc838765d server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:47.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.844+0000 7fcbdd4ed640 1 -- 172.21.15.46:0/904059178 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc030840 con 0x7fcbd80cd4a0 2023-12-02T00:28:47.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.844+0000 7fcbdd4ed640 1 -- 172.21.15.46:0/904059178 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbcc0309a0 con 0x7fcbd80cd4a0 2023-12-02T00:28:47.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.844+0000 7fcbdd4ed640 1 -- 172.21.15.46:0/904059178 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc030d30 con 0x7fcbd80cd4a0 2023-12-02T00:28:47.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.845+0000 7fcbe0753640 1 -- 172.21.15.46:0/904059178 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcbd80cd4a0 msgr2=0x7fcbd8099a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:47.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.845+0000 7fcbe0753640 1 --2- 172.21.15.46:0/904059178 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcbd80cd4a0 0x7fcbd8099a70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc009870 tx=0x7fcbcc02b980 comp rx=0 tx=0).stop 2023-12-02T00:28:47.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.845+0000 7fcbe0753640 1 -- 172.21.15.46:0/904059178 shutdown_connections 2023-12-02T00:28:47.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.845+0000 7fcbe0753640 1 --2- 172.21.15.46:0/904059178 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcbd80cd4a0 0x7fcbd8099a70 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:47.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.845+0000 7fcbe0753640 1 --2- 172.21.15.46:0/904059178 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcbd80b6c20 0x7fcbd80b1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:47.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.845+0000 7fcbe0753640 1 -- 172.21.15.46:0/904059178 >> 172.21.15.46:0/904059178 conn(0x7fcbd80d1000 msgr2=0x7fcbd80d1410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:47.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.845+0000 7fcbe0753640 1 -- 172.21.15.46:0/904059178 shutdown_connections 2023-12-02T00:28:47.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.845+0000 7fcbe0753640 1 -- 172.21.15.46:0/904059178 wait complete. 2023-12-02T00:28:47.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.846+0000 7fcbe0753640 1 Processor -- start 2023-12-02T00:28:47.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.846+0000 7fcbe0753640 1 -- start start 2023-12-02T00:28:47.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.847+0000 7fcbe0753640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcbd8099fb0 0x7fcbd80c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:47.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.847+0000 7fcbe0753640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcbd80b6c20 0x7fcbd814d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:47.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.847+0000 7fcbe0753640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbd80cd4a0 0x7fcbd80c6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:47.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.847+0000 7fcbe0753640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd80c6030 con 0x7fcbd80cd4a0 2023-12-02T00:28:47.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.847+0000 7fcbe0753640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd80c53b0 con 0x7fcbd80b6c20 2023-12-02T00:28:47.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.847+0000 7fcbe0753640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd80c5690 con 0x7fcbd8099fb0 2023-12-02T00:28:47.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.847+0000 7fcbdecf0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbd80cd4a0 0x7fcbd80c6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:47.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.847+0000 7fcbdecf0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbd80cd4a0 0x7fcbd80c6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36984/0 (socket says 172.21.15.46:36984) 2023-12-02T00:28:47.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.847+0000 7fcbdecf0640 1 -- 172.21.15.46:0/4219496678 learned_addr learned my addr 172.21.15.46:0/4219496678 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:47.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.847+0000 7fcbde4ef640 1 --2- 172.21.15.46:0/4219496678 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcbd8099fb0 0x7fcbd80c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:47.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.847+0000 7fcbddcee640 1 --2- 172.21.15.46:0/4219496678 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcbd80b6c20 0x7fcbd814d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:47.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.847+0000 7fcbdecf0640 1 -- 172.21.15.46:0/4219496678 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcbd8099fb0 msgr2=0x7fcbd80c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:47.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.847+0000 7fcbdecf0640 1 --2- 172.21.15.46:0/4219496678 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcbd8099fb0 0x7fcbd80c4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:47.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.847+0000 7fcbdecf0640 1 -- 172.21.15.46:0/4219496678 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcbd80b6c20 msgr2=0x7fcbd814d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:47.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.847+0000 7fcbdecf0640 1 --2- 172.21.15.46:0/4219496678 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcbd80b6c20 0x7fcbd814d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:47.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.847+0000 7fcbdecf0640 1 -- 172.21.15.46:0/4219496678 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd0009a70 con 0x7fcbd80cd4a0 2023-12-02T00:28:47.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.847+0000 7fcbde4ef640 1 --2- 172.21.15.46:0/4219496678 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcbd8099fb0 0x7fcbd80c4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:28:47.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.847+0000 7fcbdecf0640 1 --2- 172.21.15.46:0/4219496678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbd80cd4a0 0x7fcbd80c6450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcbc800c400 tx=0x7fcbc800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:47.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.848+0000 7fcbd6ffd640 1 -- 172.21.15.46:0/4219496678 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc80074e0 con 0x7fcbd80cd4a0 2023-12-02T00:28:47.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.848+0000 7fcbd6ffd640 1 -- 172.21.15.46:0/4219496678 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbc8007640 con 0x7fcbd80cd4a0 2023-12-02T00:28:47.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.848+0000 7fcbe0753640 1 -- 172.21.15.46:0/4219496678 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbcc0094d0 con 0x7fcbd80cd4a0 2023-12-02T00:28:47.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.848+0000 7fcbd6ffd640 1 -- 172.21.15.46:0/4219496678 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc80049e0 con 0x7fcbd80cd4a0 2023-12-02T00:28:47.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.848+0000 7fcbe0753640 1 -- 172.21.15.46:0/4219496678 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbd80c5c50 con 0x7fcbd80cd4a0 2023-12-02T00:28:47.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.849+0000 7fcbe0753640 1 -- 172.21.15.46:0/4219496678 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbd80c6170 con 0x7fcbd80cd4a0 2023-12-02T00:28:47.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.849+0000 7fcbd6ffd640 1 -- 172.21.15.46:0/4219496678 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcbc8004b40 con 0x7fcbd80cd4a0 2023-12-02T00:28:47.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.850+0000 7fcbd6ffd640 1 --2- 172.21.15.46:0/4219496678 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fcbc406ae40 0x7fcbc406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:47.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.850+0000 7fcbd6ffd640 1 -- 172.21.15.46:0/4219496678 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fcbc8088b50 con 0x7fcbd80cd4a0 2023-12-02T00:28:47.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.850+0000 7fcbde4ef640 1 --2- 172.21.15.46:0/4219496678 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fcbc406ae40 0x7fcbc406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:47.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.850+0000 7fcbde4ef640 1 --2- 172.21.15.46:0/4219496678 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fcbc406ae40 0x7fcbc406d300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0009fd0 tx=0x7fcbd000b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:47.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:47.854+0000 7fcbd6ffd640 1 -- 172.21.15.46:0/4219496678 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcbd80c6170 con 0x7fcbd80cd4a0 2023-12-02T00:28:48.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.029+0000 7fcbe0753640 1 -- 172.21.15.46:0/4219496678 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fcbd80b7000 con 0x7fcbd80cd4a0 2023-12-02T00:28:48.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.711+0000 7fcbd6ffd640 1 -- 172.21.15.46:0/4219496678 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) v1 ==== 170+0+0 (secure 0 0 0) 0x7fcbd80b7000 con 0x7fcbd80cd4a0 2023-12-02T00:28:48.712 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2023-12-02T00:28:48.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.714+0000 7fcbe0753640 1 -- 172.21.15.46:0/4219496678 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fcbc406ae40 msgr2=0x7fcbc406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:48.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.714+0000 7fcbe0753640 1 --2- 172.21.15.46:0/4219496678 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fcbc406ae40 0x7fcbc406d300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0009fd0 tx=0x7fcbd000b040 comp rx=0 tx=0).stop 2023-12-02T00:28:48.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.714+0000 7fcbe0753640 1 -- 172.21.15.46:0/4219496678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbd80cd4a0 msgr2=0x7fcbd80c6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:48.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.714+0000 7fcbe0753640 1 --2- 172.21.15.46:0/4219496678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbd80cd4a0 0x7fcbd80c6450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcbc800c400 tx=0x7fcbc800c8c0 comp rx=0 tx=0).stop 2023-12-02T00:28:48.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.714+0000 7fcbe0753640 1 -- 172.21.15.46:0/4219496678 shutdown_connections 2023-12-02T00:28:48.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.714+0000 7fcbe0753640 1 --2- 172.21.15.46:0/4219496678 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fcbc406ae40 0x7fcbc406d300 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:48.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.714+0000 7fcbe0753640 1 --2- 172.21.15.46:0/4219496678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbd80cd4a0 0x7fcbd80c6450 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:48.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.714+0000 7fcbe0753640 1 --2- 172.21.15.46:0/4219496678 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcbd80b6c20 0x7fcbd814d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:48.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.714+0000 7fcbe0753640 1 --2- 172.21.15.46:0/4219496678 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcbd8099fb0 0x7fcbd80c4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:48.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.714+0000 7fcbe0753640 1 -- 172.21.15.46:0/4219496678 >> 172.21.15.46:0/4219496678 conn(0x7fcbd80d1000 msgr2=0x7fcbd806e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:48.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.714+0000 7fcbe0753640 1 -- 172.21.15.46:0/4219496678 shutdown_connections 2023-12-02T00:28:48.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.714+0000 7fcbe0753640 1 -- 172.21.15.46:0/4219496678 wait complete. 2023-12-02T00:28:48.740 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-02T00:28:48.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.854+0000 7fc0e2696640 1 Processor -- start 2023-12-02T00:28:48.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.854+0000 7fc0e2696640 1 -- start start 2023-12-02T00:28:48.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.855+0000 7fc0e2696640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc0dc15d2d0 0x7fc0dc0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:48.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.855+0000 7fc0e2696640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0dc160f70 0x7fc0dc0cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:48.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.855+0000 7fc0e2696640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0dc1614e0 con 0x7fc0dc0ce480 2023-12-02T00:28:48.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.855+0000 7fc0e2696640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0dc1617c0 con 0x7fc0dc15d2d0 2023-12-02T00:28:48.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.855+0000 7fc0e2696640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0dc161aa0 con 0x7fc0dc160f70 2023-12-02T00:28:48.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.855+0000 7fc0e0c33640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc0dc0ce480 0x7fc0dc06b570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58048/0 (socket says 172.21.15.46:58048) 2023-12-02T00:28:48.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.855+0000 7fc0e0c33640 1 -- 172.21.15.46:0/1147275065 learned_addr learned my addr 172.21.15.46:0/1147275065 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:28:48.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.855+0000 7fc0dbfff640 1 --2- 172.21.15.46:0/1147275065 >> v2:172.21.15.162:3300/0 conn(0x7fc0dc15d2d0 0x7fc0dc0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:48.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.855+0000 7fc0db7fe640 1 --2- 172.21.15.46:0/1147275065 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0dc160f70 0x7fc0dc0cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:48.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.855+0000 7fc0daffd640 1 -- 172.21.15.46:0/1147275065 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 182384141 0 0) 0x7fc0dc1614e0 con 0x7fc0dc0ce480 2023-12-02T00:28:48.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.855+0000 7fc0daffd640 1 -- 172.21.15.46:0/1147275065 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0c4003520 con 0x7fc0dc0ce480 2023-12-02T00:28:48.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.856+0000 7fc0dbfff640 1 -- 172.21.15.46:0/1147275065 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0dc160f70 msgr2=0x7fc0dc0cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:48.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.856+0000 7fc0dbfff640 1 --2- 172.21.15.46:0/1147275065 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0dc160f70 0x7fc0dc0cdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:48.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.856+0000 7fc0dbfff640 1 -- 172.21.15.46:0/1147275065 >> v1:172.21.15.46:6789/0 conn(0x7fc0dc0ce480 legacy=0x7fc0dc06b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:48.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.856+0000 7fc0daffd640 1 -- 172.21.15.46:0/1147275065 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1525364703 0 0) 0x7fc0c4003520 con 0x7fc0dc0ce480 2023-12-02T00:28:48.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.856+0000 7fc0dbfff640 1 -- 172.21.15.46:0/1147275065 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0dc161d80 con 0x7fc0dc15d2d0 2023-12-02T00:28:48.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.856+0000 7fc0db7fe640 1 --2- 172.21.15.46:0/1147275065 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0dc160f70 0x7fc0dc0cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:28:48.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.856+0000 7fc0dbfff640 1 --2- 172.21.15.46:0/1147275065 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0dc15d2d0 0x7fc0dc0b8b90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc0d00097a0 tx=0x7fc0d002b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29f6457f26c0c20d server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:48.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.857+0000 7fc0daffd640 1 -- 172.21.15.46:0/1147275065 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d40019f0 con 0x7fc0dc15d2d0 2023-12-02T00:28:48.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.857+0000 7fc0daffd640 1 -- 172.21.15.46:0/1147275065 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0d002e050 con 0x7fc0dc15d2d0 2023-12-02T00:28:48.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.857+0000 7fc0daffd640 1 -- 172.21.15.46:0/1147275065 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d0030710 con 0x7fc0dc15d2d0 2023-12-02T00:28:48.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.857+0000 7fc0e2696640 1 -- 172.21.15.46:0/1147275065 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0dc15d2d0 msgr2=0x7fc0dc0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:48.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.857+0000 7fc0e2696640 1 --2- 172.21.15.46:0/1147275065 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0dc15d2d0 0x7fc0dc0b8b90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc0d00097a0 tx=0x7fc0d002b790 comp rx=0 tx=0).stop 2023-12-02T00:28:48.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.857+0000 7fc0e2696640 1 -- 172.21.15.46:0/1147275065 shutdown_connections 2023-12-02T00:28:48.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.857+0000 7fc0e2696640 1 --2- 172.21.15.46:0/1147275065 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0dc160f70 0x7fc0dc0cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:48.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.857+0000 7fc0e2696640 1 --2- 172.21.15.46:0/1147275065 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0dc15d2d0 0x7fc0dc0b8b90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:48.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.857+0000 7fc0e2696640 1 -- 172.21.15.46:0/1147275065 >> 172.21.15.46:0/1147275065 conn(0x7fc0dc072230 msgr2=0x7fc0dc0cfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:48.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.857+0000 7fc0e2696640 1 -- 172.21.15.46:0/1147275065 shutdown_connections 2023-12-02T00:28:48.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.857+0000 7fc0e2696640 1 -- 172.21.15.46:0/1147275065 wait complete. 2023-12-02T00:28:48.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.858+0000 7fc0e2696640 1 Processor -- start 2023-12-02T00:28:48.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.859+0000 7fc0e2696640 1 -- start start 2023-12-02T00:28:48.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.859+0000 7fc0e2696640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0dc0ce480 0x7fc0dc1588b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:48.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.859+0000 7fc0e2696640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0dc15d2d0 0x7fc0dc15bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:48.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.859+0000 7fc0e2696640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0dc160f70 0x7fc0dc15a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:48.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.859+0000 7fc0e2696640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0dc14ebf0 con 0x7fc0dc15d2d0 2023-12-02T00:28:48.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.859+0000 7fc0e2696640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0dc15ac10 con 0x7fc0dc0ce480 2023-12-02T00:28:48.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.859+0000 7fc0e2696640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0dc15aef0 con 0x7fc0dc160f70 2023-12-02T00:28:48.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.860+0000 7fc0db7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0dc15d2d0 0x7fc0dc15bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:48.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.860+0000 7fc0db7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0dc15d2d0 0x7fc0dc15bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36996/0 (socket says 172.21.15.46:36996) 2023-12-02T00:28:48.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.860+0000 7fc0db7fe640 1 -- 172.21.15.46:0/1196531035 learned_addr learned my addr 172.21.15.46:0/1196531035 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:48.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.860+0000 7fc0dbfff640 1 --2- 172.21.15.46:0/1196531035 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0dc0ce480 0x7fc0dc1588b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:48.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.860+0000 7fc0e0c33640 1 --2- 172.21.15.46:0/1196531035 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0dc160f70 0x7fc0dc15a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:48.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.860+0000 7fc0db7fe640 1 -- 172.21.15.46:0/1196531035 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0dc160f70 msgr2=0x7fc0dc15a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:48.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.860+0000 7fc0db7fe640 1 --2- 172.21.15.46:0/1196531035 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0dc160f70 0x7fc0dc15a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:48.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.860+0000 7fc0db7fe640 1 -- 172.21.15.46:0/1196531035 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0dc0ce480 msgr2=0x7fc0dc1588b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:48.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.860+0000 7fc0db7fe640 1 --2- 172.21.15.46:0/1196531035 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0dc0ce480 0x7fc0dc1588b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:48.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.860+0000 7fc0db7fe640 1 -- 172.21.15.46:0/1196531035 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0cc009400 con 0x7fc0dc15d2d0 2023-12-02T00:28:48.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.860+0000 7fc0e0c33640 1 --2- 172.21.15.46:0/1196531035 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0dc160f70 0x7fc0dc15a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:28:48.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.860+0000 7fc0db7fe640 1 --2- 172.21.15.46:0/1196531035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0dc15d2d0 0x7fc0dc15bcb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc00e960 tx=0x7fc0cc00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:48.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.860+0000 7fc0d8ff9640 1 -- 172.21.15.46:0/1196531035 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0cc00ca00 con 0x7fc0dc15d2d0 2023-12-02T00:28:48.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.860+0000 7fc0d8ff9640 1 -- 172.21.15.46:0/1196531035 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0cc00cb60 con 0x7fc0dc15d2d0 2023-12-02T00:28:48.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.860+0000 7fc0e2696640 1 -- 172.21.15.46:0/1196531035 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0d0009400 con 0x7fc0dc15d2d0 2023-12-02T00:28:48.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.860+0000 7fc0d8ff9640 1 -- 172.21.15.46:0/1196531035 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0cc013470 con 0x7fc0dc15d2d0 2023-12-02T00:28:48.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.860+0000 7fc0e2696640 1 -- 172.21.15.46:0/1196531035 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0dc15b4b0 con 0x7fc0dc15d2d0 2023-12-02T00:28:48.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.861+0000 7fc0e2696640 1 -- 172.21.15.46:0/1196531035 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0dc069400 con 0x7fc0dc15d2d0 2023-12-02T00:28:48.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.862+0000 7fc0d8ff9640 1 -- 172.21.15.46:0/1196531035 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc0cc0135d0 con 0x7fc0dc15d2d0 2023-12-02T00:28:48.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.862+0000 7fc0d8ff9640 1 --2- 172.21.15.46:0/1196531035 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc0c806ae40 0x7fc0c806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:48.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.862+0000 7fc0d8ff9640 1 -- 172.21.15.46:0/1196531035 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fc0cc089710 con 0x7fc0dc15d2d0 2023-12-02T00:28:48.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.862+0000 7fc0dbfff640 1 --2- 172.21.15.46:0/1196531035 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc0c806ae40 0x7fc0c806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:48.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.863+0000 7fc0dbfff640 1 --2- 172.21.15.46:0/1196531035 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc0c806ae40 0x7fc0c806d300 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0034000 tx=0x7fc0d002f000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:48.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:48.867+0000 7fc0d8ff9640 1 -- 172.21.15.46:0/1196531035 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc0cc054960 con 0x7fc0dc15d2d0 2023-12-02T00:28:49.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.084+0000 7fc0e2696640 1 -- 172.21.15.46:0/1196531035 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fc0dc0d0aa0 con 0x7fc0dc15d2d0 2023-12-02T00:28:49.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.726+0000 7fc0d8ff9640 1 -- 172.21.15.46:0/1196531035 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fc0cc057f80 con 0x7fc0dc15d2d0 2023-12-02T00:28:49.727 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 2 and data pool 3 2023-12-02T00:28:49.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.729+0000 7fc0e2696640 1 -- 172.21.15.46:0/1196531035 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc0c806ae40 msgr2=0x7fc0c806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:49.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.729+0000 7fc0e2696640 1 --2- 172.21.15.46:0/1196531035 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc0c806ae40 0x7fc0c806d300 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0034000 tx=0x7fc0d002f000 comp rx=0 tx=0).stop 2023-12-02T00:28:49.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.729+0000 7fc0e2696640 1 -- 172.21.15.46:0/1196531035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0dc15d2d0 msgr2=0x7fc0dc15bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:49.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.729+0000 7fc0e2696640 1 --2- 172.21.15.46:0/1196531035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0dc15d2d0 0x7fc0dc15bcb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc00e960 tx=0x7fc0cc00ee20 comp rx=0 tx=0).stop 2023-12-02T00:28:49.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.729+0000 7fc0e2696640 1 -- 172.21.15.46:0/1196531035 shutdown_connections 2023-12-02T00:28:49.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.729+0000 7fc0e2696640 1 --2- 172.21.15.46:0/1196531035 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc0c806ae40 0x7fc0c806d300 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:49.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.729+0000 7fc0e2696640 1 --2- 172.21.15.46:0/1196531035 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0dc160f70 0x7fc0dc15a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:49.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.729+0000 7fc0e2696640 1 --2- 172.21.15.46:0/1196531035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0dc15d2d0 0x7fc0dc15bcb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:49.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.729+0000 7fc0e2696640 1 --2- 172.21.15.46:0/1196531035 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0dc0ce480 0x7fc0dc1588b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:49.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.729+0000 7fc0e2696640 1 -- 172.21.15.46:0/1196531035 >> 172.21.15.46:0/1196531035 conn(0x7fc0dc072230 msgr2=0x7fc0dc15dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:49.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.730+0000 7fc0e2696640 1 -- 172.21.15.46:0/1196531035 shutdown_connections 2023-12-02T00:28:49.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.730+0000 7fc0e2696640 1 -- 172.21.15.46:0/1196531035 wait complete. 2023-12-02T00:28:49.748 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:28:49.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.859+0000 7f2ce646b640 1 Processor -- start 2023-12-02T00:28:49.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.859+0000 7f2ce646b640 1 -- start start 2023-12-02T00:28:49.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.860+0000 7f2ce646b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2ce0140310 0x7f2ce01618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:49.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.860+0000 7f2ce646b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ce006abe0 0x7f2ce006efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:49.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.860+0000 7f2ce646b640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ce01406f0 con 0x7f2ce01409d0 2023-12-02T00:28:49.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.860+0000 7f2ce646b640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ce0161e20 con 0x7f2ce0140310 2023-12-02T00:28:49.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.860+0000 7f2ce646b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce0143960 con 0x7f2ce006abe0 2023-12-02T00:28:49.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.860+0000 7f2cdffff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2ce0140310 0x7f2ce01618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:49.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.860+0000 7f2cdffff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2ce0140310 0x7f2ce01618e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37000/0 (socket says 172.21.15.46:37000) 2023-12-02T00:28:49.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.860+0000 7f2cdffff640 1 -- 172.21.15.46:0/3805598645 learned_addr learned my addr 172.21.15.46:0/3805598645 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:49.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.860+0000 7f2cdf7fe640 1 --1- 172.21.15.46:0/3805598645 >> v1:172.21.15.46:6789/0 conn(0x7f2ce01409d0 0x7f2ce006a4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58056/0 (socket says 172.21.15.46:58056) 2023-12-02T00:28:49.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.860+0000 7f2ce4a08640 1 --2- 172.21.15.46:0/3805598645 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ce006abe0 0x7f2ce006efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:49.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.860+0000 7f2cdffff640 1 -- 172.21.15.46:0/3805598645 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ce006abe0 msgr2=0x7f2ce006efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:49.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.860+0000 7f2cdffff640 1 --2- 172.21.15.46:0/3805598645 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ce006abe0 0x7f2ce006efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:49.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.860+0000 7f2cdffff640 1 -- 172.21.15.46:0/3805598645 >> v1:172.21.15.46:6789/0 conn(0x7f2ce01409d0 legacy=0x7f2ce006a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:49.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.861+0000 7f2cdeffd640 1 -- 172.21.15.46:0/3805598645 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2204210786 0 0) 0x7f2ce01406f0 con 0x7f2ce01409d0 2023-12-02T00:28:49.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.861+0000 7f2cdffff640 1 -- 172.21.15.46:0/3805598645 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce0142de0 con 0x7f2ce0140310 2023-12-02T00:28:49.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.861+0000 7f2ce4a08640 1 --2- 172.21.15.46:0/3805598645 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ce006abe0 0x7f2ce006efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:28:49.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.861+0000 7f2cdffff640 1 --2- 172.21.15.46:0/3805598645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ce0140310 0x7f2ce01618e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4009870 tx=0x7f2cd402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8046b1adc15d73c6 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:49.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.861+0000 7f2cdeffd640 1 -- 172.21.15.46:0/3805598645 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd4031640 con 0x7f2ce0140310 2023-12-02T00:28:49.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.861+0000 7f2cdeffd640 1 -- 172.21.15.46:0/3805598645 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cd40317a0 con 0x7f2ce0140310 2023-12-02T00:28:49.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.861+0000 7f2cdeffd640 1 -- 172.21.15.46:0/3805598645 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd4031b30 con 0x7f2ce0140310 2023-12-02T00:28:49.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.862+0000 7f2ce646b640 1 -- 172.21.15.46:0/3805598645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ce0140310 msgr2=0x7f2ce01618e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:49.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.862+0000 7f2ce646b640 1 --2- 172.21.15.46:0/3805598645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ce0140310 0x7f2ce01618e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4009870 tx=0x7f2cd402b8f0 comp rx=0 tx=0).stop 2023-12-02T00:28:49.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.862+0000 7f2ce646b640 1 -- 172.21.15.46:0/3805598645 shutdown_connections 2023-12-02T00:28:49.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.862+0000 7f2ce646b640 1 --2- 172.21.15.46:0/3805598645 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ce006abe0 0x7f2ce006efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:49.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.862+0000 7f2ce646b640 1 --2- 172.21.15.46:0/3805598645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ce0140310 0x7f2ce01618e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:49.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.862+0000 7f2ce646b640 1 -- 172.21.15.46:0/3805598645 >> 172.21.15.46:0/3805598645 conn(0x7f2ce00b1e50 msgr2=0x7f2ce00b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:49.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.862+0000 7f2ce646b640 1 -- 172.21.15.46:0/3805598645 shutdown_connections 2023-12-02T00:28:49.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.862+0000 7f2ce646b640 1 -- 172.21.15.46:0/3805598645 wait complete. 2023-12-02T00:28:49.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.863+0000 7f2ce646b640 1 Processor -- start 2023-12-02T00:28:49.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.863+0000 7f2ce646b640 1 -- start start 2023-12-02T00:28:49.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.864+0000 7f2ce646b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ce006abe0 0x7f2ce00aeb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:49.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.864+0000 7f2ce646b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ce0140310 0x7f2ce01f5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:49.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.864+0000 7f2ce646b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ce01409d0 0x7f2ce01f6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:49.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.864+0000 7f2ce646b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce0154520 con 0x7f2ce006abe0 2023-12-02T00:28:49.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.864+0000 7f2ce646b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce00ae760 con 0x7f2ce0140310 2023-12-02T00:28:49.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.864+0000 7f2ce646b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce0155050 con 0x7f2ce01409d0 2023-12-02T00:28:49.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.864+0000 7f2cdffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ce006abe0 0x7f2ce00aeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:49.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.864+0000 7f2cdffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ce006abe0 0x7f2ce00aeb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37006/0 (socket says 172.21.15.46:37006) 2023-12-02T00:28:49.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.864+0000 7f2cdffff640 1 -- 172.21.15.46:0/4156894589 learned_addr learned my addr 172.21.15.46:0/4156894589 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:49.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.865+0000 7f2ce4a08640 1 --2- 172.21.15.46:0/4156894589 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ce01409d0 0x7f2ce01f6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:49.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.865+0000 7f2cdffff640 1 -- 172.21.15.46:0/4156894589 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ce01409d0 msgr2=0x7f2ce01f6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:49.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.865+0000 7f2cdf7fe640 1 --2- 172.21.15.46:0/4156894589 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ce0140310 0x7f2ce01f5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:49.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.865+0000 7f2cdffff640 1 --2- 172.21.15.46:0/4156894589 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ce01409d0 0x7f2ce01f6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:49.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.865+0000 7f2cdffff640 1 -- 172.21.15.46:0/4156894589 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ce0140310 msgr2=0x7f2ce01f5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:49.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.865+0000 7f2cdffff640 1 --2- 172.21.15.46:0/4156894589 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ce0140310 0x7f2ce01f5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:49.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.865+0000 7f2cdffff640 1 -- 172.21.15.46:0/4156894589 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd40094d0 con 0x7f2ce006abe0 2023-12-02T00:28:49.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.865+0000 7f2cdf7fe640 1 --2- 172.21.15.46:0/4156894589 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ce0140310 0x7f2ce01f5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:28:49.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.865+0000 7f2cdffff640 1 --2- 172.21.15.46:0/4156894589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ce006abe0 0x7f2ce00aeb80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4009870 tx=0x7f2cd40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:49.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.865+0000 7f2cdcff9640 1 -- 172.21.15.46:0/4156894589 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd403ac60 con 0x7f2ce006abe0 2023-12-02T00:28:49.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.865+0000 7f2cdcff9640 1 -- 172.21.15.46:0/4156894589 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cd403adc0 con 0x7f2ce006abe0 2023-12-02T00:28:49.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.865+0000 7f2cdcff9640 1 -- 172.21.15.46:0/4156894589 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd4044630 con 0x7f2ce006abe0 2023-12-02T00:28:49.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.865+0000 7f2ce646b640 1 -- 172.21.15.46:0/4156894589 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ce0154d70 con 0x7f2ce006abe0 2023-12-02T00:28:49.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.865+0000 7f2ce646b640 1 -- 172.21.15.46:0/4156894589 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ce00ae380 con 0x7f2ce006abe0 2023-12-02T00:28:49.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.866+0000 7f2ce646b640 1 -- 172.21.15.46:0/4156894589 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ce00ae510 con 0x7f2ce006abe0 2023-12-02T00:28:49.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.866+0000 7f2cdcff9640 1 -- 172.21.15.46:0/4156894589 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2cd402f050 con 0x7f2ce006abe0 2023-12-02T00:28:49.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.867+0000 7f2cdcff9640 1 --2- 172.21.15.46:0/4156894589 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2ccc06ae40 0x7f2ccc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:49.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.867+0000 7f2cdcff9640 1 -- 172.21.15.46:0/4156894589 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2cd40ab3f0 con 0x7f2ce006abe0 2023-12-02T00:28:49.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.867+0000 7f2cdf7fe640 1 --2- 172.21.15.46:0/4156894589 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2ccc06ae40 0x7f2ccc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:49.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.867+0000 7f2cdf7fe640 1 --2- 172.21.15.46:0/4156894589 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2ccc06ae40 0x7f2ccc06d300 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2ce00cf360 tx=0x7f2cd0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:49.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:49.871+0000 7f2cdcff9640 1 -- 172.21.15.46:0/4156894589 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2cd4076530 con 0x7f2ce006abe0 2023-12-02T00:28:50.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.051+0000 7f2ce646b640 1 -- 172.21.15.46:0/4156894589 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ce0069400 con 0x7f2ce006abe0 2023-12-02T00:28:50.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.052+0000 7f2cdcff9640 1 -- 172.21.15.46:0/4156894589 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16494 (secure 0 0 0) 0x7f2cd4031e60 con 0x7f2ce006abe0 2023-12-02T00:28:50.054 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:28:50.054 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":20,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:28:49.086187+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-02T00:28:47.273337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-02T00:28:48.060115+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:28:50.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.054+0000 7f2ce646b640 1 -- 172.21.15.46:0/4156894589 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2ccc06ae40 msgr2=0x7f2ccc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:50.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.054+0000 7f2ce646b640 1 --2- 172.21.15.46:0/4156894589 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2ccc06ae40 0x7f2ccc06d300 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2ce00cf360 tx=0x7f2cd0009040 comp rx=0 tx=0).stop 2023-12-02T00:28:50.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.054+0000 7f2ce646b640 1 -- 172.21.15.46:0/4156894589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ce006abe0 msgr2=0x7f2ce00aeb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:50.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.054+0000 7f2ce646b640 1 --2- 172.21.15.46:0/4156894589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ce006abe0 0x7f2ce00aeb80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4009870 tx=0x7f2cd40046c0 comp rx=0 tx=0).stop 2023-12-02T00:28:50.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.054+0000 7f2ce646b640 1 -- 172.21.15.46:0/4156894589 shutdown_connections 2023-12-02T00:28:50.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.054+0000 7f2ce646b640 1 --2- 172.21.15.46:0/4156894589 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2ccc06ae40 0x7f2ccc06d300 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:50.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.054+0000 7f2ce646b640 1 --2- 172.21.15.46:0/4156894589 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ce01409d0 0x7f2ce01f6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:50.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.054+0000 7f2ce646b640 1 --2- 172.21.15.46:0/4156894589 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ce0140310 0x7f2ce01f5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:50.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.054+0000 7f2ce646b640 1 --2- 172.21.15.46:0/4156894589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ce006abe0 0x7f2ce00aeb80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:50.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.054+0000 7f2ce646b640 1 -- 172.21.15.46:0/4156894589 >> 172.21.15.46:0/4156894589 conn(0x7f2ce00b1e50 msgr2=0x7f2ce00714a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:50.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.054+0000 7f2ce646b640 1 -- 172.21.15.46:0/4156894589 shutdown_connections 2023-12-02T00:28:50.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.054+0000 7f2ce646b640 1 -- 172.21.15.46:0/4156894589 wait complete. 2023-12-02T00:28:50.066 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:28:50.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.171+0000 7f5f5d8fc640 1 Processor -- start 2023-12-02T00:28:50.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.171+0000 7f5f5d8fc640 1 -- start start 2023-12-02T00:28:50.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.172+0000 7f5f5d8fc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f581409d0 0x7f5f581618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:50.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.172+0000 7f5f5d8fc640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f581406f0 con 0x7f5f58140310 2023-12-02T00:28:50.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.172+0000 7f5f5d8fc640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f58143960 con 0x7f5f5806abe0 2023-12-02T00:28:50.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.172+0000 7f5f5d8fc640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f58161e20 con 0x7f5f581409d0 2023-12-02T00:28:50.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.172+0000 7f5f56ffd640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5f58140310 0x7f5f5806a4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58058/0 (socket says 172.21.15.46:58058) 2023-12-02T00:28:50.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.172+0000 7f5f56ffd640 1 -- 172.21.15.46:0/2140378003 learned_addr learned my addr 172.21.15.46:0/2140378003 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:28:50.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.172+0000 7f5f55ffb640 1 -- 172.21.15.46:0/2140378003 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2972874659 0 0) 0x7f5f581406f0 con 0x7f5f58140310 2023-12-02T00:28:50.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.172+0000 7f5f55ffb640 1 -- 172.21.15.46:0/2140378003 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f44003520 con 0x7f5f58140310 2023-12-02T00:28:50.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.172+0000 7f5f577fe640 1 --2- 172.21.15.46:0/2140378003 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f581409d0 0x7f5f581618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:50.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.173+0000 7f5f55ffb640 1 -- 172.21.15.46:0/2140378003 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1631360987 0 0) 0x7f5f44003520 con 0x7f5f58140310 2023-12-02T00:28:50.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.173+0000 7f5f55ffb640 1 -- 172.21.15.46:0/2140378003 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f581409d0 msgr2=0x7f5f581618e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:50.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.173+0000 7f5f55ffb640 1 --2- 172.21.15.46:0/2140378003 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f581409d0 0x7f5f581618e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:50.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.173+0000 7f5f55ffb640 1 -- 172.21.15.46:0/2140378003 >> v1:172.21.15.162:6789/0 conn(0x7f5f5806abe0 legacy=0x7f5f5806efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:50.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.173+0000 7f5f55ffb640 1 -- 172.21.15.46:0/2140378003 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f58142de0 con 0x7f5f58140310 2023-12-02T00:28:50.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.173+0000 7f5f55ffb640 1 -- 172.21.15.46:0/2140378003 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7f5f50001670 con 0x7f5f58140310 2023-12-02T00:28:50.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.173+0000 7f5f55ffb640 1 -- 172.21.15.46:0/2140378003 >> v1:172.21.15.46:6789/0 conn(0x7f5f58140310 legacy=0x7f5f5806a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:50.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.173+0000 7f5f55ffb640 1 --2- 172.21.15.46:0/2140378003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f440039a0 0x7f5f44003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:50.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.173+0000 7f5f55ffb640 1 --2- 172.21.15.46:0/2140378003 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f440047c0 0x7f5f44004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:50.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.173+0000 7f5f55ffb640 1 --2- 172.21.15.46:0/2140378003 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f581409d0 0x7f5f440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:50.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.173+0000 7f5f55ffb640 1 -- 172.21.15.46:0/2140378003 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f58161e20 con 0x7f5f440039a0 2023-12-02T00:28:50.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.173+0000 7f5f55ffb640 1 -- 172.21.15.46:0/2140378003 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4400bae0 con 0x7f5f440047c0 2023-12-02T00:28:50.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.173+0000 7f5f55ffb640 1 -- 172.21.15.46:0/2140378003 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4400bca0 con 0x7f5f581409d0 2023-12-02T00:28:50.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.173+0000 7f5f56ffd640 1 --2- 172.21.15.46:0/2140378003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f440039a0 0x7f5f44003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:50.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.173+0000 7f5f567fc640 1 --2- 172.21.15.46:0/2140378003 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f440047c0 0x7f5f44004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:50.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.173+0000 7f5f577fe640 1 --2- 172.21.15.46:0/2140378003 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f581409d0 0x7f5f440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:50.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.173+0000 7f5f56ffd640 1 -- 172.21.15.46:0/2140378003 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f581409d0 msgr2=0x7f5f440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:50.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.173+0000 7f5f56ffd640 1 --2- 172.21.15.46:0/2140378003 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f581409d0 0x7f5f440051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:50.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.173+0000 7f5f56ffd640 1 -- 172.21.15.46:0/2140378003 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f440047c0 msgr2=0x7f5f44004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:50.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.173+0000 7f5f56ffd640 1 --2- 172.21.15.46:0/2140378003 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f440047c0 0x7f5f44004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:50.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.173+0000 7f5f56ffd640 1 -- 172.21.15.46:0/2140378003 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f44003520 con 0x7f5f440039a0 2023-12-02T00:28:50.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.174+0000 7f5f56ffd640 1 --2- 172.21.15.46:0/2140378003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f440039a0 0x7f5f44003dc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5f5000c240 tx=0x7f5f440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce69a24bb898f08a server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:50.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.174+0000 7f5f55ffb640 1 -- 172.21.15.46:0/2140378003 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f50005ad0 con 0x7f5f440039a0 2023-12-02T00:28:50.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.174+0000 7f5f55ffb640 1 -- 172.21.15.46:0/2140378003 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f50005c30 con 0x7f5f440039a0 2023-12-02T00:28:50.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.174+0000 7f5f55ffb640 1 -- 172.21.15.46:0/2140378003 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5002c8b0 con 0x7f5f440039a0 2023-12-02T00:28:50.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.174+0000 7f5f5d8fc640 1 -- 172.21.15.46:0/2140378003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f440039a0 msgr2=0x7f5f44003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:50.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.174+0000 7f5f5d8fc640 1 --2- 172.21.15.46:0/2140378003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f440039a0 0x7f5f44003dc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5f5000c240 tx=0x7f5f440057f0 comp rx=0 tx=0).stop 2023-12-02T00:28:50.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.174+0000 7f5f577fe640 1 -- 172.21.15.46:0/2140378003 reap_dead start 2023-12-02T00:28:50.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.174+0000 7f5f5d8fc640 1 -- 172.21.15.46:0/2140378003 shutdown_connections 2023-12-02T00:28:50.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.174+0000 7f5f5d8fc640 1 -- 172.21.15.46:0/2140378003 >> 172.21.15.46:0/2140378003 conn(0x7f5f580b1e50 msgr2=0x7f5f580b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:50.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.175+0000 7f5f5d8fc640 1 -- 172.21.15.46:0/2140378003 shutdown_connections 2023-12-02T00:28:50.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.175+0000 7f5f5d8fc640 1 -- 172.21.15.46:0/2140378003 wait complete. 2023-12-02T00:28:50.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.176+0000 7f5f5d8fc640 1 Processor -- start 2023-12-02T00:28:50.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.176+0000 7f5f5d8fc640 1 -- start start 2023-12-02T00:28:50.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.177+0000 7f5f5d8fc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f440039a0 0x7f5f581f24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:50.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.177+0000 7f5f5d8fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f580af370 0x7f5f58146170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:50.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.177+0000 7f5f5d8fc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f581466b0 0x7f5f580afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:50.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.177+0000 7f5f5d8fc640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f58155fa0 con 0x7f5f580af370 2023-12-02T00:28:50.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.177+0000 7f5f5d8fc640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f58146e30 con 0x7f5f440039a0 2023-12-02T00:28:50.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.177+0000 7f5f5d8fc640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f58146b50 con 0x7f5f581466b0 2023-12-02T00:28:50.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.177+0000 7f5f567fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f580af370 0x7f5f58146170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:50.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.177+0000 7f5f567fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f580af370 0x7f5f58146170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37028/0 (socket says 172.21.15.46:37028) 2023-12-02T00:28:50.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.177+0000 7f5f567fc640 1 -- 172.21.15.46:0/139084251 learned_addr learned my addr 172.21.15.46:0/139084251 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:50.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.177+0000 7f5f56ffd640 1 --2- 172.21.15.46:0/139084251 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f440039a0 0x7f5f581f24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:50.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.177+0000 7f5f577fe640 1 --2- 172.21.15.46:0/139084251 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f581466b0 0x7f5f580afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:50.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.177+0000 7f5f567fc640 1 -- 172.21.15.46:0/139084251 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f581466b0 msgr2=0x7f5f580afdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:50.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.177+0000 7f5f567fc640 1 --2- 172.21.15.46:0/139084251 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f581466b0 0x7f5f580afdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:50.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.177+0000 7f5f567fc640 1 -- 172.21.15.46:0/139084251 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f440039a0 msgr2=0x7f5f581f24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:50.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.177+0000 7f5f567fc640 1 --2- 172.21.15.46:0/139084251 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f440039a0 0x7f5f581f24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:50.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.177+0000 7f5f567fc640 1 -- 172.21.15.46:0/139084251 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f581561d0 con 0x7f5f580af370 2023-12-02T00:28:50.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.177+0000 7f5f567fc640 1 --2- 172.21.15.46:0/139084251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f580af370 0x7f5f58146170 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5f4000f5f0 tx=0x7f5f4000fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:50.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.177+0000 7f5f4ffff640 1 -- 172.21.15.46:0/139084251 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f400052e0 con 0x7f5f580af370 2023-12-02T00:28:50.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.177+0000 7f5f4ffff640 1 -- 172.21.15.46:0/139084251 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f40005440 con 0x7f5f580af370 2023-12-02T00:28:50.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.177+0000 7f5f5d8fc640 1 -- 172.21.15.46:0/139084251 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f581564b0 con 0x7f5f580af370 2023-12-02T00:28:50.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.177+0000 7f5f4ffff640 1 -- 172.21.15.46:0/139084251 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f40003810 con 0x7f5f580af370 2023-12-02T00:28:50.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.177+0000 7f5f5d8fc640 1 -- 172.21.15.46:0/139084251 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f581f29f0 con 0x7f5f580af370 2023-12-02T00:28:50.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.179+0000 7f5f5d8fc640 1 -- 172.21.15.46:0/139084251 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f58161030 con 0x7f5f580af370 2023-12-02T00:28:50.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.179+0000 7f5f4ffff640 1 -- 172.21.15.46:0/139084251 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5f40003970 con 0x7f5f580af370 2023-12-02T00:28:50.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.179+0000 7f5f4ffff640 1 --2- 172.21.15.46:0/139084251 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5f3806ae40 0x7f5f3806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:50.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.179+0000 7f5f4ffff640 1 -- 172.21.15.46:0/139084251 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5f40089510 con 0x7f5f580af370 2023-12-02T00:28:50.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.179+0000 7f5f56ffd640 1 --2- 172.21.15.46:0/139084251 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5f3806ae40 0x7f5f3806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:50.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.180+0000 7f5f56ffd640 1 --2- 172.21.15.46:0/139084251 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5f3806ae40 0x7f5f3806d300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5f50034000 tx=0x7f5f50009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:50.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.183+0000 7f5f4ffff640 1 -- 172.21.15.46:0/139084251 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5f400545c0 con 0x7f5f580af370 2023-12-02T00:28:50.260 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-02T00:28:50.263 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-02T00:28:50.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.362+0000 7f5f5d8fc640 1 -- 172.21.15.46:0/139084251 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5f58069400 con 0x7f5f580af370 2023-12-02T00:28:50.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.363+0000 7f5f4ffff640 1 -- 172.21.15.46:0/139084251 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16494 (secure 0 0 0) 0x7f5f40087e60 con 0x7f5f580af370 2023-12-02T00:28:50.364 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:28:50.365 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":20,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:28:49.086187+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-02T00:28:47.273337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-02T00:28:48.060115+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:28:50.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.365+0000 7f5f5d8fc640 1 -- 172.21.15.46:0/139084251 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5f3806ae40 msgr2=0x7f5f3806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:50.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.365+0000 7f5f5d8fc640 1 --2- 172.21.15.46:0/139084251 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5f3806ae40 0x7f5f3806d300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5f50034000 tx=0x7f5f50009030 comp rx=0 tx=0).stop 2023-12-02T00:28:50.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.365+0000 7f5f5d8fc640 1 -- 172.21.15.46:0/139084251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f580af370 msgr2=0x7f5f58146170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:50.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.365+0000 7f5f5d8fc640 1 --2- 172.21.15.46:0/139084251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f580af370 0x7f5f58146170 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5f4000f5f0 tx=0x7f5f4000fab0 comp rx=0 tx=0).stop 2023-12-02T00:28:50.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.365+0000 7f5f5d8fc640 1 -- 172.21.15.46:0/139084251 shutdown_connections 2023-12-02T00:28:50.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.365+0000 7f5f5d8fc640 1 --2- 172.21.15.46:0/139084251 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5f3806ae40 0x7f5f3806d300 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:50.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.365+0000 7f5f5d8fc640 1 --2- 172.21.15.46:0/139084251 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f581466b0 0x7f5f580afdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:50.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.365+0000 7f5f5d8fc640 1 --2- 172.21.15.46:0/139084251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f580af370 0x7f5f58146170 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:50.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.365+0000 7f5f5d8fc640 1 --2- 172.21.15.46:0/139084251 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f440039a0 0x7f5f581f24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:50.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.365+0000 7f5f5d8fc640 1 -- 172.21.15.46:0/139084251 >> 172.21.15.46:0/139084251 conn(0x7f5f580b1e50 msgr2=0x7f5f5806f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:50.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.365+0000 7f5f5d8fc640 1 -- 172.21.15.46:0/139084251 shutdown_connections 2023-12-02T00:28:50.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.365+0000 7f5f5d8fc640 1 -- 172.21.15.46:0/139084251 wait complete. 2023-12-02T00:28:50.377 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-02T00:28:50.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.486+0000 7fb686b90640 1 Processor -- start 2023-12-02T00:28:50.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.486+0000 7fb686b90640 1 -- start start 2023-12-02T00:28:50.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.487+0000 7fb686b90640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb68015b5d0 0x7fb68015ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:50.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.487+0000 7fb686b90640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb68015bf40 con 0x7fb6801448f0 2023-12-02T00:28:50.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.487+0000 7fb686b90640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb68015f3a0 con 0x7fb68006a4d0 2023-12-02T00:28:50.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.487+0000 7fb686b90640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6800714c0 con 0x7fb68015b5d0 2023-12-02T00:28:50.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.487+0000 7fb68492c640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb6801448f0 0x7fb68015ebc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58066/0 (socket says 172.21.15.46:58066) 2023-12-02T00:28:50.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.487+0000 7fb68512d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb68015b5d0 0x7fb68015ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:50.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.487+0000 7fb68492c640 1 -- 172.21.15.46:0/2853774437 learned_addr learned my addr 172.21.15.46:0/2853774437 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:28:50.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.487+0000 7fb68512d640 1 -- 172.21.15.46:0/2853774437 >> v1:172.21.15.162:6789/0 conn(0x7fb68006a4d0 legacy=0x7fb680071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:50.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.487+0000 7fb68512d640 1 -- 172.21.15.46:0/2853774437 >> v1:172.21.15.46:6789/0 conn(0x7fb6801448f0 legacy=0x7fb68015ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:50.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.487+0000 7fb68512d640 1 -- 172.21.15.46:0/2853774437 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb68015f680 con 0x7fb68015b5d0 2023-12-02T00:28:50.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.487+0000 7fb67f7fe640 1 -- 172.21.15.46:0/2853774437 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1530787877 0 0) 0x7fb68015bf40 con 0x7fb6801448f0 2023-12-02T00:28:50.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.488+0000 7fb68512d640 1 --2- 172.21.15.46:0/2853774437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb68015b5d0 0x7fb68015ba00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb678009d90 tx=0x7fb67802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7b538a0965feda5 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:50.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.488+0000 7fb67f7fe640 1 -- 172.21.15.46:0/2853774437 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67802cc60 con 0x7fb68015b5d0 2023-12-02T00:28:50.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.488+0000 7fb67f7fe640 1 -- 172.21.15.46:0/2853774437 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb67802cdc0 con 0x7fb68015b5d0 2023-12-02T00:28:50.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.488+0000 7fb67f7fe640 1 -- 172.21.15.46:0/2853774437 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb678034660 con 0x7fb68015b5d0 2023-12-02T00:28:50.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.489+0000 7fb686b90640 1 -- 172.21.15.46:0/2853774437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb68015b5d0 msgr2=0x7fb68015ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:50.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.489+0000 7fb686b90640 1 --2- 172.21.15.46:0/2853774437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb68015b5d0 0x7fb68015ba00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb678009d90 tx=0x7fb67802beb0 comp rx=0 tx=0).stop 2023-12-02T00:28:50.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.489+0000 7fb686b90640 1 -- 172.21.15.46:0/2853774437 shutdown_connections 2023-12-02T00:28:50.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.489+0000 7fb686b90640 1 --2- 172.21.15.46:0/2853774437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb68015b5d0 0x7fb68015ba00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:50.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.489+0000 7fb686b90640 1 -- 172.21.15.46:0/2853774437 >> 172.21.15.46:0/2853774437 conn(0x7fb6800b1de0 msgr2=0x7fb6800b21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:50.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.489+0000 7fb686b90640 1 -- 172.21.15.46:0/2853774437 shutdown_connections 2023-12-02T00:28:50.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.489+0000 7fb686b90640 1 -- 172.21.15.46:0/2853774437 wait complete. 2023-12-02T00:28:50.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.490+0000 7fb686b90640 1 Processor -- start 2023-12-02T00:28:50.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.490+0000 7fb686b90640 1 -- start start 2023-12-02T00:28:50.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.491+0000 7fb686b90640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb68006a4d0 0x7fb68015a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:50.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.491+0000 7fb686b90640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6801448f0 0x7fb68015bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:50.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.491+0000 7fb686b90640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb68015b5d0 0x7fb68015c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:50.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.491+0000 7fb686b90640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68015aa20 con 0x7fb6801448f0 2023-12-02T00:28:50.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.491+0000 7fb686b90640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6800ca470 con 0x7fb68015b5d0 2023-12-02T00:28:50.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.491+0000 7fb686b90640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6800ca750 con 0x7fb68006a4d0 2023-12-02T00:28:50.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.491+0000 7fb67ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6801448f0 0x7fb68015bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:50.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.491+0000 7fb67ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6801448f0 0x7fb68015bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37052/0 (socket says 172.21.15.46:37052) 2023-12-02T00:28:50.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.491+0000 7fb67ffff640 1 -- 172.21.15.46:0/1352218932 learned_addr learned my addr 172.21.15.46:0/1352218932 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:50.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.491+0000 7fb68512d640 1 --2- 172.21.15.46:0/1352218932 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb68015b5d0 0x7fb68015c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:50.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.491+0000 7fb68492c640 1 --2- 172.21.15.46:0/1352218932 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb68006a4d0 0x7fb68015a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:50.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.491+0000 7fb67ffff640 1 -- 172.21.15.46:0/1352218932 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb68006a4d0 msgr2=0x7fb68015a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:50.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.491+0000 7fb67ffff640 1 --2- 172.21.15.46:0/1352218932 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb68006a4d0 0x7fb68015a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:50.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.491+0000 7fb67ffff640 1 -- 172.21.15.46:0/1352218932 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb68015b5d0 msgr2=0x7fb68015c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:50.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.491+0000 7fb67ffff640 1 --2- 172.21.15.46:0/1352218932 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb68015b5d0 0x7fb68015c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:50.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.491+0000 7fb67ffff640 1 -- 172.21.15.46:0/1352218932 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb678009a70 con 0x7fb6801448f0 2023-12-02T00:28:50.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.491+0000 7fb68492c640 1 --2- 172.21.15.46:0/1352218932 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb68006a4d0 0x7fb68015a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:28:50.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.491+0000 7fb67ffff640 1 --2- 172.21.15.46:0/1352218932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6801448f0 0x7fb68015bc80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb66c00c430 tx=0x7fb66c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:50.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.492+0000 7fb67d7fa640 1 -- 172.21.15.46:0/1352218932 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66c011790 con 0x7fb6801448f0 2023-12-02T00:28:50.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.492+0000 7fb67d7fa640 1 -- 172.21.15.46:0/1352218932 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb66c0118f0 con 0x7fb6801448f0 2023-12-02T00:28:50.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.492+0000 7fb67d7fa640 1 -- 172.21.15.46:0/1352218932 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66c011ce0 con 0x7fb6801448f0 2023-12-02T00:28:50.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.492+0000 7fb686b90640 1 -- 172.21.15.46:0/1352218932 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6800caa30 con 0x7fb6801448f0 2023-12-02T00:28:50.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.492+0000 7fb686b90640 1 -- 172.21.15.46:0/1352218932 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6800caff0 con 0x7fb6801448f0 2023-12-02T00:28:50.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.493+0000 7fb686b90640 1 -- 172.21.15.46:0/1352218932 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb680067420 con 0x7fb6801448f0 2023-12-02T00:28:50.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.493+0000 7fb67d7fa640 1 -- 172.21.15.46:0/1352218932 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb66c01c460 con 0x7fb6801448f0 2023-12-02T00:28:50.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.493+0000 7fb67d7fa640 1 --2- 172.21.15.46:0/1352218932 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb66406ae40 0x7fb66406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:50.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.493+0000 7fb67d7fa640 1 -- 172.21.15.46:0/1352218932 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb66c0895f0 con 0x7fb6801448f0 2023-12-02T00:28:50.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.494+0000 7fb68492c640 1 --2- 172.21.15.46:0/1352218932 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb66406ae40 0x7fb66406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:50.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.494+0000 7fb68492c640 1 --2- 172.21.15.46:0/1352218932 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb66406ae40 0x7fb66406d300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb674000f60 tx=0x7fb674009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:50.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.498+0000 7fb67d7fa640 1 -- 172.21.15.46:0/1352218932 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb66c054610 con 0x7fb6801448f0 2023-12-02T00:28:50.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.711+0000 7fb686b90640 1 -- 172.21.15.46:0/1352218932 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb68006fe80 con 0x7fb6801448f0 2023-12-02T00:28:50.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.731+0000 7fb67d7fa640 1 -- 172.21.15.46:0/1352218932 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb66c057c30 con 0x7fb6801448f0 2023-12-02T00:28:50.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.732+0000 7fb686b90640 1 -- 172.21.15.46:0/1352218932 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb66406ae40 msgr2=0x7fb66406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:50.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.732+0000 7fb686b90640 1 --2- 172.21.15.46:0/1352218932 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb66406ae40 0x7fb66406d300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb674000f60 tx=0x7fb674009040 comp rx=0 tx=0).stop 2023-12-02T00:28:50.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.732+0000 7fb686b90640 1 -- 172.21.15.46:0/1352218932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6801448f0 msgr2=0x7fb68015bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:50.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.732+0000 7fb686b90640 1 --2- 172.21.15.46:0/1352218932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6801448f0 0x7fb68015bc80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb66c00c430 tx=0x7fb66c00c8f0 comp rx=0 tx=0).stop 2023-12-02T00:28:50.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.733+0000 7fb686b90640 1 -- 172.21.15.46:0/1352218932 shutdown_connections 2023-12-02T00:28:50.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.733+0000 7fb686b90640 1 --2- 172.21.15.46:0/1352218932 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb66406ae40 0x7fb66406d300 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:50.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.733+0000 7fb686b90640 1 --2- 172.21.15.46:0/1352218932 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb68015b5d0 0x7fb68015c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:50.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.733+0000 7fb686b90640 1 --2- 172.21.15.46:0/1352218932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6801448f0 0x7fb68015bc80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:50.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.733+0000 7fb686b90640 1 --2- 172.21.15.46:0/1352218932 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb68006a4d0 0x7fb68015a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:50.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.733+0000 7fb686b90640 1 -- 172.21.15.46:0/1352218932 >> 172.21.15.46:0/1352218932 conn(0x7fb6800b1de0 msgr2=0x7fb68015e620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:50.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.733+0000 7fb686b90640 1 -- 172.21.15.46:0/1352218932 shutdown_connections 2023-12-02T00:28:50.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.733+0000 7fb686b90640 1 -- 172.21.15.46:0/1352218932 wait complete. 2023-12-02T00:28:50.744 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-02T00:28:50.745 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-02T00:28:50.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.852+0000 7f502eaad640 1 Processor -- start 2023-12-02T00:28:50.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.852+0000 7f502eaad640 1 -- start start 2023-12-02T00:28:50.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.853+0000 7f502eaad640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f50280b4370 0x7f50280b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:50.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.853+0000 7f502eaad640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5028071770 con 0x7f50280ceb70 2023-12-02T00:28:50.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.853+0000 7f502eaad640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50280ccb60 con 0x7f5028144980 2023-12-02T00:28:50.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.853+0000 7f502eaad640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f502815ced0 con 0x7f50280b4370 2023-12-02T00:28:50.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.853+0000 7f5027fff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f50280ceb70 0x7f50280cc450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58074/0 (socket says 172.21.15.46:58074) 2023-12-02T00:28:50.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.853+0000 7f5027fff640 1 -- 172.21.15.46:0/37839002 learned_addr learned my addr 172.21.15.46:0/37839002 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:28:50.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.853+0000 7f502d04a640 1 --2- 172.21.15.46:0/37839002 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f50280b4370 0x7f50280b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:50.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.853+0000 7f50277fe640 1 -- 172.21.15.46:0/37839002 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3196848563 0 0) 0x7f5028071770 con 0x7f50280ceb70 2023-12-02T00:28:50.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.853+0000 7f50277fe640 1 -- 172.21.15.46:0/37839002 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5014003520 con 0x7f50280ceb70 2023-12-02T00:28:50.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.854+0000 7f50277fe640 1 -- 172.21.15.46:0/37839002 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 63574969 0 0) 0x7f5014003520 con 0x7f50280ceb70 2023-12-02T00:28:50.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.854+0000 7f50277fe640 1 -- 172.21.15.46:0/37839002 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f50280b4370 msgr2=0x7f50280b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:50.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.854+0000 7f50277fe640 1 --2- 172.21.15.46:0/37839002 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f50280b4370 0x7f50280b47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:50.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.854+0000 7f50277fe640 1 -- 172.21.15.46:0/37839002 >> v1:172.21.15.162:6789/0 conn(0x7f5028144980 legacy=0x7f502815c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:50.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.854+0000 7f50277fe640 1 -- 172.21.15.46:0/37839002 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50280b4ce0 con 0x7f50280ceb70 2023-12-02T00:28:50.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.854+0000 7f50277fe640 1 -- 172.21.15.46:0/37839002 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7f5018001670 con 0x7f50280ceb70 2023-12-02T00:28:50.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.854+0000 7f50277fe640 1 -- 172.21.15.46:0/37839002 >> v1:172.21.15.46:6789/0 conn(0x7f50280ceb70 legacy=0x7f50280cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:50.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.854+0000 7f50277fe640 1 --2- 172.21.15.46:0/37839002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50140039a0 0x7f5014003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:50.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.854+0000 7f50277fe640 1 --2- 172.21.15.46:0/37839002 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5014004370 0x7f5014004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:50.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.854+0000 7f50277fe640 1 --2- 172.21.15.46:0/37839002 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f50280b4370 0x7f50140050a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:50.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.854+0000 7f50277fe640 1 -- 172.21.15.46:0/37839002 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502815ced0 con 0x7f50140039a0 2023-12-02T00:28:50.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.854+0000 7f50277fe640 1 -- 172.21.15.46:0/37839002 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f501400b9b0 con 0x7f5014004370 2023-12-02T00:28:50.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.854+0000 7f50277fe640 1 -- 172.21.15.46:0/37839002 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f501400bb70 con 0x7f50280b4370 2023-12-02T00:28:50.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.854+0000 7f502c849640 1 --2- 172.21.15.46:0/37839002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50140039a0 0x7f5014003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:50.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.854+0000 7f5027fff640 1 --2- 172.21.15.46:0/37839002 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5014004370 0x7f5014004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:50.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.854+0000 7f502d04a640 1 --2- 172.21.15.46:0/37839002 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f50280b4370 0x7f50140050a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:50.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.854+0000 7f502c849640 1 -- 172.21.15.46:0/37839002 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f50280b4370 msgr2=0x7f50140050a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:50.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.854+0000 7f502c849640 1 --2- 172.21.15.46:0/37839002 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f50280b4370 0x7f50140050a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:50.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.854+0000 7f502c849640 1 -- 172.21.15.46:0/37839002 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5014004370 msgr2=0x7f5014004840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:50.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.854+0000 7f502c849640 1 --2- 172.21.15.46:0/37839002 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5014004370 0x7f5014004840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:50.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.854+0000 7f502c849640 1 -- 172.21.15.46:0/37839002 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5014003520 con 0x7f50140039a0 2023-12-02T00:28:50.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.855+0000 7f502c849640 1 --2- 172.21.15.46:0/37839002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50140039a0 0x7f5014003da0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5020001f60 tx=0x7f50140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d2a27af807fad64 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:50.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.855+0000 7f50277fe640 1 -- 172.21.15.46:0/37839002 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502002c710 con 0x7f50140039a0 2023-12-02T00:28:50.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.855+0000 7f50277fe640 1 -- 172.21.15.46:0/37839002 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f502002c890 con 0x7f50140039a0 2023-12-02T00:28:50.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.855+0000 7f50277fe640 1 -- 172.21.15.46:0/37839002 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502002cc40 con 0x7f50140039a0 2023-12-02T00:28:50.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.855+0000 7f502eaad640 1 -- 172.21.15.46:0/37839002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50140039a0 msgr2=0x7f5014003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:50.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.855+0000 7f502eaad640 1 --2- 172.21.15.46:0/37839002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50140039a0 0x7f5014003da0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5020001f60 tx=0x7f50140057f0 comp rx=0 tx=0).stop 2023-12-02T00:28:50.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.855+0000 7f502d04a640 1 -- 172.21.15.46:0/37839002 reap_dead start 2023-12-02T00:28:50.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.856+0000 7f502eaad640 1 -- 172.21.15.46:0/37839002 shutdown_connections 2023-12-02T00:28:50.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.856+0000 7f502eaad640 1 -- 172.21.15.46:0/37839002 >> 172.21.15.46:0/37839002 conn(0x7f502806a4d0 msgr2=0x7f502806a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:50.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.856+0000 7f502eaad640 1 -- 172.21.15.46:0/37839002 shutdown_connections 2023-12-02T00:28:50.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.856+0000 7f502eaad640 1 -- 172.21.15.46:0/37839002 wait complete. 2023-12-02T00:28:50.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.857+0000 7f502eaad640 1 Processor -- start 2023-12-02T00:28:50.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.857+0000 7f502eaad640 1 -- start start 2023-12-02T00:28:50.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.858+0000 7f502eaad640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50140039a0 0x7f5028154840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:50.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.858+0000 7f502eaad640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f502815dcc0 0x7f502815e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:50.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.858+0000 7f502eaad640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50281f3c80 0x7f50281f4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:50.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.858+0000 7f502eaad640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50281f4c50 con 0x7f50140039a0 2023-12-02T00:28:50.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.858+0000 7f502eaad640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502815e680 con 0x7f50281f3c80 2023-12-02T00:28:50.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.858+0000 7f502eaad640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50280c8db0 con 0x7f502815dcc0 2023-12-02T00:28:50.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.858+0000 7f502c849640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50140039a0 0x7f5028154840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:50.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.858+0000 7f502c849640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50140039a0 0x7f5028154840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37072/0 (socket says 172.21.15.46:37072) 2023-12-02T00:28:50.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.858+0000 7f502c849640 1 -- 172.21.15.46:0/1116094283 learned_addr learned my addr 172.21.15.46:0/1116094283 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:50.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.858+0000 7f5027fff640 1 --2- 172.21.15.46:0/1116094283 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f502815dcc0 0x7f502815e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:50.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.858+0000 7f502c849640 1 -- 172.21.15.46:0/1116094283 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f502815dcc0 msgr2=0x7f502815e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:50.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.858+0000 7f502d04a640 1 --2- 172.21.15.46:0/1116094283 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50281f3c80 0x7f50281f4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:50.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.858+0000 7f502c849640 1 --2- 172.21.15.46:0/1116094283 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f502815dcc0 0x7f502815e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:50.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.858+0000 7f502c849640 1 -- 172.21.15.46:0/1116094283 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50281f3c80 msgr2=0x7f50281f4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:50.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.858+0000 7f502c849640 1 --2- 172.21.15.46:0/1116094283 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50281f3c80 0x7f50281f4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:50.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.858+0000 7f502c849640 1 -- 172.21.15.46:0/1116094283 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50281f4970 con 0x7f50140039a0 2023-12-02T00:28:50.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.858+0000 7f502c849640 1 --2- 172.21.15.46:0/1116094283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50140039a0 0x7f5028154840 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f502002c0b0 tx=0x7f50200015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:50.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.858+0000 7f50257fa640 1 -- 172.21.15.46:0/1116094283 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502002ce80 con 0x7f50140039a0 2023-12-02T00:28:50.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.858+0000 7f50257fa640 1 -- 172.21.15.46:0/1116094283 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5020003970 con 0x7f50140039a0 2023-12-02T00:28:50.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.858+0000 7f502eaad640 1 -- 172.21.15.46:0/1116094283 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5028131410 con 0x7f50140039a0 2023-12-02T00:28:50.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.858+0000 7f50257fa640 1 -- 172.21.15.46:0/1116094283 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5020003d00 con 0x7f50140039a0 2023-12-02T00:28:50.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.859+0000 7f502eaad640 1 -- 172.21.15.46:0/1116094283 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50281319d0 con 0x7f50140039a0 2023-12-02T00:28:50.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.859+0000 7f502eaad640 1 -- 172.21.15.46:0/1116094283 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5028069400 con 0x7f50140039a0 2023-12-02T00:28:50.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.860+0000 7f50257fa640 1 -- 172.21.15.46:0/1116094283 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f502003b070 con 0x7f50140039a0 2023-12-02T00:28:50.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.860+0000 7f50257fa640 1 --2- 172.21.15.46:0/1116094283 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f500806ae70 0x7f500806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:50.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.860+0000 7f50257fa640 1 -- 172.21.15.46:0/1116094283 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f50200aa4a0 con 0x7f50140039a0 2023-12-02T00:28:50.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.860+0000 7f5027fff640 1 --2- 172.21.15.46:0/1116094283 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f500806ae70 0x7f500806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:50.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.861+0000 7f5027fff640 1 --2- 172.21.15.46:0/1116094283 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f500806ae70 0x7f500806d330 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5028067740 tx=0x7f5018009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:50.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:50.865+0000 7f50257fa640 1 -- 172.21.15.46:0/1116094283 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5020075600 con 0x7f50140039a0 2023-12-02T00:28:51.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.081+0000 7f502eaad640 1 -- 172.21.15.46:0/1116094283 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f5028071f10 con 0x7f50140039a0 2023-12-02T00:28:51.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.736+0000 7f50257fa640 1 -- 172.21.15.46:0/1116094283 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5020003e60 con 0x7f50140039a0 2023-12-02T00:28:51.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.738+0000 7f502eaad640 1 -- 172.21.15.46:0/1116094283 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f500806ae70 msgr2=0x7f500806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:51.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.738+0000 7f502eaad640 1 --2- 172.21.15.46:0/1116094283 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f500806ae70 0x7f500806d330 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5028067740 tx=0x7f5018009040 comp rx=0 tx=0).stop 2023-12-02T00:28:51.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.738+0000 7f502eaad640 1 -- 172.21.15.46:0/1116094283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50140039a0 msgr2=0x7f5028154840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:51.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.738+0000 7f502eaad640 1 --2- 172.21.15.46:0/1116094283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50140039a0 0x7f5028154840 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f502002c0b0 tx=0x7f50200015b0 comp rx=0 tx=0).stop 2023-12-02T00:28:51.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.739+0000 7f502eaad640 1 -- 172.21.15.46:0/1116094283 shutdown_connections 2023-12-02T00:28:51.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.739+0000 7f502eaad640 1 --2- 172.21.15.46:0/1116094283 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f500806ae70 0x7f500806d330 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:51.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.739+0000 7f502eaad640 1 --2- 172.21.15.46:0/1116094283 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50281f3c80 0x7f50281f4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:51.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.739+0000 7f502eaad640 1 --2- 172.21.15.46:0/1116094283 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f502815dcc0 0x7f502815e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:51.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.739+0000 7f502eaad640 1 --2- 172.21.15.46:0/1116094283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50140039a0 0x7f5028154840 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:51.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.739+0000 7f502eaad640 1 -- 172.21.15.46:0/1116094283 >> 172.21.15.46:0/1116094283 conn(0x7f502806a4d0 msgr2=0x7f502815a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:51.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.739+0000 7f502eaad640 1 -- 172.21.15.46:0/1116094283 shutdown_connections 2023-12-02T00:28:51.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.739+0000 7f502eaad640 1 -- 172.21.15.46:0/1116094283 wait complete. 2023-12-02T00:28:51.753 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-02T00:28:51.753 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-02T00:28:51.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.865+0000 7fbc51097640 1 Processor -- start 2023-12-02T00:28:51.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.865+0000 7fbc51097640 1 -- start start 2023-12-02T00:28:51.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.866+0000 7fbc51097640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbc4c06ea50 0x7fbc4c06ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:51.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.866+0000 7fbc51097640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc4c161d20 con 0x7fbc4c0d0ed0 2023-12-02T00:28:51.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.866+0000 7fbc51097640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc4c06f370 con 0x7fbc4c160fb0 2023-12-02T00:28:51.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.866+0000 7fbc51097640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc4c15e760 con 0x7fbc4c06ea50 2023-12-02T00:28:51.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.866+0000 7fbc4b59e640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fbc4c0d0ed0 0x7fbc4c0d1320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58082/0 (socket says 172.21.15.46:58082) 2023-12-02T00:28:51.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.866+0000 7fbc4b59e640 1 -- 172.21.15.46:0/2740031967 learned_addr learned my addr 172.21.15.46:0/2740031967 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:28:51.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.866+0000 7fbc4ad9d640 1 --2- 172.21.15.46:0/2740031967 >> v2:172.21.15.162:3300/0 conn(0x7fbc4c06ea50 0x7fbc4c06ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:51.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.866+0000 7fbc4a59c640 1 -- 172.21.15.46:0/2740031967 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3899413945 0 0) 0x7fbc4c161d20 con 0x7fbc4c0d0ed0 2023-12-02T00:28:51.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.866+0000 7fbc4a59c640 1 -- 172.21.15.46:0/2740031967 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc38003520 con 0x7fbc4c0d0ed0 2023-12-02T00:28:51.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.866+0000 7fbc4a59c640 1 -- 172.21.15.46:0/2740031967 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1548668861 0 0) 0x7fbc38003520 con 0x7fbc4c0d0ed0 2023-12-02T00:28:51.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.866+0000 7fbc4a59c640 1 -- 172.21.15.46:0/2740031967 >> v2:172.21.15.162:3300/0 conn(0x7fbc4c06ea50 msgr2=0x7fbc4c06ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:51.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.866+0000 7fbc4a59c640 1 --2- 172.21.15.46:0/2740031967 >> v2:172.21.15.162:3300/0 conn(0x7fbc4c06ea50 0x7fbc4c06ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:51.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.866+0000 7fbc4a59c640 1 -- 172.21.15.46:0/2740031967 >> v1:172.21.15.162:6789/0 conn(0x7fbc4c160fb0 legacy=0x7fbc4c161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:51.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.866+0000 7fbc4a59c640 1 -- 172.21.15.46:0/2740031967 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc4c15d9a0 con 0x7fbc4c0d0ed0 2023-12-02T00:28:51.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.866+0000 7fbc4a59c640 1 -- 172.21.15.46:0/2740031967 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7fbc440019f0 con 0x7fbc4c0d0ed0 2023-12-02T00:28:51.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.866+0000 7fbc4a59c640 1 -- 172.21.15.46:0/2740031967 >> v1:172.21.15.46:6789/0 conn(0x7fbc4c0d0ed0 legacy=0x7fbc4c0d1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:51.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.866+0000 7fbc4a59c640 1 --2- 172.21.15.46:0/2740031967 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbc38003970 0x7fbc38003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:51.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.866+0000 7fbc4a59c640 1 --2- 172.21.15.46:0/2740031967 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc380047c0 0x7fbc38004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:51.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.866+0000 7fbc4a59c640 1 --2- 172.21.15.46:0/2740031967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc38005180 0x7fbc3800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:51.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.866+0000 7fbc4a59c640 1 -- 172.21.15.46:0/2740031967 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4c15e760 con 0x7fbc38005180 2023-12-02T00:28:51.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.866+0000 7fbc4a59c640 1 -- 172.21.15.46:0/2740031967 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3800c120 con 0x7fbc380047c0 2023-12-02T00:28:51.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.866+0000 7fbc4a59c640 1 -- 172.21.15.46:0/2740031967 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3800c2e0 con 0x7fbc38003970 2023-12-02T00:28:51.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.867+0000 7fbc4b59e640 1 --2- 172.21.15.46:0/2740031967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc38005180 0x7fbc3800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:51.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.867+0000 7fbc4ad9d640 1 --2- 172.21.15.46:0/2740031967 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbc38003970 0x7fbc38003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:51.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.867+0000 7fbc4259c640 1 --2- 172.21.15.46:0/2740031967 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc380047c0 0x7fbc38004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:51.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.867+0000 7fbc4b59e640 1 -- 172.21.15.46:0/2740031967 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbc38003970 msgr2=0x7fbc38003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:51.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.867+0000 7fbc4b59e640 1 --2- 172.21.15.46:0/2740031967 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbc38003970 0x7fbc38003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:51.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.867+0000 7fbc4b59e640 1 -- 172.21.15.46:0/2740031967 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc380047c0 msgr2=0x7fbc38004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:51.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.867+0000 7fbc4b59e640 1 --2- 172.21.15.46:0/2740031967 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc380047c0 0x7fbc38004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:51.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.867+0000 7fbc4b59e640 1 -- 172.21.15.46:0/2740031967 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc38003520 con 0x7fbc38005180 2023-12-02T00:28:51.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.867+0000 7fbc4b59e640 1 -- 172.21.15.46:0/2740031967 reap_dead start 2023-12-02T00:28:51.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.868+0000 7fbc4b59e640 1 --2- 172.21.15.46:0/2740031967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc38005180 0x7fbc3800b980 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbc4400a830 tx=0x7fbc380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=684304e06ee28e2 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:51.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.868+0000 7fbc4a59c640 1 -- 172.21.15.46:0/2740031967 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc440072d0 con 0x7fbc38005180 2023-12-02T00:28:51.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.868+0000 7fbc4a59c640 1 -- 172.21.15.46:0/2740031967 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc4402f070 con 0x7fbc38005180 2023-12-02T00:28:51.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.868+0000 7fbc4a59c640 1 -- 172.21.15.46:0/2740031967 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc440076b0 con 0x7fbc38005180 2023-12-02T00:28:51.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.868+0000 7fbc51097640 1 -- 172.21.15.46:0/2740031967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc38005180 msgr2=0x7fbc3800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:51.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.868+0000 7fbc51097640 1 --2- 172.21.15.46:0/2740031967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc38005180 0x7fbc3800b980 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbc4400a830 tx=0x7fbc380057f0 comp rx=0 tx=0).stop 2023-12-02T00:28:51.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.868+0000 7fbc51097640 1 -- 172.21.15.46:0/2740031967 shutdown_connections 2023-12-02T00:28:51.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.868+0000 7fbc51097640 1 --2- 172.21.15.46:0/2740031967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc38005180 0x7fbc3800b980 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:51.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.868+0000 7fbc51097640 1 -- 172.21.15.46:0/2740031967 >> 172.21.15.46:0/2740031967 conn(0x7fbc4c06a6b0 msgr2=0x7fbc4c06aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:51.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.868+0000 7fbc51097640 1 -- 172.21.15.46:0/2740031967 shutdown_connections 2023-12-02T00:28:51.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.868+0000 7fbc51097640 1 -- 172.21.15.46:0/2740031967 wait complete. 2023-12-02T00:28:51.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.870+0000 7fbc51097640 1 Processor -- start 2023-12-02T00:28:51.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.870+0000 7fbc51097640 1 -- start start 2023-12-02T00:28:51.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.870+0000 7fbc51097640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbc38005180 0x7fbc4c13fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:51.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.871+0000 7fbc51097640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc4c159fa0 0x7fbc4c15a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:51.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.871+0000 7fbc51097640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc4c1e7440 0x7fbc4c1e78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:51.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.871+0000 7fbc51097640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4c1f73f0 con 0x7fbc4c159fa0 2023-12-02T00:28:51.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.871+0000 7fbc51097640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4c15a960 con 0x7fbc4c1e7440 2023-12-02T00:28:51.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.871+0000 7fbc51097640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4c1e7e00 con 0x7fbc38005180 2023-12-02T00:28:51.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.871+0000 7fbc4259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc4c159fa0 0x7fbc4c15a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:51.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.871+0000 7fbc4259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc4c159fa0 0x7fbc4c15a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37078/0 (socket says 172.21.15.46:37078) 2023-12-02T00:28:51.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.871+0000 7fbc4259c640 1 -- 172.21.15.46:0/4258266136 learned_addr learned my addr 172.21.15.46:0/4258266136 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:51.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.871+0000 7fbc4ad9d640 1 --2- 172.21.15.46:0/4258266136 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbc38005180 0x7fbc4c13fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:51.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.871+0000 7fbc4b59e640 1 --2- 172.21.15.46:0/4258266136 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc4c1e7440 0x7fbc4c1e78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:51.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.871+0000 7fbc4259c640 1 -- 172.21.15.46:0/4258266136 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbc38005180 msgr2=0x7fbc4c13fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:51.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.871+0000 7fbc4259c640 1 --2- 172.21.15.46:0/4258266136 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbc38005180 0x7fbc4c13fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:51.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.871+0000 7fbc4259c640 1 -- 172.21.15.46:0/4258266136 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc4c1e7440 msgr2=0x7fbc4c1e78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:51.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.871+0000 7fbc4259c640 1 --2- 172.21.15.46:0/4258266136 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc4c1e7440 0x7fbc4c1e78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:51.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.871+0000 7fbc4259c640 1 -- 172.21.15.46:0/4258266136 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc4c1e9600 con 0x7fbc4c159fa0 2023-12-02T00:28:51.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.871+0000 7fbc4259c640 1 --2- 172.21.15.46:0/4258266136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc4c159fa0 0x7fbc4c15a420 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbc3400f620 tx=0x7fbc3400fa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:51.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.871+0000 7fbc43fff640 1 -- 172.21.15.46:0/4258266136 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc340052e0 con 0x7fbc4c159fa0 2023-12-02T00:28:51.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.871+0000 7fbc43fff640 1 -- 172.21.15.46:0/4258266136 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc34005440 con 0x7fbc4c159fa0 2023-12-02T00:28:51.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.871+0000 7fbc51097640 1 -- 172.21.15.46:0/4258266136 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc4c1585b0 con 0x7fbc4c159fa0 2023-12-02T00:28:51.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.871+0000 7fbc43fff640 1 -- 172.21.15.46:0/4258266136 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc34003810 con 0x7fbc4c159fa0 2023-12-02T00:28:51.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.871+0000 7fbc51097640 1 -- 172.21.15.46:0/4258266136 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc4c1e8020 con 0x7fbc4c159fa0 2023-12-02T00:28:51.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.872+0000 7fbc51097640 1 -- 172.21.15.46:0/4258266136 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc4c158740 con 0x7fbc4c159fa0 2023-12-02T00:28:51.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.873+0000 7fbc43fff640 1 -- 172.21.15.46:0/4258266136 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbc34003970 con 0x7fbc4c159fa0 2023-12-02T00:28:51.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.873+0000 7fbc43fff640 1 --2- 172.21.15.46:0/4258266136 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fbc2c06ae40 0x7fbc2c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:51.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.873+0000 7fbc43fff640 1 -- 172.21.15.46:0/4258266136 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbc34089210 con 0x7fbc4c159fa0 2023-12-02T00:28:51.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.873+0000 7fbc4ad9d640 1 --2- 172.21.15.46:0/4258266136 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fbc2c06ae40 0x7fbc2c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:51.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.874+0000 7fbc4ad9d640 1 --2- 172.21.15.46:0/4258266136 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fbc2c06ae40 0x7fbc2c06d300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c009b40 tx=0x7fbc3c009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:51.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:51.877+0000 7fbc43fff640 1 -- 172.21.15.46:0/4258266136 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbc34054350 con 0x7fbc4c159fa0 2023-12-02T00:28:52.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.096+0000 7fbc51097640 1 -- 172.21.15.46:0/4258266136 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbc4c069400 con 0x7fbc4c159fa0 2023-12-02T00:28:52.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.097+0000 7fbc43fff640 1 -- 172.21.15.46:0/4258266136 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4907 (secure 0 0 0) 0x7fbc34057970 con 0x7fbc4c159fa0 2023-12-02T00:28:52.098 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:28:52.098 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/2059113401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2059113401},{"type":"v1","addr":"172.21.15.46:6834","nonce":2059113401}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/987532381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":987532381},{"type":"v1","addr":"172.21.15.46:6835","nonce":987532381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/449974862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":449974862},{"type":"v1","addr":"172.21.15.162:6836","nonce":449974862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-02T00:28:49.086548+0000","modified":"2023-12-02T00:28:51.082469+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4253":{"gid":4253,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/1231844478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":1231844478},{"type":"v1","addr":"172.21.15.162:6837","nonce":1231844478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-02T00:28:52.098 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 6 2023-12-02T00:28:52.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.099+0000 7fbc51097640 1 -- 172.21.15.46:0/4258266136 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fbc2c06ae40 msgr2=0x7fbc2c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:52.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.099+0000 7fbc51097640 1 --2- 172.21.15.46:0/4258266136 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fbc2c06ae40 0x7fbc2c06d300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c009b40 tx=0x7fbc3c009190 comp rx=0 tx=0).stop 2023-12-02T00:28:52.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.099+0000 7fbc51097640 1 -- 172.21.15.46:0/4258266136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc4c159fa0 msgr2=0x7fbc4c15a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:52.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.099+0000 7fbc51097640 1 --2- 172.21.15.46:0/4258266136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc4c159fa0 0x7fbc4c15a420 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbc3400f620 tx=0x7fbc3400fa20 comp rx=0 tx=0).stop 2023-12-02T00:28:52.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.099+0000 7fbc51097640 1 -- 172.21.15.46:0/4258266136 shutdown_connections 2023-12-02T00:28:52.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.099+0000 7fbc51097640 1 --2- 172.21.15.46:0/4258266136 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fbc2c06ae40 0x7fbc2c06d300 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:52.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.099+0000 7fbc51097640 1 --2- 172.21.15.46:0/4258266136 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc4c1e7440 0x7fbc4c1e78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:52.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.099+0000 7fbc51097640 1 --2- 172.21.15.46:0/4258266136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc4c159fa0 0x7fbc4c15a420 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:52.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.099+0000 7fbc51097640 1 --2- 172.21.15.46:0/4258266136 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbc38005180 0x7fbc4c13fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:52.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.099+0000 7fbc51097640 1 -- 172.21.15.46:0/4258266136 >> 172.21.15.46:0/4258266136 conn(0x7fbc4c06a6b0 msgr2=0x7fbc4c0b3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:52.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.099+0000 7fbc51097640 1 -- 172.21.15.46:0/4258266136 shutdown_connections 2023-12-02T00:28:52.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.099+0000 7fbc51097640 1 -- 172.21.15.46:0/4258266136 wait complete. 2023-12-02T00:28:52.111 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:28:52.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.217+0000 7f5053f53640 1 Processor -- start 2023-12-02T00:28:52.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.217+0000 7f5053f53640 1 -- start start 2023-12-02T00:28:52.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.218+0000 7f5053f53640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f504c15e730 0x7f504c15abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:52.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.218+0000 7f5053f53640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f504c15e2a0 con 0x7f504c143490 2023-12-02T00:28:52.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.218+0000 7f5053f53640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f504c15b240 con 0x7f504c0b1400 2023-12-02T00:28:52.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.218+0000 7f5053f53640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f504c15b520 con 0x7f504c15e730 2023-12-02T00:28:52.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.218+0000 7f50524f0640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f504c15e730 0x7f504c15abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:52.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.218+0000 7f5051cef640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f504c143490 0x7f504c0b0cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58092/0 (socket says 172.21.15.46:58092) 2023-12-02T00:28:52.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.218+0000 7f5051cef640 1 -- 172.21.15.46:0/2485919693 learned_addr learned my addr 172.21.15.46:0/2485919693 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:28:52.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.218+0000 7f50524f0640 1 -- 172.21.15.46:0/2485919693 >> v1:172.21.15.162:6789/0 conn(0x7f504c0b1400 legacy=0x7f504c15a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:52.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.218+0000 7f50524f0640 1 -- 172.21.15.46:0/2485919693 >> v1:172.21.15.46:6789/0 conn(0x7f504c143490 legacy=0x7f504c0b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:52.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.218+0000 7f50524f0640 1 -- 172.21.15.46:0/2485919693 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f504c15dc50 con 0x7f504c15e730 2023-12-02T00:28:52.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.218+0000 7f5050ced640 1 -- 172.21.15.46:0/2485919693 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2796848391 0 0) 0x7f504c15e2a0 con 0x7f504c143490 2023-12-02T00:28:52.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.219+0000 7f50524f0640 1 --2- 172.21.15.46:0/2485919693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f504c15e730 0x7f504c15abf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5044009d90 tx=0x7f504402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4846da0f263d9e5 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:52.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.219+0000 7f5050ced640 1 -- 172.21.15.46:0/2485919693 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504402cd00 con 0x7f504c15e730 2023-12-02T00:28:52.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.219+0000 7f5050ced640 1 -- 172.21.15.46:0/2485919693 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f504402ce60 con 0x7f504c15e730 2023-12-02T00:28:52.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.219+0000 7f5050ced640 1 -- 172.21.15.46:0/2485919693 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5044034660 con 0x7f504c15e730 2023-12-02T00:28:52.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.220+0000 7f5053f53640 1 -- 172.21.15.46:0/2485919693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f504c15e730 msgr2=0x7f504c15abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:52.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.220+0000 7f5053f53640 1 --2- 172.21.15.46:0/2485919693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f504c15e730 0x7f504c15abf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5044009d90 tx=0x7f504402bf50 comp rx=0 tx=0).stop 2023-12-02T00:28:52.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.220+0000 7f5053f53640 1 -- 172.21.15.46:0/2485919693 shutdown_connections 2023-12-02T00:28:52.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.220+0000 7f5053f53640 1 --2- 172.21.15.46:0/2485919693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f504c15e730 0x7f504c15abf0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:52.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.220+0000 7f5053f53640 1 -- 172.21.15.46:0/2485919693 >> 172.21.15.46:0/2485919693 conn(0x7f504c06a4d0 msgr2=0x7f504c06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:52.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.220+0000 7f5053f53640 1 -- 172.21.15.46:0/2485919693 shutdown_connections 2023-12-02T00:28:52.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.220+0000 7f5053f53640 1 -- 172.21.15.46:0/2485919693 wait complete. 2023-12-02T00:28:52.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.221+0000 7f5053f53640 1 Processor -- start 2023-12-02T00:28:52.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.221+0000 7f5053f53640 1 -- start start 2023-12-02T00:28:52.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.222+0000 7f5053f53640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f504c0b1400 0x7f504c159450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:52.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.222+0000 7f5053f53640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f504c143490 0x7f504c13dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:52.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.222+0000 7f5053f53640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f504c15e730 0x7f504c13b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:52.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.222+0000 7f5053f53640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f504c13e230 con 0x7f504c15e730 2023-12-02T00:28:52.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.222+0000 7f5053f53640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f504c13e510 con 0x7f504c143490 2023-12-02T00:28:52.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.222+0000 7f5053f53640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f504c13e7f0 con 0x7f504c0b1400 2023-12-02T00:28:52.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.222+0000 7f50524f0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f504c15e730 0x7f504c13b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:52.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.222+0000 7f50524f0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f504c15e730 0x7f504c13b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37100/0 (socket says 172.21.15.46:37100) 2023-12-02T00:28:52.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.222+0000 7f50524f0640 1 -- 172.21.15.46:0/100096695 learned_addr learned my addr 172.21.15.46:0/100096695 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:52.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.222+0000 7f50514ee640 1 --2- 172.21.15.46:0/100096695 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f504c143490 0x7f504c13dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:52.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.222+0000 7f5051cef640 1 --2- 172.21.15.46:0/100096695 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f504c0b1400 0x7f504c159450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:52.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.222+0000 7f50524f0640 1 -- 172.21.15.46:0/100096695 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f504c0b1400 msgr2=0x7f504c159450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:52.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.222+0000 7f50524f0640 1 --2- 172.21.15.46:0/100096695 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f504c0b1400 0x7f504c159450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:52.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.222+0000 7f50524f0640 1 -- 172.21.15.46:0/100096695 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f504c143490 msgr2=0x7f504c13dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:52.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.222+0000 7f50524f0640 1 --2- 172.21.15.46:0/100096695 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f504c143490 0x7f504c13dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:52.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.222+0000 7f50524f0640 1 -- 172.21.15.46:0/100096695 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5044009a70 con 0x7f504c15e730 2023-12-02T00:28:52.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.222+0000 7f50524f0640 1 --2- 172.21.15.46:0/100096695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f504c15e730 0x7f504c13b310 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5044009d90 tx=0x7f5044004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:52.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.222+0000 7f50427fc640 1 -- 172.21.15.46:0/100096695 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5044003b90 con 0x7f504c15e730 2023-12-02T00:28:52.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.222+0000 7f50427fc640 1 -- 172.21.15.46:0/100096695 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5044003cf0 con 0x7f504c15e730 2023-12-02T00:28:52.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.222+0000 7f5053f53640 1 -- 172.21.15.46:0/100096695 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f504c13ead0 con 0x7f504c15e730 2023-12-02T00:28:52.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.222+0000 7f50427fc640 1 -- 172.21.15.46:0/100096695 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5044004230 con 0x7f504c15e730 2023-12-02T00:28:52.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.223+0000 7f5053f53640 1 -- 172.21.15.46:0/100096695 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f504c13b850 con 0x7f504c15e730 2023-12-02T00:28:52.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.223+0000 7f5053f53640 1 -- 172.21.15.46:0/100096695 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f504c069400 con 0x7f504c15e730 2023-12-02T00:28:52.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.224+0000 7f50427fc640 1 -- 172.21.15.46:0/100096695 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f504403b050 con 0x7f504c15e730 2023-12-02T00:28:52.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.224+0000 7f50427fc640 1 --2- 172.21.15.46:0/100096695 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f503006ae70 0x7f503006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:52.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.224+0000 7f50427fc640 1 -- 172.21.15.46:0/100096695 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f50440abf80 con 0x7f504c15e730 2023-12-02T00:28:52.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.225+0000 7f5051cef640 1 --2- 172.21.15.46:0/100096695 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f503006ae70 0x7f503006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:52.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.225+0000 7f5051cef640 1 --2- 172.21.15.46:0/100096695 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f503006ae70 0x7f503006d330 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5048000f60 tx=0x7f5048007bb0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:52.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.229+0000 7f50427fc640 1 -- 172.21.15.46:0/100096695 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f50440dee90 con 0x7f504c15e730 2023-12-02T00:28:52.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.404+0000 7f5053f53640 1 -- 172.21.15.46:0/100096695 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f504c0cb590 con 0x7f504c15e730 2023-12-02T00:28:52.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.405+0000 7f50427fc640 1 -- 172.21.15.46:0/100096695 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f5044039090 con 0x7f504c15e730 2023-12-02T00:28:52.407 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:28:52.407 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":21,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:28:50.700112+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-02T00:28:47.273337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-02T00:28:48.060115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:28:52.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.408+0000 7f5053f53640 1 -- 172.21.15.46:0/100096695 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f503006ae70 msgr2=0x7f503006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:52.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.408+0000 7f5053f53640 1 --2- 172.21.15.46:0/100096695 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f503006ae70 0x7f503006d330 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5048000f60 tx=0x7f5048007bb0 comp rx=0 tx=0).stop 2023-12-02T00:28:52.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.408+0000 7f5053f53640 1 -- 172.21.15.46:0/100096695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f504c15e730 msgr2=0x7f504c13b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:52.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.408+0000 7f5053f53640 1 --2- 172.21.15.46:0/100096695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f504c15e730 0x7f504c13b310 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5044009d90 tx=0x7f5044004a40 comp rx=0 tx=0).stop 2023-12-02T00:28:52.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.408+0000 7f5053f53640 1 -- 172.21.15.46:0/100096695 shutdown_connections 2023-12-02T00:28:52.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.408+0000 7f5053f53640 1 --2- 172.21.15.46:0/100096695 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f503006ae70 0x7f503006d330 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:52.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.408+0000 7f5053f53640 1 --2- 172.21.15.46:0/100096695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f504c15e730 0x7f504c13b310 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:52.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.408+0000 7f5053f53640 1 --2- 172.21.15.46:0/100096695 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f504c143490 0x7f504c13dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:52.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.408+0000 7f5053f53640 1 --2- 172.21.15.46:0/100096695 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f504c0b1400 0x7f504c159450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:52.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.408+0000 7f5053f53640 1 -- 172.21.15.46:0/100096695 >> 172.21.15.46:0/100096695 conn(0x7f504c06a4d0 msgr2=0x7f504c0b2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:52.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.408+0000 7f5053f53640 1 -- 172.21.15.46:0/100096695 shutdown_connections 2023-12-02T00:28:52.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.408+0000 7f5053f53640 1 -- 172.21.15.46:0/100096695 wait complete. 2023-12-02T00:28:52.420 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-02T00:28:52.420 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-02T00:28:52.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.525+0000 7f7e6d5d0640 1 Processor -- start 2023-12-02T00:28:52.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.525+0000 7f7e6d5d0640 1 -- start start 2023-12-02T00:28:52.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.526+0000 7f7e6d5d0640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7e68143490 0x7f7e680b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:52.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.526+0000 7f7e6d5d0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e6815e730 0x7f7e680b1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:52.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.526+0000 7f7e6d5d0640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7e6815a4e0 0x7f7e6815a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:52.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.526+0000 7f7e6d5d0640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e6815b520 con 0x7f7e68143490 2023-12-02T00:28:52.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.526+0000 7f7e6d5d0640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e68143870 con 0x7f7e6815a4e0 2023-12-02T00:28:52.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.526+0000 7f7e6d5d0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6815e2a0 con 0x7f7e6815e730 2023-12-02T00:28:52.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.526+0000 7f7e66ffd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7e68143490 0x7f7e680b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:52.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.526+0000 7f7e66ffd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7e68143490 0x7f7e680b0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37112/0 (socket says 172.21.15.46:37112) 2023-12-02T00:28:52.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.526+0000 7f7e66ffd640 1 -- 172.21.15.46:0/2678147630 learned_addr learned my addr 172.21.15.46:0/2678147630 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:52.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.526+0000 7f7e677fe640 1 --2- 172.21.15.46:0/2678147630 >> v2:172.21.15.162:3300/0 conn(0x7f7e6815a4e0 0x7f7e6815a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:52.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.526+0000 7f7e667fc640 1 --2- 172.21.15.46:0/2678147630 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e6815e730 0x7f7e680b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:52.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.526+0000 7f7e66ffd640 1 -- 172.21.15.46:0/2678147630 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e6815e730 msgr2=0x7f7e680b1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:52.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.526+0000 7f7e66ffd640 1 --2- 172.21.15.46:0/2678147630 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e6815e730 0x7f7e680b1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:52.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.526+0000 7f7e66ffd640 1 -- 172.21.15.46:0/2678147630 >> v2:172.21.15.162:3300/0 conn(0x7f7e6815a4e0 msgr2=0x7f7e6815a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:52.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.526+0000 7f7e66ffd640 1 --2- 172.21.15.46:0/2678147630 >> v2:172.21.15.162:3300/0 conn(0x7f7e6815a4e0 0x7f7e6815a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:52.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.526+0000 7f7e66ffd640 1 -- 172.21.15.46:0/2678147630 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e6815b240 con 0x7f7e68143490 2023-12-02T00:28:52.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.526+0000 7f7e677fe640 1 --2- 172.21.15.46:0/2678147630 >> v2:172.21.15.162:3300/0 conn(0x7f7e6815a4e0 0x7f7e6815a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:28:52.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.526+0000 7f7e667fc640 1 --2- 172.21.15.46:0/2678147630 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e6815e730 0x7f7e680b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:28:52.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.527+0000 7f7e66ffd640 1 --2- 172.21.15.46:0/2678147630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e68143490 0x7f7e680b0cf0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7e6815eb90 tx=0x7f7e5802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d745e393fdabf216 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:52.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.527+0000 7f7e65ffb640 1 -- 172.21.15.46:0/2678147630 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e58031860 con 0x7f7e68143490 2023-12-02T00:28:52.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.527+0000 7f7e65ffb640 1 -- 172.21.15.46:0/2678147630 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e580319c0 con 0x7f7e68143490 2023-12-02T00:28:52.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.527+0000 7f7e65ffb640 1 -- 172.21.15.46:0/2678147630 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e58031d50 con 0x7f7e68143490 2023-12-02T00:28:52.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.527+0000 7f7e6d5d0640 1 -- 172.21.15.46:0/2678147630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e68143490 msgr2=0x7f7e680b0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:52.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.527+0000 7f7e6d5d0640 1 --2- 172.21.15.46:0/2678147630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e68143490 0x7f7e680b0cf0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7e6815eb90 tx=0x7f7e5802b900 comp rx=0 tx=0).stop 2023-12-02T00:28:52.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.527+0000 7f7e6d5d0640 1 -- 172.21.15.46:0/2678147630 shutdown_connections 2023-12-02T00:28:52.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.527+0000 7f7e6d5d0640 1 --2- 172.21.15.46:0/2678147630 >> v2:172.21.15.162:3300/0 conn(0x7f7e6815a4e0 0x7f7e6815a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:52.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.527+0000 7f7e6d5d0640 1 --2- 172.21.15.46:0/2678147630 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e6815e730 0x7f7e680b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:52.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.527+0000 7f7e6d5d0640 1 --2- 172.21.15.46:0/2678147630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e68143490 0x7f7e680b0cf0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:52.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.528+0000 7f7e6d5d0640 1 -- 172.21.15.46:0/2678147630 >> 172.21.15.46:0/2678147630 conn(0x7f7e6806a4d0 msgr2=0x7f7e6806a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:52.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.528+0000 7f7e6d5d0640 1 -- 172.21.15.46:0/2678147630 shutdown_connections 2023-12-02T00:28:52.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.528+0000 7f7e6d5d0640 1 -- 172.21.15.46:0/2678147630 wait complete. 2023-12-02T00:28:52.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.529+0000 7f7e6d5d0640 1 Processor -- start 2023-12-02T00:28:52.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.529+0000 7f7e6d5d0640 1 -- start start 2023-12-02T00:28:52.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.530+0000 7f7e6d5d0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7e68143490 0x7f7e680ade30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:52.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.530+0000 7f7e6d5d0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e6815a4e0 0x7f7e6813e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:52.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.530+0000 7f7e6d5d0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e6815e730 0x7f7e6815c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:52.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.530+0000 7f7e6d5d0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e681f1400 con 0x7f7e6815a4e0 2023-12-02T00:28:52.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.530+0000 7f7e6d5d0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e681f1690 con 0x7f7e68143490 2023-12-02T00:28:52.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.530+0000 7f7e6d5d0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e681f1970 con 0x7f7e6815e730 2023-12-02T00:28:52.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.530+0000 7f7e667fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e6815a4e0 0x7f7e6813e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:52.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.530+0000 7f7e667fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e6815a4e0 0x7f7e6813e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37118/0 (socket says 172.21.15.46:37118) 2023-12-02T00:28:52.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.530+0000 7f7e667fc640 1 -- 172.21.15.46:0/1637090102 learned_addr learned my addr 172.21.15.46:0/1637090102 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:52.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.530+0000 7f7e66ffd640 1 --2- 172.21.15.46:0/1637090102 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7e68143490 0x7f7e680ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:52.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.530+0000 7f7e677fe640 1 --2- 172.21.15.46:0/1637090102 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e6815e730 0x7f7e6815c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:52.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.530+0000 7f7e667fc640 1 -- 172.21.15.46:0/1637090102 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e6815e730 msgr2=0x7f7e6815c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:52.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.530+0000 7f7e667fc640 1 --2- 172.21.15.46:0/1637090102 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e6815e730 0x7f7e6815c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:52.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.530+0000 7f7e667fc640 1 -- 172.21.15.46:0/1637090102 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7e68143490 msgr2=0x7f7e680ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:52.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.530+0000 7f7e667fc640 1 --2- 172.21.15.46:0/1637090102 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7e68143490 0x7f7e680ade30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:52.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.530+0000 7f7e667fc640 1 -- 172.21.15.46:0/1637090102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e580094d0 con 0x7f7e6815a4e0 2023-12-02T00:28:52.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.530+0000 7f7e677fe640 1 --2- 172.21.15.46:0/1637090102 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e6815e730 0x7f7e6815c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:28:52.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.530+0000 7f7e667fc640 1 --2- 172.21.15.46:0/1637090102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e6815a4e0 0x7f7e6813e920 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c00aed0 tx=0x7f7e5c00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:52.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.530+0000 7f7e4f7fe640 1 -- 172.21.15.46:0/1637090102 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c007820 con 0x7f7e6815a4e0 2023-12-02T00:28:52.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.530+0000 7f7e4f7fe640 1 -- 172.21.15.46:0/1637090102 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5c007980 con 0x7f7e6815a4e0 2023-12-02T00:28:52.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.530+0000 7f7e4f7fe640 1 -- 172.21.15.46:0/1637090102 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c007cf0 con 0x7f7e6815a4e0 2023-12-02T00:28:52.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.530+0000 7f7e6d5d0640 1 -- 172.21.15.46:0/1637090102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e681f1c50 con 0x7f7e6815a4e0 2023-12-02T00:28:52.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.530+0000 7f7e6d5d0640 1 -- 172.21.15.46:0/1637090102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e6815c750 con 0x7f7e6815a4e0 2023-12-02T00:28:52.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.531+0000 7f7e6d5d0640 1 -- 172.21.15.46:0/1637090102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e68069400 con 0x7f7e6815a4e0 2023-12-02T00:28:52.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.532+0000 7f7e4f7fe640 1 -- 172.21.15.46:0/1637090102 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7e5c010650 con 0x7f7e6815a4e0 2023-12-02T00:28:52.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.532+0000 7f7e4f7fe640 1 --2- 172.21.15.46:0/1637090102 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7e5406ae40 0x7f7e5406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:52.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.532+0000 7f7e4f7fe640 1 -- 172.21.15.46:0/1637090102 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7e5c014070 con 0x7f7e6815a4e0 2023-12-02T00:28:52.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.532+0000 7f7e66ffd640 1 --2- 172.21.15.46:0/1637090102 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7e5406ae40 0x7f7e5406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:52.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.533+0000 7f7e66ffd640 1 --2- 172.21.15.46:0/1637090102 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7e5406ae40 0x7f7e5406d300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7e58036000 tx=0x7f7e5803c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:52.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.536+0000 7f7e4f7fe640 1 -- 172.21.15.46:0/1637090102 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7e5c051e90 con 0x7f7e6815a4e0 2023-12-02T00:28:52.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.708+0000 7f7e6d5d0640 1 -- 172.21.15.46:0/1637090102 --> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7e6815aeb0 con 0x7f7e5406ae40 2023-12-02T00:28:52.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.716+0000 7f7e4f7fe640 1 -- 172.21.15.46:0/1637090102 <== mgr.4108 v2:172.21.15.162:6832/3248644722 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+249848 (secure 0 0 0) 0x7f7e58031d50 con 0x7f7e5406ae40 2023-12-02T00:28:52.719 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:28:52.721 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-12-02T00:28:52.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.718+0000 7f7e6d5d0640 1 -- 172.21.15.46:0/1637090102 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7e5406ae40 msgr2=0x7f7e5406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:52.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.718+0000 7f7e6d5d0640 1 --2- 172.21.15.46:0/1637090102 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7e5406ae40 0x7f7e5406d300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7e58036000 tx=0x7f7e5803c040 comp rx=0 tx=0).stop 2023-12-02T00:28:52.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.718+0000 7f7e6d5d0640 1 -- 172.21.15.46:0/1637090102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e6815a4e0 msgr2=0x7f7e6813e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:52.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.718+0000 7f7e6d5d0640 1 --2- 172.21.15.46:0/1637090102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e6815a4e0 0x7f7e6813e920 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c00aed0 tx=0x7f7e5c00cb90 comp rx=0 tx=0).stop 2023-12-02T00:28:52.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.718+0000 7f7e6d5d0640 1 -- 172.21.15.46:0/1637090102 shutdown_connections 2023-12-02T00:28:52.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.718+0000 7f7e6d5d0640 1 --2- 172.21.15.46:0/1637090102 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7e5406ae40 0x7f7e5406d300 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:52.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.718+0000 7f7e6d5d0640 1 --2- 172.21.15.46:0/1637090102 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e6815e730 0x7f7e6815c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:52.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.718+0000 7f7e6d5d0640 1 --2- 172.21.15.46:0/1637090102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e6815a4e0 0x7f7e6813e920 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:52.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.718+0000 7f7e6d5d0640 1 --2- 172.21.15.46:0/1637090102 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7e68143490 0x7f7e680ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:52.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.719+0000 7f7e6d5d0640 1 -- 172.21.15.46:0/1637090102 >> 172.21.15.46:0/1637090102 conn(0x7f7e6806a4d0 msgr2=0x7f7e6815cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:52.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.719+0000 7f7e6d5d0640 1 -- 172.21.15.46:0/1637090102 shutdown_connections 2023-12-02T00:28:52.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.719+0000 7f7e6d5d0640 1 -- 172.21.15.46:0/1637090102 wait complete. 2023-12-02T00:28:52.730 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-12-02T00:28:51.158255+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":236076,"kb_used_data":2464,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738644,"statfs":{"total":773094113280,"available":772852371456,"internally_reserved":0,"allocated":2523136,"data_stored":1582810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001083"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":161,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-12-02T00:28:40.992948+0000","last_change":"2023-12-02T00:28:39.379380+0000","last_active":"2023-12-02T00:28:40.992948+0000","last_peered":"2023-12-02T00:28:40.992948+0000","last_clean":"2023-12-02T00:28:40.992948+0000","last_became_active":"2023-12-02T00:28:39.378255+0000","last_became_peered":"2023-12-02T00:28:39.378255+0000","last_unstale":"2023-12-02T00:28:40.992948+0000","last_undegraded":"2023-12-02T00:28:40.992948+0000","last_fullsized":"2023-12-02T00:28:40.992948+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:38.177345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:38.177345+0000","last_clean_scrub_stamp":"2023-12-02T00:28:38.177345+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:21:08.582412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28384,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343456,"statfs":{"total":96636764160,"available":96607698944,"internally_reserved":0,"allocated":761856,"data_stored":640782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":50414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28000000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":50414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27188,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":167936,"data_stored":50414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31348,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340492,"statfs":{"total":96636764160,"available":96604663808,"internally_reserved":0,"allocated":167936,"data_stored":50414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31340,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340500,"statfs":{"total":96636764160,"available":96604672000,"internally_reserved":0,"allocated":163840,"data_stored":49795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31344,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":167936,"data_stored":50414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32104,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339736,"statfs":{"total":96636764160,"available":96603889664,"internally_reserved":0,"allocated":757760,"data_stored":640163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-02T00:28:52.735 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-02T00:28:52.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.875+0000 7f57b74bb640 1 Processor -- start 2023-12-02T00:28:52.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.876+0000 7f57b74bb640 1 -- start start 2023-12-02T00:28:52.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.876+0000 7f57b74bb640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f57b006ea50 0x7f57b006ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:52.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.877+0000 7f57b74bb640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57b0161d20 con 0x7f57b0160fb0 2023-12-02T00:28:52.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.877+0000 7f57b74bb640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57b006f370 con 0x7f57b00d0ed0 2023-12-02T00:28:52.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.877+0000 7f57b74bb640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57b015e760 con 0x7f57b006ea50 2023-12-02T00:28:52.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.877+0000 7f57b5257640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f57b006ea50 0x7f57b006ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:52.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.877+0000 7f57b4a56640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f57b0160fb0 0x7f57b0161440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:44870/0 (socket says 172.21.15.46:44870) 2023-12-02T00:28:52.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.877+0000 7f57b4a56640 1 -- 172.21.15.46:0/711216686 learned_addr learned my addr 172.21.15.46:0/711216686 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:28:52.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.877+0000 7f57b5257640 1 -- 172.21.15.46:0/711216686 >> v1:172.21.15.162:6789/0 conn(0x7f57b00d0ed0 legacy=0x7f57b00d1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:52.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.877+0000 7f57b5257640 1 -- 172.21.15.46:0/711216686 >> v1:172.21.15.46:6789/0 conn(0x7f57b0160fb0 legacy=0x7f57b0161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:52.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.877+0000 7f57b5257640 1 -- 172.21.15.46:0/711216686 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b015d9a0 con 0x7f57b006ea50 2023-12-02T00:28:52.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.877+0000 7f57a7fff640 1 -- 172.21.15.46:0/711216686 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 855500701 0 0) 0x7f57b0161d20 con 0x7f57b0160fb0 2023-12-02T00:28:52.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.878+0000 7f57b5257640 1 --2- 172.21.15.46:0/711216686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57b006ea50 0x7f57b006ee30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f57ac0097a0 tx=0x7f57ac02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f14d7b20666b028c server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:52.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.878+0000 7f57a7fff640 1 -- 172.21.15.46:0/711216686 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ac031830 con 0x7f57b006ea50 2023-12-02T00:28:52.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.878+0000 7f57a7fff640 1 -- 172.21.15.46:0/711216686 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57ac031990 con 0x7f57b006ea50 2023-12-02T00:28:52.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.878+0000 7f57a7fff640 1 -- 172.21.15.46:0/711216686 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ac031d20 con 0x7f57b006ea50 2023-12-02T00:28:52.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.878+0000 7f57b74bb640 1 -- 172.21.15.46:0/711216686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57b006ea50 msgr2=0x7f57b006ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:52.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.878+0000 7f57b74bb640 1 --2- 172.21.15.46:0/711216686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57b006ea50 0x7f57b006ee30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f57ac0097a0 tx=0x7f57ac02b8d0 comp rx=0 tx=0).stop 2023-12-02T00:28:52.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.878+0000 7f57b74bb640 1 -- 172.21.15.46:0/711216686 shutdown_connections 2023-12-02T00:28:52.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.878+0000 7f57b74bb640 1 --2- 172.21.15.46:0/711216686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57b006ea50 0x7f57b006ee30 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:52.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.878+0000 7f57b74bb640 1 -- 172.21.15.46:0/711216686 >> 172.21.15.46:0/711216686 conn(0x7f57b006a6b0 msgr2=0x7f57b006aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:52.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.879+0000 7f57b74bb640 1 -- 172.21.15.46:0/711216686 shutdown_connections 2023-12-02T00:28:52.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.879+0000 7f57b74bb640 1 -- 172.21.15.46:0/711216686 wait complete. 2023-12-02T00:28:52.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.880+0000 7f57b74bb640 1 Processor -- start 2023-12-02T00:28:52.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.880+0000 7f57b74bb640 1 -- start start 2023-12-02T00:28:52.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.880+0000 7f57b74bb640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f57b006ea50 0x7f57b00b36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:52.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.880+0000 7f57b74bb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57b00d0ed0 0x7f57b01f66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:52.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.880+0000 7f57b74bb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f57b0160fb0 0x7f57b01f6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:52.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.880+0000 7f57b74bb640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b01f47d0 con 0x7f57b00d0ed0 2023-12-02T00:28:52.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.880+0000 7f57b74bb640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b00b4680 con 0x7f57b006ea50 2023-12-02T00:28:52.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.880+0000 7f57b74bb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57b0140040 con 0x7f57b0160fb0 2023-12-02T00:28:52.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.881+0000 7f57b4a56640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57b00d0ed0 0x7f57b01f66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:52.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.881+0000 7f57b4a56640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57b00d0ed0 0x7f57b01f66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37738/0 (socket says 172.21.15.46:37738) 2023-12-02T00:28:52.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.881+0000 7f57b4a56640 1 -- 172.21.15.46:0/3525914418 learned_addr learned my addr 172.21.15.46:0/3525914418 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:52.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.881+0000 7f57b5257640 1 --2- 172.21.15.46:0/3525914418 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f57b006ea50 0x7f57b00b36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:52.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.881+0000 7f57b5a58640 1 --2- 172.21.15.46:0/3525914418 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f57b0160fb0 0x7f57b01f6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:52.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.881+0000 7f57b4a56640 1 -- 172.21.15.46:0/3525914418 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f57b0160fb0 msgr2=0x7f57b01f6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:52.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.881+0000 7f57b4a56640 1 --2- 172.21.15.46:0/3525914418 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f57b0160fb0 0x7f57b01f6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:52.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.881+0000 7f57b4a56640 1 -- 172.21.15.46:0/3525914418 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f57b006ea50 msgr2=0x7f57b00b36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:52.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.881+0000 7f57b4a56640 1 --2- 172.21.15.46:0/3525914418 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f57b006ea50 0x7f57b00b36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:52.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.881+0000 7f57b4a56640 1 -- 172.21.15.46:0/3525914418 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ac009400 con 0x7f57b00d0ed0 2023-12-02T00:28:52.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.881+0000 7f57b5257640 1 --2- 172.21.15.46:0/3525914418 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f57b006ea50 0x7f57b00b36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:28:52.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.881+0000 7f57b4a56640 1 --2- 172.21.15.46:0/3525914418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57b00d0ed0 0x7f57b01f66a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f57a000c4a0 tx=0x7f57a000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:52.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.881+0000 7f57a5ffb640 1 -- 172.21.15.46:0/3525914418 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a00074e0 con 0x7f57b00d0ed0 2023-12-02T00:28:52.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.881+0000 7f57a5ffb640 1 -- 172.21.15.46:0/3525914418 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57a0007640 con 0x7f57b00d0ed0 2023-12-02T00:28:52.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.881+0000 7f57b74bb640 1 -- 172.21.15.46:0/3525914418 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57b00b3de0 con 0x7f57b00d0ed0 2023-12-02T00:28:52.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.881+0000 7f57a5ffb640 1 -- 172.21.15.46:0/3525914418 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a0004a00 con 0x7f57b00d0ed0 2023-12-02T00:28:52.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.882+0000 7f57b74bb640 1 -- 172.21.15.46:0/3525914418 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57b00b43a0 con 0x7f57b00d0ed0 2023-12-02T00:28:52.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.882+0000 7f57b74bb640 1 -- 172.21.15.46:0/3525914418 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57b01f4910 con 0x7f57b00d0ed0 2023-12-02T00:28:52.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.883+0000 7f57a5ffb640 1 -- 172.21.15.46:0/3525914418 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f57a0004b60 con 0x7f57b00d0ed0 2023-12-02T00:28:52.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.883+0000 7f57a5ffb640 1 --2- 172.21.15.46:0/3525914418 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f579406ae40 0x7f579406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:52.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.883+0000 7f57a5ffb640 1 -- 172.21.15.46:0/3525914418 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f57a0088aa0 con 0x7f57b00d0ed0 2023-12-02T00:28:52.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.883+0000 7f57b5257640 1 --2- 172.21.15.46:0/3525914418 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f579406ae40 0x7f579406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:52.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.884+0000 7f57b5257640 1 --2- 172.21.15.46:0/3525914418 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f579406ae40 0x7f579406d300 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f57ac036000 tx=0x7f57ac03c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:52.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:52.887+0000 7f57a5ffb640 1 -- 172.21.15.46:0/3525914418 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f57b01f4910 con 0x7f57b00d0ed0 2023-12-02T00:28:53.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.059+0000 7f57b74bb640 1 -- 172.21.15.46:0/3525914418 --> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f57b015e240 con 0x7f579406ae40 2023-12-02T00:28:53.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.066+0000 7f57a5ffb640 1 -- 172.21.15.46:0/3525914418 <== mgr.4108 v2:172.21.15.162:6832/3248644722 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+249848 (secure 0 0 0) 0x7f57ac044400 con 0x7f579406ae40 2023-12-02T00:28:53.069 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:28:53.070 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-12-02T00:28:53.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.069+0000 7f57b74bb640 1 -- 172.21.15.46:0/3525914418 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f579406ae40 msgr2=0x7f579406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:53.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.069+0000 7f57b74bb640 1 --2- 172.21.15.46:0/3525914418 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f579406ae40 0x7f579406d300 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f57ac036000 tx=0x7f57ac03c040 comp rx=0 tx=0).stop 2023-12-02T00:28:53.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.069+0000 7f57b74bb640 1 -- 172.21.15.46:0/3525914418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57b00d0ed0 msgr2=0x7f57b01f66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:53.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.069+0000 7f57b74bb640 1 --2- 172.21.15.46:0/3525914418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57b00d0ed0 0x7f57b01f66a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f57a000c4a0 tx=0x7f57a000c960 comp rx=0 tx=0).stop 2023-12-02T00:28:53.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.069+0000 7f57b74bb640 1 -- 172.21.15.46:0/3525914418 shutdown_connections 2023-12-02T00:28:53.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.069+0000 7f57b74bb640 1 --2- 172.21.15.46:0/3525914418 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f579406ae40 0x7f579406d300 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:53.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.069+0000 7f57b74bb640 1 --2- 172.21.15.46:0/3525914418 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f57b0160fb0 0x7f57b01f6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:53.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.069+0000 7f57b74bb640 1 --2- 172.21.15.46:0/3525914418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57b00d0ed0 0x7f57b01f66a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:53.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.069+0000 7f57b74bb640 1 --2- 172.21.15.46:0/3525914418 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f57b006ea50 0x7f57b00b36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:53.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.069+0000 7f57b74bb640 1 -- 172.21.15.46:0/3525914418 >> 172.21.15.46:0/3525914418 conn(0x7f57b006a6b0 msgr2=0x7f57b00d1320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:53.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.069+0000 7f57b74bb640 1 -- 172.21.15.46:0/3525914418 shutdown_connections 2023-12-02T00:28:53.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.069+0000 7f57b74bb640 1 -- 172.21.15.46:0/3525914418 wait complete. 2023-12-02T00:28:53.080 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-12-02T00:28:51.158255+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":236076,"kb_used_data":2464,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738644,"statfs":{"total":773094113280,"available":772852371456,"internally_reserved":0,"allocated":2523136,"data_stored":1582810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001083"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":161,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-12-02T00:28:40.992948+0000","last_change":"2023-12-02T00:28:39.379380+0000","last_active":"2023-12-02T00:28:40.992948+0000","last_peered":"2023-12-02T00:28:40.992948+0000","last_clean":"2023-12-02T00:28:40.992948+0000","last_became_active":"2023-12-02T00:28:39.378255+0000","last_became_peered":"2023-12-02T00:28:39.378255+0000","last_unstale":"2023-12-02T00:28:40.992948+0000","last_undegraded":"2023-12-02T00:28:40.992948+0000","last_fullsized":"2023-12-02T00:28:40.992948+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:38.177345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:38.177345+0000","last_clean_scrub_stamp":"2023-12-02T00:28:38.177345+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:21:08.582412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:48.709287+0000","last_change":"2023-12-02T00:28:48.709287+0000","last_active":"2023-12-02T00:28:48.709287+0000","last_peered":"2023-12-02T00:28:48.709287+0000","last_clean":"2023-12-02T00:28:48.709287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:48.709287+0000","last_undegraded":"2023-12-02T00:28:48.709287+0000","last_fullsized":"2023-12-02T00:28:48.709287+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:28:47.701913+0000","last_change":"2023-12-02T00:28:47.701913+0000","last_active":"2023-12-02T00:28:47.701913+0000","last_peered":"2023-12-02T00:28:47.701913+0000","last_clean":"2023-12-02T00:28:47.701913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:28:47.701913+0000","last_undegraded":"2023-12-02T00:28:47.701913+0000","last_fullsized":"2023-12-02T00:28:47.701913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28384,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343456,"statfs":{"total":96636764160,"available":96607698944,"internally_reserved":0,"allocated":761856,"data_stored":640782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":50414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28000000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":50414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27188,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":167936,"data_stored":50414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31348,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340492,"statfs":{"total":96636764160,"available":96604663808,"internally_reserved":0,"allocated":167936,"data_stored":50414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31340,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340500,"statfs":{"total":96636764160,"available":96604672000,"internally_reserved":0,"allocated":163840,"data_stored":49795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31344,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":167936,"data_stored":50414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32104,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339736,"statfs":{"total":96636764160,"available":96603889664,"internally_reserved":0,"allocated":757760,"data_stored":640163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-02T00:28:53.084 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-02T00:28:53.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.226+0000 7fca833ec640 1 Processor -- start 2023-12-02T00:28:53.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.226+0000 7fca833ec640 1 -- start start 2023-12-02T00:28:53.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.227+0000 7fca833ec640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fca7c15d2d0 0x7fca7c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:53.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.227+0000 7fca833ec640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fca7c160f70 0x7fca7c0cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:53.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.227+0000 7fca833ec640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fca7c0ce480 0x7fca7c06b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:53.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.227+0000 7fca833ec640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca7c1614e0 con 0x7fca7c0ce480 2023-12-02T00:28:53.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.227+0000 7fca833ec640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca7c1617c0 con 0x7fca7c160f70 2023-12-02T00:28:53.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.227+0000 7fca833ec640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c161aa0 con 0x7fca7c15d2d0 2023-12-02T00:28:53.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.227+0000 7fca81989640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fca7c0ce480 0x7fca7c06b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:53.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.227+0000 7fca81989640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fca7c0ce480 0x7fca7c06b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37752/0 (socket says 172.21.15.46:37752) 2023-12-02T00:28:53.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.227+0000 7fca81989640 1 -- 172.21.15.46:0/82915999 learned_addr learned my addr 172.21.15.46:0/82915999 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:53.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.227+0000 7fca80987640 1 --2- 172.21.15.46:0/82915999 >> v2:172.21.15.162:3300/0 conn(0x7fca7c160f70 0x7fca7c0cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:53.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.227+0000 7fca81188640 1 --2- 172.21.15.46:0/82915999 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fca7c15d2d0 0x7fca7c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:53.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.228+0000 7fca81989640 1 -- 172.21.15.46:0/82915999 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fca7c15d2d0 msgr2=0x7fca7c0b8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:53.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.228+0000 7fca81989640 1 --2- 172.21.15.46:0/82915999 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fca7c15d2d0 0x7fca7c0b8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:53.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.228+0000 7fca81989640 1 -- 172.21.15.46:0/82915999 >> v2:172.21.15.162:3300/0 conn(0x7fca7c160f70 msgr2=0x7fca7c0cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:53.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.228+0000 7fca81989640 1 --2- 172.21.15.46:0/82915999 >> v2:172.21.15.162:3300/0 conn(0x7fca7c160f70 0x7fca7c0cdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:53.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.228+0000 7fca81989640 1 -- 172.21.15.46:0/82915999 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca7c161d80 con 0x7fca7c0ce480 2023-12-02T00:28:53.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.228+0000 7fca81188640 1 --2- 172.21.15.46:0/82915999 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fca7c15d2d0 0x7fca7c0b8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:28:53.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.228+0000 7fca80987640 1 --2- 172.21.15.46:0/82915999 >> v2:172.21.15.162:3300/0 conn(0x7fca7c160f70 0x7fca7c0cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:28:53.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.228+0000 7fca81989640 1 --2- 172.21.15.46:0/82915999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca7c0ce480 0x7fca7c06b570 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fca7c0d4260 tx=0x7fca7802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=736431dd207c3f7d server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:53.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.228+0000 7fca73fff640 1 -- 172.21.15.46:0/82915999 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7802ca70 con 0x7fca7c0ce480 2023-12-02T00:28:53.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.228+0000 7fca73fff640 1 -- 172.21.15.46:0/82915999 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca7802cbd0 con 0x7fca7c0ce480 2023-12-02T00:28:53.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.228+0000 7fca73fff640 1 -- 172.21.15.46:0/82915999 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca780343f0 con 0x7fca7c0ce480 2023-12-02T00:28:53.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.229+0000 7fca833ec640 1 -- 172.21.15.46:0/82915999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca7c0ce480 msgr2=0x7fca7c06b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:53.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.229+0000 7fca833ec640 1 --2- 172.21.15.46:0/82915999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca7c0ce480 0x7fca7c06b570 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fca7c0d4260 tx=0x7fca7802bed0 comp rx=0 tx=0).stop 2023-12-02T00:28:53.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.229+0000 7fca833ec640 1 -- 172.21.15.46:0/82915999 shutdown_connections 2023-12-02T00:28:53.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.229+0000 7fca833ec640 1 --2- 172.21.15.46:0/82915999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca7c0ce480 0x7fca7c06b570 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:53.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.229+0000 7fca833ec640 1 --2- 172.21.15.46:0/82915999 >> v2:172.21.15.162:3300/0 conn(0x7fca7c160f70 0x7fca7c0cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:53.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.229+0000 7fca833ec640 1 --2- 172.21.15.46:0/82915999 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fca7c15d2d0 0x7fca7c0b8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:53.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.229+0000 7fca833ec640 1 -- 172.21.15.46:0/82915999 >> 172.21.15.46:0/82915999 conn(0x7fca7c072230 msgr2=0x7fca7c0cfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:53.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.229+0000 7fca833ec640 1 -- 172.21.15.46:0/82915999 shutdown_connections 2023-12-02T00:28:53.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.229+0000 7fca833ec640 1 -- 172.21.15.46:0/82915999 wait complete. 2023-12-02T00:28:53.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.230+0000 7fca833ec640 1 Processor -- start 2023-12-02T00:28:53.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.230+0000 7fca833ec640 1 -- start start 2023-12-02T00:28:53.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.231+0000 7fca833ec640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca7c0ce480 0x7fca7c0c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:53.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.231+0000 7fca833ec640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca7c15d2d0 0x7fca7c1604e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:53.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.231+0000 7fca833ec640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fca7c160f70 0x7fca7c1f2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:53.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.231+0000 7fca833ec640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c160a20 con 0x7fca7c0ce480 2023-12-02T00:28:53.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.231+0000 7fca833ec640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c15dbb0 con 0x7fca7c15d2d0 2023-12-02T00:28:53.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.231+0000 7fca833ec640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c0c9210 con 0x7fca7c160f70 2023-12-02T00:28:53.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.231+0000 7fca81188640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca7c0ce480 0x7fca7c0c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:53.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.231+0000 7fca81188640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca7c0ce480 0x7fca7c0c8b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37756/0 (socket says 172.21.15.46:37756) 2023-12-02T00:28:53.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.231+0000 7fca81188640 1 -- 172.21.15.46:0/728147615 learned_addr learned my addr 172.21.15.46:0/728147615 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:53.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.231+0000 7fca80987640 1 --2- 172.21.15.46:0/728147615 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca7c15d2d0 0x7fca7c1604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:53.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.231+0000 7fca81989640 1 --2- 172.21.15.46:0/728147615 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fca7c160f70 0x7fca7c1f2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:53.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.231+0000 7fca81188640 1 -- 172.21.15.46:0/728147615 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fca7c160f70 msgr2=0x7fca7c1f2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:53.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.231+0000 7fca81188640 1 --2- 172.21.15.46:0/728147615 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fca7c160f70 0x7fca7c1f2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:53.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.231+0000 7fca81188640 1 -- 172.21.15.46:0/728147615 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca7c15d2d0 msgr2=0x7fca7c1604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:53.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.231+0000 7fca81188640 1 --2- 172.21.15.46:0/728147615 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca7c15d2d0 0x7fca7c1604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:53.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.231+0000 7fca81188640 1 -- 172.21.15.46:0/728147615 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca78009b30 con 0x7fca7c0ce480 2023-12-02T00:28:53.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.231+0000 7fca81188640 1 --2- 172.21.15.46:0/728147615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca7c0ce480 0x7fca7c0c8b30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fca7400e810 tx=0x7fca7400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:53.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.232+0000 7fca71ffb640 1 -- 172.21.15.46:0/728147615 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca74004030 con 0x7fca7c0ce480 2023-12-02T00:28:53.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.232+0000 7fca71ffb640 1 -- 172.21.15.46:0/728147615 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca74004190 con 0x7fca7c0ce480 2023-12-02T00:28:53.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.232+0000 7fca833ec640 1 -- 172.21.15.46:0/728147615 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca7c0c94f0 con 0x7fca7c0ce480 2023-12-02T00:28:53.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.232+0000 7fca71ffb640 1 -- 172.21.15.46:0/728147615 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca74004520 con 0x7fca7c0ce480 2023-12-02T00:28:53.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.232+0000 7fca833ec640 1 -- 172.21.15.46:0/728147615 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca7c0c9ab0 con 0x7fca7c0ce480 2023-12-02T00:28:53.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.233+0000 7fca833ec640 1 -- 172.21.15.46:0/728147615 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca7c069400 con 0x7fca7c0ce480 2023-12-02T00:28:53.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.233+0000 7fca71ffb640 1 -- 172.21.15.46:0/728147615 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fca7400c6d0 con 0x7fca7c0ce480 2023-12-02T00:28:53.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.234+0000 7fca71ffb640 1 --2- 172.21.15.46:0/728147615 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fca6006ae70 0x7fca6006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:53.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.234+0000 7fca71ffb640 1 -- 172.21.15.46:0/728147615 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fca74089090 con 0x7fca7c0ce480 2023-12-02T00:28:53.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.234+0000 7fca80987640 1 --2- 172.21.15.46:0/728147615 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fca6006ae70 0x7fca6006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:53.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.234+0000 7fca80987640 1 --2- 172.21.15.46:0/728147615 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fca6006ae70 0x7fca6006d330 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fca7c160080 tx=0x7fca6c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:53.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.238+0000 7fca71ffb640 1 -- 172.21.15.46:0/728147615 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fca740541d0 con 0x7fca7c0ce480 2023-12-02T00:28:53.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.411+0000 7fca833ec640 1 -- 172.21.15.46:0/728147615 --> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fca7c15dd50 con 0x7fca6006ae70 2023-12-02T00:28:53.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.418+0000 7fca71ffb640 1 -- 172.21.15.46:0/728147615 <== mgr.4108 v2:172.21.15.162:6832/3248644722 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264715 (secure 0 0 0) 0x7fca7c15dd50 con 0x7fca6006ae70 2023-12-02T00:28:53.422 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:28:53.423 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-12-02T00:28:53.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.421+0000 7fca833ec640 1 -- 172.21.15.46:0/728147615 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fca6006ae70 msgr2=0x7fca6006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:53.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.421+0000 7fca833ec640 1 --2- 172.21.15.46:0/728147615 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fca6006ae70 0x7fca6006d330 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fca7c160080 tx=0x7fca6c008040 comp rx=0 tx=0).stop 2023-12-02T00:28:53.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.421+0000 7fca833ec640 1 -- 172.21.15.46:0/728147615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca7c0ce480 msgr2=0x7fca7c0c8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:53.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.421+0000 7fca833ec640 1 --2- 172.21.15.46:0/728147615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca7c0ce480 0x7fca7c0c8b30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fca7400e810 tx=0x7fca7400ecd0 comp rx=0 tx=0).stop 2023-12-02T00:28:53.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.421+0000 7fca833ec640 1 -- 172.21.15.46:0/728147615 shutdown_connections 2023-12-02T00:28:53.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.421+0000 7fca833ec640 1 --2- 172.21.15.46:0/728147615 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fca6006ae70 0x7fca6006d330 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:53.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.421+0000 7fca833ec640 1 --2- 172.21.15.46:0/728147615 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fca7c160f70 0x7fca7c1f2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:53.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.421+0000 7fca833ec640 1 --2- 172.21.15.46:0/728147615 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca7c15d2d0 0x7fca7c1604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:53.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.421+0000 7fca833ec640 1 --2- 172.21.15.46:0/728147615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca7c0ce480 0x7fca7c0c8b30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:53.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.421+0000 7fca833ec640 1 -- 172.21.15.46:0/728147615 >> 172.21.15.46:0/728147615 conn(0x7fca7c072230 msgr2=0x7fca7c0d3a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:53.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.421+0000 7fca833ec640 1 -- 172.21.15.46:0/728147615 shutdown_connections 2023-12-02T00:28:53.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.421+0000 7fca833ec640 1 -- 172.21.15.46:0/728147615 wait complete. 2023-12-02T00:28:53.433 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-12-02T00:28:53.158780+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":172,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235644,"kb_used_data":2828,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739076,"statfs":{"total":773094113280,"available":772852813824,"internally_reserved":0,"allocated":2895872,"data_stored":1744866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.448626"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021839+0000","last_change":"2023-12-02T00:28:49.717914+0000","last_active":"2023-12-02T00:28:51.021839+0000","last_peered":"2023-12-02T00:28:51.021839+0000","last_clean":"2023-12-02T00:28:51.021839+0000","last_became_active":"2023-12-02T00:28:49.717594+0000","last_became_peered":"2023-12-02T00:28:49.717594+0000","last_unstale":"2023-12-02T00:28:51.021839+0000","last_undegraded":"2023-12-02T00:28:51.021839+0000","last_fullsized":"2023-12-02T00:28:51.021839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:48:16.850673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703744+0000","last_change":"2023-12-02T00:28:49.713605+0000","last_active":"2023-12-02T00:28:50.703744+0000","last_peered":"2023-12-02T00:28:50.703744+0000","last_clean":"2023-12-02T00:28:50.703744+0000","last_became_active":"2023-12-02T00:28:49.712497+0000","last_became_peered":"2023-12-02T00:28:49.712497+0000","last_unstale":"2023-12-02T00:28:50.703744+0000","last_undegraded":"2023-12-02T00:28:50.703744+0000","last_fullsized":"2023-12-02T00:28:50.703744+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:15:51.747629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021917+0000","last_change":"2023-12-02T00:28:49.713065+0000","last_active":"2023-12-02T00:28:51.021917+0000","last_peered":"2023-12-02T00:28:51.021917+0000","last_clean":"2023-12-02T00:28:51.021917+0000","last_became_active":"2023-12-02T00:28:49.712715+0000","last_became_peered":"2023-12-02T00:28:49.712715+0000","last_unstale":"2023-12-02T00:28:51.021917+0000","last_undegraded":"2023-12-02T00:28:51.021917+0000","last_fullsized":"2023-12-02T00:28:51.021917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:52:22.600655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706337+0000","last_change":"2023-12-02T00:28:49.727831+0000","last_active":"2023-12-02T00:28:50.706337+0000","last_peered":"2023-12-02T00:28:50.706337+0000","last_clean":"2023-12-02T00:28:50.706337+0000","last_became_active":"2023-12-02T00:28:49.725804+0000","last_became_peered":"2023-12-02T00:28:49.725804+0000","last_unstale":"2023-12-02T00:28:50.706337+0000","last_undegraded":"2023-12-02T00:28:50.706337+0000","last_fullsized":"2023-12-02T00:28:50.706337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:50:45.793304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021332+0000","last_change":"2023-12-02T00:28:49.721391+0000","last_active":"2023-12-02T00:28:51.021332+0000","last_peered":"2023-12-02T00:28:51.021332+0000","last_clean":"2023-12-02T00:28:51.021332+0000","last_became_active":"2023-12-02T00:28:49.720314+0000","last_became_peered":"2023-12-02T00:28:49.720314+0000","last_unstale":"2023-12-02T00:28:51.021332+0000","last_undegraded":"2023-12-02T00:28:51.021332+0000","last_fullsized":"2023-12-02T00:28:51.021332+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:32:13.515968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705990+0000","last_change":"2023-12-02T00:28:49.725516+0000","last_active":"2023-12-02T00:28:50.705990+0000","last_peered":"2023-12-02T00:28:50.705990+0000","last_clean":"2023-12-02T00:28:50.705990+0000","last_became_active":"2023-12-02T00:28:49.725149+0000","last_became_peered":"2023-12-02T00:28:49.725149+0000","last_unstale":"2023-12-02T00:28:50.705990+0000","last_undegraded":"2023-12-02T00:28:50.705990+0000","last_fullsized":"2023-12-02T00:28:50.705990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T10:27:29.244181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021273+0000","last_change":"2023-12-02T00:28:49.719153+0000","last_active":"2023-12-02T00:28:51.021273+0000","last_peered":"2023-12-02T00:28:51.021273+0000","last_clean":"2023-12-02T00:28:51.021273+0000","last_became_active":"2023-12-02T00:28:49.718567+0000","last_became_peered":"2023-12-02T00:28:49.718567+0000","last_unstale":"2023-12-02T00:28:51.021273+0000","last_undegraded":"2023-12-02T00:28:51.021273+0000","last_fullsized":"2023-12-02T00:28:51.021273+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T10:27:29.244181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020683+0000","last_change":"2023-12-02T00:28:49.726986+0000","last_active":"2023-12-02T00:28:51.020683+0000","last_peered":"2023-12-02T00:28:51.020683+0000","last_clean":"2023-12-02T00:28:51.020683+0000","last_became_active":"2023-12-02T00:28:49.726646+0000","last_became_peered":"2023-12-02T00:28:49.726646+0000","last_unstale":"2023-12-02T00:28:51.020683+0000","last_undegraded":"2023-12-02T00:28:51.020683+0000","last_fullsized":"2023-12-02T00:28:51.020683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:35:09.231704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021827+0000","last_change":"2023-12-02T00:28:49.725402+0000","last_active":"2023-12-02T00:28:51.021827+0000","last_peered":"2023-12-02T00:28:51.021827+0000","last_clean":"2023-12-02T00:28:51.021827+0000","last_became_active":"2023-12-02T00:28:49.724978+0000","last_became_peered":"2023-12-02T00:28:49.724978+0000","last_unstale":"2023-12-02T00:28:51.021827+0000","last_undegraded":"2023-12-02T00:28:51.021827+0000","last_fullsized":"2023-12-02T00:28:51.021827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:18:31.615464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021500+0000","last_change":"2023-12-02T00:28:49.718464+0000","last_active":"2023-12-02T00:28:51.021500+0000","last_peered":"2023-12-02T00:28:51.021500+0000","last_clean":"2023-12-02T00:28:51.021500+0000","last_became_active":"2023-12-02T00:28:49.718060+0000","last_became_peered":"2023-12-02T00:28:49.718060+0000","last_unstale":"2023-12-02T00:28:51.021500+0000","last_undegraded":"2023-12-02T00:28:51.021500+0000","last_fullsized":"2023-12-02T00:28:51.021500+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:36:22.587928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021177+0000","last_change":"2023-12-02T00:28:49.720030+0000","last_active":"2023-12-02T00:28:51.021177+0000","last_peered":"2023-12-02T00:28:51.021177+0000","last_clean":"2023-12-02T00:28:51.021177+0000","last_became_active":"2023-12-02T00:28:49.719715+0000","last_became_peered":"2023-12-02T00:28:49.719715+0000","last_unstale":"2023-12-02T00:28:51.021177+0000","last_undegraded":"2023-12-02T00:28:51.021177+0000","last_fullsized":"2023-12-02T00:28:51.021177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:07:52.802590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705569+0000","last_change":"2023-12-02T00:28:49.713687+0000","last_active":"2023-12-02T00:28:50.705569+0000","last_peered":"2023-12-02T00:28:50.705569+0000","last_clean":"2023-12-02T00:28:50.705569+0000","last_became_active":"2023-12-02T00:28:49.713359+0000","last_became_peered":"2023-12-02T00:28:49.713359+0000","last_unstale":"2023-12-02T00:28:50.705569+0000","last_undegraded":"2023-12-02T00:28:50.705569+0000","last_fullsized":"2023-12-02T00:28:50.705569+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:11:06.388890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021465+0000","last_change":"2023-12-02T00:28:49.720866+0000","last_active":"2023-12-02T00:28:51.021465+0000","last_peered":"2023-12-02T00:28:51.021465+0000","last_clean":"2023-12-02T00:28:51.021465+0000","last_became_active":"2023-12-02T00:28:49.720524+0000","last_became_peered":"2023-12-02T00:28:49.720524+0000","last_unstale":"2023-12-02T00:28:51.021465+0000","last_undegraded":"2023-12-02T00:28:51.021465+0000","last_fullsized":"2023-12-02T00:28:51.021465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:32:13.515968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021720+0000","last_change":"2023-12-02T00:28:49.711387+0000","last_active":"2023-12-02T00:28:51.021720+0000","last_peered":"2023-12-02T00:28:51.021720+0000","last_clean":"2023-12-02T00:28:51.021720+0000","last_became_active":"2023-12-02T00:28:49.711036+0000","last_became_peered":"2023-12-02T00:28:49.711036+0000","last_unstale":"2023-12-02T00:28:51.021720+0000","last_undegraded":"2023-12-02T00:28:51.021720+0000","last_fullsized":"2023-12-02T00:28:51.021720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:13:36.512466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703511+0000","last_change":"2023-12-02T00:28:49.706444+0000","last_active":"2023-12-02T00:28:50.703511+0000","last_peered":"2023-12-02T00:28:50.703511+0000","last_clean":"2023-12-02T00:28:50.703511+0000","last_became_active":"2023-12-02T00:28:49.706132+0000","last_became_peered":"2023-12-02T00:28:49.706132+0000","last_unstale":"2023-12-02T00:28:50.703511+0000","last_undegraded":"2023-12-02T00:28:50.703511+0000","last_fullsized":"2023-12-02T00:28:50.703511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:52:22.600655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705451+0000","last_change":"2023-12-02T00:28:49.725045+0000","last_active":"2023-12-02T00:28:50.705451+0000","last_peered":"2023-12-02T00:28:50.705451+0000","last_clean":"2023-12-02T00:28:50.705451+0000","last_became_active":"2023-12-02T00:28:49.722884+0000","last_became_peered":"2023-12-02T00:28:49.722884+0000","last_unstale":"2023-12-02T00:28:50.705451+0000","last_undegraded":"2023-12-02T00:28:50.705451+0000","last_fullsized":"2023-12-02T00:28:50.705451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:36:22.587928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706621+0000","last_change":"2023-12-02T00:28:49.727232+0000","last_active":"2023-12-02T00:28:50.706621+0000","last_peered":"2023-12-02T00:28:50.706621+0000","last_clean":"2023-12-02T00:28:50.706621+0000","last_became_active":"2023-12-02T00:28:49.726881+0000","last_became_peered":"2023-12-02T00:28:49.726881+0000","last_unstale":"2023-12-02T00:28:50.706621+0000","last_undegraded":"2023-12-02T00:28:50.706621+0000","last_fullsized":"2023-12-02T00:28:50.706621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:48:16.850673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021328+0000","last_change":"2023-12-02T00:28:49.711953+0000","last_active":"2023-12-02T00:28:51.021328+0000","last_peered":"2023-12-02T00:28:51.021328+0000","last_clean":"2023-12-02T00:28:51.021328+0000","last_became_active":"2023-12-02T00:28:49.711653+0000","last_became_peered":"2023-12-02T00:28:49.711653+0000","last_unstale":"2023-12-02T00:28:51.021328+0000","last_undegraded":"2023-12-02T00:28:51.021328+0000","last_fullsized":"2023-12-02T00:28:51.021328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:52:08.689376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021886+0000","last_change":"2023-12-02T00:28:49.717365+0000","last_active":"2023-12-02T00:28:51.021886+0000","last_peered":"2023-12-02T00:28:51.021886+0000","last_clean":"2023-12-02T00:28:51.021886+0000","last_became_active":"2023-12-02T00:28:49.717071+0000","last_became_peered":"2023-12-02T00:28:49.717071+0000","last_unstale":"2023-12-02T00:28:51.021886+0000","last_undegraded":"2023-12-02T00:28:51.021886+0000","last_fullsized":"2023-12-02T00:28:51.021886+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:10:26.775049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021416+0000","last_change":"2023-12-02T00:28:49.716297+0000","last_active":"2023-12-02T00:28:51.021416+0000","last_peered":"2023-12-02T00:28:51.021416+0000","last_clean":"2023-12-02T00:28:51.021416+0000","last_became_active":"2023-12-02T00:28:49.715963+0000","last_became_peered":"2023-12-02T00:28:49.715963+0000","last_unstale":"2023-12-02T00:28:51.021416+0000","last_undegraded":"2023-12-02T00:28:51.021416+0000","last_fullsized":"2023-12-02T00:28:51.021416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:58:11.596095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021739+0000","last_change":"2023-12-02T00:28:49.722259+0000","last_active":"2023-12-02T00:28:51.021739+0000","last_peered":"2023-12-02T00:28:51.021739+0000","last_clean":"2023-12-02T00:28:51.021739+0000","last_became_active":"2023-12-02T00:28:49.721989+0000","last_became_peered":"2023-12-02T00:28:49.721989+0000","last_unstale":"2023-12-02T00:28:51.021739+0000","last_undegraded":"2023-12-02T00:28:51.021739+0000","last_fullsized":"2023-12-02T00:28:51.021739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:55:55.141283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705987+0000","last_change":"2023-12-02T00:28:49.729526+0000","last_active":"2023-12-02T00:28:50.705987+0000","last_peered":"2023-12-02T00:28:50.705987+0000","last_clean":"2023-12-02T00:28:50.705987+0000","last_became_active":"2023-12-02T00:28:49.728999+0000","last_became_peered":"2023-12-02T00:28:49.728999+0000","last_unstale":"2023-12-02T00:28:50.705987+0000","last_undegraded":"2023-12-02T00:28:50.705987+0000","last_fullsized":"2023-12-02T00:28:50.705987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:58:15.388237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021070+0000","last_change":"2023-12-02T00:28:49.717936+0000","last_active":"2023-12-02T00:28:51.021070+0000","last_peered":"2023-12-02T00:28:51.021070+0000","last_clean":"2023-12-02T00:28:51.021070+0000","last_became_active":"2023-12-02T00:28:49.716770+0000","last_became_peered":"2023-12-02T00:28:49.716770+0000","last_unstale":"2023-12-02T00:28:51.021070+0000","last_undegraded":"2023-12-02T00:28:51.021070+0000","last_fullsized":"2023-12-02T00:28:51.021070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:36:22.587928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705591+0000","last_change":"2023-12-02T00:28:49.724863+0000","last_active":"2023-12-02T00:28:50.705591+0000","last_peered":"2023-12-02T00:28:50.705591+0000","last_clean":"2023-12-02T00:28:50.705591+0000","last_became_active":"2023-12-02T00:28:49.724592+0000","last_became_peered":"2023-12-02T00:28:49.724592+0000","last_unstale":"2023-12-02T00:28:50.705591+0000","last_undegraded":"2023-12-02T00:28:50.705591+0000","last_fullsized":"2023-12-02T00:28:50.705591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:15:51.747629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020786+0000","last_change":"2023-12-02T00:28:49.724283+0000","last_active":"2023-12-02T00:28:51.020786+0000","last_peered":"2023-12-02T00:28:51.020786+0000","last_clean":"2023-12-02T00:28:51.020786+0000","last_became_active":"2023-12-02T00:28:49.723940+0000","last_became_peered":"2023-12-02T00:28:49.723940+0000","last_unstale":"2023-12-02T00:28:51.020786+0000","last_undegraded":"2023-12-02T00:28:51.020786+0000","last_fullsized":"2023-12-02T00:28:51.020786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:52:22.600655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021632+0000","last_change":"2023-12-02T00:28:49.717015+0000","last_active":"2023-12-02T00:28:51.021632+0000","last_peered":"2023-12-02T00:28:51.021632+0000","last_clean":"2023-12-02T00:28:51.021632+0000","last_became_active":"2023-12-02T00:28:49.716394+0000","last_became_peered":"2023-12-02T00:28:49.716394+0000","last_unstale":"2023-12-02T00:28:51.021632+0000","last_undegraded":"2023-12-02T00:28:51.021632+0000","last_fullsized":"2023-12-02T00:28:51.021632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:15:51.747629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703687+0000","last_change":"2023-12-02T00:28:49.716707+0000","last_active":"2023-12-02T00:28:50.703687+0000","last_peered":"2023-12-02T00:28:50.703687+0000","last_clean":"2023-12-02T00:28:50.703687+0000","last_became_active":"2023-12-02T00:28:49.716392+0000","last_became_peered":"2023-12-02T00:28:49.716392+0000","last_unstale":"2023-12-02T00:28:50.703687+0000","last_undegraded":"2023-12-02T00:28:50.703687+0000","last_fullsized":"2023-12-02T00:28:50.703687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:36:22.587928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021477+0000","last_change":"2023-12-02T00:28:49.724290+0000","last_active":"2023-12-02T00:28:51.021477+0000","last_peered":"2023-12-02T00:28:51.021477+0000","last_clean":"2023-12-02T00:28:51.021477+0000","last_became_active":"2023-12-02T00:28:49.723952+0000","last_became_peered":"2023-12-02T00:28:49.723952+0000","last_unstale":"2023-12-02T00:28:51.021477+0000","last_undegraded":"2023-12-02T00:28:51.021477+0000","last_fullsized":"2023-12-02T00:28:51.021477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:15:02.988933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021881+0000","last_change":"2023-12-02T00:28:49.721218+0000","last_active":"2023-12-02T00:28:51.021881+0000","last_peered":"2023-12-02T00:28:51.021881+0000","last_clean":"2023-12-02T00:28:51.021881+0000","last_became_active":"2023-12-02T00:28:49.720894+0000","last_became_peered":"2023-12-02T00:28:49.720894+0000","last_unstale":"2023-12-02T00:28:51.021881+0000","last_undegraded":"2023-12-02T00:28:51.021881+0000","last_fullsized":"2023-12-02T00:28:51.021881+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:35:09.231704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706145+0000","last_change":"2023-12-02T00:28:49.725424+0000","last_active":"2023-12-02T00:28:50.706145+0000","last_peered":"2023-12-02T00:28:50.706145+0000","last_clean":"2023-12-02T00:28:50.706145+0000","last_became_active":"2023-12-02T00:28:49.724671+0000","last_became_peered":"2023-12-02T00:28:49.724671+0000","last_unstale":"2023-12-02T00:28:50.706145+0000","last_undegraded":"2023-12-02T00:28:50.706145+0000","last_fullsized":"2023-12-02T00:28:50.706145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:32:13.515968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021246+0000","last_change":"2023-12-02T00:28:49.722395+0000","last_active":"2023-12-02T00:28:51.021246+0000","last_peered":"2023-12-02T00:28:51.021246+0000","last_clean":"2023-12-02T00:28:51.021246+0000","last_became_active":"2023-12-02T00:28:49.722052+0000","last_became_peered":"2023-12-02T00:28:49.722052+0000","last_unstale":"2023-12-02T00:28:51.021246+0000","last_undegraded":"2023-12-02T00:28:51.021246+0000","last_fullsized":"2023-12-02T00:28:51.021246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:11:06.388890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020869+0000","last_change":"2023-12-02T00:28:49.713618+0000","last_active":"2023-12-02T00:28:51.020869+0000","last_peered":"2023-12-02T00:28:51.020869+0000","last_clean":"2023-12-02T00:28:51.020869+0000","last_became_active":"2023-12-02T00:28:49.713325+0000","last_became_peered":"2023-12-02T00:28:49.713325+0000","last_unstale":"2023-12-02T00:28:51.020869+0000","last_undegraded":"2023-12-02T00:28:51.020869+0000","last_fullsized":"2023-12-02T00:28:51.020869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:35:09.231704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705601+0000","last_change":"2023-12-02T00:28:49.710604+0000","last_active":"2023-12-02T00:28:50.705601+0000","last_peered":"2023-12-02T00:28:50.705601+0000","last_clean":"2023-12-02T00:28:50.705601+0000","last_became_active":"2023-12-02T00:28:49.710316+0000","last_became_peered":"2023-12-02T00:28:49.710316+0000","last_unstale":"2023-12-02T00:28:50.705601+0000","last_undegraded":"2023-12-02T00:28:50.705601+0000","last_fullsized":"2023-12-02T00:28:50.705601+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:52:08.689376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021549+0000","last_change":"2023-12-02T00:28:48.735813+0000","last_active":"2023-12-02T00:28:51.021549+0000","last_peered":"2023-12-02T00:28:51.021549+0000","last_clean":"2023-12-02T00:28:51.021549+0000","last_became_active":"2023-12-02T00:28:48.735462+0000","last_became_peered":"2023-12-02T00:28:48.735462+0000","last_unstale":"2023-12-02T00:28:51.021549+0000","last_undegraded":"2023-12-02T00:28:51.021549+0000","last_fullsized":"2023-12-02T00:28:51.021549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:55:54.133908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.022018+0000","last_change":"2023-12-02T00:28:49.715318+0000","last_active":"2023-12-02T00:28:51.022018+0000","last_peered":"2023-12-02T00:28:51.022018+0000","last_clean":"2023-12-02T00:28:51.022018+0000","last_became_active":"2023-12-02T00:28:49.715020+0000","last_became_peered":"2023-12-02T00:28:49.715020+0000","last_unstale":"2023-12-02T00:28:51.022018+0000","last_undegraded":"2023-12-02T00:28:51.022018+0000","last_fullsized":"2023-12-02T00:28:51.022018+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:36:22.587928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021150+0000","last_change":"2023-12-02T00:28:48.730625+0000","last_active":"2023-12-02T00:28:51.021150+0000","last_peered":"2023-12-02T00:28:51.021150+0000","last_clean":"2023-12-02T00:28:51.021150+0000","last_became_active":"2023-12-02T00:28:48.730306+0000","last_became_peered":"2023-12-02T00:28:48.730306+0000","last_unstale":"2023-12-02T00:28:51.021150+0000","last_undegraded":"2023-12-02T00:28:51.021150+0000","last_fullsized":"2023-12-02T00:28:51.021150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:11:05.381516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020857+0000","last_change":"2023-12-02T00:28:48.735160+0000","last_active":"2023-12-02T00:28:51.020857+0000","last_peered":"2023-12-02T00:28:51.020857+0000","last_clean":"2023-12-02T00:28:51.020857+0000","last_became_active":"2023-12-02T00:28:48.734860+0000","last_became_peered":"2023-12-02T00:28:48.734860+0000","last_unstale":"2023-12-02T00:28:51.020857+0000","last_undegraded":"2023-12-02T00:28:51.020857+0000","last_fullsized":"2023-12-02T00:28:51.020857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T09:17:51.234416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020811+0000","last_change":"2023-12-02T00:28:49.716640+0000","last_active":"2023-12-02T00:28:51.020811+0000","last_peered":"2023-12-02T00:28:51.020811+0000","last_clean":"2023-12-02T00:28:51.020811+0000","last_became_active":"2023-12-02T00:28:49.716325+0000","last_became_peered":"2023-12-02T00:28:49.716325+0000","last_unstale":"2023-12-02T00:28:51.020811+0000","last_undegraded":"2023-12-02T00:28:51.020811+0000","last_fullsized":"2023-12-02T00:28:51.020811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:07:52.802590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705691+0000","last_change":"2023-12-02T00:28:48.730403+0000","last_active":"2023-12-02T00:28:50.705691+0000","last_peered":"2023-12-02T00:28:50.705691+0000","last_clean":"2023-12-02T00:28:50.705691+0000","last_became_active":"2023-12-02T00:28:48.730053+0000","last_became_peered":"2023-12-02T00:28:48.730053+0000","last_unstale":"2023-12-02T00:28:50.705691+0000","last_undegraded":"2023-12-02T00:28:50.705691+0000","last_fullsized":"2023-12-02T00:28:50.705691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:13:35.505091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703512+0000","last_change":"2023-12-02T00:28:49.716925+0000","last_active":"2023-12-02T00:28:50.703512+0000","last_peered":"2023-12-02T00:28:50.703512+0000","last_clean":"2023-12-02T00:28:50.703512+0000","last_became_active":"2023-12-02T00:28:49.716530+0000","last_became_peered":"2023-12-02T00:28:49.716530+0000","last_unstale":"2023-12-02T00:28:50.703512+0000","last_undegraded":"2023-12-02T00:28:50.703512+0000","last_fullsized":"2023-12-02T00:28:50.703512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:35:09.231704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021002+0000","last_change":"2023-12-02T00:28:48.732302+0000","last_active":"2023-12-02T00:28:51.021002+0000","last_peered":"2023-12-02T00:28:51.021002+0000","last_clean":"2023-12-02T00:28:51.021002+0000","last_became_active":"2023-12-02T00:28:48.731891+0000","last_became_peered":"2023-12-02T00:28:48.731891+0000","last_unstale":"2023-12-02T00:28:51.021002+0000","last_undegraded":"2023-12-02T00:28:51.021002+0000","last_fullsized":"2023-12-02T00:28:51.021002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:44:45.468077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706568+0000","last_change":"2023-12-02T00:28:49.719085+0000","last_active":"2023-12-02T00:28:50.706568+0000","last_peered":"2023-12-02T00:28:50.706568+0000","last_clean":"2023-12-02T00:28:50.706568+0000","last_became_active":"2023-12-02T00:28:49.718606+0000","last_became_peered":"2023-12-02T00:28:49.718606+0000","last_unstale":"2023-12-02T00:28:50.706568+0000","last_undegraded":"2023-12-02T00:28:50.706568+0000","last_fullsized":"2023-12-02T00:28:50.706568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:18:31.615464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706430+0000","last_change":"2023-12-02T00:28:48.732243+0000","last_active":"2023-12-02T00:28:50.706430+0000","last_peered":"2023-12-02T00:28:50.706430+0000","last_clean":"2023-12-02T00:28:50.706430+0000","last_became_active":"2023-12-02T00:28:48.731895+0000","last_became_peered":"2023-12-02T00:28:48.731895+0000","last_unstale":"2023-12-02T00:28:50.706430+0000","last_undegraded":"2023-12-02T00:28:50.706430+0000","last_fullsized":"2023-12-02T00:28:50.706430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:55:54.133908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021998+0000","last_change":"2023-12-02T00:28:49.718319+0000","last_active":"2023-12-02T00:28:51.021998+0000","last_peered":"2023-12-02T00:28:51.021998+0000","last_clean":"2023-12-02T00:28:51.021998+0000","last_became_active":"2023-12-02T00:28:49.718045+0000","last_became_peered":"2023-12-02T00:28:49.718045+0000","last_unstale":"2023-12-02T00:28:51.021998+0000","last_undegraded":"2023-12-02T00:28:51.021998+0000","last_fullsized":"2023-12-02T00:28:51.021998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:58:11.596095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703791+0000","last_change":"2023-12-02T00:28:48.728008+0000","last_active":"2023-12-02T00:28:50.703791+0000","last_peered":"2023-12-02T00:28:50.703791+0000","last_clean":"2023-12-02T00:28:50.703791+0000","last_became_active":"2023-12-02T00:28:48.727662+0000","last_became_peered":"2023-12-02T00:28:48.727662+0000","last_unstale":"2023-12-02T00:28:50.703791+0000","last_undegraded":"2023-12-02T00:28:50.703791+0000","last_fullsized":"2023-12-02T00:28:50.703791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:13:35.505091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705692+0000","last_change":"2023-12-02T00:28:49.719391+0000","last_active":"2023-12-02T00:28:50.705692+0000","last_peered":"2023-12-02T00:28:50.705692+0000","last_clean":"2023-12-02T00:28:50.705692+0000","last_became_active":"2023-12-02T00:28:49.719126+0000","last_became_peered":"2023-12-02T00:28:49.719126+0000","last_unstale":"2023-12-02T00:28:50.705692+0000","last_undegraded":"2023-12-02T00:28:50.705692+0000","last_fullsized":"2023-12-02T00:28:50.705692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:47:09.825514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021521+0000","last_change":"2023-12-02T00:28:48.736811+0000","last_active":"2023-12-02T00:28:51.021521+0000","last_peered":"2023-12-02T00:28:51.021521+0000","last_clean":"2023-12-02T00:28:51.021521+0000","last_became_active":"2023-12-02T00:28:48.736576+0000","last_became_peered":"2023-12-02T00:28:48.736576+0000","last_unstale":"2023-12-02T00:28:51.021521+0000","last_undegraded":"2023-12-02T00:28:51.021521+0000","last_fullsized":"2023-12-02T00:28:51.021521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:52:07.682002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021600+0000","last_change":"2023-12-02T00:28:49.717541+0000","last_active":"2023-12-02T00:28:51.021600+0000","last_peered":"2023-12-02T00:28:51.021600+0000","last_clean":"2023-12-02T00:28:51.021600+0000","last_became_active":"2023-12-02T00:28:49.717225+0000","last_became_peered":"2023-12-02T00:28:49.717225+0000","last_unstale":"2023-12-02T00:28:51.021600+0000","last_undegraded":"2023-12-02T00:28:51.021600+0000","last_fullsized":"2023-12-02T00:28:51.021600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:29:16.078935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.022091+0000","last_change":"2023-12-02T00:28:48.725581+0000","last_active":"2023-12-02T00:28:51.022091+0000","last_peered":"2023-12-02T00:28:51.022091+0000","last_clean":"2023-12-02T00:28:51.022091+0000","last_became_active":"2023-12-02T00:28:48.725230+0000","last_became_peered":"2023-12-02T00:28:48.725230+0000","last_unstale":"2023-12-02T00:28:51.022091+0000","last_undegraded":"2023-12-02T00:28:51.022091+0000","last_fullsized":"2023-12-02T00:28:51.022091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:39:30.211229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021761+0000","last_change":"2023-12-02T00:28:49.716591+0000","last_active":"2023-12-02T00:28:51.021761+0000","last_peered":"2023-12-02T00:28:51.021761+0000","last_clean":"2023-12-02T00:28:51.021761+0000","last_became_active":"2023-12-02T00:28:49.716237+0000","last_became_peered":"2023-12-02T00:28:49.716237+0000","last_unstale":"2023-12-02T00:28:51.021761+0000","last_undegraded":"2023-12-02T00:28:51.021761+0000","last_fullsized":"2023-12-02T00:28:51.021761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T12:06:34.965686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706099+0000","last_change":"2023-12-02T00:28:48.734778+0000","last_active":"2023-12-02T00:28:50.706099+0000","last_peered":"2023-12-02T00:28:50.706099+0000","last_clean":"2023-12-02T00:28:50.706099+0000","last_became_active":"2023-12-02T00:28:48.734474+0000","last_became_peered":"2023-12-02T00:28:48.734474+0000","last_unstale":"2023-12-02T00:28:50.706099+0000","last_undegraded":"2023-12-02T00:28:50.706099+0000","last_fullsized":"2023-12-02T00:28:50.706099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:07:51.795216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021342+0000","last_change":"2023-12-02T00:28:49.721274+0000","last_active":"2023-12-02T00:28:51.021342+0000","last_peered":"2023-12-02T00:28:51.021342+0000","last_clean":"2023-12-02T00:28:51.021342+0000","last_became_active":"2023-12-02T00:28:49.719979+0000","last_became_peered":"2023-12-02T00:28:49.719979+0000","last_unstale":"2023-12-02T00:28:51.021342+0000","last_undegraded":"2023-12-02T00:28:51.021342+0000","last_fullsized":"2023-12-02T00:28:51.021342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:50:45.793304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020850+0000","last_change":"2023-12-02T00:28:48.732958+0000","last_active":"2023-12-02T00:28:51.020850+0000","last_peered":"2023-12-02T00:28:51.020850+0000","last_clean":"2023-12-02T00:28:51.020850+0000","last_became_active":"2023-12-02T00:28:48.732651+0000","last_became_peered":"2023-12-02T00:28:48.732651+0000","last_unstale":"2023-12-02T00:28:51.020850+0000","last_undegraded":"2023-12-02T00:28:51.020850+0000","last_fullsized":"2023-12-02T00:28:51.020850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:29:26.381170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020869+0000","last_change":"2023-12-02T00:28:49.712859+0000","last_active":"2023-12-02T00:28:51.020869+0000","last_peered":"2023-12-02T00:28:51.020869+0000","last_clean":"2023-12-02T00:28:51.020869+0000","last_became_active":"2023-12-02T00:28:49.712585+0000","last_became_peered":"2023-12-02T00:28:49.712585+0000","last_unstale":"2023-12-02T00:28:51.020869+0000","last_undegraded":"2023-12-02T00:28:51.020869+0000","last_fullsized":"2023-12-02T00:28:51.020869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T12:06:34.965686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020660+0000","last_change":"2023-12-02T00:28:48.734113+0000","last_active":"2023-12-02T00:28:51.020660+0000","last_peered":"2023-12-02T00:28:51.020660+0000","last_clean":"2023-12-02T00:28:51.020660+0000","last_became_active":"2023-12-02T00:28:48.733749+0000","last_became_peered":"2023-12-02T00:28:48.733749+0000","last_unstale":"2023-12-02T00:28:51.020660+0000","last_undegraded":"2023-12-02T00:28:51.020660+0000","last_fullsized":"2023-12-02T00:28:51.020660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T10:56:02.550120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021416+0000","last_change":"2023-12-02T00:28:49.719477+0000","last_active":"2023-12-02T00:28:51.021416+0000","last_peered":"2023-12-02T00:28:51.021416+0000","last_clean":"2023-12-02T00:28:51.021416+0000","last_became_active":"2023-12-02T00:28:49.718537+0000","last_became_peered":"2023-12-02T00:28:49.718537+0000","last_unstale":"2023-12-02T00:28:51.021416+0000","last_undegraded":"2023-12-02T00:28:51.021416+0000","last_fullsized":"2023-12-02T00:28:51.021416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:15:02.988933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705801+0000","last_change":"2023-12-02T00:28:48.730372+0000","last_active":"2023-12-02T00:28:50.705801+0000","last_peered":"2023-12-02T00:28:50.705801+0000","last_clean":"2023-12-02T00:28:50.705801+0000","last_became_active":"2023-12-02T00:28:48.730051+0000","last_became_peered":"2023-12-02T00:28:48.730051+0000","last_unstale":"2023-12-02T00:28:50.705801+0000","last_undegraded":"2023-12-02T00:28:50.705801+0000","last_fullsized":"2023-12-02T00:28:50.705801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:29:10.550270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021515+0000","last_change":"2023-12-02T00:28:49.714467+0000","last_active":"2023-12-02T00:28:51.021515+0000","last_peered":"2023-12-02T00:28:51.021515+0000","last_clean":"2023-12-02T00:28:51.021515+0000","last_became_active":"2023-12-02T00:28:49.713964+0000","last_became_peered":"2023-12-02T00:28:49.713964+0000","last_unstale":"2023-12-02T00:28:51.021515+0000","last_undegraded":"2023-12-02T00:28:51.021515+0000","last_fullsized":"2023-12-02T00:28:51.021515+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T12:22:41.666941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021286+0000","last_change":"2023-12-02T00:28:48.736447+0000","last_active":"2023-12-02T00:28:51.021286+0000","last_peered":"2023-12-02T00:28:51.021286+0000","last_clean":"2023-12-02T00:28:51.021286+0000","last_became_active":"2023-12-02T00:28:48.736169+0000","last_became_peered":"2023-12-02T00:28:48.736169+0000","last_unstale":"2023-12-02T00:28:51.021286+0000","last_undegraded":"2023-12-02T00:28:51.021286+0000","last_fullsized":"2023-12-02T00:28:51.021286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T09:17:51.234416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021954+0000","last_change":"2023-12-02T00:28:49.719154+0000","last_active":"2023-12-02T00:28:51.021954+0000","last_peered":"2023-12-02T00:28:51.021954+0000","last_clean":"2023-12-02T00:28:51.021954+0000","last_became_active":"2023-12-02T00:28:49.718713+0000","last_became_peered":"2023-12-02T00:28:49.718713+0000","last_unstale":"2023-12-02T00:28:51.021954+0000","last_undegraded":"2023-12-02T00:28:51.021954+0000","last_fullsized":"2023-12-02T00:28:51.021954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:50:17.285633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703845+0000","last_change":"2023-12-02T00:28:48.731504+0000","last_active":"2023-12-02T00:28:50.703845+0000","last_peered":"2023-12-02T00:28:50.703845+0000","last_clean":"2023-12-02T00:28:50.703845+0000","last_became_active":"2023-12-02T00:28:48.731224+0000","last_became_peered":"2023-12-02T00:28:48.731224+0000","last_unstale":"2023-12-02T00:28:50.703845+0000","last_undegraded":"2023-12-02T00:28:50.703845+0000","last_fullsized":"2023-12-02T00:28:50.703845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:55:54.133908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706503+0000","last_change":"2023-12-02T00:28:49.726089+0000","last_active":"2023-12-02T00:28:50.706503+0000","last_peered":"2023-12-02T00:28:50.706503+0000","last_clean":"2023-12-02T00:28:50.706503+0000","last_became_active":"2023-12-02T00:28:49.725482+0000","last_became_peered":"2023-12-02T00:28:49.725482+0000","last_unstale":"2023-12-02T00:28:50.706503+0000","last_undegraded":"2023-12-02T00:28:50.706503+0000","last_fullsized":"2023-12-02T00:28:50.706503+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T12:22:41.666941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020880+0000","last_change":"2023-12-02T00:28:48.729936+0000","last_active":"2023-12-02T00:28:51.020880+0000","last_peered":"2023-12-02T00:28:51.020880+0000","last_clean":"2023-12-02T00:28:51.020880+0000","last_became_active":"2023-12-02T00:28:48.729617+0000","last_became_peered":"2023-12-02T00:28:48.729617+0000","last_unstale":"2023-12-02T00:28:51.020880+0000","last_undegraded":"2023-12-02T00:28:51.020880+0000","last_fullsized":"2023-12-02T00:28:51.020880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:29:26.381170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021803+0000","last_change":"2023-12-02T00:28:49.725211+0000","last_active":"2023-12-02T00:28:51.021803+0000","last_peered":"2023-12-02T00:28:51.021803+0000","last_clean":"2023-12-02T00:28:51.021803+0000","last_became_active":"2023-12-02T00:28:49.724928+0000","last_became_peered":"2023-12-02T00:28:49.724928+0000","last_unstale":"2023-12-02T00:28:51.021803+0000","last_undegraded":"2023-12-02T00:28:51.021803+0000","last_fullsized":"2023-12-02T00:28:51.021803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:33:43.085260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":165,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-02T00:28:49.708269+0000","last_change":"2023-12-02T00:28:39.379380+0000","last_active":"2023-12-02T00:28:49.708269+0000","last_peered":"2023-12-02T00:28:49.708269+0000","last_clean":"2023-12-02T00:28:49.708269+0000","last_became_active":"2023-12-02T00:28:39.378255+0000","last_became_peered":"2023-12-02T00:28:39.378255+0000","last_unstale":"2023-12-02T00:28:49.708269+0000","last_undegraded":"2023-12-02T00:28:49.708269+0000","last_fullsized":"2023-12-02T00:28:49.708269+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:38.177345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:38.177345+0000","last_clean_scrub_stamp":"2023-12-02T00:28:38.177345+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:21:08.582412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706107+0000","last_change":"2023-12-02T00:28:48.731326+0000","last_active":"2023-12-02T00:28:50.706107+0000","last_peered":"2023-12-02T00:28:50.706107+0000","last_clean":"2023-12-02T00:28:50.706107+0000","last_became_active":"2023-12-02T00:28:48.730970+0000","last_became_peered":"2023-12-02T00:28:48.730970+0000","last_unstale":"2023-12-02T00:28:50.706107+0000","last_undegraded":"2023-12-02T00:28:50.706107+0000","last_fullsized":"2023-12-02T00:28:50.706107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:44:45.468077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021403+0000","last_change":"2023-12-02T00:28:49.727341+0000","last_active":"2023-12-02T00:28:51.021403+0000","last_peered":"2023-12-02T00:28:51.021403+0000","last_clean":"2023-12-02T00:28:51.021403+0000","last_became_active":"2023-12-02T00:28:49.727045+0000","last_became_peered":"2023-12-02T00:28:49.727045+0000","last_unstale":"2023-12-02T00:28:51.021403+0000","last_undegraded":"2023-12-02T00:28:51.021403+0000","last_fullsized":"2023-12-02T00:28:51.021403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:58:15.388237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020921+0000","last_change":"2023-12-02T00:28:48.732719+0000","last_active":"2023-12-02T00:28:51.020921+0000","last_peered":"2023-12-02T00:28:51.020921+0000","last_clean":"2023-12-02T00:28:51.020921+0000","last_became_active":"2023-12-02T00:28:48.732466+0000","last_became_peered":"2023-12-02T00:28:48.732466+0000","last_unstale":"2023-12-02T00:28:51.020921+0000","last_undegraded":"2023-12-02T00:28:51.020921+0000","last_fullsized":"2023-12-02T00:28:51.020921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:52:21.593280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021611+0000","last_change":"2023-12-02T00:28:49.716043+0000","last_active":"2023-12-02T00:28:51.021611+0000","last_peered":"2023-12-02T00:28:51.021611+0000","last_clean":"2023-12-02T00:28:51.021611+0000","last_became_active":"2023-12-02T00:28:49.715747+0000","last_became_peered":"2023-12-02T00:28:49.715747+0000","last_unstale":"2023-12-02T00:28:51.021611+0000","last_undegraded":"2023-12-02T00:28:51.021611+0000","last_fullsized":"2023-12-02T00:28:51.021611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:44:46.475452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021208+0000","last_change":"2023-12-02T00:28:48.736024+0000","last_active":"2023-12-02T00:28:51.021208+0000","last_peered":"2023-12-02T00:28:51.021208+0000","last_clean":"2023-12-02T00:28:51.021208+0000","last_became_active":"2023-12-02T00:28:48.735736+0000","last_became_peered":"2023-12-02T00:28:48.735736+0000","last_unstale":"2023-12-02T00:28:51.021208+0000","last_undegraded":"2023-12-02T00:28:51.021208+0000","last_fullsized":"2023-12-02T00:28:51.021208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:55:48.025090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703534+0000","last_change":"2023-12-02T00:28:49.707755+0000","last_active":"2023-12-02T00:28:50.703534+0000","last_peered":"2023-12-02T00:28:50.703534+0000","last_clean":"2023-12-02T00:28:50.703534+0000","last_became_active":"2023-12-02T00:28:49.707449+0000","last_became_peered":"2023-12-02T00:28:49.707449+0000","last_unstale":"2023-12-02T00:28:50.703534+0000","last_undegraded":"2023-12-02T00:28:50.703534+0000","last_fullsized":"2023-12-02T00:28:50.703534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:07:52.802590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706217+0000","last_change":"2023-12-02T00:28:48.729078+0000","last_active":"2023-12-02T00:28:50.706217+0000","last_peered":"2023-12-02T00:28:50.706217+0000","last_clean":"2023-12-02T00:28:50.706217+0000","last_became_active":"2023-12-02T00:28:48.728567+0000","last_became_peered":"2023-12-02T00:28:48.728567+0000","last_unstale":"2023-12-02T00:28:50.706217+0000","last_undegraded":"2023-12-02T00:28:50.706217+0000","last_fullsized":"2023-12-02T00:28:50.706217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:58:10.588720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705483+0000","last_change":"2023-12-02T00:28:49.712801+0000","last_active":"2023-12-02T00:28:50.705483+0000","last_peered":"2023-12-02T00:28:50.705483+0000","last_clean":"2023-12-02T00:28:50.705483+0000","last_became_active":"2023-12-02T00:28:49.712511+0000","last_became_peered":"2023-12-02T00:28:49.712511+0000","last_unstale":"2023-12-02T00:28:50.705483+0000","last_undegraded":"2023-12-02T00:28:50.705483+0000","last_fullsized":"2023-12-02T00:28:50.705483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:55:49.032465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703749+0000","last_change":"2023-12-02T00:28:48.730774+0000","last_active":"2023-12-02T00:28:50.703749+0000","last_peered":"2023-12-02T00:28:50.703749+0000","last_clean":"2023-12-02T00:28:50.703749+0000","last_became_active":"2023-12-02T00:28:48.730498+0000","last_became_peered":"2023-12-02T00:28:48.730498+0000","last_unstale":"2023-12-02T00:28:50.703749+0000","last_undegraded":"2023-12-02T00:28:50.703749+0000","last_fullsized":"2023-12-02T00:28:50.703749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:29:10.550270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706537+0000","last_change":"2023-12-02T00:28:49.726928+0000","last_active":"2023-12-02T00:28:50.706537+0000","last_peered":"2023-12-02T00:28:50.706537+0000","last_clean":"2023-12-02T00:28:50.706537+0000","last_became_active":"2023-12-02T00:28:49.725033+0000","last_became_peered":"2023-12-02T00:28:49.725033+0000","last_unstale":"2023-12-02T00:28:50.706537+0000","last_undegraded":"2023-12-02T00:28:50.706537+0000","last_fullsized":"2023-12-02T00:28:50.706537+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:33:43.085260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021906+0000","last_change":"2023-12-02T00:28:48.735559+0000","last_active":"2023-12-02T00:28:51.021906+0000","last_peered":"2023-12-02T00:28:51.021906+0000","last_clean":"2023-12-02T00:28:51.021906+0000","last_became_active":"2023-12-02T00:28:48.735243+0000","last_became_peered":"2023-12-02T00:28:48.735243+0000","last_unstale":"2023-12-02T00:28:51.021906+0000","last_undegraded":"2023-12-02T00:28:51.021906+0000","last_fullsized":"2023-12-02T00:28:51.021906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:07:51.795216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703595+0000","last_change":"2023-12-02T00:28:49.710546+0000","last_active":"2023-12-02T00:28:50.703595+0000","last_peered":"2023-12-02T00:28:50.703595+0000","last_clean":"2023-12-02T00:28:50.703595+0000","last_became_active":"2023-12-02T00:28:49.710010+0000","last_became_peered":"2023-12-02T00:28:49.710010+0000","last_unstale":"2023-12-02T00:28:50.703595+0000","last_undegraded":"2023-12-02T00:28:50.703595+0000","last_fullsized":"2023-12-02T00:28:50.703595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:58:11.596095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703835+0000","last_change":"2023-12-02T00:28:48.731556+0000","last_active":"2023-12-02T00:28:50.703835+0000","last_peered":"2023-12-02T00:28:50.703835+0000","last_clean":"2023-12-02T00:28:50.703835+0000","last_became_active":"2023-12-02T00:28:48.731291+0000","last_became_peered":"2023-12-02T00:28:48.731291+0000","last_unstale":"2023-12-02T00:28:50.703835+0000","last_undegraded":"2023-12-02T00:28:50.703835+0000","last_fullsized":"2023-12-02T00:28:50.703835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:39:30.211229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020929+0000","last_change":"2023-12-02T00:28:49.723678+0000","last_active":"2023-12-02T00:28:51.020929+0000","last_peered":"2023-12-02T00:28:51.020929+0000","last_clean":"2023-12-02T00:28:51.020929+0000","last_became_active":"2023-12-02T00:28:49.723299+0000","last_became_peered":"2023-12-02T00:28:49.723299+0000","last_unstale":"2023-12-02T00:28:51.020929+0000","last_undegraded":"2023-12-02T00:28:51.020929+0000","last_fullsized":"2023-12-02T00:28:51.020929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:58:11.596095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021044+0000","last_change":"2023-12-02T00:28:48.735741+0000","last_active":"2023-12-02T00:28:51.021044+0000","last_peered":"2023-12-02T00:28:51.021044+0000","last_clean":"2023-12-02T00:28:51.021044+0000","last_became_active":"2023-12-02T00:28:48.735400+0000","last_became_peered":"2023-12-02T00:28:48.735400+0000","last_unstale":"2023-12-02T00:28:51.021044+0000","last_undegraded":"2023-12-02T00:28:51.021044+0000","last_fullsized":"2023-12-02T00:28:51.021044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:13:35.505091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.022028+0000","last_change":"2023-12-02T00:28:49.711592+0000","last_active":"2023-12-02T00:28:51.022028+0000","last_peered":"2023-12-02T00:28:51.022028+0000","last_clean":"2023-12-02T00:28:51.022028+0000","last_became_active":"2023-12-02T00:28:49.711211+0000","last_became_peered":"2023-12-02T00:28:49.711211+0000","last_unstale":"2023-12-02T00:28:51.022028+0000","last_undegraded":"2023-12-02T00:28:51.022028+0000","last_fullsized":"2023-12-02T00:28:51.022028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:15:51.747629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020816+0000","last_change":"2023-12-02T00:28:48.735722+0000","last_active":"2023-12-02T00:28:51.020816+0000","last_peered":"2023-12-02T00:28:51.020816+0000","last_clean":"2023-12-02T00:28:51.020816+0000","last_became_active":"2023-12-02T00:28:48.735212+0000","last_became_peered":"2023-12-02T00:28:48.735212+0000","last_unstale":"2023-12-02T00:28:51.020816+0000","last_undegraded":"2023-12-02T00:28:51.020816+0000","last_fullsized":"2023-12-02T00:28:51.020816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:47:08.818139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705309+0000","last_change":"2023-12-02T00:28:49.711824+0000","last_active":"2023-12-02T00:28:50.705309+0000","last_peered":"2023-12-02T00:28:50.705309+0000","last_clean":"2023-12-02T00:28:50.705309+0000","last_became_active":"2023-12-02T00:28:49.710946+0000","last_became_peered":"2023-12-02T00:28:49.710946+0000","last_unstale":"2023-12-02T00:28:50.705309+0000","last_undegraded":"2023-12-02T00:28:50.705309+0000","last_fullsized":"2023-12-02T00:28:50.705309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T09:17:52.241790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703622+0000","last_change":"2023-12-02T00:28:48.727127+0000","last_active":"2023-12-02T00:28:50.703622+0000","last_peered":"2023-12-02T00:28:50.703622+0000","last_clean":"2023-12-02T00:28:50.703622+0000","last_became_active":"2023-12-02T00:28:48.726750+0000","last_became_peered":"2023-12-02T00:28:48.726750+0000","last_unstale":"2023-12-02T00:28:50.703622+0000","last_undegraded":"2023-12-02T00:28:50.703622+0000","last_fullsized":"2023-12-02T00:28:50.703622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:44:45.468077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021415+0000","last_change":"2023-12-02T00:28:49.718158+0000","last_active":"2023-12-02T00:28:51.021415+0000","last_peered":"2023-12-02T00:28:51.021415+0000","last_clean":"2023-12-02T00:28:51.021415+0000","last_became_active":"2023-12-02T00:28:49.717721+0000","last_became_peered":"2023-12-02T00:28:49.717721+0000","last_unstale":"2023-12-02T00:28:51.021415+0000","last_undegraded":"2023-12-02T00:28:51.021415+0000","last_fullsized":"2023-12-02T00:28:51.021415+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T10:27:29.244181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703765+0000","last_change":"2023-12-02T00:28:48.735894+0000","last_active":"2023-12-02T00:28:50.703765+0000","last_peered":"2023-12-02T00:28:50.703765+0000","last_clean":"2023-12-02T00:28:50.703765+0000","last_became_active":"2023-12-02T00:28:48.735613+0000","last_became_peered":"2023-12-02T00:28:48.735613+0000","last_unstale":"2023-12-02T00:28:50.703765+0000","last_undegraded":"2023-12-02T00:28:50.703765+0000","last_fullsized":"2023-12-02T00:28:50.703765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:52:07.682002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021464+0000","last_change":"2023-12-02T00:28:49.719314+0000","last_active":"2023-12-02T00:28:51.021464+0000","last_peered":"2023-12-02T00:28:51.021464+0000","last_clean":"2023-12-02T00:28:51.021464+0000","last_became_active":"2023-12-02T00:28:49.718940+0000","last_became_peered":"2023-12-02T00:28:49.718940+0000","last_unstale":"2023-12-02T00:28:51.021464+0000","last_undegraded":"2023-12-02T00:28:51.021464+0000","last_fullsized":"2023-12-02T00:28:51.021464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:50:45.793304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021457+0000","last_change":"2023-12-02T00:28:48.731948+0000","last_active":"2023-12-02T00:28:51.021457+0000","last_peered":"2023-12-02T00:28:51.021457+0000","last_clean":"2023-12-02T00:28:51.021457+0000","last_became_active":"2023-12-02T00:28:48.731641+0000","last_became_peered":"2023-12-02T00:28:48.731641+0000","last_unstale":"2023-12-02T00:28:51.021457+0000","last_undegraded":"2023-12-02T00:28:51.021457+0000","last_fullsized":"2023-12-02T00:28:51.021457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:36:21.580553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021037+0000","last_change":"2023-12-02T00:28:49.715513+0000","last_active":"2023-12-02T00:28:51.021037+0000","last_peered":"2023-12-02T00:28:51.021037+0000","last_clean":"2023-12-02T00:28:51.021037+0000","last_became_active":"2023-12-02T00:28:49.715240+0000","last_became_peered":"2023-12-02T00:28:49.715240+0000","last_unstale":"2023-12-02T00:28:51.021037+0000","last_undegraded":"2023-12-02T00:28:51.021037+0000","last_fullsized":"2023-12-02T00:28:51.021037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:52:22.600655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706521+0000","last_change":"2023-12-02T00:28:49.717785+0000","last_active":"2023-12-02T00:28:50.706521+0000","last_peered":"2023-12-02T00:28:50.706521+0000","last_clean":"2023-12-02T00:28:50.706521+0000","last_became_active":"2023-12-02T00:28:49.717408+0000","last_became_peered":"2023-12-02T00:28:49.717408+0000","last_unstale":"2023-12-02T00:28:50.706521+0000","last_undegraded":"2023-12-02T00:28:50.706521+0000","last_fullsized":"2023-12-02T00:28:50.706521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:29:16.078935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021429+0000","last_change":"2023-12-02T00:28:48.729768+0000","last_active":"2023-12-02T00:28:51.021429+0000","last_peered":"2023-12-02T00:28:51.021429+0000","last_clean":"2023-12-02T00:28:51.021429+0000","last_became_active":"2023-12-02T00:28:48.729466+0000","last_became_peered":"2023-12-02T00:28:48.729466+0000","last_unstale":"2023-12-02T00:28:51.021429+0000","last_undegraded":"2023-12-02T00:28:51.021429+0000","last_fullsized":"2023-12-02T00:28:51.021429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T12:06:33.958311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706378+0000","last_change":"2023-12-02T00:28:49.727325+0000","last_active":"2023-12-02T00:28:50.706378+0000","last_peered":"2023-12-02T00:28:50.706378+0000","last_clean":"2023-12-02T00:28:50.706378+0000","last_became_active":"2023-12-02T00:28:49.727062+0000","last_became_peered":"2023-12-02T00:28:49.727062+0000","last_unstale":"2023-12-02T00:28:50.706378+0000","last_undegraded":"2023-12-02T00:28:50.706378+0000","last_fullsized":"2023-12-02T00:28:50.706378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:15:02.988933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021734+0000","last_change":"2023-12-02T00:28:48.732989+0000","last_active":"2023-12-02T00:28:51.021734+0000","last_peered":"2023-12-02T00:28:51.021734+0000","last_clean":"2023-12-02T00:28:51.021734+0000","last_became_active":"2023-12-02T00:28:48.731645+0000","last_became_peered":"2023-12-02T00:28:48.731645+0000","last_unstale":"2023-12-02T00:28:51.021734+0000","last_undegraded":"2023-12-02T00:28:51.021734+0000","last_fullsized":"2023-12-02T00:28:51.021734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:44:45.468077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021923+0000","last_change":"2023-12-02T00:28:49.725372+0000","last_active":"2023-12-02T00:28:51.021923+0000","last_peered":"2023-12-02T00:28:51.021923+0000","last_clean":"2023-12-02T00:28:51.021923+0000","last_became_active":"2023-12-02T00:28:49.724926+0000","last_became_peered":"2023-12-02T00:28:49.724926+0000","last_unstale":"2023-12-02T00:28:51.021923+0000","last_undegraded":"2023-12-02T00:28:51.021923+0000","last_fullsized":"2023-12-02T00:28:51.021923+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:07:52.802590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021204+0000","last_change":"2023-12-02T00:28:48.731302+0000","last_active":"2023-12-02T00:28:51.021204+0000","last_peered":"2023-12-02T00:28:51.021204+0000","last_clean":"2023-12-02T00:28:51.021204+0000","last_became_active":"2023-12-02T00:28:48.731002+0000","last_became_peered":"2023-12-02T00:28:48.731002+0000","last_unstale":"2023-12-02T00:28:51.021204+0000","last_undegraded":"2023-12-02T00:28:51.021204+0000","last_fullsized":"2023-12-02T00:28:51.021204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:29:26.381170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021221+0000","last_change":"2023-12-02T00:28:49.721605+0000","last_active":"2023-12-02T00:28:51.021221+0000","last_peered":"2023-12-02T00:28:51.021221+0000","last_clean":"2023-12-02T00:28:51.021221+0000","last_became_active":"2023-12-02T00:28:49.721035+0000","last_became_peered":"2023-12-02T00:28:49.721035+0000","last_unstale":"2023-12-02T00:28:51.021221+0000","last_undegraded":"2023-12-02T00:28:51.021221+0000","last_fullsized":"2023-12-02T00:28:51.021221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:15:51.747629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.022018+0000","last_change":"2023-12-02T00:28:48.729331+0000","last_active":"2023-12-02T00:28:51.022018+0000","last_peered":"2023-12-02T00:28:51.022018+0000","last_clean":"2023-12-02T00:28:51.022018+0000","last_became_active":"2023-12-02T00:28:48.728469+0000","last_became_peered":"2023-12-02T00:28:48.728469+0000","last_unstale":"2023-12-02T00:28:51.022018+0000","last_undegraded":"2023-12-02T00:28:51.022018+0000","last_fullsized":"2023-12-02T00:28:51.022018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:47:08.818139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":660231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":70893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":69371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":69407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":77193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31376,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340464,"statfs":{"total":96636764160,"available":96604635136,"internally_reserved":0,"allocated":200704,"data_stored":68662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":212992,"data_stored":69441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":806912,"data_stored":659668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-02T00:28:53.440 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-12-02T00:28:53.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.578+0000 7fae18056640 1 Processor -- start 2023-12-02T00:28:53.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.578+0000 7fae18056640 1 -- start start 2023-12-02T00:28:53.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.579+0000 7fae18056640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fae1015c2f0 0x7fae1006eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:53.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.579+0000 7fae18056640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fae1006f040 0x7fae10140800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:53.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.579+0000 7fae18056640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae1015ba40 con 0x7fae10140d40 2023-12-02T00:28:53.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.579+0000 7fae18056640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae1015c6d0 con 0x7fae1015c2f0 2023-12-02T00:28:53.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.579+0000 7fae18056640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae1013e530 con 0x7fae1006f040 2023-12-02T00:28:53.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.579+0000 7fae15df2640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fae1015c2f0 0x7fae1006eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:53.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.579+0000 7fae15df2640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fae1015c2f0 0x7fae1006eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37768/0 (socket says 172.21.15.46:37768) 2023-12-02T00:28:53.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.580+0000 7fae15df2640 1 -- 172.21.15.46:0/2209778047 learned_addr learned my addr 172.21.15.46:0/2209778047 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:53.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.580+0000 7fae155f1640 1 --2- 172.21.15.46:0/2209778047 >> v2:172.21.15.162:3300/0 conn(0x7fae1006f040 0x7fae10140800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:53.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.580+0000 7fae15df2640 1 -- 172.21.15.46:0/2209778047 >> v2:172.21.15.162:3300/0 conn(0x7fae1006f040 msgr2=0x7fae10140800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:53.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.580+0000 7fae15df2640 1 --2- 172.21.15.46:0/2209778047 >> v2:172.21.15.162:3300/0 conn(0x7fae1006f040 0x7fae10140800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:53.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.580+0000 7fae15df2640 1 -- 172.21.15.46:0/2209778047 >> v1:172.21.15.162:6789/0 conn(0x7fae10140d40 legacy=0x7fae1015ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:53.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.580+0000 7fae15df2640 1 -- 172.21.15.46:0/2209778047 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae1006f4a0 con 0x7fae1015c2f0 2023-12-02T00:28:53.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.581+0000 7fae15df2640 1 --2- 172.21.15.46:0/2209778047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae1015c2f0 0x7fae1006eb00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fae100d20f0 tx=0x7fae0802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b182f47cd2f96668 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:53.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.581+0000 7fae14df0640 1 -- 172.21.15.46:0/2209778047 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae08030860 con 0x7fae1015c2f0 2023-12-02T00:28:53.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.581+0000 7fae14df0640 1 -- 172.21.15.46:0/2209778047 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae080309c0 con 0x7fae1015c2f0 2023-12-02T00:28:53.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.581+0000 7fae14df0640 1 -- 172.21.15.46:0/2209778047 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae08030d50 con 0x7fae1015c2f0 2023-12-02T00:28:53.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.581+0000 7fae18056640 1 -- 172.21.15.46:0/2209778047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae1015c2f0 msgr2=0x7fae1006eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:53.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.581+0000 7fae18056640 1 --2- 172.21.15.46:0/2209778047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae1015c2f0 0x7fae1006eb00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fae100d20f0 tx=0x7fae0802b9a0 comp rx=0 tx=0).stop 2023-12-02T00:28:53.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.581+0000 7fae18056640 1 -- 172.21.15.46:0/2209778047 shutdown_connections 2023-12-02T00:28:53.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.581+0000 7fae18056640 1 --2- 172.21.15.46:0/2209778047 >> v2:172.21.15.162:3300/0 conn(0x7fae1006f040 0x7fae10140800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:53.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.581+0000 7fae18056640 1 --2- 172.21.15.46:0/2209778047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae1015c2f0 0x7fae1006eb00 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:53.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.581+0000 7fae18056640 1 -- 172.21.15.46:0/2209778047 >> 172.21.15.46:0/2209778047 conn(0x7fae1006a6b0 msgr2=0x7fae1006aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:53.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.581+0000 7fae18056640 1 -- 172.21.15.46:0/2209778047 shutdown_connections 2023-12-02T00:28:53.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.581+0000 7fae18056640 1 -- 172.21.15.46:0/2209778047 wait complete. 2023-12-02T00:28:53.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.582+0000 7fae18056640 1 Processor -- start 2023-12-02T00:28:53.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.583+0000 7fae18056640 1 -- start start 2023-12-02T00:28:53.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.583+0000 7fae18056640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae1006f040 0x7fae1013d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:53.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.583+0000 7fae18056640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae10140d40 0x7fae1013ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:53.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.583+0000 7fae18056640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae1015c2f0 0x7fae1015d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:53.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.583+0000 7fae18056640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae10155480 con 0x7fae1006f040 2023-12-02T00:28:53.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.583+0000 7fae18056640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1013d3a0 con 0x7fae1015c2f0 2023-12-02T00:28:53.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.583+0000 7fae18056640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae100d0cb0 con 0x7fae10140d40 2023-12-02T00:28:53.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.583+0000 7fae15df2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae1006f040 0x7fae1013d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:53.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.583+0000 7fae15df2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae1006f040 0x7fae1013d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37774/0 (socket says 172.21.15.46:37774) 2023-12-02T00:28:53.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.583+0000 7fae15df2640 1 -- 172.21.15.46:0/1346242588 learned_addr learned my addr 172.21.15.46:0/1346242588 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:53.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.583+0000 7fae165f3640 1 --2- 172.21.15.46:0/1346242588 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae1015c2f0 0x7fae1015d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:53.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.583+0000 7fae155f1640 1 --2- 172.21.15.46:0/1346242588 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae10140d40 0x7fae1013ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:53.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.584+0000 7fae15df2640 1 -- 172.21.15.46:0/1346242588 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae10140d40 msgr2=0x7fae1013ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:53.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.584+0000 7fae15df2640 1 --2- 172.21.15.46:0/1346242588 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae10140d40 0x7fae1013ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:53.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.584+0000 7fae15df2640 1 -- 172.21.15.46:0/1346242588 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae1015c2f0 msgr2=0x7fae1015d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:53.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.584+0000 7fae15df2640 1 --2- 172.21.15.46:0/1346242588 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae1015c2f0 0x7fae1015d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:53.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.584+0000 7fae15df2640 1 -- 172.21.15.46:0/1346242588 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae080094d0 con 0x7fae1006f040 2023-12-02T00:28:53.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.584+0000 7fae15df2640 1 --2- 172.21.15.46:0/1346242588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae1006f040 0x7fae1013d5b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fae08000c00 tx=0x7fae08004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:53.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.584+0000 7fae0e7fc640 1 -- 172.21.15.46:0/1346242588 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae08039a90 con 0x7fae1006f040 2023-12-02T00:28:53.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.584+0000 7fae0e7fc640 1 -- 172.21.15.46:0/1346242588 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae08039bf0 con 0x7fae1006f040 2023-12-02T00:28:53.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.584+0000 7fae18056640 1 -- 172.21.15.46:0/1346242588 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae100d0f90 con 0x7fae1006f040 2023-12-02T00:28:53.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.584+0000 7fae0e7fc640 1 -- 172.21.15.46:0/1346242588 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae08043410 con 0x7fae1006f040 2023-12-02T00:28:53.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.584+0000 7fae18056640 1 -- 172.21.15.46:0/1346242588 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae101549f0 con 0x7fae1006f040 2023-12-02T00:28:53.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.585+0000 7fae18056640 1 -- 172.21.15.46:0/1346242588 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae100d18c0 con 0x7fae1006f040 2023-12-02T00:28:53.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.585+0000 7fae0e7fc640 1 -- 172.21.15.46:0/1346242588 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fae0804c3b0 con 0x7fae1006f040 2023-12-02T00:28:53.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.586+0000 7fae0e7fc640 1 --2- 172.21.15.46:0/1346242588 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fadfc06ae40 0x7fadfc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:53.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.586+0000 7fae0e7fc640 1 -- 172.21.15.46:0/1346242588 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fae080ab9f0 con 0x7fae1006f040 2023-12-02T00:28:53.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.586+0000 7fae155f1640 1 --2- 172.21.15.46:0/1346242588 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fadfc06ae40 0x7fadfc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:53.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.587+0000 7fae155f1640 1 --2- 172.21.15.46:0/1346242588 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fadfc06ae40 0x7fadfc06d300 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fae1013fd70 tx=0x7fae04009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:53.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.590+0000 7fae0e7fc640 1 -- 172.21.15.46:0/1346242588 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fae08076b50 con 0x7fae1006f040 2023-12-02T00:28:53.783 INFO:tasks.ceph.mds.d.smithi162.stderr:2023-12-02T00:28:53.782+0000 7f738bf8b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-02T00:28:53.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.835+0000 7fae18056640 1 -- 172.21.15.46:0/1346242588 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fae100d0df0 con 0x7fae1006f040 2023-12-02T00:28:53.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.835+0000 7fae0e7fc640 1 -- 172.21.15.46:0/1346242588 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1010 (secure 0 0 0) 0x7fae100d0df0 con 0x7fae1006f040 2023-12-02T00:28:53.837 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:28:53.837 INFO:teuthology.orchestra.run.smithi046.stdout:{"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":28,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":21,"num_osds":8,"num_up_osds":8,"osd_up_since":1701476915,"num_in_osds":8,"osd_in_since":1701476907,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":23,"data_bytes":592778,"bytes_used":241299456,"bytes_avail":772852813824,"bytes_total":773094113280,"write_bytes_sec":2631,"read_op_per_sec":0,"write_op_per_sec":8},"fsmap":{"epoch":6,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"d","status":"up:active","gid":4253}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2023-12-02T00:28:24.930758+0000","services":{}},"progress_events":{}} 2023-12-02T00:28:53.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.837+0000 7fae18056640 1 -- 172.21.15.46:0/1346242588 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fadfc06ae40 msgr2=0x7fadfc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:53.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.837+0000 7fae18056640 1 --2- 172.21.15.46:0/1346242588 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fadfc06ae40 0x7fadfc06d300 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fae1013fd70 tx=0x7fae04009190 comp rx=0 tx=0).stop 2023-12-02T00:28:53.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.837+0000 7fae18056640 1 -- 172.21.15.46:0/1346242588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae1006f040 msgr2=0x7fae1013d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:53.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.837+0000 7fae18056640 1 --2- 172.21.15.46:0/1346242588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae1006f040 0x7fae1013d5b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fae08000c00 tx=0x7fae08004730 comp rx=0 tx=0).stop 2023-12-02T00:28:53.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.837+0000 7fae18056640 1 -- 172.21.15.46:0/1346242588 shutdown_connections 2023-12-02T00:28:53.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.837+0000 7fae18056640 1 --2- 172.21.15.46:0/1346242588 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fadfc06ae40 0x7fadfc06d300 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:53.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.837+0000 7fae18056640 1 --2- 172.21.15.46:0/1346242588 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae1015c2f0 0x7fae1015d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:53.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.837+0000 7fae18056640 1 --2- 172.21.15.46:0/1346242588 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae10140d40 0x7fae1013ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:53.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.837+0000 7fae18056640 1 --2- 172.21.15.46:0/1346242588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae1006f040 0x7fae1013d5b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:53.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.837+0000 7fae18056640 1 -- 172.21.15.46:0/1346242588 >> 172.21.15.46:0/1346242588 conn(0x7fae1006a6b0 msgr2=0x7fae1006aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:53.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.837+0000 7fae18056640 1 -- 172.21.15.46:0/1346242588 shutdown_connections 2023-12-02T00:28:53.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.837+0000 7fae18056640 1 -- 172.21.15.46:0/1346242588 wait complete. 2023-12-02T00:28:53.849 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-12-02T00:28:53.849 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-02T00:28:53.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.954+0000 7f4a06ed3640 1 Processor -- start 2023-12-02T00:28:53.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.955+0000 7f4a06ed3640 1 -- start start 2023-12-02T00:28:53.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.955+0000 7f4a06ed3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a000d2910 0x7f4a000d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:53.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.955+0000 7f4a06ed3640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4a000d3730 0x7f4a000b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:53.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.955+0000 7f4a06ed3640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a000b2b10 con 0x7f4a0015d330 2023-12-02T00:28:53.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.955+0000 7f4a06ed3640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a000d3b50 con 0x7f4a000d3730 2023-12-02T00:28:53.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.955+0000 7f4a06ed3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a000d1590 con 0x7f4a000d2910 2023-12-02T00:28:53.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.955+0000 7f4a04c6f640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4a0015d330 0x7f4a000b8b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:44876/0 (socket says 172.21.15.46:44876) 2023-12-02T00:28:53.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.955+0000 7f4a04c6f640 1 -- 172.21.15.46:0/958505065 learned_addr learned my addr 172.21.15.46:0/958505065 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:28:53.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.956+0000 7f49fffff640 1 --2- 172.21.15.46:0/958505065 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a000d2910 0x7f4a000d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:53.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.956+0000 7f4a05470640 1 --2- 172.21.15.46:0/958505065 >> v2:172.21.15.162:3300/0 conn(0x7f4a000d3730 0x7f4a000b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:53.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.956+0000 7f49ff7fe640 1 -- 172.21.15.46:0/958505065 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3062092273 0 0) 0x7f4a000b2b10 con 0x7f4a0015d330 2023-12-02T00:28:53.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.956+0000 7f49ff7fe640 1 -- 172.21.15.46:0/958505065 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49ec003520 con 0x7f4a0015d330 2023-12-02T00:28:53.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.956+0000 7f49ff7fe640 1 -- 172.21.15.46:0/958505065 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 502928146 0 0) 0x7f49ec003520 con 0x7f4a0015d330 2023-12-02T00:28:53.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.956+0000 7f49ff7fe640 1 -- 172.21.15.46:0/958505065 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a000d2910 msgr2=0x7f4a000d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:53.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.956+0000 7f49ff7fe640 1 --2- 172.21.15.46:0/958505065 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a000d2910 0x7f4a000d31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:53.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.956+0000 7f49ff7fe640 1 -- 172.21.15.46:0/958505065 >> v2:172.21.15.162:3300/0 conn(0x7f4a000d3730 msgr2=0x7f4a000b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:53.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.956+0000 7f49ff7fe640 1 --2- 172.21.15.46:0/958505065 >> v2:172.21.15.162:3300/0 conn(0x7f4a000d3730 0x7f4a000b6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:53.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.956+0000 7f49ff7fe640 1 -- 172.21.15.46:0/958505065 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a000d1870 con 0x7f4a0015d330 2023-12-02T00:28:53.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.956+0000 7f49ff7fe640 1 -- 172.21.15.46:0/958505065 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7f49f80019f0 con 0x7f4a0015d330 2023-12-02T00:28:53.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.956+0000 7f49fffff640 1 --2- 172.21.15.46:0/958505065 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a000d2910 0x7f4a000d31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:28:53.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.956+0000 7f4a05470640 1 --2- 172.21.15.46:0/958505065 >> v2:172.21.15.162:3300/0 conn(0x7f4a000d3730 0x7f4a000b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:28:53.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.956+0000 7f49ff7fe640 1 -- 172.21.15.46:0/958505065 >> v1:172.21.15.46:6789/0 conn(0x7f4a0015d330 legacy=0x7f4a000b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:53.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.956+0000 7f49ff7fe640 1 --2- 172.21.15.46:0/958505065 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49ec003970 0x7f49ec003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:53.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.956+0000 7f49ff7fe640 1 --2- 172.21.15.46:0/958505065 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a000d2910 0x7f49ec0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:53.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.956+0000 7f49ff7fe640 1 --2- 172.21.15.46:0/958505065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49ec004d00 0x7f49ec005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:53.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.956+0000 7f49ff7fe640 1 -- 172.21.15.46:0/958505065 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a000d3b50 con 0x7f49ec004d00 2023-12-02T00:28:53.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.956+0000 7f49ff7fe640 1 -- 172.21.15.46:0/958505065 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a000d1590 con 0x7f49ec003970 2023-12-02T00:28:53.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.956+0000 7f49ff7fe640 1 -- 172.21.15.46:0/958505065 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec00b980 con 0x7f4a000d2910 2023-12-02T00:28:53.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.956+0000 7f4a05470640 1 --2- 172.21.15.46:0/958505065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49ec004d00 0x7f49ec005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:53.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.956+0000 7f49fffff640 1 --2- 172.21.15.46:0/958505065 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a000d2910 0x7f49ec0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:53.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.956+0000 7f4a04c6f640 1 --2- 172.21.15.46:0/958505065 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49ec003970 0x7f49ec003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:53.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.956+0000 7f4a05470640 1 -- 172.21.15.46:0/958505065 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a000d2910 msgr2=0x7f49ec0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:53.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.956+0000 7f4a05470640 1 --2- 172.21.15.46:0/958505065 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a000d2910 0x7f49ec0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:53.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.957+0000 7f4a05470640 1 -- 172.21.15.46:0/958505065 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49ec003970 msgr2=0x7f49ec003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:53.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.957+0000 7f4a05470640 1 --2- 172.21.15.46:0/958505065 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49ec003970 0x7f49ec003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:53.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.957+0000 7f4a05470640 1 -- 172.21.15.46:0/958505065 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f0009400 con 0x7f49ec004d00 2023-12-02T00:28:53.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.957+0000 7f4a05470640 1 --2- 172.21.15.46:0/958505065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49ec004d00 0x7f49ec005180 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f49e8009c90 tx=0x7f49ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=530ffba8b0675401 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:53.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.957+0000 7f49ff7fe640 1 -- 172.21.15.46:0/958505065 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e802c980 con 0x7f49ec004d00 2023-12-02T00:28:53.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.957+0000 7f49ff7fe640 1 -- 172.21.15.46:0/958505065 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49e802cae0 con 0x7f49ec004d00 2023-12-02T00:28:53.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.957+0000 7f49ff7fe640 1 -- 172.21.15.46:0/958505065 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e802ce90 con 0x7f49ec004d00 2023-12-02T00:28:53.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.958+0000 7f4a06ed3640 1 -- 172.21.15.46:0/958505065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49ec004d00 msgr2=0x7f49ec005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:53.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.958+0000 7f4a06ed3640 1 --2- 172.21.15.46:0/958505065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49ec004d00 0x7f49ec005180 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f49e8009c90 tx=0x7f49ec0057f0 comp rx=0 tx=0).stop 2023-12-02T00:28:53.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.958+0000 7f4a05470640 1 -- 172.21.15.46:0/958505065 reap_dead start 2023-12-02T00:28:53.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.958+0000 7f4a06ed3640 1 -- 172.21.15.46:0/958505065 shutdown_connections 2023-12-02T00:28:53.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.958+0000 7f4a06ed3640 1 -- 172.21.15.46:0/958505065 >> 172.21.15.46:0/958505065 conn(0x7f4a00072020 msgr2=0x7f4a00072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:53.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.958+0000 7f4a06ed3640 1 -- 172.21.15.46:0/958505065 shutdown_connections 2023-12-02T00:28:53.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.958+0000 7f4a06ed3640 1 -- 172.21.15.46:0/958505065 wait complete. 2023-12-02T00:28:53.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.959+0000 7f4a06ed3640 1 Processor -- start 2023-12-02T00:28:53.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.959+0000 7f4a06ed3640 1 -- start start 2023-12-02T00:28:53.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.960+0000 7f4a06ed3640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a001562c0 0x7f4a00160300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:53.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.960+0000 7f4a06ed3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a001566a0 0x7f4a001537a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:53.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.960+0000 7f4a06ed3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a001f6280 0x7f4a001f6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:53.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.960+0000 7f4a06ed3640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0015fe70 con 0x7f4a001566a0 2023-12-02T00:28:53.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.960+0000 7f4a06ed3640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a000cd670 con 0x7f4a001562c0 2023-12-02T00:28:53.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.960+0000 7f4a06ed3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a000cd390 con 0x7f4a001f6280 2023-12-02T00:28:53.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.960+0000 7f49fffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a001566a0 0x7f4a001537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:53.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.960+0000 7f49fffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a001566a0 0x7f4a001537a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37786/0 (socket says 172.21.15.46:37786) 2023-12-02T00:28:53.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.960+0000 7f49fffff640 1 -- 172.21.15.46:0/4051059887 learned_addr learned my addr 172.21.15.46:0/4051059887 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:53.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.960+0000 7f4a04c6f640 1 --2- 172.21.15.46:0/4051059887 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a001562c0 0x7f4a00160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:53.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.960+0000 7f4a05470640 1 --2- 172.21.15.46:0/4051059887 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a001f6280 0x7f4a001f6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:53.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.960+0000 7f49fffff640 1 -- 172.21.15.46:0/4051059887 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a001f6280 msgr2=0x7f4a001f6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:53.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.960+0000 7f49fffff640 1 --2- 172.21.15.46:0/4051059887 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a001f6280 0x7f4a001f6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:53.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.960+0000 7f49fffff640 1 -- 172.21.15.46:0/4051059887 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a001562c0 msgr2=0x7f4a00160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:53.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.960+0000 7f49fffff640 1 --2- 172.21.15.46:0/4051059887 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a001562c0 0x7f4a00160300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:53.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.960+0000 7f49fffff640 1 -- 172.21.15.46:0/4051059887 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a0015f820 con 0x7f4a001566a0 2023-12-02T00:28:53.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.961+0000 7f49fffff640 1 --2- 172.21.15.46:0/4051059887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a001566a0 0x7f4a001537a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f49f0011d00 tx=0x7f49f000f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:53.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.961+0000 7f49fd7fa640 1 -- 172.21.15.46:0/4051059887 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f000fd20 con 0x7f4a001566a0 2023-12-02T00:28:53.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.961+0000 7f49fd7fa640 1 -- 172.21.15.46:0/4051059887 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49f000fe80 con 0x7f4a001566a0 2023-12-02T00:28:53.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.961+0000 7f49fd7fa640 1 -- 172.21.15.46:0/4051059887 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f0004240 con 0x7f4a001566a0 2023-12-02T00:28:53.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.961+0000 7f4a06ed3640 1 -- 172.21.15.46:0/4051059887 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a0015fb00 con 0x7f4a001566a0 2023-12-02T00:28:53.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.961+0000 7f4a06ed3640 1 -- 172.21.15.46:0/4051059887 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a001f6c40 con 0x7f4a001566a0 2023-12-02T00:28:53.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.962+0000 7f4a06ed3640 1 -- 172.21.15.46:0/4051059887 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a00160020 con 0x7f4a001566a0 2023-12-02T00:28:53.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.962+0000 7f49fd7fa640 1 -- 172.21.15.46:0/4051059887 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f49f00043a0 con 0x7f4a001566a0 2023-12-02T00:28:53.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.963+0000 7f49fd7fa640 1 --2- 172.21.15.46:0/4051059887 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f49e006ae40 0x7f49e006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:53.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.963+0000 7f49fd7fa640 1 -- 172.21.15.46:0/4051059887 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f49f0089c80 con 0x7f4a001566a0 2023-12-02T00:28:53.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.963+0000 7f4a04c6f640 1 --2- 172.21.15.46:0/4051059887 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f49e006ae40 0x7f49e006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:53.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.963+0000 7f4a04c6f640 1 --2- 172.21.15.46:0/4051059887 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f49e006ae40 0x7f49e006d300 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f49f80013d0 tx=0x7f49f8006040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:53.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:53.967+0000 7f49fd7fa640 1 -- 172.21.15.46:0/4051059887 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4a00160020 con 0x7f4a001566a0 2023-12-02T00:28:54.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:54.139+0000 7f4a06ed3640 1 -- 172.21.15.46:0/4051059887 --> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4a00073c20 con 0x7f49e006ae40 2023-12-02T00:28:54.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:54.146+0000 7f49fd7fa640 1 -- 172.21.15.46:0/4051059887 <== mgr.4108 v2:172.21.15.162:6832/3248644722 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264715 (secure 0 0 0) 0x7f4a00073c20 con 0x7f49e006ae40 2023-12-02T00:28:54.150 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:28:54.151 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-12-02T00:28:54.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:54.149+0000 7f4a06ed3640 1 -- 172.21.15.46:0/4051059887 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f49e006ae40 msgr2=0x7f49e006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:54.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:54.149+0000 7f4a06ed3640 1 --2- 172.21.15.46:0/4051059887 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f49e006ae40 0x7f49e006d300 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f49f80013d0 tx=0x7f49f8006040 comp rx=0 tx=0).stop 2023-12-02T00:28:54.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:54.149+0000 7f4a06ed3640 1 -- 172.21.15.46:0/4051059887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a001566a0 msgr2=0x7f4a001537a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:54.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:54.149+0000 7f4a06ed3640 1 --2- 172.21.15.46:0/4051059887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a001566a0 0x7f4a001537a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f49f0011d00 tx=0x7f49f000f610 comp rx=0 tx=0).stop 2023-12-02T00:28:54.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:54.149+0000 7f4a06ed3640 1 -- 172.21.15.46:0/4051059887 shutdown_connections 2023-12-02T00:28:54.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:54.149+0000 7f4a06ed3640 1 --2- 172.21.15.46:0/4051059887 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f49e006ae40 0x7f49e006d300 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:54.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:54.149+0000 7f4a06ed3640 1 --2- 172.21.15.46:0/4051059887 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a001f6280 0x7f4a001f6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:54.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:54.149+0000 7f4a06ed3640 1 --2- 172.21.15.46:0/4051059887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a001566a0 0x7f4a001537a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:54.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:54.149+0000 7f4a06ed3640 1 --2- 172.21.15.46:0/4051059887 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a001562c0 0x7f4a00160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:54.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:54.149+0000 7f4a06ed3640 1 -- 172.21.15.46:0/4051059887 >> 172.21.15.46:0/4051059887 conn(0x7f4a00072020 msgr2=0x7f4a000c7970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:54.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:54.149+0000 7f4a06ed3640 1 -- 172.21.15.46:0/4051059887 shutdown_connections 2023-12-02T00:28:54.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:54.149+0000 7f4a06ed3640 1 -- 172.21.15.46:0/4051059887 wait complete. 2023-12-02T00:28:54.161 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-12-02T00:28:53.158780+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":172,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235644,"kb_used_data":2828,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739076,"statfs":{"total":773094113280,"available":772852813824,"internally_reserved":0,"allocated":2895872,"data_stored":1744866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.448626"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021839+0000","last_change":"2023-12-02T00:28:49.717914+0000","last_active":"2023-12-02T00:28:51.021839+0000","last_peered":"2023-12-02T00:28:51.021839+0000","last_clean":"2023-12-02T00:28:51.021839+0000","last_became_active":"2023-12-02T00:28:49.717594+0000","last_became_peered":"2023-12-02T00:28:49.717594+0000","last_unstale":"2023-12-02T00:28:51.021839+0000","last_undegraded":"2023-12-02T00:28:51.021839+0000","last_fullsized":"2023-12-02T00:28:51.021839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:48:16.850673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703744+0000","last_change":"2023-12-02T00:28:49.713605+0000","last_active":"2023-12-02T00:28:50.703744+0000","last_peered":"2023-12-02T00:28:50.703744+0000","last_clean":"2023-12-02T00:28:50.703744+0000","last_became_active":"2023-12-02T00:28:49.712497+0000","last_became_peered":"2023-12-02T00:28:49.712497+0000","last_unstale":"2023-12-02T00:28:50.703744+0000","last_undegraded":"2023-12-02T00:28:50.703744+0000","last_fullsized":"2023-12-02T00:28:50.703744+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:15:51.747629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021917+0000","last_change":"2023-12-02T00:28:49.713065+0000","last_active":"2023-12-02T00:28:51.021917+0000","last_peered":"2023-12-02T00:28:51.021917+0000","last_clean":"2023-12-02T00:28:51.021917+0000","last_became_active":"2023-12-02T00:28:49.712715+0000","last_became_peered":"2023-12-02T00:28:49.712715+0000","last_unstale":"2023-12-02T00:28:51.021917+0000","last_undegraded":"2023-12-02T00:28:51.021917+0000","last_fullsized":"2023-12-02T00:28:51.021917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:52:22.600655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706337+0000","last_change":"2023-12-02T00:28:49.727831+0000","last_active":"2023-12-02T00:28:50.706337+0000","last_peered":"2023-12-02T00:28:50.706337+0000","last_clean":"2023-12-02T00:28:50.706337+0000","last_became_active":"2023-12-02T00:28:49.725804+0000","last_became_peered":"2023-12-02T00:28:49.725804+0000","last_unstale":"2023-12-02T00:28:50.706337+0000","last_undegraded":"2023-12-02T00:28:50.706337+0000","last_fullsized":"2023-12-02T00:28:50.706337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:50:45.793304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021332+0000","last_change":"2023-12-02T00:28:49.721391+0000","last_active":"2023-12-02T00:28:51.021332+0000","last_peered":"2023-12-02T00:28:51.021332+0000","last_clean":"2023-12-02T00:28:51.021332+0000","last_became_active":"2023-12-02T00:28:49.720314+0000","last_became_peered":"2023-12-02T00:28:49.720314+0000","last_unstale":"2023-12-02T00:28:51.021332+0000","last_undegraded":"2023-12-02T00:28:51.021332+0000","last_fullsized":"2023-12-02T00:28:51.021332+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:32:13.515968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705990+0000","last_change":"2023-12-02T00:28:49.725516+0000","last_active":"2023-12-02T00:28:50.705990+0000","last_peered":"2023-12-02T00:28:50.705990+0000","last_clean":"2023-12-02T00:28:50.705990+0000","last_became_active":"2023-12-02T00:28:49.725149+0000","last_became_peered":"2023-12-02T00:28:49.725149+0000","last_unstale":"2023-12-02T00:28:50.705990+0000","last_undegraded":"2023-12-02T00:28:50.705990+0000","last_fullsized":"2023-12-02T00:28:50.705990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T10:27:29.244181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021273+0000","last_change":"2023-12-02T00:28:49.719153+0000","last_active":"2023-12-02T00:28:51.021273+0000","last_peered":"2023-12-02T00:28:51.021273+0000","last_clean":"2023-12-02T00:28:51.021273+0000","last_became_active":"2023-12-02T00:28:49.718567+0000","last_became_peered":"2023-12-02T00:28:49.718567+0000","last_unstale":"2023-12-02T00:28:51.021273+0000","last_undegraded":"2023-12-02T00:28:51.021273+0000","last_fullsized":"2023-12-02T00:28:51.021273+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T10:27:29.244181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020683+0000","last_change":"2023-12-02T00:28:49.726986+0000","last_active":"2023-12-02T00:28:51.020683+0000","last_peered":"2023-12-02T00:28:51.020683+0000","last_clean":"2023-12-02T00:28:51.020683+0000","last_became_active":"2023-12-02T00:28:49.726646+0000","last_became_peered":"2023-12-02T00:28:49.726646+0000","last_unstale":"2023-12-02T00:28:51.020683+0000","last_undegraded":"2023-12-02T00:28:51.020683+0000","last_fullsized":"2023-12-02T00:28:51.020683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:35:09.231704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021827+0000","last_change":"2023-12-02T00:28:49.725402+0000","last_active":"2023-12-02T00:28:51.021827+0000","last_peered":"2023-12-02T00:28:51.021827+0000","last_clean":"2023-12-02T00:28:51.021827+0000","last_became_active":"2023-12-02T00:28:49.724978+0000","last_became_peered":"2023-12-02T00:28:49.724978+0000","last_unstale":"2023-12-02T00:28:51.021827+0000","last_undegraded":"2023-12-02T00:28:51.021827+0000","last_fullsized":"2023-12-02T00:28:51.021827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:18:31.615464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021500+0000","last_change":"2023-12-02T00:28:49.718464+0000","last_active":"2023-12-02T00:28:51.021500+0000","last_peered":"2023-12-02T00:28:51.021500+0000","last_clean":"2023-12-02T00:28:51.021500+0000","last_became_active":"2023-12-02T00:28:49.718060+0000","last_became_peered":"2023-12-02T00:28:49.718060+0000","last_unstale":"2023-12-02T00:28:51.021500+0000","last_undegraded":"2023-12-02T00:28:51.021500+0000","last_fullsized":"2023-12-02T00:28:51.021500+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:36:22.587928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021177+0000","last_change":"2023-12-02T00:28:49.720030+0000","last_active":"2023-12-02T00:28:51.021177+0000","last_peered":"2023-12-02T00:28:51.021177+0000","last_clean":"2023-12-02T00:28:51.021177+0000","last_became_active":"2023-12-02T00:28:49.719715+0000","last_became_peered":"2023-12-02T00:28:49.719715+0000","last_unstale":"2023-12-02T00:28:51.021177+0000","last_undegraded":"2023-12-02T00:28:51.021177+0000","last_fullsized":"2023-12-02T00:28:51.021177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:07:52.802590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705569+0000","last_change":"2023-12-02T00:28:49.713687+0000","last_active":"2023-12-02T00:28:50.705569+0000","last_peered":"2023-12-02T00:28:50.705569+0000","last_clean":"2023-12-02T00:28:50.705569+0000","last_became_active":"2023-12-02T00:28:49.713359+0000","last_became_peered":"2023-12-02T00:28:49.713359+0000","last_unstale":"2023-12-02T00:28:50.705569+0000","last_undegraded":"2023-12-02T00:28:50.705569+0000","last_fullsized":"2023-12-02T00:28:50.705569+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:11:06.388890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021465+0000","last_change":"2023-12-02T00:28:49.720866+0000","last_active":"2023-12-02T00:28:51.021465+0000","last_peered":"2023-12-02T00:28:51.021465+0000","last_clean":"2023-12-02T00:28:51.021465+0000","last_became_active":"2023-12-02T00:28:49.720524+0000","last_became_peered":"2023-12-02T00:28:49.720524+0000","last_unstale":"2023-12-02T00:28:51.021465+0000","last_undegraded":"2023-12-02T00:28:51.021465+0000","last_fullsized":"2023-12-02T00:28:51.021465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:32:13.515968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021720+0000","last_change":"2023-12-02T00:28:49.711387+0000","last_active":"2023-12-02T00:28:51.021720+0000","last_peered":"2023-12-02T00:28:51.021720+0000","last_clean":"2023-12-02T00:28:51.021720+0000","last_became_active":"2023-12-02T00:28:49.711036+0000","last_became_peered":"2023-12-02T00:28:49.711036+0000","last_unstale":"2023-12-02T00:28:51.021720+0000","last_undegraded":"2023-12-02T00:28:51.021720+0000","last_fullsized":"2023-12-02T00:28:51.021720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:13:36.512466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703511+0000","last_change":"2023-12-02T00:28:49.706444+0000","last_active":"2023-12-02T00:28:50.703511+0000","last_peered":"2023-12-02T00:28:50.703511+0000","last_clean":"2023-12-02T00:28:50.703511+0000","last_became_active":"2023-12-02T00:28:49.706132+0000","last_became_peered":"2023-12-02T00:28:49.706132+0000","last_unstale":"2023-12-02T00:28:50.703511+0000","last_undegraded":"2023-12-02T00:28:50.703511+0000","last_fullsized":"2023-12-02T00:28:50.703511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:52:22.600655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705451+0000","last_change":"2023-12-02T00:28:49.725045+0000","last_active":"2023-12-02T00:28:50.705451+0000","last_peered":"2023-12-02T00:28:50.705451+0000","last_clean":"2023-12-02T00:28:50.705451+0000","last_became_active":"2023-12-02T00:28:49.722884+0000","last_became_peered":"2023-12-02T00:28:49.722884+0000","last_unstale":"2023-12-02T00:28:50.705451+0000","last_undegraded":"2023-12-02T00:28:50.705451+0000","last_fullsized":"2023-12-02T00:28:50.705451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:36:22.587928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706621+0000","last_change":"2023-12-02T00:28:49.727232+0000","last_active":"2023-12-02T00:28:50.706621+0000","last_peered":"2023-12-02T00:28:50.706621+0000","last_clean":"2023-12-02T00:28:50.706621+0000","last_became_active":"2023-12-02T00:28:49.726881+0000","last_became_peered":"2023-12-02T00:28:49.726881+0000","last_unstale":"2023-12-02T00:28:50.706621+0000","last_undegraded":"2023-12-02T00:28:50.706621+0000","last_fullsized":"2023-12-02T00:28:50.706621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:48:16.850673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021328+0000","last_change":"2023-12-02T00:28:49.711953+0000","last_active":"2023-12-02T00:28:51.021328+0000","last_peered":"2023-12-02T00:28:51.021328+0000","last_clean":"2023-12-02T00:28:51.021328+0000","last_became_active":"2023-12-02T00:28:49.711653+0000","last_became_peered":"2023-12-02T00:28:49.711653+0000","last_unstale":"2023-12-02T00:28:51.021328+0000","last_undegraded":"2023-12-02T00:28:51.021328+0000","last_fullsized":"2023-12-02T00:28:51.021328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:52:08.689376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021886+0000","last_change":"2023-12-02T00:28:49.717365+0000","last_active":"2023-12-02T00:28:51.021886+0000","last_peered":"2023-12-02T00:28:51.021886+0000","last_clean":"2023-12-02T00:28:51.021886+0000","last_became_active":"2023-12-02T00:28:49.717071+0000","last_became_peered":"2023-12-02T00:28:49.717071+0000","last_unstale":"2023-12-02T00:28:51.021886+0000","last_undegraded":"2023-12-02T00:28:51.021886+0000","last_fullsized":"2023-12-02T00:28:51.021886+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:10:26.775049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021416+0000","last_change":"2023-12-02T00:28:49.716297+0000","last_active":"2023-12-02T00:28:51.021416+0000","last_peered":"2023-12-02T00:28:51.021416+0000","last_clean":"2023-12-02T00:28:51.021416+0000","last_became_active":"2023-12-02T00:28:49.715963+0000","last_became_peered":"2023-12-02T00:28:49.715963+0000","last_unstale":"2023-12-02T00:28:51.021416+0000","last_undegraded":"2023-12-02T00:28:51.021416+0000","last_fullsized":"2023-12-02T00:28:51.021416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:58:11.596095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021739+0000","last_change":"2023-12-02T00:28:49.722259+0000","last_active":"2023-12-02T00:28:51.021739+0000","last_peered":"2023-12-02T00:28:51.021739+0000","last_clean":"2023-12-02T00:28:51.021739+0000","last_became_active":"2023-12-02T00:28:49.721989+0000","last_became_peered":"2023-12-02T00:28:49.721989+0000","last_unstale":"2023-12-02T00:28:51.021739+0000","last_undegraded":"2023-12-02T00:28:51.021739+0000","last_fullsized":"2023-12-02T00:28:51.021739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:55:55.141283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705987+0000","last_change":"2023-12-02T00:28:49.729526+0000","last_active":"2023-12-02T00:28:50.705987+0000","last_peered":"2023-12-02T00:28:50.705987+0000","last_clean":"2023-12-02T00:28:50.705987+0000","last_became_active":"2023-12-02T00:28:49.728999+0000","last_became_peered":"2023-12-02T00:28:49.728999+0000","last_unstale":"2023-12-02T00:28:50.705987+0000","last_undegraded":"2023-12-02T00:28:50.705987+0000","last_fullsized":"2023-12-02T00:28:50.705987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:58:15.388237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021070+0000","last_change":"2023-12-02T00:28:49.717936+0000","last_active":"2023-12-02T00:28:51.021070+0000","last_peered":"2023-12-02T00:28:51.021070+0000","last_clean":"2023-12-02T00:28:51.021070+0000","last_became_active":"2023-12-02T00:28:49.716770+0000","last_became_peered":"2023-12-02T00:28:49.716770+0000","last_unstale":"2023-12-02T00:28:51.021070+0000","last_undegraded":"2023-12-02T00:28:51.021070+0000","last_fullsized":"2023-12-02T00:28:51.021070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:36:22.587928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705591+0000","last_change":"2023-12-02T00:28:49.724863+0000","last_active":"2023-12-02T00:28:50.705591+0000","last_peered":"2023-12-02T00:28:50.705591+0000","last_clean":"2023-12-02T00:28:50.705591+0000","last_became_active":"2023-12-02T00:28:49.724592+0000","last_became_peered":"2023-12-02T00:28:49.724592+0000","last_unstale":"2023-12-02T00:28:50.705591+0000","last_undegraded":"2023-12-02T00:28:50.705591+0000","last_fullsized":"2023-12-02T00:28:50.705591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:15:51.747629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020786+0000","last_change":"2023-12-02T00:28:49.724283+0000","last_active":"2023-12-02T00:28:51.020786+0000","last_peered":"2023-12-02T00:28:51.020786+0000","last_clean":"2023-12-02T00:28:51.020786+0000","last_became_active":"2023-12-02T00:28:49.723940+0000","last_became_peered":"2023-12-02T00:28:49.723940+0000","last_unstale":"2023-12-02T00:28:51.020786+0000","last_undegraded":"2023-12-02T00:28:51.020786+0000","last_fullsized":"2023-12-02T00:28:51.020786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:52:22.600655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021632+0000","last_change":"2023-12-02T00:28:49.717015+0000","last_active":"2023-12-02T00:28:51.021632+0000","last_peered":"2023-12-02T00:28:51.021632+0000","last_clean":"2023-12-02T00:28:51.021632+0000","last_became_active":"2023-12-02T00:28:49.716394+0000","last_became_peered":"2023-12-02T00:28:49.716394+0000","last_unstale":"2023-12-02T00:28:51.021632+0000","last_undegraded":"2023-12-02T00:28:51.021632+0000","last_fullsized":"2023-12-02T00:28:51.021632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:15:51.747629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703687+0000","last_change":"2023-12-02T00:28:49.716707+0000","last_active":"2023-12-02T00:28:50.703687+0000","last_peered":"2023-12-02T00:28:50.703687+0000","last_clean":"2023-12-02T00:28:50.703687+0000","last_became_active":"2023-12-02T00:28:49.716392+0000","last_became_peered":"2023-12-02T00:28:49.716392+0000","last_unstale":"2023-12-02T00:28:50.703687+0000","last_undegraded":"2023-12-02T00:28:50.703687+0000","last_fullsized":"2023-12-02T00:28:50.703687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:36:22.587928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021477+0000","last_change":"2023-12-02T00:28:49.724290+0000","last_active":"2023-12-02T00:28:51.021477+0000","last_peered":"2023-12-02T00:28:51.021477+0000","last_clean":"2023-12-02T00:28:51.021477+0000","last_became_active":"2023-12-02T00:28:49.723952+0000","last_became_peered":"2023-12-02T00:28:49.723952+0000","last_unstale":"2023-12-02T00:28:51.021477+0000","last_undegraded":"2023-12-02T00:28:51.021477+0000","last_fullsized":"2023-12-02T00:28:51.021477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:15:02.988933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021881+0000","last_change":"2023-12-02T00:28:49.721218+0000","last_active":"2023-12-02T00:28:51.021881+0000","last_peered":"2023-12-02T00:28:51.021881+0000","last_clean":"2023-12-02T00:28:51.021881+0000","last_became_active":"2023-12-02T00:28:49.720894+0000","last_became_peered":"2023-12-02T00:28:49.720894+0000","last_unstale":"2023-12-02T00:28:51.021881+0000","last_undegraded":"2023-12-02T00:28:51.021881+0000","last_fullsized":"2023-12-02T00:28:51.021881+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:35:09.231704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706145+0000","last_change":"2023-12-02T00:28:49.725424+0000","last_active":"2023-12-02T00:28:50.706145+0000","last_peered":"2023-12-02T00:28:50.706145+0000","last_clean":"2023-12-02T00:28:50.706145+0000","last_became_active":"2023-12-02T00:28:49.724671+0000","last_became_peered":"2023-12-02T00:28:49.724671+0000","last_unstale":"2023-12-02T00:28:50.706145+0000","last_undegraded":"2023-12-02T00:28:50.706145+0000","last_fullsized":"2023-12-02T00:28:50.706145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:32:13.515968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021246+0000","last_change":"2023-12-02T00:28:49.722395+0000","last_active":"2023-12-02T00:28:51.021246+0000","last_peered":"2023-12-02T00:28:51.021246+0000","last_clean":"2023-12-02T00:28:51.021246+0000","last_became_active":"2023-12-02T00:28:49.722052+0000","last_became_peered":"2023-12-02T00:28:49.722052+0000","last_unstale":"2023-12-02T00:28:51.021246+0000","last_undegraded":"2023-12-02T00:28:51.021246+0000","last_fullsized":"2023-12-02T00:28:51.021246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:11:06.388890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020869+0000","last_change":"2023-12-02T00:28:49.713618+0000","last_active":"2023-12-02T00:28:51.020869+0000","last_peered":"2023-12-02T00:28:51.020869+0000","last_clean":"2023-12-02T00:28:51.020869+0000","last_became_active":"2023-12-02T00:28:49.713325+0000","last_became_peered":"2023-12-02T00:28:49.713325+0000","last_unstale":"2023-12-02T00:28:51.020869+0000","last_undegraded":"2023-12-02T00:28:51.020869+0000","last_fullsized":"2023-12-02T00:28:51.020869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:35:09.231704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705601+0000","last_change":"2023-12-02T00:28:49.710604+0000","last_active":"2023-12-02T00:28:50.705601+0000","last_peered":"2023-12-02T00:28:50.705601+0000","last_clean":"2023-12-02T00:28:50.705601+0000","last_became_active":"2023-12-02T00:28:49.710316+0000","last_became_peered":"2023-12-02T00:28:49.710316+0000","last_unstale":"2023-12-02T00:28:50.705601+0000","last_undegraded":"2023-12-02T00:28:50.705601+0000","last_fullsized":"2023-12-02T00:28:50.705601+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:52:08.689376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021549+0000","last_change":"2023-12-02T00:28:48.735813+0000","last_active":"2023-12-02T00:28:51.021549+0000","last_peered":"2023-12-02T00:28:51.021549+0000","last_clean":"2023-12-02T00:28:51.021549+0000","last_became_active":"2023-12-02T00:28:48.735462+0000","last_became_peered":"2023-12-02T00:28:48.735462+0000","last_unstale":"2023-12-02T00:28:51.021549+0000","last_undegraded":"2023-12-02T00:28:51.021549+0000","last_fullsized":"2023-12-02T00:28:51.021549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:55:54.133908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.022018+0000","last_change":"2023-12-02T00:28:49.715318+0000","last_active":"2023-12-02T00:28:51.022018+0000","last_peered":"2023-12-02T00:28:51.022018+0000","last_clean":"2023-12-02T00:28:51.022018+0000","last_became_active":"2023-12-02T00:28:49.715020+0000","last_became_peered":"2023-12-02T00:28:49.715020+0000","last_unstale":"2023-12-02T00:28:51.022018+0000","last_undegraded":"2023-12-02T00:28:51.022018+0000","last_fullsized":"2023-12-02T00:28:51.022018+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:36:22.587928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021150+0000","last_change":"2023-12-02T00:28:48.730625+0000","last_active":"2023-12-02T00:28:51.021150+0000","last_peered":"2023-12-02T00:28:51.021150+0000","last_clean":"2023-12-02T00:28:51.021150+0000","last_became_active":"2023-12-02T00:28:48.730306+0000","last_became_peered":"2023-12-02T00:28:48.730306+0000","last_unstale":"2023-12-02T00:28:51.021150+0000","last_undegraded":"2023-12-02T00:28:51.021150+0000","last_fullsized":"2023-12-02T00:28:51.021150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:11:05.381516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020857+0000","last_change":"2023-12-02T00:28:48.735160+0000","last_active":"2023-12-02T00:28:51.020857+0000","last_peered":"2023-12-02T00:28:51.020857+0000","last_clean":"2023-12-02T00:28:51.020857+0000","last_became_active":"2023-12-02T00:28:48.734860+0000","last_became_peered":"2023-12-02T00:28:48.734860+0000","last_unstale":"2023-12-02T00:28:51.020857+0000","last_undegraded":"2023-12-02T00:28:51.020857+0000","last_fullsized":"2023-12-02T00:28:51.020857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T09:17:51.234416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020811+0000","last_change":"2023-12-02T00:28:49.716640+0000","last_active":"2023-12-02T00:28:51.020811+0000","last_peered":"2023-12-02T00:28:51.020811+0000","last_clean":"2023-12-02T00:28:51.020811+0000","last_became_active":"2023-12-02T00:28:49.716325+0000","last_became_peered":"2023-12-02T00:28:49.716325+0000","last_unstale":"2023-12-02T00:28:51.020811+0000","last_undegraded":"2023-12-02T00:28:51.020811+0000","last_fullsized":"2023-12-02T00:28:51.020811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:07:52.802590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705691+0000","last_change":"2023-12-02T00:28:48.730403+0000","last_active":"2023-12-02T00:28:50.705691+0000","last_peered":"2023-12-02T00:28:50.705691+0000","last_clean":"2023-12-02T00:28:50.705691+0000","last_became_active":"2023-12-02T00:28:48.730053+0000","last_became_peered":"2023-12-02T00:28:48.730053+0000","last_unstale":"2023-12-02T00:28:50.705691+0000","last_undegraded":"2023-12-02T00:28:50.705691+0000","last_fullsized":"2023-12-02T00:28:50.705691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:13:35.505091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703512+0000","last_change":"2023-12-02T00:28:49.716925+0000","last_active":"2023-12-02T00:28:50.703512+0000","last_peered":"2023-12-02T00:28:50.703512+0000","last_clean":"2023-12-02T00:28:50.703512+0000","last_became_active":"2023-12-02T00:28:49.716530+0000","last_became_peered":"2023-12-02T00:28:49.716530+0000","last_unstale":"2023-12-02T00:28:50.703512+0000","last_undegraded":"2023-12-02T00:28:50.703512+0000","last_fullsized":"2023-12-02T00:28:50.703512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:35:09.231704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021002+0000","last_change":"2023-12-02T00:28:48.732302+0000","last_active":"2023-12-02T00:28:51.021002+0000","last_peered":"2023-12-02T00:28:51.021002+0000","last_clean":"2023-12-02T00:28:51.021002+0000","last_became_active":"2023-12-02T00:28:48.731891+0000","last_became_peered":"2023-12-02T00:28:48.731891+0000","last_unstale":"2023-12-02T00:28:51.021002+0000","last_undegraded":"2023-12-02T00:28:51.021002+0000","last_fullsized":"2023-12-02T00:28:51.021002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:44:45.468077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706568+0000","last_change":"2023-12-02T00:28:49.719085+0000","last_active":"2023-12-02T00:28:50.706568+0000","last_peered":"2023-12-02T00:28:50.706568+0000","last_clean":"2023-12-02T00:28:50.706568+0000","last_became_active":"2023-12-02T00:28:49.718606+0000","last_became_peered":"2023-12-02T00:28:49.718606+0000","last_unstale":"2023-12-02T00:28:50.706568+0000","last_undegraded":"2023-12-02T00:28:50.706568+0000","last_fullsized":"2023-12-02T00:28:50.706568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:18:31.615464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706430+0000","last_change":"2023-12-02T00:28:48.732243+0000","last_active":"2023-12-02T00:28:50.706430+0000","last_peered":"2023-12-02T00:28:50.706430+0000","last_clean":"2023-12-02T00:28:50.706430+0000","last_became_active":"2023-12-02T00:28:48.731895+0000","last_became_peered":"2023-12-02T00:28:48.731895+0000","last_unstale":"2023-12-02T00:28:50.706430+0000","last_undegraded":"2023-12-02T00:28:50.706430+0000","last_fullsized":"2023-12-02T00:28:50.706430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:55:54.133908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021998+0000","last_change":"2023-12-02T00:28:49.718319+0000","last_active":"2023-12-02T00:28:51.021998+0000","last_peered":"2023-12-02T00:28:51.021998+0000","last_clean":"2023-12-02T00:28:51.021998+0000","last_became_active":"2023-12-02T00:28:49.718045+0000","last_became_peered":"2023-12-02T00:28:49.718045+0000","last_unstale":"2023-12-02T00:28:51.021998+0000","last_undegraded":"2023-12-02T00:28:51.021998+0000","last_fullsized":"2023-12-02T00:28:51.021998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:58:11.596095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703791+0000","last_change":"2023-12-02T00:28:48.728008+0000","last_active":"2023-12-02T00:28:50.703791+0000","last_peered":"2023-12-02T00:28:50.703791+0000","last_clean":"2023-12-02T00:28:50.703791+0000","last_became_active":"2023-12-02T00:28:48.727662+0000","last_became_peered":"2023-12-02T00:28:48.727662+0000","last_unstale":"2023-12-02T00:28:50.703791+0000","last_undegraded":"2023-12-02T00:28:50.703791+0000","last_fullsized":"2023-12-02T00:28:50.703791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:13:35.505091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705692+0000","last_change":"2023-12-02T00:28:49.719391+0000","last_active":"2023-12-02T00:28:50.705692+0000","last_peered":"2023-12-02T00:28:50.705692+0000","last_clean":"2023-12-02T00:28:50.705692+0000","last_became_active":"2023-12-02T00:28:49.719126+0000","last_became_peered":"2023-12-02T00:28:49.719126+0000","last_unstale":"2023-12-02T00:28:50.705692+0000","last_undegraded":"2023-12-02T00:28:50.705692+0000","last_fullsized":"2023-12-02T00:28:50.705692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:47:09.825514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021521+0000","last_change":"2023-12-02T00:28:48.736811+0000","last_active":"2023-12-02T00:28:51.021521+0000","last_peered":"2023-12-02T00:28:51.021521+0000","last_clean":"2023-12-02T00:28:51.021521+0000","last_became_active":"2023-12-02T00:28:48.736576+0000","last_became_peered":"2023-12-02T00:28:48.736576+0000","last_unstale":"2023-12-02T00:28:51.021521+0000","last_undegraded":"2023-12-02T00:28:51.021521+0000","last_fullsized":"2023-12-02T00:28:51.021521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:52:07.682002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021600+0000","last_change":"2023-12-02T00:28:49.717541+0000","last_active":"2023-12-02T00:28:51.021600+0000","last_peered":"2023-12-02T00:28:51.021600+0000","last_clean":"2023-12-02T00:28:51.021600+0000","last_became_active":"2023-12-02T00:28:49.717225+0000","last_became_peered":"2023-12-02T00:28:49.717225+0000","last_unstale":"2023-12-02T00:28:51.021600+0000","last_undegraded":"2023-12-02T00:28:51.021600+0000","last_fullsized":"2023-12-02T00:28:51.021600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:29:16.078935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.022091+0000","last_change":"2023-12-02T00:28:48.725581+0000","last_active":"2023-12-02T00:28:51.022091+0000","last_peered":"2023-12-02T00:28:51.022091+0000","last_clean":"2023-12-02T00:28:51.022091+0000","last_became_active":"2023-12-02T00:28:48.725230+0000","last_became_peered":"2023-12-02T00:28:48.725230+0000","last_unstale":"2023-12-02T00:28:51.022091+0000","last_undegraded":"2023-12-02T00:28:51.022091+0000","last_fullsized":"2023-12-02T00:28:51.022091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:39:30.211229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021761+0000","last_change":"2023-12-02T00:28:49.716591+0000","last_active":"2023-12-02T00:28:51.021761+0000","last_peered":"2023-12-02T00:28:51.021761+0000","last_clean":"2023-12-02T00:28:51.021761+0000","last_became_active":"2023-12-02T00:28:49.716237+0000","last_became_peered":"2023-12-02T00:28:49.716237+0000","last_unstale":"2023-12-02T00:28:51.021761+0000","last_undegraded":"2023-12-02T00:28:51.021761+0000","last_fullsized":"2023-12-02T00:28:51.021761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T12:06:34.965686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706099+0000","last_change":"2023-12-02T00:28:48.734778+0000","last_active":"2023-12-02T00:28:50.706099+0000","last_peered":"2023-12-02T00:28:50.706099+0000","last_clean":"2023-12-02T00:28:50.706099+0000","last_became_active":"2023-12-02T00:28:48.734474+0000","last_became_peered":"2023-12-02T00:28:48.734474+0000","last_unstale":"2023-12-02T00:28:50.706099+0000","last_undegraded":"2023-12-02T00:28:50.706099+0000","last_fullsized":"2023-12-02T00:28:50.706099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:07:51.795216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021342+0000","last_change":"2023-12-02T00:28:49.721274+0000","last_active":"2023-12-02T00:28:51.021342+0000","last_peered":"2023-12-02T00:28:51.021342+0000","last_clean":"2023-12-02T00:28:51.021342+0000","last_became_active":"2023-12-02T00:28:49.719979+0000","last_became_peered":"2023-12-02T00:28:49.719979+0000","last_unstale":"2023-12-02T00:28:51.021342+0000","last_undegraded":"2023-12-02T00:28:51.021342+0000","last_fullsized":"2023-12-02T00:28:51.021342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:50:45.793304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020850+0000","last_change":"2023-12-02T00:28:48.732958+0000","last_active":"2023-12-02T00:28:51.020850+0000","last_peered":"2023-12-02T00:28:51.020850+0000","last_clean":"2023-12-02T00:28:51.020850+0000","last_became_active":"2023-12-02T00:28:48.732651+0000","last_became_peered":"2023-12-02T00:28:48.732651+0000","last_unstale":"2023-12-02T00:28:51.020850+0000","last_undegraded":"2023-12-02T00:28:51.020850+0000","last_fullsized":"2023-12-02T00:28:51.020850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:29:26.381170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020869+0000","last_change":"2023-12-02T00:28:49.712859+0000","last_active":"2023-12-02T00:28:51.020869+0000","last_peered":"2023-12-02T00:28:51.020869+0000","last_clean":"2023-12-02T00:28:51.020869+0000","last_became_active":"2023-12-02T00:28:49.712585+0000","last_became_peered":"2023-12-02T00:28:49.712585+0000","last_unstale":"2023-12-02T00:28:51.020869+0000","last_undegraded":"2023-12-02T00:28:51.020869+0000","last_fullsized":"2023-12-02T00:28:51.020869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T12:06:34.965686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020660+0000","last_change":"2023-12-02T00:28:48.734113+0000","last_active":"2023-12-02T00:28:51.020660+0000","last_peered":"2023-12-02T00:28:51.020660+0000","last_clean":"2023-12-02T00:28:51.020660+0000","last_became_active":"2023-12-02T00:28:48.733749+0000","last_became_peered":"2023-12-02T00:28:48.733749+0000","last_unstale":"2023-12-02T00:28:51.020660+0000","last_undegraded":"2023-12-02T00:28:51.020660+0000","last_fullsized":"2023-12-02T00:28:51.020660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T10:56:02.550120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021416+0000","last_change":"2023-12-02T00:28:49.719477+0000","last_active":"2023-12-02T00:28:51.021416+0000","last_peered":"2023-12-02T00:28:51.021416+0000","last_clean":"2023-12-02T00:28:51.021416+0000","last_became_active":"2023-12-02T00:28:49.718537+0000","last_became_peered":"2023-12-02T00:28:49.718537+0000","last_unstale":"2023-12-02T00:28:51.021416+0000","last_undegraded":"2023-12-02T00:28:51.021416+0000","last_fullsized":"2023-12-02T00:28:51.021416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:15:02.988933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705801+0000","last_change":"2023-12-02T00:28:48.730372+0000","last_active":"2023-12-02T00:28:50.705801+0000","last_peered":"2023-12-02T00:28:50.705801+0000","last_clean":"2023-12-02T00:28:50.705801+0000","last_became_active":"2023-12-02T00:28:48.730051+0000","last_became_peered":"2023-12-02T00:28:48.730051+0000","last_unstale":"2023-12-02T00:28:50.705801+0000","last_undegraded":"2023-12-02T00:28:50.705801+0000","last_fullsized":"2023-12-02T00:28:50.705801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:29:10.550270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021515+0000","last_change":"2023-12-02T00:28:49.714467+0000","last_active":"2023-12-02T00:28:51.021515+0000","last_peered":"2023-12-02T00:28:51.021515+0000","last_clean":"2023-12-02T00:28:51.021515+0000","last_became_active":"2023-12-02T00:28:49.713964+0000","last_became_peered":"2023-12-02T00:28:49.713964+0000","last_unstale":"2023-12-02T00:28:51.021515+0000","last_undegraded":"2023-12-02T00:28:51.021515+0000","last_fullsized":"2023-12-02T00:28:51.021515+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T12:22:41.666941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021286+0000","last_change":"2023-12-02T00:28:48.736447+0000","last_active":"2023-12-02T00:28:51.021286+0000","last_peered":"2023-12-02T00:28:51.021286+0000","last_clean":"2023-12-02T00:28:51.021286+0000","last_became_active":"2023-12-02T00:28:48.736169+0000","last_became_peered":"2023-12-02T00:28:48.736169+0000","last_unstale":"2023-12-02T00:28:51.021286+0000","last_undegraded":"2023-12-02T00:28:51.021286+0000","last_fullsized":"2023-12-02T00:28:51.021286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T09:17:51.234416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021954+0000","last_change":"2023-12-02T00:28:49.719154+0000","last_active":"2023-12-02T00:28:51.021954+0000","last_peered":"2023-12-02T00:28:51.021954+0000","last_clean":"2023-12-02T00:28:51.021954+0000","last_became_active":"2023-12-02T00:28:49.718713+0000","last_became_peered":"2023-12-02T00:28:49.718713+0000","last_unstale":"2023-12-02T00:28:51.021954+0000","last_undegraded":"2023-12-02T00:28:51.021954+0000","last_fullsized":"2023-12-02T00:28:51.021954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:50:17.285633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703845+0000","last_change":"2023-12-02T00:28:48.731504+0000","last_active":"2023-12-02T00:28:50.703845+0000","last_peered":"2023-12-02T00:28:50.703845+0000","last_clean":"2023-12-02T00:28:50.703845+0000","last_became_active":"2023-12-02T00:28:48.731224+0000","last_became_peered":"2023-12-02T00:28:48.731224+0000","last_unstale":"2023-12-02T00:28:50.703845+0000","last_undegraded":"2023-12-02T00:28:50.703845+0000","last_fullsized":"2023-12-02T00:28:50.703845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:55:54.133908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706503+0000","last_change":"2023-12-02T00:28:49.726089+0000","last_active":"2023-12-02T00:28:50.706503+0000","last_peered":"2023-12-02T00:28:50.706503+0000","last_clean":"2023-12-02T00:28:50.706503+0000","last_became_active":"2023-12-02T00:28:49.725482+0000","last_became_peered":"2023-12-02T00:28:49.725482+0000","last_unstale":"2023-12-02T00:28:50.706503+0000","last_undegraded":"2023-12-02T00:28:50.706503+0000","last_fullsized":"2023-12-02T00:28:50.706503+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T12:22:41.666941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020880+0000","last_change":"2023-12-02T00:28:48.729936+0000","last_active":"2023-12-02T00:28:51.020880+0000","last_peered":"2023-12-02T00:28:51.020880+0000","last_clean":"2023-12-02T00:28:51.020880+0000","last_became_active":"2023-12-02T00:28:48.729617+0000","last_became_peered":"2023-12-02T00:28:48.729617+0000","last_unstale":"2023-12-02T00:28:51.020880+0000","last_undegraded":"2023-12-02T00:28:51.020880+0000","last_fullsized":"2023-12-02T00:28:51.020880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:29:26.381170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021803+0000","last_change":"2023-12-02T00:28:49.725211+0000","last_active":"2023-12-02T00:28:51.021803+0000","last_peered":"2023-12-02T00:28:51.021803+0000","last_clean":"2023-12-02T00:28:51.021803+0000","last_became_active":"2023-12-02T00:28:49.724928+0000","last_became_peered":"2023-12-02T00:28:49.724928+0000","last_unstale":"2023-12-02T00:28:51.021803+0000","last_undegraded":"2023-12-02T00:28:51.021803+0000","last_fullsized":"2023-12-02T00:28:51.021803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:33:43.085260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":165,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-02T00:28:49.708269+0000","last_change":"2023-12-02T00:28:39.379380+0000","last_active":"2023-12-02T00:28:49.708269+0000","last_peered":"2023-12-02T00:28:49.708269+0000","last_clean":"2023-12-02T00:28:49.708269+0000","last_became_active":"2023-12-02T00:28:39.378255+0000","last_became_peered":"2023-12-02T00:28:39.378255+0000","last_unstale":"2023-12-02T00:28:49.708269+0000","last_undegraded":"2023-12-02T00:28:49.708269+0000","last_fullsized":"2023-12-02T00:28:49.708269+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:38.177345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:38.177345+0000","last_clean_scrub_stamp":"2023-12-02T00:28:38.177345+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:21:08.582412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706107+0000","last_change":"2023-12-02T00:28:48.731326+0000","last_active":"2023-12-02T00:28:50.706107+0000","last_peered":"2023-12-02T00:28:50.706107+0000","last_clean":"2023-12-02T00:28:50.706107+0000","last_became_active":"2023-12-02T00:28:48.730970+0000","last_became_peered":"2023-12-02T00:28:48.730970+0000","last_unstale":"2023-12-02T00:28:50.706107+0000","last_undegraded":"2023-12-02T00:28:50.706107+0000","last_fullsized":"2023-12-02T00:28:50.706107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:44:45.468077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021403+0000","last_change":"2023-12-02T00:28:49.727341+0000","last_active":"2023-12-02T00:28:51.021403+0000","last_peered":"2023-12-02T00:28:51.021403+0000","last_clean":"2023-12-02T00:28:51.021403+0000","last_became_active":"2023-12-02T00:28:49.727045+0000","last_became_peered":"2023-12-02T00:28:49.727045+0000","last_unstale":"2023-12-02T00:28:51.021403+0000","last_undegraded":"2023-12-02T00:28:51.021403+0000","last_fullsized":"2023-12-02T00:28:51.021403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:58:15.388237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020921+0000","last_change":"2023-12-02T00:28:48.732719+0000","last_active":"2023-12-02T00:28:51.020921+0000","last_peered":"2023-12-02T00:28:51.020921+0000","last_clean":"2023-12-02T00:28:51.020921+0000","last_became_active":"2023-12-02T00:28:48.732466+0000","last_became_peered":"2023-12-02T00:28:48.732466+0000","last_unstale":"2023-12-02T00:28:51.020921+0000","last_undegraded":"2023-12-02T00:28:51.020921+0000","last_fullsized":"2023-12-02T00:28:51.020921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:52:21.593280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021611+0000","last_change":"2023-12-02T00:28:49.716043+0000","last_active":"2023-12-02T00:28:51.021611+0000","last_peered":"2023-12-02T00:28:51.021611+0000","last_clean":"2023-12-02T00:28:51.021611+0000","last_became_active":"2023-12-02T00:28:49.715747+0000","last_became_peered":"2023-12-02T00:28:49.715747+0000","last_unstale":"2023-12-02T00:28:51.021611+0000","last_undegraded":"2023-12-02T00:28:51.021611+0000","last_fullsized":"2023-12-02T00:28:51.021611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:44:46.475452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021208+0000","last_change":"2023-12-02T00:28:48.736024+0000","last_active":"2023-12-02T00:28:51.021208+0000","last_peered":"2023-12-02T00:28:51.021208+0000","last_clean":"2023-12-02T00:28:51.021208+0000","last_became_active":"2023-12-02T00:28:48.735736+0000","last_became_peered":"2023-12-02T00:28:48.735736+0000","last_unstale":"2023-12-02T00:28:51.021208+0000","last_undegraded":"2023-12-02T00:28:51.021208+0000","last_fullsized":"2023-12-02T00:28:51.021208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:55:48.025090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703534+0000","last_change":"2023-12-02T00:28:49.707755+0000","last_active":"2023-12-02T00:28:50.703534+0000","last_peered":"2023-12-02T00:28:50.703534+0000","last_clean":"2023-12-02T00:28:50.703534+0000","last_became_active":"2023-12-02T00:28:49.707449+0000","last_became_peered":"2023-12-02T00:28:49.707449+0000","last_unstale":"2023-12-02T00:28:50.703534+0000","last_undegraded":"2023-12-02T00:28:50.703534+0000","last_fullsized":"2023-12-02T00:28:50.703534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:07:52.802590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706217+0000","last_change":"2023-12-02T00:28:48.729078+0000","last_active":"2023-12-02T00:28:50.706217+0000","last_peered":"2023-12-02T00:28:50.706217+0000","last_clean":"2023-12-02T00:28:50.706217+0000","last_became_active":"2023-12-02T00:28:48.728567+0000","last_became_peered":"2023-12-02T00:28:48.728567+0000","last_unstale":"2023-12-02T00:28:50.706217+0000","last_undegraded":"2023-12-02T00:28:50.706217+0000","last_fullsized":"2023-12-02T00:28:50.706217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:58:10.588720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705483+0000","last_change":"2023-12-02T00:28:49.712801+0000","last_active":"2023-12-02T00:28:50.705483+0000","last_peered":"2023-12-02T00:28:50.705483+0000","last_clean":"2023-12-02T00:28:50.705483+0000","last_became_active":"2023-12-02T00:28:49.712511+0000","last_became_peered":"2023-12-02T00:28:49.712511+0000","last_unstale":"2023-12-02T00:28:50.705483+0000","last_undegraded":"2023-12-02T00:28:50.705483+0000","last_fullsized":"2023-12-02T00:28:50.705483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:55:49.032465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703749+0000","last_change":"2023-12-02T00:28:48.730774+0000","last_active":"2023-12-02T00:28:50.703749+0000","last_peered":"2023-12-02T00:28:50.703749+0000","last_clean":"2023-12-02T00:28:50.703749+0000","last_became_active":"2023-12-02T00:28:48.730498+0000","last_became_peered":"2023-12-02T00:28:48.730498+0000","last_unstale":"2023-12-02T00:28:50.703749+0000","last_undegraded":"2023-12-02T00:28:50.703749+0000","last_fullsized":"2023-12-02T00:28:50.703749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:29:10.550270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706537+0000","last_change":"2023-12-02T00:28:49.726928+0000","last_active":"2023-12-02T00:28:50.706537+0000","last_peered":"2023-12-02T00:28:50.706537+0000","last_clean":"2023-12-02T00:28:50.706537+0000","last_became_active":"2023-12-02T00:28:49.725033+0000","last_became_peered":"2023-12-02T00:28:49.725033+0000","last_unstale":"2023-12-02T00:28:50.706537+0000","last_undegraded":"2023-12-02T00:28:50.706537+0000","last_fullsized":"2023-12-02T00:28:50.706537+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:33:43.085260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021906+0000","last_change":"2023-12-02T00:28:48.735559+0000","last_active":"2023-12-02T00:28:51.021906+0000","last_peered":"2023-12-02T00:28:51.021906+0000","last_clean":"2023-12-02T00:28:51.021906+0000","last_became_active":"2023-12-02T00:28:48.735243+0000","last_became_peered":"2023-12-02T00:28:48.735243+0000","last_unstale":"2023-12-02T00:28:51.021906+0000","last_undegraded":"2023-12-02T00:28:51.021906+0000","last_fullsized":"2023-12-02T00:28:51.021906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:07:51.795216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703595+0000","last_change":"2023-12-02T00:28:49.710546+0000","last_active":"2023-12-02T00:28:50.703595+0000","last_peered":"2023-12-02T00:28:50.703595+0000","last_clean":"2023-12-02T00:28:50.703595+0000","last_became_active":"2023-12-02T00:28:49.710010+0000","last_became_peered":"2023-12-02T00:28:49.710010+0000","last_unstale":"2023-12-02T00:28:50.703595+0000","last_undegraded":"2023-12-02T00:28:50.703595+0000","last_fullsized":"2023-12-02T00:28:50.703595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:58:11.596095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703835+0000","last_change":"2023-12-02T00:28:48.731556+0000","last_active":"2023-12-02T00:28:50.703835+0000","last_peered":"2023-12-02T00:28:50.703835+0000","last_clean":"2023-12-02T00:28:50.703835+0000","last_became_active":"2023-12-02T00:28:48.731291+0000","last_became_peered":"2023-12-02T00:28:48.731291+0000","last_unstale":"2023-12-02T00:28:50.703835+0000","last_undegraded":"2023-12-02T00:28:50.703835+0000","last_fullsized":"2023-12-02T00:28:50.703835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:39:30.211229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020929+0000","last_change":"2023-12-02T00:28:49.723678+0000","last_active":"2023-12-02T00:28:51.020929+0000","last_peered":"2023-12-02T00:28:51.020929+0000","last_clean":"2023-12-02T00:28:51.020929+0000","last_became_active":"2023-12-02T00:28:49.723299+0000","last_became_peered":"2023-12-02T00:28:49.723299+0000","last_unstale":"2023-12-02T00:28:51.020929+0000","last_undegraded":"2023-12-02T00:28:51.020929+0000","last_fullsized":"2023-12-02T00:28:51.020929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:58:11.596095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021044+0000","last_change":"2023-12-02T00:28:48.735741+0000","last_active":"2023-12-02T00:28:51.021044+0000","last_peered":"2023-12-02T00:28:51.021044+0000","last_clean":"2023-12-02T00:28:51.021044+0000","last_became_active":"2023-12-02T00:28:48.735400+0000","last_became_peered":"2023-12-02T00:28:48.735400+0000","last_unstale":"2023-12-02T00:28:51.021044+0000","last_undegraded":"2023-12-02T00:28:51.021044+0000","last_fullsized":"2023-12-02T00:28:51.021044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:13:35.505091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.022028+0000","last_change":"2023-12-02T00:28:49.711592+0000","last_active":"2023-12-02T00:28:51.022028+0000","last_peered":"2023-12-02T00:28:51.022028+0000","last_clean":"2023-12-02T00:28:51.022028+0000","last_became_active":"2023-12-02T00:28:49.711211+0000","last_became_peered":"2023-12-02T00:28:49.711211+0000","last_unstale":"2023-12-02T00:28:51.022028+0000","last_undegraded":"2023-12-02T00:28:51.022028+0000","last_fullsized":"2023-12-02T00:28:51.022028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:15:51.747629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020816+0000","last_change":"2023-12-02T00:28:48.735722+0000","last_active":"2023-12-02T00:28:51.020816+0000","last_peered":"2023-12-02T00:28:51.020816+0000","last_clean":"2023-12-02T00:28:51.020816+0000","last_became_active":"2023-12-02T00:28:48.735212+0000","last_became_peered":"2023-12-02T00:28:48.735212+0000","last_unstale":"2023-12-02T00:28:51.020816+0000","last_undegraded":"2023-12-02T00:28:51.020816+0000","last_fullsized":"2023-12-02T00:28:51.020816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:47:08.818139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705309+0000","last_change":"2023-12-02T00:28:49.711824+0000","last_active":"2023-12-02T00:28:50.705309+0000","last_peered":"2023-12-02T00:28:50.705309+0000","last_clean":"2023-12-02T00:28:50.705309+0000","last_became_active":"2023-12-02T00:28:49.710946+0000","last_became_peered":"2023-12-02T00:28:49.710946+0000","last_unstale":"2023-12-02T00:28:50.705309+0000","last_undegraded":"2023-12-02T00:28:50.705309+0000","last_fullsized":"2023-12-02T00:28:50.705309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T09:17:52.241790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703622+0000","last_change":"2023-12-02T00:28:48.727127+0000","last_active":"2023-12-02T00:28:50.703622+0000","last_peered":"2023-12-02T00:28:50.703622+0000","last_clean":"2023-12-02T00:28:50.703622+0000","last_became_active":"2023-12-02T00:28:48.726750+0000","last_became_peered":"2023-12-02T00:28:48.726750+0000","last_unstale":"2023-12-02T00:28:50.703622+0000","last_undegraded":"2023-12-02T00:28:50.703622+0000","last_fullsized":"2023-12-02T00:28:50.703622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:44:45.468077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021415+0000","last_change":"2023-12-02T00:28:49.718158+0000","last_active":"2023-12-02T00:28:51.021415+0000","last_peered":"2023-12-02T00:28:51.021415+0000","last_clean":"2023-12-02T00:28:51.021415+0000","last_became_active":"2023-12-02T00:28:49.717721+0000","last_became_peered":"2023-12-02T00:28:49.717721+0000","last_unstale":"2023-12-02T00:28:51.021415+0000","last_undegraded":"2023-12-02T00:28:51.021415+0000","last_fullsized":"2023-12-02T00:28:51.021415+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T10:27:29.244181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703765+0000","last_change":"2023-12-02T00:28:48.735894+0000","last_active":"2023-12-02T00:28:50.703765+0000","last_peered":"2023-12-02T00:28:50.703765+0000","last_clean":"2023-12-02T00:28:50.703765+0000","last_became_active":"2023-12-02T00:28:48.735613+0000","last_became_peered":"2023-12-02T00:28:48.735613+0000","last_unstale":"2023-12-02T00:28:50.703765+0000","last_undegraded":"2023-12-02T00:28:50.703765+0000","last_fullsized":"2023-12-02T00:28:50.703765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:52:07.682002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021464+0000","last_change":"2023-12-02T00:28:49.719314+0000","last_active":"2023-12-02T00:28:51.021464+0000","last_peered":"2023-12-02T00:28:51.021464+0000","last_clean":"2023-12-02T00:28:51.021464+0000","last_became_active":"2023-12-02T00:28:49.718940+0000","last_became_peered":"2023-12-02T00:28:49.718940+0000","last_unstale":"2023-12-02T00:28:51.021464+0000","last_undegraded":"2023-12-02T00:28:51.021464+0000","last_fullsized":"2023-12-02T00:28:51.021464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:50:45.793304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021457+0000","last_change":"2023-12-02T00:28:48.731948+0000","last_active":"2023-12-02T00:28:51.021457+0000","last_peered":"2023-12-02T00:28:51.021457+0000","last_clean":"2023-12-02T00:28:51.021457+0000","last_became_active":"2023-12-02T00:28:48.731641+0000","last_became_peered":"2023-12-02T00:28:48.731641+0000","last_unstale":"2023-12-02T00:28:51.021457+0000","last_undegraded":"2023-12-02T00:28:51.021457+0000","last_fullsized":"2023-12-02T00:28:51.021457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:36:21.580553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021037+0000","last_change":"2023-12-02T00:28:49.715513+0000","last_active":"2023-12-02T00:28:51.021037+0000","last_peered":"2023-12-02T00:28:51.021037+0000","last_clean":"2023-12-02T00:28:51.021037+0000","last_became_active":"2023-12-02T00:28:49.715240+0000","last_became_peered":"2023-12-02T00:28:49.715240+0000","last_unstale":"2023-12-02T00:28:51.021037+0000","last_undegraded":"2023-12-02T00:28:51.021037+0000","last_fullsized":"2023-12-02T00:28:51.021037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:52:22.600655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706521+0000","last_change":"2023-12-02T00:28:49.717785+0000","last_active":"2023-12-02T00:28:50.706521+0000","last_peered":"2023-12-02T00:28:50.706521+0000","last_clean":"2023-12-02T00:28:50.706521+0000","last_became_active":"2023-12-02T00:28:49.717408+0000","last_became_peered":"2023-12-02T00:28:49.717408+0000","last_unstale":"2023-12-02T00:28:50.706521+0000","last_undegraded":"2023-12-02T00:28:50.706521+0000","last_fullsized":"2023-12-02T00:28:50.706521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:29:16.078935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021429+0000","last_change":"2023-12-02T00:28:48.729768+0000","last_active":"2023-12-02T00:28:51.021429+0000","last_peered":"2023-12-02T00:28:51.021429+0000","last_clean":"2023-12-02T00:28:51.021429+0000","last_became_active":"2023-12-02T00:28:48.729466+0000","last_became_peered":"2023-12-02T00:28:48.729466+0000","last_unstale":"2023-12-02T00:28:51.021429+0000","last_undegraded":"2023-12-02T00:28:51.021429+0000","last_fullsized":"2023-12-02T00:28:51.021429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T12:06:33.958311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706378+0000","last_change":"2023-12-02T00:28:49.727325+0000","last_active":"2023-12-02T00:28:50.706378+0000","last_peered":"2023-12-02T00:28:50.706378+0000","last_clean":"2023-12-02T00:28:50.706378+0000","last_became_active":"2023-12-02T00:28:49.727062+0000","last_became_peered":"2023-12-02T00:28:49.727062+0000","last_unstale":"2023-12-02T00:28:50.706378+0000","last_undegraded":"2023-12-02T00:28:50.706378+0000","last_fullsized":"2023-12-02T00:28:50.706378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:15:02.988933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021734+0000","last_change":"2023-12-02T00:28:48.732989+0000","last_active":"2023-12-02T00:28:51.021734+0000","last_peered":"2023-12-02T00:28:51.021734+0000","last_clean":"2023-12-02T00:28:51.021734+0000","last_became_active":"2023-12-02T00:28:48.731645+0000","last_became_peered":"2023-12-02T00:28:48.731645+0000","last_unstale":"2023-12-02T00:28:51.021734+0000","last_undegraded":"2023-12-02T00:28:51.021734+0000","last_fullsized":"2023-12-02T00:28:51.021734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:44:45.468077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021923+0000","last_change":"2023-12-02T00:28:49.725372+0000","last_active":"2023-12-02T00:28:51.021923+0000","last_peered":"2023-12-02T00:28:51.021923+0000","last_clean":"2023-12-02T00:28:51.021923+0000","last_became_active":"2023-12-02T00:28:49.724926+0000","last_became_peered":"2023-12-02T00:28:49.724926+0000","last_unstale":"2023-12-02T00:28:51.021923+0000","last_undegraded":"2023-12-02T00:28:51.021923+0000","last_fullsized":"2023-12-02T00:28:51.021923+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:07:52.802590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021204+0000","last_change":"2023-12-02T00:28:48.731302+0000","last_active":"2023-12-02T00:28:51.021204+0000","last_peered":"2023-12-02T00:28:51.021204+0000","last_clean":"2023-12-02T00:28:51.021204+0000","last_became_active":"2023-12-02T00:28:48.731002+0000","last_became_peered":"2023-12-02T00:28:48.731002+0000","last_unstale":"2023-12-02T00:28:51.021204+0000","last_undegraded":"2023-12-02T00:28:51.021204+0000","last_fullsized":"2023-12-02T00:28:51.021204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:29:26.381170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021221+0000","last_change":"2023-12-02T00:28:49.721605+0000","last_active":"2023-12-02T00:28:51.021221+0000","last_peered":"2023-12-02T00:28:51.021221+0000","last_clean":"2023-12-02T00:28:51.021221+0000","last_became_active":"2023-12-02T00:28:49.721035+0000","last_became_peered":"2023-12-02T00:28:49.721035+0000","last_unstale":"2023-12-02T00:28:51.021221+0000","last_undegraded":"2023-12-02T00:28:51.021221+0000","last_fullsized":"2023-12-02T00:28:51.021221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:15:51.747629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.022018+0000","last_change":"2023-12-02T00:28:48.729331+0000","last_active":"2023-12-02T00:28:51.022018+0000","last_peered":"2023-12-02T00:28:51.022018+0000","last_clean":"2023-12-02T00:28:51.022018+0000","last_became_active":"2023-12-02T00:28:48.728469+0000","last_became_peered":"2023-12-02T00:28:48.728469+0000","last_unstale":"2023-12-02T00:28:51.022018+0000","last_undegraded":"2023-12-02T00:28:51.022018+0000","last_fullsized":"2023-12-02T00:28:51.022018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:47:08.818139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":660231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":70893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":69371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":69407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":77193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31376,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340464,"statfs":{"total":96636764160,"available":96604635136,"internally_reserved":0,"allocated":200704,"data_stored":68662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":212992,"data_stored":69441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":806912,"data_stored":659668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-02T00:28:57.168 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-02T00:28:57.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.286+0000 7f7325626640 1 Processor -- start 2023-12-02T00:28:57.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.286+0000 7f7325626640 1 -- start start 2023-12-02T00:28:57.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.287+0000 7f7325626640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7320144980 0x7f732015c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:57.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.287+0000 7f7325626640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f73200b4370 0x7f73200b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:57.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.287+0000 7f7325626640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7320071770 con 0x7f73200ceb70 2023-12-02T00:28:57.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.287+0000 7f7325626640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73200ccb60 con 0x7f7320144980 2023-12-02T00:28:57.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.287+0000 7f7325626640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73200b4ce0 con 0x7f73200b4370 2023-12-02T00:28:57.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.287+0000 7f731effd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7320144980 0x7f732015c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:57.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.287+0000 7f731effd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7320144980 0x7f732015c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37792/0 (socket says 172.21.15.46:37792) 2023-12-02T00:28:57.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.287+0000 7f731effd640 1 -- 172.21.15.46:0/2009319400 learned_addr learned my addr 172.21.15.46:0/2009319400 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:57.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.287+0000 7f731f7fe640 1 --2- 172.21.15.46:0/2009319400 >> v2:172.21.15.162:3300/0 conn(0x7f73200b4370 0x7f73200b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:57.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.287+0000 7f731effd640 1 -- 172.21.15.46:0/2009319400 >> v2:172.21.15.162:3300/0 conn(0x7f73200b4370 msgr2=0x7f73200b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:57.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.287+0000 7f731effd640 1 --2- 172.21.15.46:0/2009319400 >> v2:172.21.15.162:3300/0 conn(0x7f73200b4370 0x7f73200b47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:57.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.287+0000 7f731effd640 1 -- 172.21.15.46:0/2009319400 >> v1:172.21.15.162:6789/0 conn(0x7f73200ceb70 legacy=0x7f73200cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:57.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.287+0000 7f731effd640 1 -- 172.21.15.46:0/2009319400 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73200ccdc0 con 0x7f7320144980 2023-12-02T00:28:57.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.288+0000 7f731effd640 1 --2- 172.21.15.46:0/2009319400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7320144980 0x7f732015c7c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f73200716e0 tx=0x7f731802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d961de846b2b8f45 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:57.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.288+0000 7f731dffb640 1 -- 172.21.15.46:0/2009319400 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7318030860 con 0x7f7320144980 2023-12-02T00:28:57.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.288+0000 7f731dffb640 1 -- 172.21.15.46:0/2009319400 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73180309c0 con 0x7f7320144980 2023-12-02T00:28:57.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.288+0000 7f731dffb640 1 -- 172.21.15.46:0/2009319400 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7318030d50 con 0x7f7320144980 2023-12-02T00:28:57.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.289+0000 7f7325626640 1 -- 172.21.15.46:0/2009319400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7320144980 msgr2=0x7f732015c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:57.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.289+0000 7f7325626640 1 --2- 172.21.15.46:0/2009319400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7320144980 0x7f732015c7c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f73200716e0 tx=0x7f731802b9a0 comp rx=0 tx=0).stop 2023-12-02T00:28:57.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.289+0000 7f7325626640 1 -- 172.21.15.46:0/2009319400 shutdown_connections 2023-12-02T00:28:57.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.289+0000 7f7325626640 1 --2- 172.21.15.46:0/2009319400 >> v2:172.21.15.162:3300/0 conn(0x7f73200b4370 0x7f73200b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:57.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.289+0000 7f7325626640 1 --2- 172.21.15.46:0/2009319400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7320144980 0x7f732015c7c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:57.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.289+0000 7f7325626640 1 -- 172.21.15.46:0/2009319400 >> 172.21.15.46:0/2009319400 conn(0x7f732006a4d0 msgr2=0x7f732006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:57.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.289+0000 7f7325626640 1 -- 172.21.15.46:0/2009319400 shutdown_connections 2023-12-02T00:28:57.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.289+0000 7f7325626640 1 -- 172.21.15.46:0/2009319400 wait complete. 2023-12-02T00:28:57.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.290+0000 7f7325626640 1 Processor -- start 2023-12-02T00:28:57.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.290+0000 7f7325626640 1 -- start start 2023-12-02T00:28:57.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.291+0000 7f7325626640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f73200b4370 0x7f7320158790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:57.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.291+0000 7f7325626640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73200ceb70 0x7f732013deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:57.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.291+0000 7f7325626640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7320144980 0x7f73201f1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:57.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.291+0000 7f7325626640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f732013e890 con 0x7f7320144980 2023-12-02T00:28:57.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.291+0000 7f7325626640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73201f20a0 con 0x7f73200ceb70 2023-12-02T00:28:57.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.291+0000 7f7325626640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f732013e5b0 con 0x7f73200b4370 2023-12-02T00:28:57.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.291+0000 7f731f7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7320144980 0x7f73201f1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:57.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.291+0000 7f731f7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7320144980 0x7f73201f1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37806/0 (socket says 172.21.15.46:37806) 2023-12-02T00:28:57.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.291+0000 7f731f7fe640 1 -- 172.21.15.46:0/206370847 learned_addr learned my addr 172.21.15.46:0/206370847 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:57.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.291+0000 7f731e7fc640 1 --2- 172.21.15.46:0/206370847 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73200ceb70 0x7f732013deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:57.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.291+0000 7f731effd640 1 --2- 172.21.15.46:0/206370847 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f73200b4370 0x7f7320158790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:57.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.292+0000 7f731f7fe640 1 -- 172.21.15.46:0/206370847 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f73200b4370 msgr2=0x7f7320158790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:57.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.292+0000 7f731f7fe640 1 --2- 172.21.15.46:0/206370847 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f73200b4370 0x7f7320158790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:57.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.292+0000 7f731f7fe640 1 -- 172.21.15.46:0/206370847 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73200ceb70 msgr2=0x7f732013deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:57.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.292+0000 7f731f7fe640 1 --2- 172.21.15.46:0/206370847 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73200ceb70 0x7f732013deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:57.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.292+0000 7f731f7fe640 1 -- 172.21.15.46:0/206370847 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73180094d0 con 0x7f7320144980 2023-12-02T00:28:57.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.292+0000 7f731f7fe640 1 --2- 172.21.15.46:0/206370847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7320144980 0x7f73201f1b60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f731000efc0 tx=0x7f731000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:57.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.292+0000 7f73177fe640 1 -- 172.21.15.46:0/206370847 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7310014480 con 0x7f7320144980 2023-12-02T00:28:57.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.292+0000 7f73177fe640 1 -- 172.21.15.46:0/206370847 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f731000f040 con 0x7f7320144980 2023-12-02T00:28:57.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.292+0000 7f73177fe640 1 -- 172.21.15.46:0/206370847 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7310014820 con 0x7f7320144980 2023-12-02T00:28:57.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.292+0000 7f7325626640 1 -- 172.21.15.46:0/206370847 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7320155e60 con 0x7f7320144980 2023-12-02T00:28:57.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.292+0000 7f7325626640 1 -- 172.21.15.46:0/206370847 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7320156420 con 0x7f7320144980 2023-12-02T00:28:57.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.293+0000 7f7325626640 1 -- 172.21.15.46:0/206370847 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7320156640 con 0x7f7320144980 2023-12-02T00:28:57.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.294+0000 7f73177fe640 1 -- 172.21.15.46:0/206370847 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7310014980 con 0x7f7320144980 2023-12-02T00:28:57.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.294+0000 7f73177fe640 1 --2- 172.21.15.46:0/206370847 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f730006ae40 0x7f730006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:57.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.294+0000 7f73177fe640 1 -- 172.21.15.46:0/206370847 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f731008a2f0 con 0x7f7320144980 2023-12-02T00:28:57.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.294+0000 7f731effd640 1 --2- 172.21.15.46:0/206370847 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f730006ae40 0x7f730006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:57.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.295+0000 7f731effd640 1 --2- 172.21.15.46:0/206370847 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f730006ae40 0x7f730006d300 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7318036000 tx=0x7f7318008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:57.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.299+0000 7f73177fe640 1 -- 172.21.15.46:0/206370847 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f73100554e0 con 0x7f7320144980 2023-12-02T00:28:57.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.471+0000 7f7325626640 1 -- 172.21.15.46:0/206370847 --> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f73200aeaf0 con 0x7f730006ae40 2023-12-02T00:28:57.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.478+0000 7f73177fe640 1 -- 172.21.15.46:0/206370847 <== mgr.4108 v2:172.21.15.162:6832/3248644722 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264592 (secure 0 0 0) 0x7f73200aeaf0 con 0x7f730006ae40 2023-12-02T00:28:57.484 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:28:57.486 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-12-02T00:28:57.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.481+0000 7f7325626640 1 -- 172.21.15.46:0/206370847 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f730006ae40 msgr2=0x7f730006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:57.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.481+0000 7f7325626640 1 --2- 172.21.15.46:0/206370847 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f730006ae40 0x7f730006d300 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7318036000 tx=0x7f7318008030 comp rx=0 tx=0).stop 2023-12-02T00:28:57.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.481+0000 7f7325626640 1 -- 172.21.15.46:0/206370847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7320144980 msgr2=0x7f73201f1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:57.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.481+0000 7f7325626640 1 --2- 172.21.15.46:0/206370847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7320144980 0x7f73201f1b60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f731000efc0 tx=0x7f731000c490 comp rx=0 tx=0).stop 2023-12-02T00:28:57.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.481+0000 7f7325626640 1 -- 172.21.15.46:0/206370847 shutdown_connections 2023-12-02T00:28:57.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.481+0000 7f7325626640 1 --2- 172.21.15.46:0/206370847 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f730006ae40 0x7f730006d300 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:57.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.481+0000 7f7325626640 1 --2- 172.21.15.46:0/206370847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7320144980 0x7f73201f1b60 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:57.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.481+0000 7f7325626640 1 --2- 172.21.15.46:0/206370847 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73200ceb70 0x7f732013deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:57.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.481+0000 7f7325626640 1 --2- 172.21.15.46:0/206370847 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f73200b4370 0x7f7320158790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:57.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.481+0000 7f7325626640 1 -- 172.21.15.46:0/206370847 >> 172.21.15.46:0/206370847 conn(0x7f732006a4d0 msgr2=0x7f732015ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:57.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.481+0000 7f7325626640 1 -- 172.21.15.46:0/206370847 shutdown_connections 2023-12-02T00:28:57.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.481+0000 7f7325626640 1 -- 172.21.15.46:0/206370847 wait complete. 2023-12-02T00:28:57.493 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-12-02T00:28:57.159750+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235744,"kb_used_data":2912,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738976,"statfs":{"total":773094113280,"available":772852711424,"internally_reserved":0,"allocated":2981888,"data_stored":1788966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002183"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021839+0000","last_change":"2023-12-02T00:28:49.717914+0000","last_active":"2023-12-02T00:28:51.021839+0000","last_peered":"2023-12-02T00:28:51.021839+0000","last_clean":"2023-12-02T00:28:51.021839+0000","last_became_active":"2023-12-02T00:28:49.717594+0000","last_became_peered":"2023-12-02T00:28:49.717594+0000","last_unstale":"2023-12-02T00:28:51.021839+0000","last_undegraded":"2023-12-02T00:28:51.021839+0000","last_fullsized":"2023-12-02T00:28:51.021839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:48:16.850673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703744+0000","last_change":"2023-12-02T00:28:49.713605+0000","last_active":"2023-12-02T00:28:50.703744+0000","last_peered":"2023-12-02T00:28:50.703744+0000","last_clean":"2023-12-02T00:28:50.703744+0000","last_became_active":"2023-12-02T00:28:49.712497+0000","last_became_peered":"2023-12-02T00:28:49.712497+0000","last_unstale":"2023-12-02T00:28:50.703744+0000","last_undegraded":"2023-12-02T00:28:50.703744+0000","last_fullsized":"2023-12-02T00:28:50.703744+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:15:51.747629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021917+0000","last_change":"2023-12-02T00:28:49.713065+0000","last_active":"2023-12-02T00:28:51.021917+0000","last_peered":"2023-12-02T00:28:51.021917+0000","last_clean":"2023-12-02T00:28:51.021917+0000","last_became_active":"2023-12-02T00:28:49.712715+0000","last_became_peered":"2023-12-02T00:28:49.712715+0000","last_unstale":"2023-12-02T00:28:51.021917+0000","last_undegraded":"2023-12-02T00:28:51.021917+0000","last_fullsized":"2023-12-02T00:28:51.021917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:52:22.600655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706337+0000","last_change":"2023-12-02T00:28:49.727831+0000","last_active":"2023-12-02T00:28:50.706337+0000","last_peered":"2023-12-02T00:28:50.706337+0000","last_clean":"2023-12-02T00:28:50.706337+0000","last_became_active":"2023-12-02T00:28:49.725804+0000","last_became_peered":"2023-12-02T00:28:49.725804+0000","last_unstale":"2023-12-02T00:28:50.706337+0000","last_undegraded":"2023-12-02T00:28:50.706337+0000","last_fullsized":"2023-12-02T00:28:50.706337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:50:45.793304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021332+0000","last_change":"2023-12-02T00:28:49.721391+0000","last_active":"2023-12-02T00:28:51.021332+0000","last_peered":"2023-12-02T00:28:51.021332+0000","last_clean":"2023-12-02T00:28:51.021332+0000","last_became_active":"2023-12-02T00:28:49.720314+0000","last_became_peered":"2023-12-02T00:28:49.720314+0000","last_unstale":"2023-12-02T00:28:51.021332+0000","last_undegraded":"2023-12-02T00:28:51.021332+0000","last_fullsized":"2023-12-02T00:28:51.021332+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:32:13.515968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705990+0000","last_change":"2023-12-02T00:28:49.725516+0000","last_active":"2023-12-02T00:28:50.705990+0000","last_peered":"2023-12-02T00:28:50.705990+0000","last_clean":"2023-12-02T00:28:50.705990+0000","last_became_active":"2023-12-02T00:28:49.725149+0000","last_became_peered":"2023-12-02T00:28:49.725149+0000","last_unstale":"2023-12-02T00:28:50.705990+0000","last_undegraded":"2023-12-02T00:28:50.705990+0000","last_fullsized":"2023-12-02T00:28:50.705990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T10:27:29.244181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021273+0000","last_change":"2023-12-02T00:28:49.719153+0000","last_active":"2023-12-02T00:28:51.021273+0000","last_peered":"2023-12-02T00:28:51.021273+0000","last_clean":"2023-12-02T00:28:51.021273+0000","last_became_active":"2023-12-02T00:28:49.718567+0000","last_became_peered":"2023-12-02T00:28:49.718567+0000","last_unstale":"2023-12-02T00:28:51.021273+0000","last_undegraded":"2023-12-02T00:28:51.021273+0000","last_fullsized":"2023-12-02T00:28:51.021273+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T10:27:29.244181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020683+0000","last_change":"2023-12-02T00:28:49.726986+0000","last_active":"2023-12-02T00:28:51.020683+0000","last_peered":"2023-12-02T00:28:51.020683+0000","last_clean":"2023-12-02T00:28:51.020683+0000","last_became_active":"2023-12-02T00:28:49.726646+0000","last_became_peered":"2023-12-02T00:28:49.726646+0000","last_unstale":"2023-12-02T00:28:51.020683+0000","last_undegraded":"2023-12-02T00:28:51.020683+0000","last_fullsized":"2023-12-02T00:28:51.020683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:35:09.231704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021827+0000","last_change":"2023-12-02T00:28:49.725402+0000","last_active":"2023-12-02T00:28:51.021827+0000","last_peered":"2023-12-02T00:28:51.021827+0000","last_clean":"2023-12-02T00:28:51.021827+0000","last_became_active":"2023-12-02T00:28:49.724978+0000","last_became_peered":"2023-12-02T00:28:49.724978+0000","last_unstale":"2023-12-02T00:28:51.021827+0000","last_undegraded":"2023-12-02T00:28:51.021827+0000","last_fullsized":"2023-12-02T00:28:51.021827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:18:31.615464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021500+0000","last_change":"2023-12-02T00:28:49.718464+0000","last_active":"2023-12-02T00:28:51.021500+0000","last_peered":"2023-12-02T00:28:51.021500+0000","last_clean":"2023-12-02T00:28:51.021500+0000","last_became_active":"2023-12-02T00:28:49.718060+0000","last_became_peered":"2023-12-02T00:28:49.718060+0000","last_unstale":"2023-12-02T00:28:51.021500+0000","last_undegraded":"2023-12-02T00:28:51.021500+0000","last_fullsized":"2023-12-02T00:28:51.021500+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:36:22.587928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021177+0000","last_change":"2023-12-02T00:28:49.720030+0000","last_active":"2023-12-02T00:28:51.021177+0000","last_peered":"2023-12-02T00:28:51.021177+0000","last_clean":"2023-12-02T00:28:51.021177+0000","last_became_active":"2023-12-02T00:28:49.719715+0000","last_became_peered":"2023-12-02T00:28:49.719715+0000","last_unstale":"2023-12-02T00:28:51.021177+0000","last_undegraded":"2023-12-02T00:28:51.021177+0000","last_fullsized":"2023-12-02T00:28:51.021177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:07:52.802590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705569+0000","last_change":"2023-12-02T00:28:49.713687+0000","last_active":"2023-12-02T00:28:50.705569+0000","last_peered":"2023-12-02T00:28:50.705569+0000","last_clean":"2023-12-02T00:28:50.705569+0000","last_became_active":"2023-12-02T00:28:49.713359+0000","last_became_peered":"2023-12-02T00:28:49.713359+0000","last_unstale":"2023-12-02T00:28:50.705569+0000","last_undegraded":"2023-12-02T00:28:50.705569+0000","last_fullsized":"2023-12-02T00:28:50.705569+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:11:06.388890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021465+0000","last_change":"2023-12-02T00:28:49.720866+0000","last_active":"2023-12-02T00:28:51.021465+0000","last_peered":"2023-12-02T00:28:51.021465+0000","last_clean":"2023-12-02T00:28:51.021465+0000","last_became_active":"2023-12-02T00:28:49.720524+0000","last_became_peered":"2023-12-02T00:28:49.720524+0000","last_unstale":"2023-12-02T00:28:51.021465+0000","last_undegraded":"2023-12-02T00:28:51.021465+0000","last_fullsized":"2023-12-02T00:28:51.021465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:32:13.515968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021720+0000","last_change":"2023-12-02T00:28:49.711387+0000","last_active":"2023-12-02T00:28:51.021720+0000","last_peered":"2023-12-02T00:28:51.021720+0000","last_clean":"2023-12-02T00:28:51.021720+0000","last_became_active":"2023-12-02T00:28:49.711036+0000","last_became_peered":"2023-12-02T00:28:49.711036+0000","last_unstale":"2023-12-02T00:28:51.021720+0000","last_undegraded":"2023-12-02T00:28:51.021720+0000","last_fullsized":"2023-12-02T00:28:51.021720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:13:36.512466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703511+0000","last_change":"2023-12-02T00:28:49.706444+0000","last_active":"2023-12-02T00:28:50.703511+0000","last_peered":"2023-12-02T00:28:50.703511+0000","last_clean":"2023-12-02T00:28:50.703511+0000","last_became_active":"2023-12-02T00:28:49.706132+0000","last_became_peered":"2023-12-02T00:28:49.706132+0000","last_unstale":"2023-12-02T00:28:50.703511+0000","last_undegraded":"2023-12-02T00:28:50.703511+0000","last_fullsized":"2023-12-02T00:28:50.703511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:52:22.600655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705451+0000","last_change":"2023-12-02T00:28:49.725045+0000","last_active":"2023-12-02T00:28:50.705451+0000","last_peered":"2023-12-02T00:28:50.705451+0000","last_clean":"2023-12-02T00:28:50.705451+0000","last_became_active":"2023-12-02T00:28:49.722884+0000","last_became_peered":"2023-12-02T00:28:49.722884+0000","last_unstale":"2023-12-02T00:28:50.705451+0000","last_undegraded":"2023-12-02T00:28:50.705451+0000","last_fullsized":"2023-12-02T00:28:50.705451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:36:22.587928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706621+0000","last_change":"2023-12-02T00:28:49.727232+0000","last_active":"2023-12-02T00:28:50.706621+0000","last_peered":"2023-12-02T00:28:50.706621+0000","last_clean":"2023-12-02T00:28:50.706621+0000","last_became_active":"2023-12-02T00:28:49.726881+0000","last_became_peered":"2023-12-02T00:28:49.726881+0000","last_unstale":"2023-12-02T00:28:50.706621+0000","last_undegraded":"2023-12-02T00:28:50.706621+0000","last_fullsized":"2023-12-02T00:28:50.706621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:48:16.850673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021328+0000","last_change":"2023-12-02T00:28:49.711953+0000","last_active":"2023-12-02T00:28:51.021328+0000","last_peered":"2023-12-02T00:28:51.021328+0000","last_clean":"2023-12-02T00:28:51.021328+0000","last_became_active":"2023-12-02T00:28:49.711653+0000","last_became_peered":"2023-12-02T00:28:49.711653+0000","last_unstale":"2023-12-02T00:28:51.021328+0000","last_undegraded":"2023-12-02T00:28:51.021328+0000","last_fullsized":"2023-12-02T00:28:51.021328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:52:08.689376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021886+0000","last_change":"2023-12-02T00:28:49.717365+0000","last_active":"2023-12-02T00:28:51.021886+0000","last_peered":"2023-12-02T00:28:51.021886+0000","last_clean":"2023-12-02T00:28:51.021886+0000","last_became_active":"2023-12-02T00:28:49.717071+0000","last_became_peered":"2023-12-02T00:28:49.717071+0000","last_unstale":"2023-12-02T00:28:51.021886+0000","last_undegraded":"2023-12-02T00:28:51.021886+0000","last_fullsized":"2023-12-02T00:28:51.021886+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:10:26.775049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021416+0000","last_change":"2023-12-02T00:28:49.716297+0000","last_active":"2023-12-02T00:28:51.021416+0000","last_peered":"2023-12-02T00:28:51.021416+0000","last_clean":"2023-12-02T00:28:51.021416+0000","last_became_active":"2023-12-02T00:28:49.715963+0000","last_became_peered":"2023-12-02T00:28:49.715963+0000","last_unstale":"2023-12-02T00:28:51.021416+0000","last_undegraded":"2023-12-02T00:28:51.021416+0000","last_fullsized":"2023-12-02T00:28:51.021416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:58:11.596095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021739+0000","last_change":"2023-12-02T00:28:49.722259+0000","last_active":"2023-12-02T00:28:51.021739+0000","last_peered":"2023-12-02T00:28:51.021739+0000","last_clean":"2023-12-02T00:28:51.021739+0000","last_became_active":"2023-12-02T00:28:49.721989+0000","last_became_peered":"2023-12-02T00:28:49.721989+0000","last_unstale":"2023-12-02T00:28:51.021739+0000","last_undegraded":"2023-12-02T00:28:51.021739+0000","last_fullsized":"2023-12-02T00:28:51.021739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:55:55.141283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705987+0000","last_change":"2023-12-02T00:28:49.729526+0000","last_active":"2023-12-02T00:28:50.705987+0000","last_peered":"2023-12-02T00:28:50.705987+0000","last_clean":"2023-12-02T00:28:50.705987+0000","last_became_active":"2023-12-02T00:28:49.728999+0000","last_became_peered":"2023-12-02T00:28:49.728999+0000","last_unstale":"2023-12-02T00:28:50.705987+0000","last_undegraded":"2023-12-02T00:28:50.705987+0000","last_fullsized":"2023-12-02T00:28:50.705987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:58:15.388237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021070+0000","last_change":"2023-12-02T00:28:49.717936+0000","last_active":"2023-12-02T00:28:51.021070+0000","last_peered":"2023-12-02T00:28:51.021070+0000","last_clean":"2023-12-02T00:28:51.021070+0000","last_became_active":"2023-12-02T00:28:49.716770+0000","last_became_peered":"2023-12-02T00:28:49.716770+0000","last_unstale":"2023-12-02T00:28:51.021070+0000","last_undegraded":"2023-12-02T00:28:51.021070+0000","last_fullsized":"2023-12-02T00:28:51.021070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:36:22.587928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705591+0000","last_change":"2023-12-02T00:28:49.724863+0000","last_active":"2023-12-02T00:28:50.705591+0000","last_peered":"2023-12-02T00:28:50.705591+0000","last_clean":"2023-12-02T00:28:50.705591+0000","last_became_active":"2023-12-02T00:28:49.724592+0000","last_became_peered":"2023-12-02T00:28:49.724592+0000","last_unstale":"2023-12-02T00:28:50.705591+0000","last_undegraded":"2023-12-02T00:28:50.705591+0000","last_fullsized":"2023-12-02T00:28:50.705591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:15:51.747629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020786+0000","last_change":"2023-12-02T00:28:49.724283+0000","last_active":"2023-12-02T00:28:51.020786+0000","last_peered":"2023-12-02T00:28:51.020786+0000","last_clean":"2023-12-02T00:28:51.020786+0000","last_became_active":"2023-12-02T00:28:49.723940+0000","last_became_peered":"2023-12-02T00:28:49.723940+0000","last_unstale":"2023-12-02T00:28:51.020786+0000","last_undegraded":"2023-12-02T00:28:51.020786+0000","last_fullsized":"2023-12-02T00:28:51.020786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:52:22.600655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021632+0000","last_change":"2023-12-02T00:28:49.717015+0000","last_active":"2023-12-02T00:28:51.021632+0000","last_peered":"2023-12-02T00:28:51.021632+0000","last_clean":"2023-12-02T00:28:51.021632+0000","last_became_active":"2023-12-02T00:28:49.716394+0000","last_became_peered":"2023-12-02T00:28:49.716394+0000","last_unstale":"2023-12-02T00:28:51.021632+0000","last_undegraded":"2023-12-02T00:28:51.021632+0000","last_fullsized":"2023-12-02T00:28:51.021632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:15:51.747629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703687+0000","last_change":"2023-12-02T00:28:49.716707+0000","last_active":"2023-12-02T00:28:50.703687+0000","last_peered":"2023-12-02T00:28:50.703687+0000","last_clean":"2023-12-02T00:28:50.703687+0000","last_became_active":"2023-12-02T00:28:49.716392+0000","last_became_peered":"2023-12-02T00:28:49.716392+0000","last_unstale":"2023-12-02T00:28:50.703687+0000","last_undegraded":"2023-12-02T00:28:50.703687+0000","last_fullsized":"2023-12-02T00:28:50.703687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:36:22.587928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021477+0000","last_change":"2023-12-02T00:28:49.724290+0000","last_active":"2023-12-02T00:28:51.021477+0000","last_peered":"2023-12-02T00:28:51.021477+0000","last_clean":"2023-12-02T00:28:51.021477+0000","last_became_active":"2023-12-02T00:28:49.723952+0000","last_became_peered":"2023-12-02T00:28:49.723952+0000","last_unstale":"2023-12-02T00:28:51.021477+0000","last_undegraded":"2023-12-02T00:28:51.021477+0000","last_fullsized":"2023-12-02T00:28:51.021477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:15:02.988933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021881+0000","last_change":"2023-12-02T00:28:49.721218+0000","last_active":"2023-12-02T00:28:51.021881+0000","last_peered":"2023-12-02T00:28:51.021881+0000","last_clean":"2023-12-02T00:28:51.021881+0000","last_became_active":"2023-12-02T00:28:49.720894+0000","last_became_peered":"2023-12-02T00:28:49.720894+0000","last_unstale":"2023-12-02T00:28:51.021881+0000","last_undegraded":"2023-12-02T00:28:51.021881+0000","last_fullsized":"2023-12-02T00:28:51.021881+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:35:09.231704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706145+0000","last_change":"2023-12-02T00:28:49.725424+0000","last_active":"2023-12-02T00:28:50.706145+0000","last_peered":"2023-12-02T00:28:50.706145+0000","last_clean":"2023-12-02T00:28:50.706145+0000","last_became_active":"2023-12-02T00:28:49.724671+0000","last_became_peered":"2023-12-02T00:28:49.724671+0000","last_unstale":"2023-12-02T00:28:50.706145+0000","last_undegraded":"2023-12-02T00:28:50.706145+0000","last_fullsized":"2023-12-02T00:28:50.706145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:32:13.515968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021246+0000","last_change":"2023-12-02T00:28:49.722395+0000","last_active":"2023-12-02T00:28:51.021246+0000","last_peered":"2023-12-02T00:28:51.021246+0000","last_clean":"2023-12-02T00:28:51.021246+0000","last_became_active":"2023-12-02T00:28:49.722052+0000","last_became_peered":"2023-12-02T00:28:49.722052+0000","last_unstale":"2023-12-02T00:28:51.021246+0000","last_undegraded":"2023-12-02T00:28:51.021246+0000","last_fullsized":"2023-12-02T00:28:51.021246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:11:06.388890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020869+0000","last_change":"2023-12-02T00:28:49.713618+0000","last_active":"2023-12-02T00:28:51.020869+0000","last_peered":"2023-12-02T00:28:51.020869+0000","last_clean":"2023-12-02T00:28:51.020869+0000","last_became_active":"2023-12-02T00:28:49.713325+0000","last_became_peered":"2023-12-02T00:28:49.713325+0000","last_unstale":"2023-12-02T00:28:51.020869+0000","last_undegraded":"2023-12-02T00:28:51.020869+0000","last_fullsized":"2023-12-02T00:28:51.020869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:35:09.231704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705601+0000","last_change":"2023-12-02T00:28:49.710604+0000","last_active":"2023-12-02T00:28:50.705601+0000","last_peered":"2023-12-02T00:28:50.705601+0000","last_clean":"2023-12-02T00:28:50.705601+0000","last_became_active":"2023-12-02T00:28:49.710316+0000","last_became_peered":"2023-12-02T00:28:49.710316+0000","last_unstale":"2023-12-02T00:28:50.705601+0000","last_undegraded":"2023-12-02T00:28:50.705601+0000","last_fullsized":"2023-12-02T00:28:50.705601+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:52:08.689376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021549+0000","last_change":"2023-12-02T00:28:48.735813+0000","last_active":"2023-12-02T00:28:51.021549+0000","last_peered":"2023-12-02T00:28:51.021549+0000","last_clean":"2023-12-02T00:28:51.021549+0000","last_became_active":"2023-12-02T00:28:48.735462+0000","last_became_peered":"2023-12-02T00:28:48.735462+0000","last_unstale":"2023-12-02T00:28:51.021549+0000","last_undegraded":"2023-12-02T00:28:51.021549+0000","last_fullsized":"2023-12-02T00:28:51.021549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:55:54.133908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.022018+0000","last_change":"2023-12-02T00:28:49.715318+0000","last_active":"2023-12-02T00:28:51.022018+0000","last_peered":"2023-12-02T00:28:51.022018+0000","last_clean":"2023-12-02T00:28:51.022018+0000","last_became_active":"2023-12-02T00:28:49.715020+0000","last_became_peered":"2023-12-02T00:28:49.715020+0000","last_unstale":"2023-12-02T00:28:51.022018+0000","last_undegraded":"2023-12-02T00:28:51.022018+0000","last_fullsized":"2023-12-02T00:28:51.022018+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:36:22.587928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021150+0000","last_change":"2023-12-02T00:28:48.730625+0000","last_active":"2023-12-02T00:28:51.021150+0000","last_peered":"2023-12-02T00:28:51.021150+0000","last_clean":"2023-12-02T00:28:51.021150+0000","last_became_active":"2023-12-02T00:28:48.730306+0000","last_became_peered":"2023-12-02T00:28:48.730306+0000","last_unstale":"2023-12-02T00:28:51.021150+0000","last_undegraded":"2023-12-02T00:28:51.021150+0000","last_fullsized":"2023-12-02T00:28:51.021150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:11:05.381516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020857+0000","last_change":"2023-12-02T00:28:48.735160+0000","last_active":"2023-12-02T00:28:51.020857+0000","last_peered":"2023-12-02T00:28:51.020857+0000","last_clean":"2023-12-02T00:28:51.020857+0000","last_became_active":"2023-12-02T00:28:48.734860+0000","last_became_peered":"2023-12-02T00:28:48.734860+0000","last_unstale":"2023-12-02T00:28:51.020857+0000","last_undegraded":"2023-12-02T00:28:51.020857+0000","last_fullsized":"2023-12-02T00:28:51.020857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T09:17:51.234416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020811+0000","last_change":"2023-12-02T00:28:49.716640+0000","last_active":"2023-12-02T00:28:51.020811+0000","last_peered":"2023-12-02T00:28:51.020811+0000","last_clean":"2023-12-02T00:28:51.020811+0000","last_became_active":"2023-12-02T00:28:49.716325+0000","last_became_peered":"2023-12-02T00:28:49.716325+0000","last_unstale":"2023-12-02T00:28:51.020811+0000","last_undegraded":"2023-12-02T00:28:51.020811+0000","last_fullsized":"2023-12-02T00:28:51.020811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:07:52.802590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705691+0000","last_change":"2023-12-02T00:28:48.730403+0000","last_active":"2023-12-02T00:28:50.705691+0000","last_peered":"2023-12-02T00:28:50.705691+0000","last_clean":"2023-12-02T00:28:50.705691+0000","last_became_active":"2023-12-02T00:28:48.730053+0000","last_became_peered":"2023-12-02T00:28:48.730053+0000","last_unstale":"2023-12-02T00:28:50.705691+0000","last_undegraded":"2023-12-02T00:28:50.705691+0000","last_fullsized":"2023-12-02T00:28:50.705691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:13:35.505091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703512+0000","last_change":"2023-12-02T00:28:49.716925+0000","last_active":"2023-12-02T00:28:50.703512+0000","last_peered":"2023-12-02T00:28:50.703512+0000","last_clean":"2023-12-02T00:28:50.703512+0000","last_became_active":"2023-12-02T00:28:49.716530+0000","last_became_peered":"2023-12-02T00:28:49.716530+0000","last_unstale":"2023-12-02T00:28:50.703512+0000","last_undegraded":"2023-12-02T00:28:50.703512+0000","last_fullsized":"2023-12-02T00:28:50.703512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:35:09.231704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021002+0000","last_change":"2023-12-02T00:28:48.732302+0000","last_active":"2023-12-02T00:28:51.021002+0000","last_peered":"2023-12-02T00:28:51.021002+0000","last_clean":"2023-12-02T00:28:51.021002+0000","last_became_active":"2023-12-02T00:28:48.731891+0000","last_became_peered":"2023-12-02T00:28:48.731891+0000","last_unstale":"2023-12-02T00:28:51.021002+0000","last_undegraded":"2023-12-02T00:28:51.021002+0000","last_fullsized":"2023-12-02T00:28:51.021002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:44:45.468077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706568+0000","last_change":"2023-12-02T00:28:49.719085+0000","last_active":"2023-12-02T00:28:50.706568+0000","last_peered":"2023-12-02T00:28:50.706568+0000","last_clean":"2023-12-02T00:28:50.706568+0000","last_became_active":"2023-12-02T00:28:49.718606+0000","last_became_peered":"2023-12-02T00:28:49.718606+0000","last_unstale":"2023-12-02T00:28:50.706568+0000","last_undegraded":"2023-12-02T00:28:50.706568+0000","last_fullsized":"2023-12-02T00:28:50.706568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:18:31.615464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706430+0000","last_change":"2023-12-02T00:28:48.732243+0000","last_active":"2023-12-02T00:28:50.706430+0000","last_peered":"2023-12-02T00:28:50.706430+0000","last_clean":"2023-12-02T00:28:50.706430+0000","last_became_active":"2023-12-02T00:28:48.731895+0000","last_became_peered":"2023-12-02T00:28:48.731895+0000","last_unstale":"2023-12-02T00:28:50.706430+0000","last_undegraded":"2023-12-02T00:28:50.706430+0000","last_fullsized":"2023-12-02T00:28:50.706430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:55:54.133908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021998+0000","last_change":"2023-12-02T00:28:49.718319+0000","last_active":"2023-12-02T00:28:51.021998+0000","last_peered":"2023-12-02T00:28:51.021998+0000","last_clean":"2023-12-02T00:28:51.021998+0000","last_became_active":"2023-12-02T00:28:49.718045+0000","last_became_peered":"2023-12-02T00:28:49.718045+0000","last_unstale":"2023-12-02T00:28:51.021998+0000","last_undegraded":"2023-12-02T00:28:51.021998+0000","last_fullsized":"2023-12-02T00:28:51.021998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:58:11.596095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703791+0000","last_change":"2023-12-02T00:28:48.728008+0000","last_active":"2023-12-02T00:28:50.703791+0000","last_peered":"2023-12-02T00:28:50.703791+0000","last_clean":"2023-12-02T00:28:50.703791+0000","last_became_active":"2023-12-02T00:28:48.727662+0000","last_became_peered":"2023-12-02T00:28:48.727662+0000","last_unstale":"2023-12-02T00:28:50.703791+0000","last_undegraded":"2023-12-02T00:28:50.703791+0000","last_fullsized":"2023-12-02T00:28:50.703791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:13:35.505091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705692+0000","last_change":"2023-12-02T00:28:49.719391+0000","last_active":"2023-12-02T00:28:50.705692+0000","last_peered":"2023-12-02T00:28:50.705692+0000","last_clean":"2023-12-02T00:28:50.705692+0000","last_became_active":"2023-12-02T00:28:49.719126+0000","last_became_peered":"2023-12-02T00:28:49.719126+0000","last_unstale":"2023-12-02T00:28:50.705692+0000","last_undegraded":"2023-12-02T00:28:50.705692+0000","last_fullsized":"2023-12-02T00:28:50.705692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:47:09.825514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021521+0000","last_change":"2023-12-02T00:28:48.736811+0000","last_active":"2023-12-02T00:28:51.021521+0000","last_peered":"2023-12-02T00:28:51.021521+0000","last_clean":"2023-12-02T00:28:51.021521+0000","last_became_active":"2023-12-02T00:28:48.736576+0000","last_became_peered":"2023-12-02T00:28:48.736576+0000","last_unstale":"2023-12-02T00:28:51.021521+0000","last_undegraded":"2023-12-02T00:28:51.021521+0000","last_fullsized":"2023-12-02T00:28:51.021521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:52:07.682002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021600+0000","last_change":"2023-12-02T00:28:49.717541+0000","last_active":"2023-12-02T00:28:51.021600+0000","last_peered":"2023-12-02T00:28:51.021600+0000","last_clean":"2023-12-02T00:28:51.021600+0000","last_became_active":"2023-12-02T00:28:49.717225+0000","last_became_peered":"2023-12-02T00:28:49.717225+0000","last_unstale":"2023-12-02T00:28:51.021600+0000","last_undegraded":"2023-12-02T00:28:51.021600+0000","last_fullsized":"2023-12-02T00:28:51.021600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:29:16.078935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.022091+0000","last_change":"2023-12-02T00:28:48.725581+0000","last_active":"2023-12-02T00:28:51.022091+0000","last_peered":"2023-12-02T00:28:51.022091+0000","last_clean":"2023-12-02T00:28:51.022091+0000","last_became_active":"2023-12-02T00:28:48.725230+0000","last_became_peered":"2023-12-02T00:28:48.725230+0000","last_unstale":"2023-12-02T00:28:51.022091+0000","last_undegraded":"2023-12-02T00:28:51.022091+0000","last_fullsized":"2023-12-02T00:28:51.022091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:39:30.211229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021761+0000","last_change":"2023-12-02T00:28:49.716591+0000","last_active":"2023-12-02T00:28:51.021761+0000","last_peered":"2023-12-02T00:28:51.021761+0000","last_clean":"2023-12-02T00:28:51.021761+0000","last_became_active":"2023-12-02T00:28:49.716237+0000","last_became_peered":"2023-12-02T00:28:49.716237+0000","last_unstale":"2023-12-02T00:28:51.021761+0000","last_undegraded":"2023-12-02T00:28:51.021761+0000","last_fullsized":"2023-12-02T00:28:51.021761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T12:06:34.965686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706099+0000","last_change":"2023-12-02T00:28:48.734778+0000","last_active":"2023-12-02T00:28:50.706099+0000","last_peered":"2023-12-02T00:28:50.706099+0000","last_clean":"2023-12-02T00:28:50.706099+0000","last_became_active":"2023-12-02T00:28:48.734474+0000","last_became_peered":"2023-12-02T00:28:48.734474+0000","last_unstale":"2023-12-02T00:28:50.706099+0000","last_undegraded":"2023-12-02T00:28:50.706099+0000","last_fullsized":"2023-12-02T00:28:50.706099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:07:51.795216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021342+0000","last_change":"2023-12-02T00:28:49.721274+0000","last_active":"2023-12-02T00:28:51.021342+0000","last_peered":"2023-12-02T00:28:51.021342+0000","last_clean":"2023-12-02T00:28:51.021342+0000","last_became_active":"2023-12-02T00:28:49.719979+0000","last_became_peered":"2023-12-02T00:28:49.719979+0000","last_unstale":"2023-12-02T00:28:51.021342+0000","last_undegraded":"2023-12-02T00:28:51.021342+0000","last_fullsized":"2023-12-02T00:28:51.021342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:50:45.793304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020850+0000","last_change":"2023-12-02T00:28:48.732958+0000","last_active":"2023-12-02T00:28:51.020850+0000","last_peered":"2023-12-02T00:28:51.020850+0000","last_clean":"2023-12-02T00:28:51.020850+0000","last_became_active":"2023-12-02T00:28:48.732651+0000","last_became_peered":"2023-12-02T00:28:48.732651+0000","last_unstale":"2023-12-02T00:28:51.020850+0000","last_undegraded":"2023-12-02T00:28:51.020850+0000","last_fullsized":"2023-12-02T00:28:51.020850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:29:26.381170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020869+0000","last_change":"2023-12-02T00:28:49.712859+0000","last_active":"2023-12-02T00:28:51.020869+0000","last_peered":"2023-12-02T00:28:51.020869+0000","last_clean":"2023-12-02T00:28:51.020869+0000","last_became_active":"2023-12-02T00:28:49.712585+0000","last_became_peered":"2023-12-02T00:28:49.712585+0000","last_unstale":"2023-12-02T00:28:51.020869+0000","last_undegraded":"2023-12-02T00:28:51.020869+0000","last_fullsized":"2023-12-02T00:28:51.020869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T12:06:34.965686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020660+0000","last_change":"2023-12-02T00:28:48.734113+0000","last_active":"2023-12-02T00:28:51.020660+0000","last_peered":"2023-12-02T00:28:51.020660+0000","last_clean":"2023-12-02T00:28:51.020660+0000","last_became_active":"2023-12-02T00:28:48.733749+0000","last_became_peered":"2023-12-02T00:28:48.733749+0000","last_unstale":"2023-12-02T00:28:51.020660+0000","last_undegraded":"2023-12-02T00:28:51.020660+0000","last_fullsized":"2023-12-02T00:28:51.020660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T10:56:02.550120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021416+0000","last_change":"2023-12-02T00:28:49.719477+0000","last_active":"2023-12-02T00:28:51.021416+0000","last_peered":"2023-12-02T00:28:51.021416+0000","last_clean":"2023-12-02T00:28:51.021416+0000","last_became_active":"2023-12-02T00:28:49.718537+0000","last_became_peered":"2023-12-02T00:28:49.718537+0000","last_unstale":"2023-12-02T00:28:51.021416+0000","last_undegraded":"2023-12-02T00:28:51.021416+0000","last_fullsized":"2023-12-02T00:28:51.021416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:15:02.988933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705801+0000","last_change":"2023-12-02T00:28:48.730372+0000","last_active":"2023-12-02T00:28:50.705801+0000","last_peered":"2023-12-02T00:28:50.705801+0000","last_clean":"2023-12-02T00:28:50.705801+0000","last_became_active":"2023-12-02T00:28:48.730051+0000","last_became_peered":"2023-12-02T00:28:48.730051+0000","last_unstale":"2023-12-02T00:28:50.705801+0000","last_undegraded":"2023-12-02T00:28:50.705801+0000","last_fullsized":"2023-12-02T00:28:50.705801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:29:10.550270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021515+0000","last_change":"2023-12-02T00:28:49.714467+0000","last_active":"2023-12-02T00:28:51.021515+0000","last_peered":"2023-12-02T00:28:51.021515+0000","last_clean":"2023-12-02T00:28:51.021515+0000","last_became_active":"2023-12-02T00:28:49.713964+0000","last_became_peered":"2023-12-02T00:28:49.713964+0000","last_unstale":"2023-12-02T00:28:51.021515+0000","last_undegraded":"2023-12-02T00:28:51.021515+0000","last_fullsized":"2023-12-02T00:28:51.021515+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T12:22:41.666941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021286+0000","last_change":"2023-12-02T00:28:48.736447+0000","last_active":"2023-12-02T00:28:51.021286+0000","last_peered":"2023-12-02T00:28:51.021286+0000","last_clean":"2023-12-02T00:28:51.021286+0000","last_became_active":"2023-12-02T00:28:48.736169+0000","last_became_peered":"2023-12-02T00:28:48.736169+0000","last_unstale":"2023-12-02T00:28:51.021286+0000","last_undegraded":"2023-12-02T00:28:51.021286+0000","last_fullsized":"2023-12-02T00:28:51.021286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T09:17:51.234416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021954+0000","last_change":"2023-12-02T00:28:49.719154+0000","last_active":"2023-12-02T00:28:51.021954+0000","last_peered":"2023-12-02T00:28:51.021954+0000","last_clean":"2023-12-02T00:28:51.021954+0000","last_became_active":"2023-12-02T00:28:49.718713+0000","last_became_peered":"2023-12-02T00:28:49.718713+0000","last_unstale":"2023-12-02T00:28:51.021954+0000","last_undegraded":"2023-12-02T00:28:51.021954+0000","last_fullsized":"2023-12-02T00:28:51.021954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:50:17.285633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703845+0000","last_change":"2023-12-02T00:28:48.731504+0000","last_active":"2023-12-02T00:28:50.703845+0000","last_peered":"2023-12-02T00:28:50.703845+0000","last_clean":"2023-12-02T00:28:50.703845+0000","last_became_active":"2023-12-02T00:28:48.731224+0000","last_became_peered":"2023-12-02T00:28:48.731224+0000","last_unstale":"2023-12-02T00:28:50.703845+0000","last_undegraded":"2023-12-02T00:28:50.703845+0000","last_fullsized":"2023-12-02T00:28:50.703845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:55:54.133908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706503+0000","last_change":"2023-12-02T00:28:49.726089+0000","last_active":"2023-12-02T00:28:50.706503+0000","last_peered":"2023-12-02T00:28:50.706503+0000","last_clean":"2023-12-02T00:28:50.706503+0000","last_became_active":"2023-12-02T00:28:49.725482+0000","last_became_peered":"2023-12-02T00:28:49.725482+0000","last_unstale":"2023-12-02T00:28:50.706503+0000","last_undegraded":"2023-12-02T00:28:50.706503+0000","last_fullsized":"2023-12-02T00:28:50.706503+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T12:22:41.666941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020880+0000","last_change":"2023-12-02T00:28:48.729936+0000","last_active":"2023-12-02T00:28:51.020880+0000","last_peered":"2023-12-02T00:28:51.020880+0000","last_clean":"2023-12-02T00:28:51.020880+0000","last_became_active":"2023-12-02T00:28:48.729617+0000","last_became_peered":"2023-12-02T00:28:48.729617+0000","last_unstale":"2023-12-02T00:28:51.020880+0000","last_undegraded":"2023-12-02T00:28:51.020880+0000","last_fullsized":"2023-12-02T00:28:51.020880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:29:26.381170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021803+0000","last_change":"2023-12-02T00:28:49.725211+0000","last_active":"2023-12-02T00:28:51.021803+0000","last_peered":"2023-12-02T00:28:51.021803+0000","last_clean":"2023-12-02T00:28:51.021803+0000","last_became_active":"2023-12-02T00:28:49.724928+0000","last_became_peered":"2023-12-02T00:28:49.724928+0000","last_unstale":"2023-12-02T00:28:51.021803+0000","last_undegraded":"2023-12-02T00:28:51.021803+0000","last_fullsized":"2023-12-02T00:28:51.021803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:33:43.085260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":165,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-02T00:28:49.708269+0000","last_change":"2023-12-02T00:28:39.379380+0000","last_active":"2023-12-02T00:28:49.708269+0000","last_peered":"2023-12-02T00:28:49.708269+0000","last_clean":"2023-12-02T00:28:49.708269+0000","last_became_active":"2023-12-02T00:28:39.378255+0000","last_became_peered":"2023-12-02T00:28:39.378255+0000","last_unstale":"2023-12-02T00:28:49.708269+0000","last_undegraded":"2023-12-02T00:28:49.708269+0000","last_fullsized":"2023-12-02T00:28:49.708269+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:38.177345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:38.177345+0000","last_clean_scrub_stamp":"2023-12-02T00:28:38.177345+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:21:08.582412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706107+0000","last_change":"2023-12-02T00:28:48.731326+0000","last_active":"2023-12-02T00:28:50.706107+0000","last_peered":"2023-12-02T00:28:50.706107+0000","last_clean":"2023-12-02T00:28:50.706107+0000","last_became_active":"2023-12-02T00:28:48.730970+0000","last_became_peered":"2023-12-02T00:28:48.730970+0000","last_unstale":"2023-12-02T00:28:50.706107+0000","last_undegraded":"2023-12-02T00:28:50.706107+0000","last_fullsized":"2023-12-02T00:28:50.706107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:44:45.468077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021403+0000","last_change":"2023-12-02T00:28:49.727341+0000","last_active":"2023-12-02T00:28:51.021403+0000","last_peered":"2023-12-02T00:28:51.021403+0000","last_clean":"2023-12-02T00:28:51.021403+0000","last_became_active":"2023-12-02T00:28:49.727045+0000","last_became_peered":"2023-12-02T00:28:49.727045+0000","last_unstale":"2023-12-02T00:28:51.021403+0000","last_undegraded":"2023-12-02T00:28:51.021403+0000","last_fullsized":"2023-12-02T00:28:51.021403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:58:15.388237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020921+0000","last_change":"2023-12-02T00:28:48.732719+0000","last_active":"2023-12-02T00:28:51.020921+0000","last_peered":"2023-12-02T00:28:51.020921+0000","last_clean":"2023-12-02T00:28:51.020921+0000","last_became_active":"2023-12-02T00:28:48.732466+0000","last_became_peered":"2023-12-02T00:28:48.732466+0000","last_unstale":"2023-12-02T00:28:51.020921+0000","last_undegraded":"2023-12-02T00:28:51.020921+0000","last_fullsized":"2023-12-02T00:28:51.020921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:52:21.593280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021611+0000","last_change":"2023-12-02T00:28:49.716043+0000","last_active":"2023-12-02T00:28:51.021611+0000","last_peered":"2023-12-02T00:28:51.021611+0000","last_clean":"2023-12-02T00:28:51.021611+0000","last_became_active":"2023-12-02T00:28:49.715747+0000","last_became_peered":"2023-12-02T00:28:49.715747+0000","last_unstale":"2023-12-02T00:28:51.021611+0000","last_undegraded":"2023-12-02T00:28:51.021611+0000","last_fullsized":"2023-12-02T00:28:51.021611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:44:46.475452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021208+0000","last_change":"2023-12-02T00:28:48.736024+0000","last_active":"2023-12-02T00:28:51.021208+0000","last_peered":"2023-12-02T00:28:51.021208+0000","last_clean":"2023-12-02T00:28:51.021208+0000","last_became_active":"2023-12-02T00:28:48.735736+0000","last_became_peered":"2023-12-02T00:28:48.735736+0000","last_unstale":"2023-12-02T00:28:51.021208+0000","last_undegraded":"2023-12-02T00:28:51.021208+0000","last_fullsized":"2023-12-02T00:28:51.021208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:55:48.025090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703534+0000","last_change":"2023-12-02T00:28:49.707755+0000","last_active":"2023-12-02T00:28:50.703534+0000","last_peered":"2023-12-02T00:28:50.703534+0000","last_clean":"2023-12-02T00:28:50.703534+0000","last_became_active":"2023-12-02T00:28:49.707449+0000","last_became_peered":"2023-12-02T00:28:49.707449+0000","last_unstale":"2023-12-02T00:28:50.703534+0000","last_undegraded":"2023-12-02T00:28:50.703534+0000","last_fullsized":"2023-12-02T00:28:50.703534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:07:52.802590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706217+0000","last_change":"2023-12-02T00:28:48.729078+0000","last_active":"2023-12-02T00:28:50.706217+0000","last_peered":"2023-12-02T00:28:50.706217+0000","last_clean":"2023-12-02T00:28:50.706217+0000","last_became_active":"2023-12-02T00:28:48.728567+0000","last_became_peered":"2023-12-02T00:28:48.728567+0000","last_unstale":"2023-12-02T00:28:50.706217+0000","last_undegraded":"2023-12-02T00:28:50.706217+0000","last_fullsized":"2023-12-02T00:28:50.706217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:58:10.588720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705483+0000","last_change":"2023-12-02T00:28:49.712801+0000","last_active":"2023-12-02T00:28:50.705483+0000","last_peered":"2023-12-02T00:28:50.705483+0000","last_clean":"2023-12-02T00:28:50.705483+0000","last_became_active":"2023-12-02T00:28:49.712511+0000","last_became_peered":"2023-12-02T00:28:49.712511+0000","last_unstale":"2023-12-02T00:28:50.705483+0000","last_undegraded":"2023-12-02T00:28:50.705483+0000","last_fullsized":"2023-12-02T00:28:50.705483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:55:49.032465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703749+0000","last_change":"2023-12-02T00:28:48.730774+0000","last_active":"2023-12-02T00:28:50.703749+0000","last_peered":"2023-12-02T00:28:50.703749+0000","last_clean":"2023-12-02T00:28:50.703749+0000","last_became_active":"2023-12-02T00:28:48.730498+0000","last_became_peered":"2023-12-02T00:28:48.730498+0000","last_unstale":"2023-12-02T00:28:50.703749+0000","last_undegraded":"2023-12-02T00:28:50.703749+0000","last_fullsized":"2023-12-02T00:28:50.703749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:29:10.550270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706537+0000","last_change":"2023-12-02T00:28:49.726928+0000","last_active":"2023-12-02T00:28:50.706537+0000","last_peered":"2023-12-02T00:28:50.706537+0000","last_clean":"2023-12-02T00:28:50.706537+0000","last_became_active":"2023-12-02T00:28:49.725033+0000","last_became_peered":"2023-12-02T00:28:49.725033+0000","last_unstale":"2023-12-02T00:28:50.706537+0000","last_undegraded":"2023-12-02T00:28:50.706537+0000","last_fullsized":"2023-12-02T00:28:50.706537+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:33:43.085260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021906+0000","last_change":"2023-12-02T00:28:48.735559+0000","last_active":"2023-12-02T00:28:51.021906+0000","last_peered":"2023-12-02T00:28:51.021906+0000","last_clean":"2023-12-02T00:28:51.021906+0000","last_became_active":"2023-12-02T00:28:48.735243+0000","last_became_peered":"2023-12-02T00:28:48.735243+0000","last_unstale":"2023-12-02T00:28:51.021906+0000","last_undegraded":"2023-12-02T00:28:51.021906+0000","last_fullsized":"2023-12-02T00:28:51.021906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:07:51.795216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703595+0000","last_change":"2023-12-02T00:28:49.710546+0000","last_active":"2023-12-02T00:28:50.703595+0000","last_peered":"2023-12-02T00:28:50.703595+0000","last_clean":"2023-12-02T00:28:50.703595+0000","last_became_active":"2023-12-02T00:28:49.710010+0000","last_became_peered":"2023-12-02T00:28:49.710010+0000","last_unstale":"2023-12-02T00:28:50.703595+0000","last_undegraded":"2023-12-02T00:28:50.703595+0000","last_fullsized":"2023-12-02T00:28:50.703595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:58:11.596095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.172997+0000","last_change":"2023-12-02T00:28:48.731556+0000","last_active":"2023-12-02T00:28:51.172997+0000","last_peered":"2023-12-02T00:28:51.172997+0000","last_clean":"2023-12-02T00:28:51.172997+0000","last_became_active":"2023-12-02T00:28:48.731291+0000","last_became_peered":"2023-12-02T00:28:48.731291+0000","last_unstale":"2023-12-02T00:28:51.172997+0000","last_undegraded":"2023-12-02T00:28:51.172997+0000","last_fullsized":"2023-12-02T00:28:51.172997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:39:30.211229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020929+0000","last_change":"2023-12-02T00:28:49.723678+0000","last_active":"2023-12-02T00:28:51.020929+0000","last_peered":"2023-12-02T00:28:51.020929+0000","last_clean":"2023-12-02T00:28:51.020929+0000","last_became_active":"2023-12-02T00:28:49.723299+0000","last_became_peered":"2023-12-02T00:28:49.723299+0000","last_unstale":"2023-12-02T00:28:51.020929+0000","last_undegraded":"2023-12-02T00:28:51.020929+0000","last_fullsized":"2023-12-02T00:28:51.020929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:58:11.596095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021044+0000","last_change":"2023-12-02T00:28:48.735741+0000","last_active":"2023-12-02T00:28:51.021044+0000","last_peered":"2023-12-02T00:28:51.021044+0000","last_clean":"2023-12-02T00:28:51.021044+0000","last_became_active":"2023-12-02T00:28:48.735400+0000","last_became_peered":"2023-12-02T00:28:48.735400+0000","last_unstale":"2023-12-02T00:28:51.021044+0000","last_undegraded":"2023-12-02T00:28:51.021044+0000","last_fullsized":"2023-12-02T00:28:51.021044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:13:35.505091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.022028+0000","last_change":"2023-12-02T00:28:49.711592+0000","last_active":"2023-12-02T00:28:51.022028+0000","last_peered":"2023-12-02T00:28:51.022028+0000","last_clean":"2023-12-02T00:28:51.022028+0000","last_became_active":"2023-12-02T00:28:49.711211+0000","last_became_peered":"2023-12-02T00:28:49.711211+0000","last_unstale":"2023-12-02T00:28:51.022028+0000","last_undegraded":"2023-12-02T00:28:51.022028+0000","last_fullsized":"2023-12-02T00:28:51.022028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:15:51.747629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020816+0000","last_change":"2023-12-02T00:28:48.735722+0000","last_active":"2023-12-02T00:28:51.020816+0000","last_peered":"2023-12-02T00:28:51.020816+0000","last_clean":"2023-12-02T00:28:51.020816+0000","last_became_active":"2023-12-02T00:28:48.735212+0000","last_became_peered":"2023-12-02T00:28:48.735212+0000","last_unstale":"2023-12-02T00:28:51.020816+0000","last_undegraded":"2023-12-02T00:28:51.020816+0000","last_fullsized":"2023-12-02T00:28:51.020816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:47:08.818139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705309+0000","last_change":"2023-12-02T00:28:49.711824+0000","last_active":"2023-12-02T00:28:50.705309+0000","last_peered":"2023-12-02T00:28:50.705309+0000","last_clean":"2023-12-02T00:28:50.705309+0000","last_became_active":"2023-12-02T00:28:49.710946+0000","last_became_peered":"2023-12-02T00:28:49.710946+0000","last_unstale":"2023-12-02T00:28:50.705309+0000","last_undegraded":"2023-12-02T00:28:50.705309+0000","last_fullsized":"2023-12-02T00:28:50.705309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T09:17:52.241790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703622+0000","last_change":"2023-12-02T00:28:48.727127+0000","last_active":"2023-12-02T00:28:50.703622+0000","last_peered":"2023-12-02T00:28:50.703622+0000","last_clean":"2023-12-02T00:28:50.703622+0000","last_became_active":"2023-12-02T00:28:48.726750+0000","last_became_peered":"2023-12-02T00:28:48.726750+0000","last_unstale":"2023-12-02T00:28:50.703622+0000","last_undegraded":"2023-12-02T00:28:50.703622+0000","last_fullsized":"2023-12-02T00:28:50.703622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:44:45.468077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021415+0000","last_change":"2023-12-02T00:28:49.718158+0000","last_active":"2023-12-02T00:28:51.021415+0000","last_peered":"2023-12-02T00:28:51.021415+0000","last_clean":"2023-12-02T00:28:51.021415+0000","last_became_active":"2023-12-02T00:28:49.717721+0000","last_became_peered":"2023-12-02T00:28:49.717721+0000","last_unstale":"2023-12-02T00:28:51.021415+0000","last_undegraded":"2023-12-02T00:28:51.021415+0000","last_fullsized":"2023-12-02T00:28:51.021415+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T10:27:29.244181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703765+0000","last_change":"2023-12-02T00:28:48.735894+0000","last_active":"2023-12-02T00:28:50.703765+0000","last_peered":"2023-12-02T00:28:50.703765+0000","last_clean":"2023-12-02T00:28:50.703765+0000","last_became_active":"2023-12-02T00:28:48.735613+0000","last_became_peered":"2023-12-02T00:28:48.735613+0000","last_unstale":"2023-12-02T00:28:50.703765+0000","last_undegraded":"2023-12-02T00:28:50.703765+0000","last_fullsized":"2023-12-02T00:28:50.703765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:52:07.682002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021464+0000","last_change":"2023-12-02T00:28:49.719314+0000","last_active":"2023-12-02T00:28:51.021464+0000","last_peered":"2023-12-02T00:28:51.021464+0000","last_clean":"2023-12-02T00:28:51.021464+0000","last_became_active":"2023-12-02T00:28:49.718940+0000","last_became_peered":"2023-12-02T00:28:49.718940+0000","last_unstale":"2023-12-02T00:28:51.021464+0000","last_undegraded":"2023-12-02T00:28:51.021464+0000","last_fullsized":"2023-12-02T00:28:51.021464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:50:45.793304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021457+0000","last_change":"2023-12-02T00:28:48.731948+0000","last_active":"2023-12-02T00:28:51.021457+0000","last_peered":"2023-12-02T00:28:51.021457+0000","last_clean":"2023-12-02T00:28:51.021457+0000","last_became_active":"2023-12-02T00:28:48.731641+0000","last_became_peered":"2023-12-02T00:28:48.731641+0000","last_unstale":"2023-12-02T00:28:51.021457+0000","last_undegraded":"2023-12-02T00:28:51.021457+0000","last_fullsized":"2023-12-02T00:28:51.021457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:36:21.580553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021037+0000","last_change":"2023-12-02T00:28:49.715513+0000","last_active":"2023-12-02T00:28:51.021037+0000","last_peered":"2023-12-02T00:28:51.021037+0000","last_clean":"2023-12-02T00:28:51.021037+0000","last_became_active":"2023-12-02T00:28:49.715240+0000","last_became_peered":"2023-12-02T00:28:49.715240+0000","last_unstale":"2023-12-02T00:28:51.021037+0000","last_undegraded":"2023-12-02T00:28:51.021037+0000","last_fullsized":"2023-12-02T00:28:51.021037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:52:22.600655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706521+0000","last_change":"2023-12-02T00:28:49.717785+0000","last_active":"2023-12-02T00:28:50.706521+0000","last_peered":"2023-12-02T00:28:50.706521+0000","last_clean":"2023-12-02T00:28:50.706521+0000","last_became_active":"2023-12-02T00:28:49.717408+0000","last_became_peered":"2023-12-02T00:28:49.717408+0000","last_unstale":"2023-12-02T00:28:50.706521+0000","last_undegraded":"2023-12-02T00:28:50.706521+0000","last_fullsized":"2023-12-02T00:28:50.706521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:29:16.078935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021429+0000","last_change":"2023-12-02T00:28:48.729768+0000","last_active":"2023-12-02T00:28:51.021429+0000","last_peered":"2023-12-02T00:28:51.021429+0000","last_clean":"2023-12-02T00:28:51.021429+0000","last_became_active":"2023-12-02T00:28:48.729466+0000","last_became_peered":"2023-12-02T00:28:48.729466+0000","last_unstale":"2023-12-02T00:28:51.021429+0000","last_undegraded":"2023-12-02T00:28:51.021429+0000","last_fullsized":"2023-12-02T00:28:51.021429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T12:06:33.958311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706378+0000","last_change":"2023-12-02T00:28:49.727325+0000","last_active":"2023-12-02T00:28:50.706378+0000","last_peered":"2023-12-02T00:28:50.706378+0000","last_clean":"2023-12-02T00:28:50.706378+0000","last_became_active":"2023-12-02T00:28:49.727062+0000","last_became_peered":"2023-12-02T00:28:49.727062+0000","last_unstale":"2023-12-02T00:28:50.706378+0000","last_undegraded":"2023-12-02T00:28:50.706378+0000","last_fullsized":"2023-12-02T00:28:50.706378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:15:02.988933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021734+0000","last_change":"2023-12-02T00:28:48.732989+0000","last_active":"2023-12-02T00:28:51.021734+0000","last_peered":"2023-12-02T00:28:51.021734+0000","last_clean":"2023-12-02T00:28:51.021734+0000","last_became_active":"2023-12-02T00:28:48.731645+0000","last_became_peered":"2023-12-02T00:28:48.731645+0000","last_unstale":"2023-12-02T00:28:51.021734+0000","last_undegraded":"2023-12-02T00:28:51.021734+0000","last_fullsized":"2023-12-02T00:28:51.021734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:44:45.468077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021923+0000","last_change":"2023-12-02T00:28:49.725372+0000","last_active":"2023-12-02T00:28:51.021923+0000","last_peered":"2023-12-02T00:28:51.021923+0000","last_clean":"2023-12-02T00:28:51.021923+0000","last_became_active":"2023-12-02T00:28:49.724926+0000","last_became_peered":"2023-12-02T00:28:49.724926+0000","last_unstale":"2023-12-02T00:28:51.021923+0000","last_undegraded":"2023-12-02T00:28:51.021923+0000","last_fullsized":"2023-12-02T00:28:51.021923+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:07:52.802590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021204+0000","last_change":"2023-12-02T00:28:48.731302+0000","last_active":"2023-12-02T00:28:51.021204+0000","last_peered":"2023-12-02T00:28:51.021204+0000","last_clean":"2023-12-02T00:28:51.021204+0000","last_became_active":"2023-12-02T00:28:48.731002+0000","last_became_peered":"2023-12-02T00:28:48.731002+0000","last_unstale":"2023-12-02T00:28:51.021204+0000","last_undegraded":"2023-12-02T00:28:51.021204+0000","last_fullsized":"2023-12-02T00:28:51.021204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:29:26.381170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021221+0000","last_change":"2023-12-02T00:28:49.721605+0000","last_active":"2023-12-02T00:28:51.021221+0000","last_peered":"2023-12-02T00:28:51.021221+0000","last_clean":"2023-12-02T00:28:51.021221+0000","last_became_active":"2023-12-02T00:28:49.721035+0000","last_became_peered":"2023-12-02T00:28:49.721035+0000","last_unstale":"2023-12-02T00:28:51.021221+0000","last_undegraded":"2023-12-02T00:28:51.021221+0000","last_fullsized":"2023-12-02T00:28:51.021221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:15:51.747629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.022018+0000","last_change":"2023-12-02T00:28:48.729331+0000","last_active":"2023-12-02T00:28:51.022018+0000","last_peered":"2023-12-02T00:28:51.022018+0000","last_clean":"2023-12-02T00:28:51.022018+0000","last_became_active":"2023-12-02T00:28:48.728469+0000","last_became_peered":"2023-12-02T00:28:48.728469+0000","last_unstale":"2023-12-02T00:28:51.022018+0000","last_undegraded":"2023-12-02T00:28:51.022018+0000","last_fullsized":"2023-12-02T00:28:51.022018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:47:08.818139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":815104,"data_stored":666531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95199999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":77193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":75671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":75707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":77193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28299999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":212992,"data_stored":74962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":225280,"data_stored":75741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":819200,"data_stored":665968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84199999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-02T00:28:57.497 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-02T00:28:57.497 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-12-02T00:28:57.498 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-12-02T00:28:57.498 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-12-02T00:28:57.498 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-12-02T00:28:57.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.639+0000 7f88b7a87640 1 Processor -- start 2023-12-02T00:28:57.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.639+0000 7f88b7a87640 1 -- start start 2023-12-02T00:28:57.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.640+0000 7f88b7a87640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f88b015d1d0 0x7f88b006b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:57.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.640+0000 7f88b7a87640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f88b00ad640 0x7f88b006ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:57.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.640+0000 7f88b7a87640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88b00adae0 con 0x7f88b00aeb80 2023-12-02T00:28:57.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.640+0000 7f88b7a87640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88b00addc0 con 0x7f88b015d1d0 2023-12-02T00:28:57.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.640+0000 7f88b7a87640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88b00ae0a0 con 0x7f88b00ad640 2023-12-02T00:28:57.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.640+0000 7f88b5022640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f88b00ad640 0x7f88b006ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:57.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.640+0000 7f88b5823640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f88b015d1d0 0x7f88b006b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:57.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.640+0000 7f88b5022640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f88b00ad640 0x7f88b006ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.46:34832/0 (socket says 172.21.15.46:34832) 2023-12-02T00:28:57.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.640+0000 7f88b6024640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f88b00aeb80 0x7f88b0160ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.46:36746/0 (socket says 172.21.15.46:36746) 2023-12-02T00:28:57.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.640+0000 7f88b5022640 1 -- 172.21.15.46:0/457916580 learned_addr learned my addr 172.21.15.46:0/457916580 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:57.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.640+0000 7f88b4821640 1 -- 172.21.15.46:0/457916580 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2366887035 0 0) 0x7f88b00adae0 con 0x7f88b00aeb80 2023-12-02T00:28:57.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.641+0000 7f88b5022640 1 -- 172.21.15.46:0/457916580 >> v2:172.21.15.162:3300/0 conn(0x7f88b015d1d0 msgr2=0x7f88b006b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:57.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.641+0000 7f88b5022640 1 --2- 172.21.15.46:0/457916580 >> v2:172.21.15.162:3300/0 conn(0x7f88b015d1d0 0x7f88b006b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:57.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.641+0000 7f88b5022640 1 -- 172.21.15.46:0/457916580 >> v1:172.21.15.162:6789/0 conn(0x7f88b00aeb80 legacy=0x7f88b0160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:57.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.641+0000 7f88b5022640 1 -- 172.21.15.46:0/457916580 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b00ae380 con 0x7f88b00ad640 2023-12-02T00:28:57.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.641+0000 7f88b5823640 1 --2- 172.21.15.46:0/457916580 >> v2:172.21.15.162:3300/0 conn(0x7f88b015d1d0 0x7f88b006b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:28:57.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.641+0000 7f88b5022640 1 --2- 172.21.15.46:0/457916580 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f88b00ad640 0x7f88b006ba20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f88a0009870 tx=0x7f88a002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aac0bf8ffdc0cbf7 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:57.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.641+0000 7f88b4821640 1 -- 172.21.15.46:0/457916580 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a0030840 con 0x7f88b00ad640 2023-12-02T00:28:57.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.642+0000 7f88b4821640 1 -- 172.21.15.46:0/457916580 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a00309a0 con 0x7f88b00ad640 2023-12-02T00:28:57.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.642+0000 7f88b4821640 1 -- 172.21.15.46:0/457916580 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a0030d30 con 0x7f88b00ad640 2023-12-02T00:28:57.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.642+0000 7f88b7a87640 1 -- 172.21.15.46:0/457916580 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f88b00ad640 msgr2=0x7f88b006ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:57.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.642+0000 7f88b7a87640 1 --2- 172.21.15.46:0/457916580 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f88b00ad640 0x7f88b006ba20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f88a0009870 tx=0x7f88a002b980 comp rx=0 tx=0).stop 2023-12-02T00:28:57.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.642+0000 7f88b7a87640 1 -- 172.21.15.46:0/457916580 shutdown_connections 2023-12-02T00:28:57.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.642+0000 7f88b7a87640 1 --2- 172.21.15.46:0/457916580 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f88b00ad640 0x7f88b006ba20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:57.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.642+0000 7f88b7a87640 1 --2- 172.21.15.46:0/457916580 >> v2:172.21.15.162:3300/0 conn(0x7f88b015d1d0 0x7f88b006b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:57.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.642+0000 7f88b7a87640 1 -- 172.21.15.46:0/457916580 >> 172.21.15.46:0/457916580 conn(0x7f88b0070e70 msgr2=0x7f88b0071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:57.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.642+0000 7f88b7a87640 1 -- 172.21.15.46:0/457916580 shutdown_connections 2023-12-02T00:28:57.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.642+0000 7f88b7a87640 1 -- 172.21.15.46:0/457916580 wait complete. 2023-12-02T00:28:57.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.643+0000 7f88b7a87640 1 Processor -- start 2023-12-02T00:28:57.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.643+0000 7f88b7a87640 1 -- start start 2023-12-02T00:28:57.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.644+0000 7f88b7a87640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f88b00ad640 0x7f88b0160330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:57.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.644+0000 7f88b7a87640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f88b00aeb80 0x7f88b015d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:57.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.644+0000 7f88b7a87640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88b015d1d0 0x7f88b0140ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:57.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.644+0000 7f88b7a87640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b015c1c0 con 0x7f88b015d1d0 2023-12-02T00:28:57.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.644+0000 7f88b7a87640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b01f7270 con 0x7f88b00ad640 2023-12-02T00:28:57.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.644+0000 7f88b7a87640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88b015fea0 con 0x7f88b00aeb80 2023-12-02T00:28:57.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.644+0000 7f88b6024640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88b015d1d0 0x7f88b0140ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:57.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.644+0000 7f88b6024640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88b015d1d0 0x7f88b0140ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37808/0 (socket says 172.21.15.46:37808) 2023-12-02T00:28:57.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.644+0000 7f88b6024640 1 -- 172.21.15.46:0/410858933 learned_addr learned my addr 172.21.15.46:0/410858933 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:57.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.644+0000 7f88b5823640 1 --2- 172.21.15.46:0/410858933 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f88b00ad640 0x7f88b0160330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:57.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.644+0000 7f88b5022640 1 --2- 172.21.15.46:0/410858933 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f88b00aeb80 0x7f88b015d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:57.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.645+0000 7f88b6024640 1 -- 172.21.15.46:0/410858933 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f88b00aeb80 msgr2=0x7f88b015d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:57.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.645+0000 7f88b6024640 1 --2- 172.21.15.46:0/410858933 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f88b00aeb80 0x7f88b015d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:57.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.645+0000 7f88b6024640 1 -- 172.21.15.46:0/410858933 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f88b00ad640 msgr2=0x7f88b0160330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:57.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.645+0000 7f88b6024640 1 --2- 172.21.15.46:0/410858933 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f88b00ad640 0x7f88b0160330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:57.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.645+0000 7f88b6024640 1 -- 172.21.15.46:0/410858933 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88ac009400 con 0x7f88b015d1d0 2023-12-02T00:28:57.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.645+0000 7f88b6024640 1 --2- 172.21.15.46:0/410858933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88b015d1d0 0x7f88b0140ce0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f88a800c7d0 tx=0x7f88a800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:57.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.645+0000 7f88a67fc640 1 -- 172.21.15.46:0/410858933 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a800a640 con 0x7f88b015d1d0 2023-12-02T00:28:57.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.645+0000 7f88a67fc640 1 -- 172.21.15.46:0/410858933 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a800a7a0 con 0x7f88b015d1d0 2023-12-02T00:28:57.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.645+0000 7f88b7a87640 1 -- 172.21.15.46:0/410858933 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88a00094d0 con 0x7f88b015d1d0 2023-12-02T00:28:57.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.645+0000 7f88a67fc640 1 -- 172.21.15.46:0/410858933 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a800ab90 con 0x7f88b015d1d0 2023-12-02T00:28:57.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.645+0000 7f88b7a87640 1 -- 172.21.15.46:0/410858933 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88b0160870 con 0x7f88b015d1d0 2023-12-02T00:28:57.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.646+0000 7f88b7a87640 1 -- 172.21.15.46:0/410858933 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88b015de80 con 0x7f88b015d1d0 2023-12-02T00:28:57.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.646+0000 7f88a67fc640 1 -- 172.21.15.46:0/410858933 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f88a800acf0 con 0x7f88b015d1d0 2023-12-02T00:28:57.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.647+0000 7f88a67fc640 1 --2- 172.21.15.46:0/410858933 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f889406ae40 0x7f889406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:57.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.647+0000 7f88a67fc640 1 -- 172.21.15.46:0/410858933 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f88a8088d80 con 0x7f88b015d1d0 2023-12-02T00:28:57.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.647+0000 7f88b5823640 1 --2- 172.21.15.46:0/410858933 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f889406ae40 0x7f889406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:57.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.647+0000 7f88b5823640 1 --2- 172.21.15.46:0/410858933 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f889406ae40 0x7f889406d300 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f88b00cf540 tx=0x7f88ac009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:57.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.651+0000 7f88a67fc640 1 -- 172.21.15.46:0/410858933 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f88a8053ed0 con 0x7f88b015d1d0 2023-12-02T00:28:57.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.878+0000 7f88b7a87640 1 -- 172.21.15.46:0/410858933 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f88b0069400 con 0x7f88b015d1d0 2023-12-02T00:28:57.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.881+0000 7f88a67fc640 1 -- 172.21.15.46:0/410858933 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171039 (secure 0 0 0) 0x7f88a8018020 con 0x7f88b015d1d0 2023-12-02T00:28:57.885 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:28:57.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.883+0000 7f88b7a87640 1 -- 172.21.15.46:0/410858933 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f889406ae40 msgr2=0x7f889406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:57.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.883+0000 7f88b7a87640 1 --2- 172.21.15.46:0/410858933 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f889406ae40 0x7f889406d300 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f88b00cf540 tx=0x7f88ac009190 comp rx=0 tx=0).stop 2023-12-02T00:28:57.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.883+0000 7f88b7a87640 1 -- 172.21.15.46:0/410858933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88b015d1d0 msgr2=0x7f88b0140ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:57.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.883+0000 7f88b7a87640 1 --2- 172.21.15.46:0/410858933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88b015d1d0 0x7f88b0140ce0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f88a800c7d0 tx=0x7f88a800cc90 comp rx=0 tx=0).stop 2023-12-02T00:28:57.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.883+0000 7f88b7a87640 1 -- 172.21.15.46:0/410858933 shutdown_connections 2023-12-02T00:28:57.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.883+0000 7f88b7a87640 1 --2- 172.21.15.46:0/410858933 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f889406ae40 0x7f889406d300 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:57.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.883+0000 7f88b7a87640 1 --2- 172.21.15.46:0/410858933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88b015d1d0 0x7f88b0140ce0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:57.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.883+0000 7f88b7a87640 1 --2- 172.21.15.46:0/410858933 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f88b00aeb80 0x7f88b015d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:57.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.883+0000 7f88b7a87640 1 --2- 172.21.15.46:0/410858933 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f88b00ad640 0x7f88b0160330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:57.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.883+0000 7f88b7a87640 1 -- 172.21.15.46:0/410858933 >> 172.21.15.46:0/410858933 conn(0x7f88b0070e70 msgr2=0x7f88b006e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:57.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.883+0000 7f88b7a87640 1 -- 172.21.15.46:0/410858933 shutdown_connections 2023-12-02T00:28:57.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:57.883+0000 7f88b7a87640 1 -- 172.21.15.46:0/410858933 wait complete. 2023-12-02T00:28:57.895 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":5,"active_gid":4108,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3248644722},{"type":"v1","addr":"172.21.15.162:6833","nonce":3248644722}]},"active_addr":"172.21.15.162:6833/3248644722","active_change":"2023-12-02T00:28:29.144332+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4103,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.162:0","nonce":3064835927}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.162:0","nonce":2627055356}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.162:0","nonce":4055188929}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.162:0","nonce":3859399983}]}]} 2023-12-02T00:28:57.901 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-12-02T00:28:57.901 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-12-02T00:28:57.901 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:28:58.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.007+0000 7f12a1490640 1 Processor -- start 2023-12-02T00:28:58.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.007+0000 7f12a1490640 1 -- start start 2023-12-02T00:28:58.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.007+0000 7f12a1490640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f129c0d2910 0x7f129c0d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.007+0000 7f12a1490640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f129c0d3730 0x7f129c0b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.007+0000 7f12a1490640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f129c0b2b10 con 0x7f129c15d330 2023-12-02T00:28:58.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.007+0000 7f12a1490640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f129c0d3b50 con 0x7f129c0d3730 2023-12-02T00:28:58.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.007+0000 7f12a1490640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f129c0d1590 con 0x7f129c0d2910 2023-12-02T00:28:58.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.008+0000 7f129b7fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f129c0d3730 0x7f129c0b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.008+0000 7f129b7fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f129c0d3730 0x7f129c0b6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37812/0 (socket says 172.21.15.46:37812) 2023-12-02T00:28:58.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.008+0000 7f129affd640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f129c15d330 0x7f129c0b8b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:44892/0 (socket says 172.21.15.46:44892) 2023-12-02T00:28:58.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.008+0000 7f129b7fe640 1 -- 172.21.15.46:0/850859996 learned_addr learned my addr 172.21.15.46:0/850859996 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:58.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.008+0000 7f1293fff640 1 --2- 172.21.15.46:0/850859996 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f129c0d2910 0x7f129c0d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.008+0000 7f129b7fe640 1 -- 172.21.15.46:0/850859996 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f129c0d2910 msgr2=0x7f129c0d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:58.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.008+0000 7f129b7fe640 1 --2- 172.21.15.46:0/850859996 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f129c0d2910 0x7f129c0d31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.008+0000 7f129b7fe640 1 -- 172.21.15.46:0/850859996 >> v1:172.21.15.46:6789/0 conn(0x7f129c15d330 legacy=0x7f129c0b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.008+0000 7f129b7fe640 1 -- 172.21.15.46:0/850859996 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f129c0d1870 con 0x7f129c0d3730 2023-12-02T00:28:58.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.008+0000 7f129a7fc640 1 -- 172.21.15.46:0/850859996 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4146336769 0 0) 0x7f129c0b2b10 con 0x7f129c15d330 2023-12-02T00:28:58.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.009+0000 7f129b7fe640 1 --2- 172.21.15.46:0/850859996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f129c0d3730 0x7f129c0b6390 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1294009e50 tx=0x7f129402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ee20a31d8a34ce server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:58.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.009+0000 7f129a7fc640 1 -- 172.21.15.46:0/850859996 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129402ca70 con 0x7f129c0d3730 2023-12-02T00:28:58.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.009+0000 7f129a7fc640 1 -- 172.21.15.46:0/850859996 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f129402cbd0 con 0x7f129c0d3730 2023-12-02T00:28:58.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.009+0000 7f129a7fc640 1 -- 172.21.15.46:0/850859996 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12940343f0 con 0x7f129c0d3730 2023-12-02T00:28:58.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.009+0000 7f12a1490640 1 -- 172.21.15.46:0/850859996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f129c0d3730 msgr2=0x7f129c0b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.009+0000 7f12a1490640 1 --2- 172.21.15.46:0/850859996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f129c0d3730 0x7f129c0b6390 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1294009e50 tx=0x7f129402bed0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.009+0000 7f12a1490640 1 -- 172.21.15.46:0/850859996 shutdown_connections 2023-12-02T00:28:58.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.009+0000 7f12a1490640 1 --2- 172.21.15.46:0/850859996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f129c0d3730 0x7f129c0b6390 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.009+0000 7f12a1490640 1 --2- 172.21.15.46:0/850859996 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f129c0d2910 0x7f129c0d31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.009+0000 7f12a1490640 1 -- 172.21.15.46:0/850859996 >> 172.21.15.46:0/850859996 conn(0x7f129c072020 msgr2=0x7f129c072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:58.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.009+0000 7f12a1490640 1 -- 172.21.15.46:0/850859996 shutdown_connections 2023-12-02T00:28:58.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.009+0000 7f12a1490640 1 -- 172.21.15.46:0/850859996 wait complete. 2023-12-02T00:28:58.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.011+0000 7f12a1490640 1 Processor -- start 2023-12-02T00:28:58.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.011+0000 7f12a1490640 1 -- start start 2023-12-02T00:28:58.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.011+0000 7f12a1490640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f129c0d2910 0x7f129c149530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.011+0000 7f12a1490640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f129c0d3730 0x7f129c0cbd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.011+0000 7f12a1490640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f129c15d330 0x7f129c15d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.011+0000 7f12a1490640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129c0ccdd0 con 0x7f129c0d2910 2023-12-02T00:28:58.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.011+0000 7f12a1490640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129c0c79d0 con 0x7f129c15d330 2023-12-02T00:28:58.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.011+0000 7f12a1490640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f129c149a70 con 0x7f129c0d3730 2023-12-02T00:28:58.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.011+0000 7f129affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f129c0d2910 0x7f129c149530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.011+0000 7f129affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f129c0d2910 0x7f129c149530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37826/0 (socket says 172.21.15.46:37826) 2023-12-02T00:28:58.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.011+0000 7f129affd640 1 -- 172.21.15.46:0/1904415892 learned_addr learned my addr 172.21.15.46:0/1904415892 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:58.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.011+0000 7f1293fff640 1 --2- 172.21.15.46:0/1904415892 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f129c0d3730 0x7f129c0cbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.011+0000 7f129b7fe640 1 --2- 172.21.15.46:0/1904415892 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f129c15d330 0x7f129c15d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.012+0000 7f129affd640 1 -- 172.21.15.46:0/1904415892 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f129c0d3730 msgr2=0x7f129c0cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.012+0000 7f129affd640 1 --2- 172.21.15.46:0/1904415892 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f129c0d3730 0x7f129c0cbd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.012+0000 7f129affd640 1 -- 172.21.15.46:0/1904415892 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f129c15d330 msgr2=0x7f129c15d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.012+0000 7f129affd640 1 --2- 172.21.15.46:0/1904415892 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f129c15d330 0x7f129c15d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.012+0000 7f129affd640 1 -- 172.21.15.46:0/1904415892 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1294009b30 con 0x7f129c0d2910 2023-12-02T00:28:58.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.012+0000 7f129affd640 1 --2- 172.21.15.46:0/1904415892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f129c0d2910 0x7f129c149530 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f128c00c4d0 tx=0x7f128c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:58.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.012+0000 7f12937fe640 1 -- 172.21.15.46:0/1904415892 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128c0074e0 con 0x7f129c0d2910 2023-12-02T00:28:58.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.012+0000 7f12937fe640 1 -- 172.21.15.46:0/1904415892 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f128c007660 con 0x7f129c0d2910 2023-12-02T00:28:58.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.012+0000 7f12a1490640 1 -- 172.21.15.46:0/1904415892 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f129c0cc810 con 0x7f129c0d2910 2023-12-02T00:28:58.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.012+0000 7f12937fe640 1 -- 172.21.15.46:0/1904415892 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128c004a20 con 0x7f129c0d2910 2023-12-02T00:28:58.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.012+0000 7f12a1490640 1 -- 172.21.15.46:0/1904415892 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f129c0c7660 con 0x7f129c0d2910 2023-12-02T00:28:58.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.013+0000 7f12a1490640 1 -- 172.21.15.46:0/1904415892 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f129c0c74c0 con 0x7f129c0d2910 2023-12-02T00:28:58.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.013+0000 7f12937fe640 1 -- 172.21.15.46:0/1904415892 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f128c004b80 con 0x7f129c0d2910 2023-12-02T00:28:58.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.014+0000 7f12937fe640 1 --2- 172.21.15.46:0/1904415892 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f127c06ae70 0x7f127c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.014+0000 7f12937fe640 1 -- 172.21.15.46:0/1904415892 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f128c088a20 con 0x7f129c0d2910 2023-12-02T00:28:58.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.014+0000 7f1293fff640 1 --2- 172.21.15.46:0/1904415892 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f127c06ae70 0x7f127c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.014+0000 7f1293fff640 1 --2- 172.21.15.46:0/1904415892 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f127c06ae70 0x7f127c06d330 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f129c0b6700 tx=0x7f1284008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:58.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.018+0000 7f12937fe640 1 -- 172.21.15.46:0/1904415892 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f129c0c74c0 con 0x7f129c0d2910 2023-12-02T00:28:58.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.194+0000 7f12a1490640 1 -- 172.21.15.46:0/1904415892 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f129c0c74c0 con 0x7f129c0d2910 2023-12-02T00:28:58.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.195+0000 7f12937fe640 1 -- 172.21.15.46:0/1904415892 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f129c0c74c0 con 0x7f129c0d2910 2023-12-02T00:28:58.196 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:28:58.196 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":21,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:28:50.700112+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-02T00:28:47.273337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-02T00:28:48.060115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:28:58.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.197+0000 7f12a1490640 1 -- 172.21.15.46:0/1904415892 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f127c06ae70 msgr2=0x7f127c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.197+0000 7f12a1490640 1 --2- 172.21.15.46:0/1904415892 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f127c06ae70 0x7f127c06d330 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f129c0b6700 tx=0x7f1284008040 comp rx=0 tx=0).stop 2023-12-02T00:28:58.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.197+0000 7f12a1490640 1 -- 172.21.15.46:0/1904415892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f129c0d2910 msgr2=0x7f129c149530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.197+0000 7f12a1490640 1 --2- 172.21.15.46:0/1904415892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f129c0d2910 0x7f129c149530 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f128c00c4d0 tx=0x7f128c00c990 comp rx=0 tx=0).stop 2023-12-02T00:28:58.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.197+0000 7f12a1490640 1 -- 172.21.15.46:0/1904415892 shutdown_connections 2023-12-02T00:28:58.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.197+0000 7f12a1490640 1 --2- 172.21.15.46:0/1904415892 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f127c06ae70 0x7f127c06d330 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.197+0000 7f12a1490640 1 --2- 172.21.15.46:0/1904415892 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f129c15d330 0x7f129c15d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.197+0000 7f12a1490640 1 --2- 172.21.15.46:0/1904415892 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f129c0d3730 0x7f129c0cbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.197+0000 7f12a1490640 1 --2- 172.21.15.46:0/1904415892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f129c0d2910 0x7f129c149530 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.197+0000 7f12a1490640 1 -- 172.21.15.46:0/1904415892 >> 172.21.15.46:0/1904415892 conn(0x7f129c072020 msgr2=0x7f129c0d0030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:58.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.197+0000 7f12a1490640 1 -- 172.21.15.46:0/1904415892 shutdown_connections 2023-12-02T00:28:58.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.197+0000 7f12a1490640 1 -- 172.21.15.46:0/1904415892 wait complete. 2023-12-02T00:28:58.210 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-12-02T00:28:58.211 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:28:58.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.316+0000 7f40a2768640 1 Processor -- start 2023-12-02T00:28:58.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.316+0000 7f40a2768640 1 -- start start 2023-12-02T00:28:58.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.317+0000 7f40a2768640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f409c160f70 0x7f409c0cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.317+0000 7f40a2768640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f409c0ce480 0x7f409c0d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.317+0000 7f40a2768640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f409c1614e0 con 0x7f409c15d2d0 2023-12-02T00:28:58.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.317+0000 7f40a2768640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f409c1617c0 con 0x7f409c160f70 2023-12-02T00:28:58.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.317+0000 7f40a2768640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f409c161aa0 con 0x7f409c0ce480 2023-12-02T00:28:58.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.317+0000 7f40a0d05640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f409c0ce480 0x7f409c0d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.317+0000 7f409b7fe640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f409c160f70 0x7f409c0cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.317+0000 7f409bfff640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f409c15d2d0 0x7f409c0b8b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.46:36752/0 (socket says 172.21.15.46:36752) 2023-12-02T00:28:58.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.317+0000 7f409bfff640 1 -- 172.21.15.46:0/929568829 learned_addr learned my addr 172.21.15.46:0/929568829 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:28:58.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.318+0000 7f409affd640 1 -- 172.21.15.46:0/929568829 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4085505832 0 0) 0x7f409c1614e0 con 0x7f409c15d2d0 2023-12-02T00:28:58.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.318+0000 7f409affd640 1 -- 172.21.15.46:0/929568829 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4084003520 con 0x7f409c15d2d0 2023-12-02T00:28:58.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.318+0000 7f409b7fe640 1 -- 172.21.15.46:0/929568829 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f409c0ce480 msgr2=0x7f409c0d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:58.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.318+0000 7f409b7fe640 1 --2- 172.21.15.46:0/929568829 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f409c0ce480 0x7f409c0d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.318+0000 7f409b7fe640 1 -- 172.21.15.46:0/929568829 >> v1:172.21.15.162:6789/0 conn(0x7f409c15d2d0 legacy=0x7f409c0b8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.318+0000 7f409b7fe640 1 -- 172.21.15.46:0/929568829 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f409c161d80 con 0x7f409c160f70 2023-12-02T00:28:58.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.318+0000 7f40a0d05640 1 --2- 172.21.15.46:0/929568829 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f409c0ce480 0x7f409c0d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:28:58.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.318+0000 7f409b7fe640 1 --2- 172.21.15.46:0/929568829 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f409c160f70 0x7f409c0cdf40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f40900097a0 tx=0x7f409002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44ddbcd2dfb99995 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:58.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.319+0000 7f409affd640 1 -- 172.21.15.46:0/929568829 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4090031830 con 0x7f409c160f70 2023-12-02T00:28:58.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.319+0000 7f409affd640 1 -- 172.21.15.46:0/929568829 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4090031990 con 0x7f409c160f70 2023-12-02T00:28:58.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.319+0000 7f409affd640 1 -- 172.21.15.46:0/929568829 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4090031d20 con 0x7f409c160f70 2023-12-02T00:28:58.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.319+0000 7f40a2768640 1 -- 172.21.15.46:0/929568829 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f409c160f70 msgr2=0x7f409c0cdf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.319+0000 7f40a2768640 1 --2- 172.21.15.46:0/929568829 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f409c160f70 0x7f409c0cdf40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f40900097a0 tx=0x7f409002b8d0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.319+0000 7f40a2768640 1 -- 172.21.15.46:0/929568829 shutdown_connections 2023-12-02T00:28:58.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.319+0000 7f40a2768640 1 --2- 172.21.15.46:0/929568829 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f409c0ce480 0x7f409c0d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.319+0000 7f40a2768640 1 --2- 172.21.15.46:0/929568829 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f409c160f70 0x7f409c0cdf40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.319+0000 7f40a2768640 1 -- 172.21.15.46:0/929568829 >> 172.21.15.46:0/929568829 conn(0x7f409c072230 msgr2=0x7f409c0cfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:58.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.319+0000 7f40a2768640 1 -- 172.21.15.46:0/929568829 shutdown_connections 2023-12-02T00:28:58.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.319+0000 7f40a2768640 1 -- 172.21.15.46:0/929568829 wait complete. 2023-12-02T00:28:58.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.320+0000 7f40a2768640 1 Processor -- start 2023-12-02T00:28:58.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.320+0000 7f40a2768640 1 -- start start 2023-12-02T00:28:58.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.321+0000 7f40a2768640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f409c0ce480 0x7f409c0c91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.321+0000 7f40a2768640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f409c15d2d0 0x7f409c0cace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.321+0000 7f40a2768640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f409c160f70 0x7f409c1f3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.321+0000 7f40a2768640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409c15dc10 con 0x7f409c0ce480 2023-12-02T00:28:58.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.321+0000 7f40a2768640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409c15def0 con 0x7f409c160f70 2023-12-02T00:28:58.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.321+0000 7f40a2768640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f409c15e1d0 con 0x7f409c15d2d0 2023-12-02T00:28:58.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.321+0000 7f409bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f409c0ce480 0x7f409c0c91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.321+0000 7f409bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f409c0ce480 0x7f409c0c91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37838/0 (socket says 172.21.15.46:37838) 2023-12-02T00:28:58.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.321+0000 7f409bfff640 1 -- 172.21.15.46:0/59687389 learned_addr learned my addr 172.21.15.46:0/59687389 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:58.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.322+0000 7f40a0d05640 1 --2- 172.21.15.46:0/59687389 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f409c160f70 0x7f409c1f3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.322+0000 7f409bfff640 1 -- 172.21.15.46:0/59687389 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f409c15d2d0 msgr2=0x7f409c0cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.322+0000 7f409b7fe640 1 --2- 172.21.15.46:0/59687389 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f409c15d2d0 0x7f409c0cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.322+0000 7f409bfff640 1 --2- 172.21.15.46:0/59687389 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f409c15d2d0 0x7f409c0cace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.322+0000 7f409bfff640 1 -- 172.21.15.46:0/59687389 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f409c160f70 msgr2=0x7f409c1f3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.322+0000 7f409bfff640 1 --2- 172.21.15.46:0/59687389 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f409c160f70 0x7f409c1f3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.322+0000 7f409bfff640 1 -- 172.21.15.46:0/59687389 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4094009a70 con 0x7f409c0ce480 2023-12-02T00:28:58.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.322+0000 7f40a0d05640 1 --2- 172.21.15.46:0/59687389 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f409c160f70 0x7f409c1f3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:28:58.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.322+0000 7f409b7fe640 1 --2- 172.21.15.46:0/59687389 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f409c15d2d0 0x7f409c0cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:28:58.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.322+0000 7f409bfff640 1 --2- 172.21.15.46:0/59687389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f409c0ce480 0x7f409c0c91e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f408c00c500 tx=0x7f408c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:58.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.322+0000 7f4098ff9640 1 -- 172.21.15.46:0/59687389 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408c005ce0 con 0x7f409c0ce480 2023-12-02T00:28:58.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.322+0000 7f4098ff9640 1 -- 172.21.15.46:0/59687389 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f408c005e40 con 0x7f409c0ce480 2023-12-02T00:28:58.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.322+0000 7f40a2768640 1 -- 172.21.15.46:0/59687389 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4090009400 con 0x7f409c0ce480 2023-12-02T00:28:58.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.322+0000 7f4098ff9640 1 -- 172.21.15.46:0/59687389 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408c007330 con 0x7f409c0ce480 2023-12-02T00:28:58.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.322+0000 7f40a2768640 1 -- 172.21.15.46:0/59687389 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f409c15e790 con 0x7f409c0ce480 2023-12-02T00:28:58.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.323+0000 7f40a2768640 1 -- 172.21.15.46:0/59687389 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f409c15e980 con 0x7f409c0ce480 2023-12-02T00:28:58.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.323+0000 7f4098ff9640 1 -- 172.21.15.46:0/59687389 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f408c007550 con 0x7f409c0ce480 2023-12-02T00:28:58.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.324+0000 7f4098ff9640 1 --2- 172.21.15.46:0/59687389 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f408806ae40 0x7f408806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.324+0000 7f4098ff9640 1 -- 172.21.15.46:0/59687389 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f408c088cb0 con 0x7f409c0ce480 2023-12-02T00:28:58.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.324+0000 7f409b7fe640 1 --2- 172.21.15.46:0/59687389 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f408806ae40 0x7f408806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.324+0000 7f409b7fe640 1 --2- 172.21.15.46:0/59687389 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f408806ae40 0x7f408806d300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4090036000 tx=0x7f409003c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:58.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.328+0000 7f4098ff9640 1 -- 172.21.15.46:0/59687389 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f408c053e20 con 0x7f409c0ce480 2023-12-02T00:28:58.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.503+0000 7f40a2768640 1 -- 172.21.15.46:0/59687389 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f409c069400 con 0x7f409c0ce480 2023-12-02T00:28:58.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.505+0000 7f4098ff9640 1 -- 172.21.15.46:0/59687389 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f408c017020 con 0x7f409c0ce480 2023-12-02T00:28:58.506 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:28:58.506 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":21,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:28:50.700112+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-02T00:28:47.273337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-02T00:28:48.060115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:28:58.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.507+0000 7f40a2768640 1 -- 172.21.15.46:0/59687389 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f408806ae40 msgr2=0x7f408806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.507+0000 7f40a2768640 1 --2- 172.21.15.46:0/59687389 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f408806ae40 0x7f408806d300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4090036000 tx=0x7f409003c040 comp rx=0 tx=0).stop 2023-12-02T00:28:58.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.507+0000 7f40a2768640 1 -- 172.21.15.46:0/59687389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f409c0ce480 msgr2=0x7f409c0c91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.507+0000 7f40a2768640 1 --2- 172.21.15.46:0/59687389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f409c0ce480 0x7f409c0c91e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f408c00c500 tx=0x7f408c00c9c0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.507+0000 7f40a2768640 1 -- 172.21.15.46:0/59687389 shutdown_connections 2023-12-02T00:28:58.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.507+0000 7f40a2768640 1 --2- 172.21.15.46:0/59687389 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f408806ae40 0x7f408806d300 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.507+0000 7f40a2768640 1 --2- 172.21.15.46:0/59687389 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f409c160f70 0x7f409c1f3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.507+0000 7f40a2768640 1 --2- 172.21.15.46:0/59687389 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f409c15d2d0 0x7f409c0cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.507+0000 7f40a2768640 1 --2- 172.21.15.46:0/59687389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f409c0ce480 0x7f409c0c91e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.507+0000 7f40a2768640 1 -- 172.21.15.46:0/59687389 >> 172.21.15.46:0/59687389 conn(0x7f409c072230 msgr2=0x7f409c157850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:58.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.507+0000 7f40a2768640 1 -- 172.21.15.46:0/59687389 shutdown_connections 2023-12-02T00:28:58.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.507+0000 7f40a2768640 1 -- 172.21.15.46:0/59687389 wait complete. 2023-12-02T00:28:58.519 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-12-02T00:28:58.519 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-12-02T00:28:58.520 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-12-02T00:28:58.520 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-12-02T00:28:58.520 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-12-02T00:28:58.521 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-12-02T00:28:58.521 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-12-02T00:28:58.522 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-12-02T00:28:58.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.703+0000 7fb88249b640 1 Processor -- start 2023-12-02T00:28:58.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.703+0000 7fb88249b640 1 -- start start 2023-12-02T00:28:58.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.704+0000 7fb88249b640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb87c0b3e90 0x7fb87c0cc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.704+0000 7fb88249b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb87c08a850 0x7fb87c0cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.704+0000 7fb88249b640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb87c0cee20 con 0x7fb87c071600 2023-12-02T00:28:58.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.704+0000 7fb88249b640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb87c0711e0 con 0x7fb87c0b3e90 2023-12-02T00:28:58.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.704+0000 7fb88249b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c0b4270 con 0x7fb87c08a850 2023-12-02T00:28:58.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.704+0000 7fb880a38640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb87c071600 0x7fb87c0b2060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:44894/0 (socket says 172.21.15.46:44894) 2023-12-02T00:28:58.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.704+0000 7fb880a38640 1 -- 172.21.15.46:0/3071224137 learned_addr learned my addr 172.21.15.46:0/3071224137 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:28:58.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.705+0000 7fb87b7fe640 1 --2- 172.21.15.46:0/3071224137 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb87c08a850 0x7fb87c0cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.705+0000 7fb87affd640 1 -- 172.21.15.46:0/3071224137 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1009651197 0 0) 0x7fb87c0cee20 con 0x7fb87c071600 2023-12-02T00:28:58.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.705+0000 7fb87affd640 1 -- 172.21.15.46:0/3071224137 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb864003520 con 0x7fb87c071600 2023-12-02T00:28:58.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.705+0000 7fb87affd640 1 -- 172.21.15.46:0/3071224137 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1339485957 0 0) 0x7fb864003520 con 0x7fb87c071600 2023-12-02T00:28:58.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.705+0000 7fb87affd640 1 -- 172.21.15.46:0/3071224137 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb87c08a850 msgr2=0x7fb87c0cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:58.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.705+0000 7fb87affd640 1 --2- 172.21.15.46:0/3071224137 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb87c08a850 0x7fb87c0cccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.705+0000 7fb87affd640 1 -- 172.21.15.46:0/3071224137 >> v2:172.21.15.162:3300/0 conn(0x7fb87c0b3e90 msgr2=0x7fb87c0cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:58.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.705+0000 7fb87affd640 1 --2- 172.21.15.46:0/3071224137 >> v2:172.21.15.162:3300/0 conn(0x7fb87c0b3e90 0x7fb87c0cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.705+0000 7fb87affd640 1 -- 172.21.15.46:0/3071224137 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb87c170c60 con 0x7fb87c071600 2023-12-02T00:28:58.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.706+0000 7fb87affd640 1 -- 172.21.15.46:0/3071224137 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7fb8740019f0 con 0x7fb87c071600 2023-12-02T00:28:58.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.706+0000 7fb87affd640 1 -- 172.21.15.46:0/3071224137 >> v1:172.21.15.46:6789/0 conn(0x7fb87c071600 legacy=0x7fb87c0b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.706+0000 7f32d7939640 1 Processor -- start 2023-12-02T00:28:58.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.706+0000 7fb87affd640 1 --2- 172.21.15.46:0/3071224137 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb87c08a850 0x7fb8640039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.706+0000 7fb87affd640 1 --2- 172.21.15.46:0/3071224137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb864003fe0 0x7fb8640047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.706+0000 7f32d7939640 1 -- start start 2023-12-02T00:28:58.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.706+0000 7fb87affd640 1 --2- 172.21.15.46:0/3071224137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb864004d00 0x7fb864005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.706+0000 7fb880a38640 1 --2- 172.21.15.46:0/3071224137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb864004d00 0x7fb864005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.706+0000 7fb87bfff640 1 --2- 172.21.15.46:0/3071224137 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb87c08a850 0x7fb8640039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.706+0000 7fb87b7fe640 1 --2- 172.21.15.46:0/3071224137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb864003fe0 0x7fb8640047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.706+0000 7fb87affd640 1 -- 172.21.15.46:0/3071224137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c0711e0 con 0x7fb864004d00 2023-12-02T00:28:58.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.706+0000 7fb87affd640 1 -- 172.21.15.46:0/3071224137 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86400bb40 con 0x7fb864003fe0 2023-12-02T00:28:58.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.706+0000 7fb87affd640 1 -- 172.21.15.46:0/3071224137 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb86400bd00 con 0x7fb87c08a850 2023-12-02T00:28:58.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.707+0000 7f70e29b7640 1 Processor -- start 2023-12-02T00:28:58.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.707+0000 7fb87bfff640 1 -- 172.21.15.46:0/3071224137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb864003fe0 msgr2=0x7fb8640047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:58.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.707+0000 7fb87bfff640 1 --2- 172.21.15.46:0/3071224137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb864003fe0 0x7fb8640047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.707+0000 7fb87bfff640 1 -- 172.21.15.46:0/3071224137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb864004d00 msgr2=0x7fb864005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:58.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.707+0000 7fb87bfff640 1 --2- 172.21.15.46:0/3071224137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb864004d00 0x7fb864005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.707+0000 7f70e29b7640 1 -- start start 2023-12-02T00:28:58.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.707+0000 7f63e5d1d640 1 Processor -- start 2023-12-02T00:28:58.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.707+0000 7f92873c4640 1 Processor -- start 2023-12-02T00:28:58.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.707+0000 7f92873c4640 1 -- start start 2023-12-02T00:28:58.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.707+0000 7fb87bfff640 1 -- 172.21.15.46:0/3071224137 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb864003520 con 0x7fb87c08a850 2023-12-02T00:28:58.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.707+0000 7fb880a38640 1 --2- 172.21.15.46:0/3071224137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb864004d00 0x7fb864005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:28:58.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.707+0000 7fb87b7fe640 1 --2- 172.21.15.46:0/3071224137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb864003fe0 0x7fb8640047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:28:58.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.707+0000 7f32d7939640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32d0159210 0x7f32d00b40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.707+0000 7f32d7939640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32d00cd020 con 0x7f32d00b45f0 2023-12-02T00:28:58.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.707+0000 7f32d7939640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32d00cd300 con 0x7f32d00b1880 2023-12-02T00:28:58.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.707+0000 7f32d7939640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d00b4a80 con 0x7f32d0159210 2023-12-02T00:28:58.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.707+0000 7f32d5ed6640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f32d00b45f0 0x7f32d0163fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:44896/0 (socket says 172.21.15.46:44896) 2023-12-02T00:28:58.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.707+0000 7f32d5ed6640 1 -- 172.21.15.46:0/1984202398 learned_addr learned my addr 172.21.15.46:0/1984202398 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:28:58.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.708+0000 7f32c7fff640 1 -- 172.21.15.46:0/1984202398 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3506519 0 0) 0x7f32d00cd020 con 0x7f32d00b45f0 2023-12-02T00:28:58.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.708+0000 7f32c7fff640 1 -- 172.21.15.46:0/1984202398 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32bc005730 con 0x7f32d00b45f0 2023-12-02T00:28:58.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.708+0000 7f32d4ed4640 1 --2- 172.21.15.46:0/1984202398 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32d0159210 0x7f32d00b40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.708+0000 7f5f7c17c640 1 Processor -- start 2023-12-02T00:28:58.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.708+0000 7f32c7fff640 1 -- 172.21.15.46:0/1984202398 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2791903760 0 0) 0x7f32bc005730 con 0x7f32d00b45f0 2023-12-02T00:28:58.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.708+0000 7f32c7fff640 1 -- 172.21.15.46:0/1984202398 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32d0159210 msgr2=0x7f32d00b40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:58.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.708+0000 7f32c7fff640 1 --2- 172.21.15.46:0/1984202398 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32d0159210 0x7f32d00b40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.708+0000 7f32c7fff640 1 -- 172.21.15.46:0/1984202398 >> v1:172.21.15.162:6789/0 conn(0x7f32d00b1880 legacy=0x7f32d015a2f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-02T00:28:58.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.708+0000 7f32c7fff640 1 -- 172.21.15.46:0/1984202398 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d00cd5e0 con 0x7f32d00b45f0 2023-12-02T00:28:58.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.708+0000 7f32c7fff640 1 -- 172.21.15.46:0/1984202398 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7f32cc001670 con 0x7f32d00b45f0 2023-12-02T00:28:58.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.708+0000 7f32c7fff640 1 -- 172.21.15.46:0/1984202398 >> v1:172.21.15.46:6789/0 conn(0x7f32d00b45f0 legacy=0x7f32d0163fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.709+0000 7f32c7fff640 1 --2- 172.21.15.46:0/1984202398 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32bc005bb0 0x7f32bc005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.709+0000 7f70e29b7640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f70dc06cf40 0x7f70dc0b1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.709+0000 7f32c7fff640 1 --2- 172.21.15.46:0/1984202398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32bc0069d0 0x7f32bc006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.709+0000 7f32c7fff640 1 --2- 172.21.15.46:0/1984202398 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32d0159210 0x7f32bc007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.709+0000 7f70e29b7640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f70dc0cc770 0x7f70dc15a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.709+0000 7f32c7fff640 1 -- 172.21.15.46:0/1984202398 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d00b4a80 con 0x7f32bc0069d0 2023-12-02T00:28:58.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.709+0000 7f32c7fff640 1 -- 172.21.15.46:0/1984202398 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc00dc40 con 0x7f32bc005bb0 2023-12-02T00:28:58.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.709+0000 7f32c7fff640 1 -- 172.21.15.46:0/1984202398 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc00de00 con 0x7f32d0159210 2023-12-02T00:28:58.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.707+0000 7f63e5d1d640 1 -- start start 2023-12-02T00:28:58.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.709+0000 7fb87bfff640 1 --2- 172.21.15.46:0/3071224137 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb87c08a850 0x7fb8640039f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb870009860 tx=0x7fb8640057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f8160806fc96144 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:58.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.709+0000 7f63e5d1d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f63d804fc30 0x7f63d8053060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.709+0000 7f63e5d1d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f63d8034210 0x7f63d80535a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.709+0000 7f5f7c17c640 1 -- start start 2023-12-02T00:28:58.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.709+0000 7f63dffff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f63d8034210 0x7f63d80535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.709+0000 7f92873c4640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f92800714e0 0x7f9280071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.709+0000 7f32d5ed6640 1 --2- 172.21.15.46:0/1984202398 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32d0159210 0x7f32bc007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.709+0000 7f32d4ed4640 1 --2- 172.21.15.46:0/1984202398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32bc0069d0 0x7f32bc006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.709+0000 7f92873c4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92800ad940 0x7f92800adda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.709+0000 7f92873c4640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92801434f0 con 0x7f928013fec0 2023-12-02T00:28:58.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.709+0000 7f92873c4640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9280071eb0 con 0x7f92800714e0 2023-12-02T00:28:58.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.709+0000 7f92873c4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f928015e610 con 0x7f92800ad940 2023-12-02T00:28:58.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.709+0000 7fb87affd640 1 -- 172.21.15.46:0/3071224137 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb870032600 con 0x7fb87c08a850 2023-12-02T00:28:58.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.709+0000 7fb87affd640 1 -- 172.21.15.46:0/3071224137 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb87002e050 con 0x7fb87c08a850 2023-12-02T00:28:58.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.709+0000 7f70db7fe640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f70dc06cf40 0x7f70dc0b1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.709+0000 7f70db7fe640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f70dc06cf40 0x7f70dc0b1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.46:44310/0 (socket says 172.21.15.46:44310) 2023-12-02T00:28:58.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.709+0000 7f70dbfff640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f70dc142aa0 0x7f70dc06c830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.46:36770/0 (socket says 172.21.15.46:36770) 2023-12-02T00:28:58.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.709+0000 7f9286bc3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92800ad940 0x7f92800adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.709+0000 7f9286bc3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92800ad940 0x7f92800adda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.46:34910/0 (socket says 172.21.15.46:34910) 2023-12-02T00:28:58.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.709+0000 7f9286bc3640 1 -- 172.21.15.46:0/3838004796 learned_addr learned my addr 172.21.15.46:0/3838004796 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:58.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.709+0000 7fe3332fc640 1 Processor -- start 2023-12-02T00:28:58.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.709+0000 7f70e0f54640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f70dc0cc770 0x7f70dc15a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.709+0000 7f32d56d5640 1 --2- 172.21.15.46:0/1984202398 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32bc005bb0 0x7f32bc005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.709+0000 7fe3332fc640 1 -- start start 2023-12-02T00:28:58.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.710+0000 7f5f7c17c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5f74158690 0x7f5f740b44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.710+0000 7fb88249b640 1 -- 172.21.15.46:0/3071224137 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb87c08a850 msgr2=0x7fb8640039f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.710+0000 7fb88249b640 1 --2- 172.21.15.46:0/3071224137 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb87c08a850 0x7fb8640039f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb870009860 tx=0x7fb8640057f0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.709+0000 7f32d4ed4640 1 -- 172.21.15.46:0/1984202398 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32d0159210 msgr2=0x7f32bc007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:58.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.709+0000 7f32d4ed4640 1 --2- 172.21.15.46:0/1984202398 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32d0159210 0x7f32bc007390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.709+0000 7f32d4ed4640 1 -- 172.21.15.46:0/1984202398 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32bc005bb0 msgr2=0x7f32bc005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:58.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.709+0000 7f32d4ed4640 1 --2- 172.21.15.46:0/1984202398 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32bc005bb0 0x7f32bc005ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.709+0000 7f32d4ed4640 1 -- 172.21.15.46:0/1984202398 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d00cd300 con 0x7f32bc0069d0 2023-12-02T00:28:58.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.709+0000 7f70e0f54640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f70dc0cc770 0x7f70dc15a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37864/0 (socket says 172.21.15.46:37864) 2023-12-02T00:28:58.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.709+0000 7f70db7fe640 1 -- 172.21.15.46:0/1547368191 learned_addr learned my addr 172.21.15.46:0/1547368191 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:58.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.709+0000 7f70e29b7640 1 -- 172.21.15.46:0/1547368191 --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70dc15a640 con 0x7f70dc142aa0 2023-12-02T00:28:58.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.710+0000 7f70e29b7640 1 -- 172.21.15.46:0/1547368191 --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70dc0ccbe0 con 0x7f70dc0cc770 2023-12-02T00:28:58.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.710+0000 7f70e29b7640 1 -- 172.21.15.46:0/1547368191 --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70dc0b2d40 con 0x7f70dc06cf40 2023-12-02T00:28:58.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.710+0000 7f9286bc3640 1 -- 172.21.15.46:0/3838004796 >> v2:172.21.15.46:3300/0 conn(0x7f92800714e0 msgr2=0x7f9280071970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-02T00:28:58.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.710+0000 7f9286bc3640 1 --2- 172.21.15.46:0/3838004796 >> v2:172.21.15.46:3300/0 conn(0x7f92800714e0 0x7f9280071970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.710+0000 7f9286bc3640 1 -- 172.21.15.46:0/3838004796 >> v1:172.21.15.46:6789/0 conn(0x7f928013fec0 legacy=0x7f928015df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:58.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.710+0000 7f9286bc3640 1 -- 172.21.15.46:0/3838004796 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92801428e0 con 0x7f92800ad940 2023-12-02T00:28:58.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.709+0000 7f63dffff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f63d8034210 0x7f63d80535a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37862/0 (socket says 172.21.15.46:37862) 2023-12-02T00:28:58.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.709+0000 7f63dffff640 1 -- 172.21.15.46:0/1044041406 learned_addr learned my addr 172.21.15.46:0/1044041406 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:58.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.709+0000 7f63e5d1d640 1 -- 172.21.15.46:0/1044041406 --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63d8052bd0 con 0x7f63d80e7490 2023-12-02T00:28:58.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.709+0000 7f63e5d1d640 1 -- 172.21.15.46:0/1044041406 --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63d8007270 con 0x7f63d8034210 2023-12-02T00:28:58.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.709+0000 7f63e5d1d640 1 -- 172.21.15.46:0/1044041406 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d8007550 con 0x7f63d804fc30 2023-12-02T00:28:58.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.711+0000 7f5f7c17c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5f740b0cf0 0x7f5f740b1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.711+0000 7fb880a38640 1 -- 172.21.15.46:0/3071224137 reap_dead start 2023-12-02T00:28:58.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.711+0000 7f5f7c17c640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f74162d80 con 0x7f5f741651b0 2023-12-02T00:28:58.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.711+0000 7f8235da0640 1 Processor -- start 2023-12-02T00:28:58.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.711+0000 7fe3332fc640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe32c0b47c0 0x7fe32c0b2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.711+0000 7f5f79717640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5f740b0cf0 0x7f5f740b1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.711+0000 7f5f7a719640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f5f741651b0 0x7f5f74165620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.46:36780/0 (socket says 172.21.15.46:36780) 2023-12-02T00:28:58.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.711+0000 7f5f7a719640 1 -- 172.21.15.46:0/1479749765 learned_addr learned my addr 172.21.15.46:0/1479749765 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:28:58.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.711+0000 7f5f7c17c640 1 -- 172.21.15.46:0/1479749765 --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f740b16c0 con 0x7f5f74158690 2023-12-02T00:28:58.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.711+0000 7fe3332fc640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe32c0cd020 con 0x7fe32c159210 2023-12-02T00:28:58.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.711+0000 7fe3332fc640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe32c0cd300 con 0x7fe32c0b1880 2023-12-02T00:28:58.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.710+0000 7f63dffff640 1 -- 172.21.15.46:0/1044041406 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f63d804fc30 msgr2=0x7f63d8053060 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-02T00:28:58.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.710+0000 7f63dffff640 1 --2- 172.21.15.46:0/1044041406 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f63d804fc30 0x7f63d8053060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.710+0000 7f63dffff640 1 -- 172.21.15.46:0/1044041406 >> v1:172.21.15.162:6789/0 conn(0x7f63d80e7490 legacy=0x7f63d804f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:58.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.710+0000 7f63dffff640 1 -- 172.21.15.46:0/1044041406 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d8007830 con 0x7f63d8034210 2023-12-02T00:28:58.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.711+0000 7fb88249b640 1 -- 172.21.15.46:0/3071224137 shutdown_connections 2023-12-02T00:28:58.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.711+0000 7fb88249b640 1 -- 172.21.15.46:0/3071224137 >> 172.21.15.46:0/3071224137 conn(0x7fb87c0b0d60 msgr2=0x7fb87c0b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:58.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.711+0000 7fe3332fc640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe32c0cd5e0 con 0x7fe32c0b47c0 2023-12-02T00:28:58.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.710+0000 7f70e0f54640 1 -- 172.21.15.46:0/1547368191 >> v2:172.21.15.162:3300/0 conn(0x7f70dc06cf40 msgr2=0x7f70dc0b1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:58.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.710+0000 7f70e0f54640 1 --2- 172.21.15.46:0/1547368191 >> v2:172.21.15.162:3300/0 conn(0x7f70dc06cf40 0x7f70dc0b1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.710+0000 7f70e0f54640 1 -- 172.21.15.46:0/1547368191 >> v1:172.21.15.162:6789/0 conn(0x7f70dc142aa0 legacy=0x7f70dc06c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.710+0000 7f70e0f54640 1 -- 172.21.15.46:0/1547368191 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70dc0b24a0 con 0x7f70dc0cc770 2023-12-02T00:28:58.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.711+0000 7fe331098640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fe32c0b1880 0x7fe32c15a2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.46:36788/0 (socket says 172.21.15.46:36788) 2023-12-02T00:28:58.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.711+0000 7fe331098640 1 -- 172.21.15.46:0/1526221015 learned_addr learned my addr 172.21.15.46:0/1526221015 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:28:58.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.711+0000 7f9286bc3640 1 --2- 172.21.15.46:0/3838004796 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92800ad940 0x7f92800adda0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9270009870 tx=0x7f927002b7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cf69a61704ae0d2 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:58.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.711+0000 7fb88249b640 1 -- 172.21.15.46:0/3071224137 shutdown_connections 2023-12-02T00:28:58.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.711+0000 7fb88249b640 1 -- 172.21.15.46:0/3071224137 wait complete. 2023-12-02T00:28:58.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.711+0000 7f5f7c17c640 1 -- 172.21.15.46:0/1479749765 --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f74070390 con 0x7f5f740b0cf0 2023-12-02T00:28:58.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.711+0000 7f8235da0640 1 -- start start 2023-12-02T00:28:58.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.711+0000 7f5f79f18640 1 --2- 172.21.15.46:0/1479749765 >> v2:172.21.15.46:3300/0 conn(0x7f5f74158690 0x7f5f740b44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.712+0000 7f5f78f16640 1 -- 172.21.15.46:0/1479749765 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1979092088 0 0) 0x7f5f74162d80 con 0x7f5f741651b0 2023-12-02T00:28:58.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.712+0000 7f5f78f16640 1 -- 172.21.15.46:0/1479749765 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f60003520 con 0x7f5f741651b0 2023-12-02T00:28:58.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.712+0000 7f63dffff640 1 --2- 172.21.15.46:0/1044041406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63d8034210 0x7f63d80535a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f63cc009870 tx=0x7f63cc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=857a71c0bd9cb79a server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:58.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.712+0000 7fe331899640 1 --2- 172.21.15.46:0/1526221015 >> v2:172.21.15.46:3300/0 conn(0x7fe32c0b47c0 0x7fe32c0b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.712+0000 7fe323fff640 1 -- 172.21.15.46:0/1526221015 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 455235219 0 0) 0x7fe32c0cd300 con 0x7fe32c0b1880 2023-12-02T00:28:58.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.712+0000 7fe323fff640 1 -- 172.21.15.46:0/1526221015 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe318003520 con 0x7fe32c0b1880 2023-12-02T00:28:58.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.712+0000 7f5f79717640 1 -- 172.21.15.46:0/1479749765 >> v2:172.21.15.46:3300/0 conn(0x7f5f74158690 msgr2=0x7f5f740b44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:58.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.712+0000 7f5f79717640 1 --2- 172.21.15.46:0/1479749765 >> v2:172.21.15.46:3300/0 conn(0x7f5f74158690 0x7f5f740b44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.712+0000 7f5f79717640 1 -- 172.21.15.46:0/1479749765 >> v1:172.21.15.162:6789/0 conn(0x7f5f741651b0 legacy=0x7f5f74165620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.712+0000 7f5f79717640 1 -- 172.21.15.46:0/1479749765 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f740705a0 con 0x7f5f740b0cf0 2023-12-02T00:28:58.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.712+0000 7f70e0f54640 1 --2- 172.21.15.46:0/1547368191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70dc0cc770 0x7f70dc15a100 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f70dc06dd20 tx=0x7f70d402cfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da497418afa7caa0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:58.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.712+0000 7f92853c0640 1 -- 172.21.15.46:0/3838004796 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9270031750 con 0x7f92800ad940 2023-12-02T00:28:58.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.712+0000 7fe323fff640 1 -- 172.21.15.46:0/1526221015 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3299105808 0 0) 0x7fe318003520 con 0x7fe32c0b1880 2023-12-02T00:28:58.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.712+0000 7fe323fff640 1 -- 172.21.15.46:0/1526221015 >> v2:172.21.15.46:3300/0 conn(0x7fe32c0b47c0 msgr2=0x7fe32c0b2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:58.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.712+0000 7fe323fff640 1 --2- 172.21.15.46:0/1526221015 >> v2:172.21.15.46:3300/0 conn(0x7fe32c0b47c0 0x7fe32c0b2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.712+0000 7fe323fff640 1 -- 172.21.15.46:0/1526221015 >> v1:172.21.15.46:6789/0 conn(0x7fe32c159210 legacy=0x7fe32c0b40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.712+0000 7fe323fff640 1 -- 172.21.15.46:0/1526221015 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe32c0cd8c0 con 0x7fe32c0b1880 2023-12-02T00:28:58.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.712+0000 7f63de7fc640 1 -- 172.21.15.46:0/1044041406 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63cc030860 con 0x7f63d8034210 2023-12-02T00:28:58.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.712+0000 7f63de7fc640 1 -- 172.21.15.46:0/1044041406 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63cc0309c0 con 0x7f63d8034210 2023-12-02T00:28:58.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.712+0000 7f92853c0640 1 -- 172.21.15.46:0/3838004796 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92700318b0 con 0x7f92800ad940 2023-12-02T00:28:58.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.712+0000 7fe323fff640 1 -- 172.21.15.46:0/1526221015 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7fe328001670 con 0x7fe32c0b1880 2023-12-02T00:28:58.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.712+0000 7fe323fff640 1 -- 172.21.15.46:0/1526221015 >> v1:172.21.15.162:6789/0 conn(0x7fe32c0b1880 legacy=0x7fe32c15a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.712+0000 7fe323fff640 1 --2- 172.21.15.46:0/1526221015 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe3180039c0 0x7fe318003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.712+0000 7fe323fff640 1 --2- 172.21.15.46:0/1526221015 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3180047c0 0x7fe318004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.712+0000 7fe323fff640 1 --2- 172.21.15.46:0/1526221015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3180051d0 0x7fe31800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.712+0000 7fe323fff640 1 -- 172.21.15.46:0/1526221015 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c0cd5e0 con 0x7fe3180051d0 2023-12-02T00:28:58.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.712+0000 7fe323fff640 1 -- 172.21.15.46:0/1526221015 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe318010530 con 0x7fe3180047c0 2023-12-02T00:28:58.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.712+0000 7fe323fff640 1 -- 172.21.15.46:0/1526221015 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3180106f0 con 0x7fe3180039c0 2023-12-02T00:28:58.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.712+0000 7f92853c0640 1 -- 172.21.15.46:0/3838004796 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9270031c40 con 0x7f92800ad940 2023-12-02T00:28:58.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.712+0000 7fe331098640 1 --2- 172.21.15.46:0/1526221015 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe3180039c0 0x7fe318003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.712+0000 7f70daffd640 1 -- 172.21.15.46:0/1547368191 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d402db70 con 0x7f70dc0cc770 2023-12-02T00:28:58.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.712+0000 7f70daffd640 1 -- 172.21.15.46:0/1547368191 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d402dcd0 con 0x7f70dc0cc770 2023-12-02T00:28:58.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.712+0000 7f8235da0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8230154580 0x7f823006f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.712+0000 7f70daffd640 1 -- 172.21.15.46:0/1547368191 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d4008660 con 0x7f70dc0cc770 2023-12-02T00:28:58.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.712+0000 7f8235da0640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f82300b45f0 0x7f823006a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.713+0000 7f8235da0640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f823015acb0 con 0x7f823006d0b0 2023-12-02T00:28:58.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.713+0000 7f8235da0640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82300addc0 con 0x7f82300b45f0 2023-12-02T00:28:58.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.713+0000 7f8235da0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f823006d820 con 0x7f8230154580 2023-12-02T00:28:58.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.713+0000 7f32d4ed4640 1 --2- 172.21.15.46:0/1984202398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32bc0069d0 0x7f32bc006e50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f32c8009f90 tx=0x7f32bc007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e96f6f598034bdf4 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:58.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.713+0000 7f823559f640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f82300b45f0 0x7f823006a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.713+0000 7fe330897640 1 --2- 172.21.15.46:0/1526221015 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3180047c0 0x7fe318004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.713+0000 7f823559f640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f82300b45f0 0x7f823006a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.46:44340/0 (socket says 172.21.15.46:44340) 2023-12-02T00:28:58.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.713+0000 7f823559f640 1 -- 172.21.15.46:0/3778380071 learned_addr learned my addr 172.21.15.46:0/3778380071 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:58.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.713+0000 7f32c7fff640 1 -- 172.21.15.46:0/1984202398 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c802cba0 con 0x7f32bc0069d0 2023-12-02T00:28:58.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.713+0000 7f32c7fff640 1 -- 172.21.15.46:0/1984202398 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c800f040 con 0x7f32bc0069d0 2023-12-02T00:28:58.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.713+0000 7f32c7fff640 1 -- 172.21.15.46:0/1984202398 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c8034440 con 0x7f32bc0069d0 2023-12-02T00:28:58.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.713+0000 7f8234d9e640 1 --2- 172.21.15.46:0/3778380071 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8230154580 0x7f823006f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.713+0000 7f5f79717640 1 --2- 172.21.15.46:0/1479749765 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f740b0cf0 0x7f5f740b1180 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5f7415c230 tx=0x7f5f6402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92294fb663bdcfb9 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:58.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.713+0000 7f63e5d1d640 1 -- 172.21.15.46:0/1044041406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63d8034210 msgr2=0x7f63d80535a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.713+0000 7f63e5d1d640 1 --2- 172.21.15.46:0/1044041406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63d8034210 0x7f63d80535a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f63cc009870 tx=0x7f63cc02b9a0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.713+0000 7f63e5d1d640 1 -- 172.21.15.46:0/1044041406 shutdown_connections 2023-12-02T00:28:58.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.713+0000 7f63e5d1d640 1 --2- 172.21.15.46:0/1044041406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63d8034210 0x7f63d80535a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.713+0000 7f63e5d1d640 1 --2- 172.21.15.46:0/1044041406 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f63d804fc30 0x7f63d8053060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.713+0000 7f63e5d1d640 1 -- 172.21.15.46:0/1044041406 >> 172.21.15.46:0/1044041406 conn(0x7f63d8054ed0 msgr2=0x7f63d80552e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:58.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.713+0000 7f92873c4640 1 -- 172.21.15.46:0/3838004796 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92800ad940 msgr2=0x7f92800adda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.713+0000 7f63e5d1d640 1 -- 172.21.15.46:0/1044041406 shutdown_connections 2023-12-02T00:28:58.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.713+0000 7f92873c4640 1 --2- 172.21.15.46:0/3838004796 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92800ad940 0x7f92800adda0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9270009870 tx=0x7f927002b7f0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.713+0000 7f70e29b7640 1 -- 172.21.15.46:0/1547368191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70dc0cc770 msgr2=0x7f70dc15a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.713+0000 7f92873c4640 1 -- 172.21.15.46:0/3838004796 shutdown_connections 2023-12-02T00:28:58.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.714+0000 7f70e29b7640 1 --2- 172.21.15.46:0/1547368191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70dc0cc770 0x7f70dc15a100 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f70dc06dd20 tx=0x7f70d402cfd0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.713+0000 7fb88249b640 1 Processor -- start 2023-12-02T00:28:58.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.713+0000 7fb88249b640 1 -- start start 2023-12-02T00:28:58.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.714+0000 7f92873c4640 1 --2- 172.21.15.46:0/3838004796 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92800ad940 0x7f92800adda0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.714+0000 7f92873c4640 1 --2- 172.21.15.46:0/3838004796 >> v2:172.21.15.46:3300/0 conn(0x7f92800714e0 0x7f9280071970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.714+0000 7f92873c4640 1 -- 172.21.15.46:0/3838004796 >> 172.21.15.46:0/3838004796 conn(0x7f928006a4d0 msgr2=0x7f928006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:58.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.713+0000 7fe331098640 1 -- 172.21.15.46:0/1526221015 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3180047c0 msgr2=0x7fe318004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:58.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.713+0000 7fe331098640 1 --2- 172.21.15.46:0/1526221015 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3180047c0 0x7fe318004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.713+0000 7fe331098640 1 -- 172.21.15.46:0/1526221015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3180051d0 msgr2=0x7fe31800b980 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-02T00:28:58.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.713+0000 7fe331098640 1 --2- 172.21.15.46:0/1526221015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3180051d0 0x7fe31800b980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.713+0000 7fe331098640 1 -- 172.21.15.46:0/1526221015 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe318003520 con 0x7fe3180039c0 2023-12-02T00:28:58.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.713+0000 7fe331899640 1 -- 172.21.15.46:0/1526221015 reap_dead start 2023-12-02T00:28:58.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.714+0000 7f70e29b7640 1 -- 172.21.15.46:0/1547368191 shutdown_connections 2023-12-02T00:28:58.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.714+0000 7f70e29b7640 1 --2- 172.21.15.46:0/1547368191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70dc0cc770 0x7f70dc15a100 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.714+0000 7f70e29b7640 1 --2- 172.21.15.46:0/1547368191 >> v2:172.21.15.162:3300/0 conn(0x7f70dc06cf40 0x7f70dc0b1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.713+0000 7f63e5d1d640 1 -- 172.21.15.46:0/1044041406 wait complete. 2023-12-02T00:28:58.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.714+0000 7f70e29b7640 1 -- 172.21.15.46:0/1547368191 >> 172.21.15.46:0/1547368191 conn(0x7f70dc06a6b0 msgr2=0x7f70dc06aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:58.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.714+0000 7f823559f640 1 -- 172.21.15.46:0/3778380071 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8230154580 msgr2=0x7f823006f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:58.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.714+0000 7f823559f640 1 --2- 172.21.15.46:0/3778380071 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8230154580 0x7f823006f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.714+0000 7f823559f640 1 -- 172.21.15.46:0/3778380071 >> v1:172.21.15.46:6789/0 conn(0x7f823006d0b0 legacy=0x7f82300b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.714+0000 7f5f78f16640 1 -- 172.21.15.46:0/1479749765 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f64031830 con 0x7f5f740b0cf0 2023-12-02T00:28:58.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.714+0000 7f5f78f16640 1 -- 172.21.15.46:0/1479749765 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f64031990 con 0x7f5f740b0cf0 2023-12-02T00:28:58.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.714+0000 7f5f78f16640 1 -- 172.21.15.46:0/1479749765 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f64031d20 con 0x7f5f740b0cf0 2023-12-02T00:28:58.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.714+0000 7f823559f640 1 -- 172.21.15.46:0/3778380071 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f823015a660 con 0x7f82300b45f0 2023-12-02T00:28:58.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.714+0000 7f32d7939640 1 -- 172.21.15.46:0/1984202398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32bc0069d0 msgr2=0x7f32bc006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.714+0000 7f32d7939640 1 --2- 172.21.15.46:0/1984202398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32bc0069d0 0x7f32bc006e50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f32c8009f90 tx=0x7f32bc007a00 comp rx=0 tx=0).stop 2023-12-02T00:28:58.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.714+0000 7f32d5ed6640 1 -- 172.21.15.46:0/1984202398 reap_dead start 2023-12-02T00:28:58.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.714+0000 7f32d7939640 1 -- 172.21.15.46:0/1984202398 shutdown_connections 2023-12-02T00:28:58.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.714+0000 7f32d7939640 1 -- 172.21.15.46:0/1984202398 >> 172.21.15.46:0/1984202398 conn(0x7f32d00b0d60 msgr2=0x7f32d00b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:58.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.714+0000 7f32d7939640 1 -- 172.21.15.46:0/1984202398 shutdown_connections 2023-12-02T00:28:58.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.714+0000 7f32d7939640 1 -- 172.21.15.46:0/1984202398 wait complete. 2023-12-02T00:28:58.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.714+0000 7f70e29b7640 1 -- 172.21.15.46:0/1547368191 shutdown_connections 2023-12-02T00:28:58.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.714+0000 7f70e29b7640 1 -- 172.21.15.46:0/1547368191 wait complete. 2023-12-02T00:28:58.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.714+0000 7f92873c4640 1 -- 172.21.15.46:0/3838004796 shutdown_connections 2023-12-02T00:28:58.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.714+0000 7f92873c4640 1 -- 172.21.15.46:0/3838004796 wait complete. 2023-12-02T00:28:58.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.715+0000 7fb88249b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb87c08a850 0x7fb87c16fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.715+0000 7fb88249b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb87c0b2630 0x7fb87c158380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.715+0000 7f8234d9e640 1 --2- 172.21.15.46:0/3778380071 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8230154580 0x7f823006f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:28:58.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.715+0000 7fb88249b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb87c1588c0 0x7fb87c0a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.715+0000 7fb88249b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c089c70 con 0x7fb87c1588c0 2023-12-02T00:28:58.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.715+0000 7fb88249b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c0ad560 con 0x7fb87c08a850 2023-12-02T00:28:58.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.715+0000 7f63e5d1d640 1 Processor -- start 2023-12-02T00:28:58.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.715+0000 7f5f7c17c640 1 -- 172.21.15.46:0/1479749765 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f740b0cf0 msgr2=0x7f5f740b1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.715+0000 7fb88249b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c08a130 con 0x7fb87c0b2630 2023-12-02T00:28:58.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.715+0000 7fb87bfff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb87c08a850 0x7fb87c16fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.715+0000 7fb87bfff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb87c08a850 0x7fb87c16fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.46:44342/0 (socket says 172.21.15.46:44342) 2023-12-02T00:28:58.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.715+0000 7fb87bfff640 1 -- 172.21.15.46:0/1743745081 learned_addr learned my addr 172.21.15.46:0/1743745081 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:58.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.715+0000 7f5f7c17c640 1 --2- 172.21.15.46:0/1479749765 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f740b0cf0 0x7f5f740b1180 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5f7415c230 tx=0x7f5f6402b8d0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.715+0000 7fe331098640 1 --2- 172.21.15.46:0/1526221015 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe3180039c0 0x7fe318003de0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe32800c190 tx=0x7fe3180057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b356f10e547eded7 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:58.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.716+0000 7f5f7c17c640 1 -- 172.21.15.46:0/1479749765 shutdown_connections 2023-12-02T00:28:58.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.716+0000 7f5f7c17c640 1 --2- 172.21.15.46:0/1479749765 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f740b0cf0 0x7f5f740b1180 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.716+0000 7f5f7c17c640 1 --2- 172.21.15.46:0/1479749765 >> v2:172.21.15.46:3300/0 conn(0x7f5f74158690 0x7f5f740b44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.716+0000 7f5f7c17c640 1 -- 172.21.15.46:0/1479749765 >> 172.21.15.46:0/1479749765 conn(0x7f5f7406a4d0 msgr2=0x7f5f7406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:58.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.716+0000 7f823559f640 1 --2- 172.21.15.46:0/3778380071 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82300b45f0 0x7f823006a4d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8220009870 tx=0x7f822002b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3f12a368c5370bb server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:58.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.715+0000 7fb880a38640 1 --2- 172.21.15.46:0/1743745081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb87c1588c0 0x7fb87c0a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.716+0000 7fe323fff640 1 -- 172.21.15.46:0/1526221015 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe328005ad0 con 0x7fe3180039c0 2023-12-02T00:28:58.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.716+0000 7fe323fff640 1 -- 172.21.15.46:0/1526221015 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe328005c50 con 0x7fe3180039c0 2023-12-02T00:28:58.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.716+0000 7fe323fff640 1 -- 172.21.15.46:0/1526221015 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32802c830 con 0x7fe3180039c0 2023-12-02T00:28:58.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.716+0000 7f92873c4640 1 Processor -- start 2023-12-02T00:28:58.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.716+0000 7f63e5d1d640 1 -- start start 2023-12-02T00:28:58.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.716+0000 7f822f7fe640 1 -- 172.21.15.46:0/3778380071 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8220031870 con 0x7f82300b45f0 2023-12-02T00:28:58.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.716+0000 7f822f7fe640 1 -- 172.21.15.46:0/3778380071 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82200319d0 con 0x7f82300b45f0 2023-12-02T00:28:58.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.716+0000 7f822f7fe640 1 -- 172.21.15.46:0/3778380071 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8220031870 con 0x7f82300b45f0 2023-12-02T00:28:58.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.716+0000 7f92873c4640 1 -- start start 2023-12-02T00:28:58.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.716+0000 7fb87b7fe640 1 --2- 172.21.15.46:0/1743745081 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb87c0b2630 0x7fb87c158380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.716+0000 7f32d7939640 1 Processor -- start 2023-12-02T00:28:58.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.716+0000 7fb87bfff640 1 -- 172.21.15.46:0/1743745081 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb87c0b2630 msgr2=0x7fb87c158380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.716+0000 7fb87bfff640 1 --2- 172.21.15.46:0/1743745081 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb87c0b2630 0x7fb87c158380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.716+0000 7fb87bfff640 1 -- 172.21.15.46:0/1743745081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb87c1588c0 msgr2=0x7fb87c0a84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.716+0000 7fb87bfff640 1 --2- 172.21.15.46:0/1743745081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb87c1588c0 0x7fb87c0a84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.716+0000 7fb87bfff640 1 -- 172.21.15.46:0/1743745081 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb87c089e50 con 0x7fb87c08a850 2023-12-02T00:28:58.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.716+0000 7fb880a38640 1 --2- 172.21.15.46:0/1743745081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb87c1588c0 0x7fb87c0a84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:28:58.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.716+0000 7f32d7939640 1 -- start start 2023-12-02T00:28:58.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.716+0000 7f5f7c17c640 1 -- 172.21.15.46:0/1479749765 shutdown_connections 2023-12-02T00:28:58.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.716+0000 7f70e29b7640 1 Processor -- start 2023-12-02T00:28:58.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.717+0000 7f92873c4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92800714e0 0x7f92800b1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.717+0000 7f70e29b7640 1 -- start start 2023-12-02T00:28:58.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.717+0000 7f92873c4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92800ad940 0x7f9280155110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.717+0000 7f92873c4640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f928013fec0 0x7f928015e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.717+0000 7f92873c4640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9280154cf0 con 0x7f92800714e0 2023-12-02T00:28:58.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.717+0000 7f92873c4640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92801546c0 con 0x7f928013fec0 2023-12-02T00:28:58.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.717+0000 7f92873c4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f928015f210 con 0x7f92800ad940 2023-12-02T00:28:58.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.717+0000 7fb87bfff640 1 --2- 172.21.15.46:0/1743745081 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb87c08a850 0x7fb87c16fed0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb87002bc90 tx=0x7fb8700047d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:58.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.717+0000 7f5f7c17c640 1 -- 172.21.15.46:0/1479749765 wait complete. 2023-12-02T00:28:58.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.717+0000 7f92863c2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92800714e0 0x7f92800b1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.717+0000 7fe3332fc640 1 -- 172.21.15.46:0/1526221015 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe3180039c0 msgr2=0x7fe318003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.717+0000 7fe3332fc640 1 --2- 172.21.15.46:0/1526221015 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe3180039c0 0x7fe318003de0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe32800c190 tx=0x7fe3180057f0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.717+0000 7f92863c2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92800714e0 0x7f92800b1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37916/0 (socket says 172.21.15.46:37916) 2023-12-02T00:28:58.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.717+0000 7f92863c2640 1 -- 172.21.15.46:0/4190831256 learned_addr learned my addr 172.21.15.46:0/4190831256 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:58.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.717+0000 7f63e5d1d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f63d804fc30 0x7f63d8037a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.717+0000 7f63e5d1d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63d80e7490 0x7f63d804adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.717+0000 7f63e5d1d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63d80d4680 0x7f63d804d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.717+0000 7f63e5d1d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d8049d20 con 0x7f63d80d4680 2023-12-02T00:28:58.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.717+0000 7f63e5d1d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d804a000 con 0x7f63d80e7490 2023-12-02T00:28:58.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.717+0000 7f63e5d1d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d804a2e0 con 0x7f63d804fc30 2023-12-02T00:28:58.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.717+0000 7f9286bc3640 1 --2- 172.21.15.46:0/4190831256 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f928013fec0 0x7f928015e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.717+0000 7f9285bc1640 1 --2- 172.21.15.46:0/4190831256 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92800ad940 0x7f9280155110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.717+0000 7f8235da0640 1 -- 172.21.15.46:0/3778380071 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82300b45f0 msgr2=0x7f823006a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.717+0000 7f8235da0640 1 --2- 172.21.15.46:0/3778380071 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82300b45f0 0x7f823006a4d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8220009870 tx=0x7f822002b910 comp rx=0 tx=0).stop 2023-12-02T00:28:58.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.718+0000 7f32d7939640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32bc0069d0 0x7f32d00a87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.718+0000 7f32d7939640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32d015a1f0 0x7f32d014e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.718+0000 7f32d7939640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32d015ba30 0x7f32d00ad990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.718+0000 7fe3332fc640 1 -- 172.21.15.46:0/1526221015 shutdown_connections 2023-12-02T00:28:58.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.718+0000 7fe3332fc640 1 --2- 172.21.15.46:0/1526221015 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe3180039c0 0x7fe318003de0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.718+0000 7fe3332fc640 1 -- 172.21.15.46:0/1526221015 >> 172.21.15.46:0/1526221015 conn(0x7fe32c0b0d60 msgr2=0x7fe32c0b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:58.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.718+0000 7fb878ff9640 1 -- 172.21.15.46:0/1743745081 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb870003a00 con 0x7fb87c08a850 2023-12-02T00:28:58.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.718+0000 7fb878ff9640 1 -- 172.21.15.46:0/1743745081 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb870003b60 con 0x7fb87c08a850 2023-12-02T00:28:58.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.718+0000 7f63df7fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f63d804fc30 0x7f63d8037a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.718+0000 7f63dffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63d80d4680 0x7f63d804d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.718+0000 7f63df7fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f63d804fc30 0x7f63d8037a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.46:34980/0 (socket says 172.21.15.46:34980) 2023-12-02T00:28:58.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.718+0000 7f63df7fe640 1 -- 172.21.15.46:0/4197687937 learned_addr learned my addr 172.21.15.46:0/4197687937 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:58.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.718+0000 7fb878ff9640 1 -- 172.21.15.46:0/1743745081 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8700325f0 con 0x7fb87c08a850 2023-12-02T00:28:58.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.718+0000 7fb88249b640 1 -- 172.21.15.46:0/1743745081 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb87c0ad770 con 0x7fb87c08a850 2023-12-02T00:28:58.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.718+0000 7fb88249b640 1 -- 172.21.15.46:0/1743745081 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb87c0add30 con 0x7fb87c08a850 2023-12-02T00:28:58.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.718+0000 7f92863c2640 1 -- 172.21.15.46:0/4190831256 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92800ad940 msgr2=0x7f9280155110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.718+0000 7f92863c2640 1 --2- 172.21.15.46:0/4190831256 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92800ad940 0x7f9280155110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.718+0000 7f92863c2640 1 -- 172.21.15.46:0/4190831256 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f928013fec0 msgr2=0x7f928015e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.718+0000 7f92863c2640 1 --2- 172.21.15.46:0/4190831256 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f928013fec0 0x7f928015e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.718+0000 7f92863c2640 1 -- 172.21.15.46:0/4190831256 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92700094d0 con 0x7f92800714e0 2023-12-02T00:28:58.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.718+0000 7f32d7939640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d014eb30 con 0x7f32bc0069d0 2023-12-02T00:28:58.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.718+0000 7f32d7939640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d015a990 con 0x7f32d015ba30 2023-12-02T00:28:58.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.718+0000 7f32d7939640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d015ac70 con 0x7f32d015a1f0 2023-12-02T00:28:58.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.718+0000 7f32d56d5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32bc0069d0 0x7f32d00a87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.718+0000 7f32d5ed6640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32d015ba30 0x7f32d00ad990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.718+0000 7f32d5ed6640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32d015ba30 0x7f32d00ad990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.46:44354/0 (socket says 172.21.15.46:44354) 2023-12-02T00:28:58.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.718+0000 7f32d5ed6640 1 -- 172.21.15.46:0/2043782315 learned_addr learned my addr 172.21.15.46:0/2043782315 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:58.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.718+0000 7f9286bc3640 1 --2- 172.21.15.46:0/4190831256 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f928013fec0 0x7f928015e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:28:58.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.718+0000 7f8235da0640 1 -- 172.21.15.46:0/3778380071 shutdown_connections 2023-12-02T00:28:58.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.718+0000 7f8235da0640 1 --2- 172.21.15.46:0/3778380071 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82300b45f0 0x7f823006a4d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.718+0000 7f8235da0640 1 --2- 172.21.15.46:0/3778380071 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8230154580 0x7f823006f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.718+0000 7f8235da0640 1 -- 172.21.15.46:0/3778380071 >> 172.21.15.46:0/3778380071 conn(0x7f8230071330 msgr2=0x7f8230071740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:58.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.718+0000 7f70e29b7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f70dc06cf40 0x7f70dc1eee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.718+0000 7f32d4ed4640 1 --2- 172.21.15.46:0/2043782315 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32d015a1f0 0x7f32d014e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.719+0000 7f70e29b7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70dc0cc770 0x7f70dc0a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.718+0000 7f63dffff640 1 -- 172.21.15.46:0/4197687937 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f63d804fc30 msgr2=0x7f63d8037a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.718+0000 7f63dffff640 1 --2- 172.21.15.46:0/4197687937 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f63d804fc30 0x7f63d8037a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.718+0000 7f63dffff640 1 -- 172.21.15.46:0/4197687937 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63d80e7490 msgr2=0x7f63d804adc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-02T00:28:58.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.718+0000 7f63dffff640 1 --2- 172.21.15.46:0/4197687937 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63d80e7490 0x7f63d804adc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.718+0000 7f63dffff640 1 -- 172.21.15.46:0/4197687937 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63cc0094d0 con 0x7f63d80d4680 2023-12-02T00:28:58.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.718+0000 7f32d5ed6640 1 -- 172.21.15.46:0/2043782315 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32d015a1f0 msgr2=0x7f32d014e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.719+0000 7f92863c2640 1 --2- 172.21.15.46:0/4190831256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92800714e0 0x7f92800b1540 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f927c00b3e0 tx=0x7f927c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:58.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.719+0000 7f70e29b7640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f70dc142aa0 0x7f70dc0ca4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.719+0000 7f70e29b7640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70dc0a97d0 con 0x7f70dc0cc770 2023-12-02T00:28:58.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.719+0000 7f70e29b7640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70dc0a8a10 con 0x7f70dc142aa0 2023-12-02T00:28:58.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.719+0000 7f70e29b7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70dc0a8c50 con 0x7f70dc06cf40 2023-12-02T00:28:58.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.719+0000 7f70dbfff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f70dc06cf40 0x7f70dc1eee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.719+0000 7fe3332fc640 1 -- 172.21.15.46:0/1526221015 shutdown_connections 2023-12-02T00:28:58.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.719+0000 7f70dbfff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f70dc06cf40 0x7f70dc1eee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.46:35002/0 (socket says 172.21.15.46:35002) 2023-12-02T00:28:58.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.719+0000 7f70db7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70dc0cc770 0x7f70dc0a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.719+0000 7f70db7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70dc0cc770 0x7f70dc0a84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37944/0 (socket says 172.21.15.46:37944) 2023-12-02T00:28:58.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.719+0000 7f5f7c17c640 1 Processor -- start 2023-12-02T00:28:58.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.719+0000 7f70dbfff640 1 -- 172.21.15.46:0/2535793539 learned_addr learned my addr 172.21.15.46:0/2535793539 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:58.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.719+0000 7f5f7c17c640 1 -- start start 2023-12-02T00:28:58.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.719+0000 7f8235da0640 1 -- 172.21.15.46:0/3778380071 shutdown_connections 2023-12-02T00:28:58.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.719+0000 7fb88249b640 1 -- 172.21.15.46:0/1743745081 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb87c0b4740 con 0x7fb87c08a850 2023-12-02T00:28:58.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.719+0000 7f63dffff640 1 --2- 172.21.15.46:0/4197687937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63d80d4680 0x7f63d804d470 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f63cc033000 tx=0x7f63cc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:58.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.719+0000 7f32d5ed6640 1 --2- 172.21.15.46:0/2043782315 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32d015a1f0 0x7f32d014e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.719+0000 7f32d5ed6640 1 -- 172.21.15.46:0/2043782315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32bc0069d0 msgr2=0x7f32d00a87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.719+0000 7f32d5ed6640 1 --2- 172.21.15.46:0/2043782315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32bc0069d0 0x7f32d00a87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.719+0000 7f32d5ed6640 1 -- 172.21.15.46:0/2043782315 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d015af50 con 0x7f32d015ba30 2023-12-02T00:28:58.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.719+0000 7f70e0f54640 1 --2- 172.21.15.46:0/2535793539 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f70dc142aa0 0x7f70dc0ca4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.719+0000 7f9276ffd640 1 -- 172.21.15.46:0/4190831256 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c011830 con 0x7f92800714e0 2023-12-02T00:28:58.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.719+0000 7f70dbfff640 1 -- 172.21.15.46:0/2535793539 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f70dc142aa0 msgr2=0x7f70dc0ca4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.719+0000 7f9276ffd640 1 -- 172.21.15.46:0/4190831256 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f927c0119b0 con 0x7f92800714e0 2023-12-02T00:28:58.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.719+0000 7f70dbfff640 1 --2- 172.21.15.46:0/2535793539 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f70dc142aa0 0x7f70dc0ca4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.719+0000 7f70dbfff640 1 -- 172.21.15.46:0/2535793539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70dc0cc770 msgr2=0x7f70dc0a84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.719+0000 7f70dbfff640 1 --2- 172.21.15.46:0/2535793539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70dc0cc770 0x7f70dc0a84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.719+0000 7f32d5ed6640 1 --2- 172.21.15.46:0/2043782315 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32d015ba30 0x7f32d00ad990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f32d00b1c60 tx=0x7f32cc00fb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:58.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.720+0000 7f70dbfff640 1 -- 172.21.15.46:0/2535793539 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d400ab80 con 0x7f70dc06cf40 2023-12-02T00:28:58.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.719+0000 7f92873c4640 1 -- 172.21.15.46:0/4190831256 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f928015f4f0 con 0x7f92800714e0 2023-12-02T00:28:58.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.719+0000 7f92873c4640 1 -- 172.21.15.46:0/4190831256 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92801600e0 con 0x7f92800714e0 2023-12-02T00:28:58.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.720+0000 7f9276ffd640 1 -- 172.21.15.46:0/4190831256 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c011d60 con 0x7f92800714e0 2023-12-02T00:28:58.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.720+0000 7f70dbfff640 1 --2- 172.21.15.46:0/2535793539 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f70dc06cf40 0x7f70dc1eee40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f70cc00c3c0 tx=0x7f70cc00c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:58.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.720+0000 7f5f7c17c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f740b0cf0 0x7f5f74151fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.720+0000 7f5f7c17c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f74158690 0x7f5f7415d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.720+0000 7f5f7c17c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f741651b0 0x7f5f7415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.720+0000 7f5f7c17c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f740cc8d0 con 0x7f5f74158690 2023-12-02T00:28:58.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.720+0000 7f5f7c17c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f740ccbb0 con 0x7f5f741651b0 2023-12-02T00:28:58.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.720+0000 7f5f7c17c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f740cce90 con 0x7f5f740b0cf0 2023-12-02T00:28:58.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.720+0000 7f5f79f18640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f740b0cf0 0x7f5f74151fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.721+0000 7f5f7a719640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f741651b0 0x7f5f7415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.720+0000 7fb878ff9640 1 -- 172.21.15.46:0/1743745081 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb87003e430 con 0x7fb87c08a850 2023-12-02T00:28:58.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.720+0000 7f32c5ffb640 1 -- 172.21.15.46:0/2043782315 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32cc0047e0 con 0x7f32d015ba30 2023-12-02T00:28:58.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.720+0000 7f32c5ffb640 1 -- 172.21.15.46:0/2043782315 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32cc004960 con 0x7f32d015ba30 2023-12-02T00:28:58.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.720+0000 7f32c5ffb640 1 -- 172.21.15.46:0/2043782315 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32cc004d10 con 0x7f32d015ba30 2023-12-02T00:28:58.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.720+0000 7f70d8ff9640 1 -- 172.21.15.46:0/2535793539 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70cc0074e0 con 0x7f70dc06cf40 2023-12-02T00:28:58.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.720+0000 7f70d8ff9640 1 -- 172.21.15.46:0/2535793539 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70cc007640 con 0x7f70dc06cf40 2023-12-02T00:28:58.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.720+0000 7f70e29b7640 1 -- 172.21.15.46:0/2535793539 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70dc0a8f30 con 0x7f70dc06cf40 2023-12-02T00:28:58.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.721+0000 7f92873c4640 1 -- 172.21.15.46:0/4190831256 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9268000fc0 con 0x7f92800714e0 2023-12-02T00:28:58.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.719+0000 7fe3332fc640 1 -- 172.21.15.46:0/1526221015 wait complete. 2023-12-02T00:28:58.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.721+0000 7fe3332fc640 1 Processor -- start 2023-12-02T00:28:58.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.721+0000 7f5f79f18640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f740b0cf0 0x7f5f74151fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.46:35018/0 (socket says 172.21.15.46:35018) 2023-12-02T00:28:58.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.721+0000 7f5f79f18640 1 -- 172.21.15.46:0/3616434915 learned_addr learned my addr 172.21.15.46:0/3616434915 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:58.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.721+0000 7fb878ff9640 1 --2- 172.21.15.46:0/1743745081 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb86806af10 0x7fb86806d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.720+0000 7f32d7939640 1 -- 172.21.15.46:0/2043782315 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32d015b230 con 0x7f32d015ba30 2023-12-02T00:28:58.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.721+0000 7f32d7939640 1 -- 172.21.15.46:0/2043782315 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32d015a650 con 0x7f32d015ba30 2023-12-02T00:28:58.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.721+0000 7fb878ff9640 1 -- 172.21.15.46:0/1743745081 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb8700ab4d0 con 0x7fb87c08a850 2023-12-02T00:28:58.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.721+0000 7f70d8ff9640 1 -- 172.21.15.46:0/2535793539 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70cc004a00 con 0x7f70dc06cf40 2023-12-02T00:28:58.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.721+0000 7f70e29b7640 1 -- 172.21.15.46:0/2535793539 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70dc0a94f0 con 0x7f70dc06cf40 2023-12-02T00:28:58.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.721+0000 7fb878ff9640 1 --2- 172.21.15.46:0/1743745081 >> [v2:172.21.15.46:6824/878860153,v1:172.21.15.46:6825/878860153] conn(0x7fb868073130 0x7fb868075550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.721+0000 7fb878ff9640 1 -- 172.21.15.46:0/1743745081 --> [v2:172.21.15.46:6824/878860153,v1:172.21.15.46:6825/878860153] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb868075c00 con 0x7fb868073130 2023-12-02T00:28:58.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.721+0000 7fb878ff9640 1 -- 172.21.15.46:0/1743745081 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8700adaf0 con 0x7fb87c08a850 2023-12-02T00:28:58.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.721+0000 7f5f79f18640 1 -- 172.21.15.46:0/3616434915 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f741651b0 msgr2=0x7f5f7415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.721+0000 7f5f79f18640 1 --2- 172.21.15.46:0/3616434915 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f741651b0 0x7f5f7415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.721+0000 7f5f79f18640 1 -- 172.21.15.46:0/3616434915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f74158690 msgr2=0x7f5f7415d630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-02T00:28:58.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.721+0000 7fb87b7fe640 1 --2- 172.21.15.46:0/1743745081 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb86806af10 0x7fb86806d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.721+0000 7f5f79f18640 1 --2- 172.21.15.46:0/3616434915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f74158690 0x7f5f7415d630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.721+0000 7f5f79f18640 1 -- 172.21.15.46:0/3616434915 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f64009400 con 0x7f5f740b0cf0 2023-12-02T00:28:58.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.722+0000 7fb880a38640 1 --2- 172.21.15.46:0/1743745081 >> [v2:172.21.15.46:6824/878860153,v1:172.21.15.46:6825/878860153] conn(0x7fb868073130 0x7fb868075550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.722+0000 7f8235da0640 1 -- 172.21.15.46:0/3778380071 wait complete. 2023-12-02T00:28:58.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.721+0000 7fe3332fc640 1 -- start start 2023-12-02T00:28:58.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.722+0000 7f63cbfff640 1 -- 172.21.15.46:0/4197687937 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63cc039d90 con 0x7f63d80d4680 2023-12-02T00:28:58.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.722+0000 7f5f79f18640 1 --2- 172.21.15.46:0/3616434915 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f740b0cf0 0x7f5f74151fe0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5f7000e9b0 tx=0x7f5f7000ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:58.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.722+0000 7fe3332fc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3180039c0 0x7fe32c148a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.722+0000 7f32d7939640 1 -- 172.21.15.46:0/2043782315 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f32d0159690 con 0x7f32d015ba30 2023-12-02T00:28:58.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.722+0000 7fe3332fc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe32c157ec0 0x7fe32c1f8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.722+0000 7fe3332fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe32c15b560 0x7fe32c15b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.722+0000 7f5f6a7fc640 1 -- 172.21.15.46:0/3616434915 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f70009670 con 0x7f5f740b0cf0 2023-12-02T00:28:58.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.722+0000 7f5f6a7fc640 1 -- 172.21.15.46:0/3616434915 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f7000c9a0 con 0x7f5f740b0cf0 2023-12-02T00:28:58.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.722+0000 7f5f6a7fc640 1 -- 172.21.15.46:0/3616434915 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7000cd50 con 0x7f5f740b0cf0 2023-12-02T00:28:58.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.722+0000 7f5f7c17c640 1 -- 172.21.15.46:0/3616434915 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f740cd170 con 0x7f5f740b0cf0 2023-12-02T00:28:58.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.722+0000 7fe3332fc640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c148f80 con 0x7fe32c15b560 2023-12-02T00:28:58.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.722+0000 7fe3332fc640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c15bf30 con 0x7fe3180039c0 2023-12-02T00:28:58.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.722+0000 7fe3332fc640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c157870 con 0x7fe32c157ec0 2023-12-02T00:28:58.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.722+0000 7f5f7c17c640 1 -- 172.21.15.46:0/3616434915 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f74152520 con 0x7f5f740b0cf0 2023-12-02T00:28:58.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.722+0000 7fb880a38640 1 --2- 172.21.15.46:0/1743745081 >> [v2:172.21.15.46:6824/878860153,v1:172.21.15.46:6825/878860153] conn(0x7fb868073130 0x7fb868075550 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:58.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.722+0000 7fe330897640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe32c157ec0 0x7fe32c1f8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.722+0000 7fe330897640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe32c157ec0 0x7fe32c1f8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.46:35024/0 (socket says 172.21.15.46:35024) 2023-12-02T00:28:58.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.722+0000 7fe330897640 1 -- 172.21.15.46:0/3201185982 learned_addr learned my addr 172.21.15.46:0/3201185982 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:58.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.722+0000 7fe331098640 1 --2- 172.21.15.46:0/3201185982 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3180039c0 0x7fe32c148a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.723+0000 7fe331899640 1 --2- 172.21.15.46:0/3201185982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe32c15b560 0x7fe32c15b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.723+0000 7fe331098640 1 -- 172.21.15.46:0/3201185982 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe32c157ec0 msgr2=0x7fe32c1f8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.723+0000 7fe331098640 1 --2- 172.21.15.46:0/3201185982 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe32c157ec0 0x7fe32c1f8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.723+0000 7fe331098640 1 -- 172.21.15.46:0/3201185982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe32c15b560 msgr2=0x7fe32c15b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.723+0000 7fe331098640 1 --2- 172.21.15.46:0/3201185982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe32c15b560 0x7fe32c15b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.723+0000 7fe331098640 1 -- 172.21.15.46:0/3201185982 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe32c1f8550 con 0x7fe3180039c0 2023-12-02T00:28:58.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.723+0000 7f9276ffd640 1 -- 172.21.15.46:0/4190831256 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f927c01a400 con 0x7f92800714e0 2023-12-02T00:28:58.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.723+0000 7fb878ff9640 1 -- 172.21.15.46:0/1743745081 <== osd.2 v2:172.21.15.46:6824/878860153 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fb868075c00 con 0x7fb868073130 2023-12-02T00:28:58.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.722+0000 7f63cbfff640 1 -- 172.21.15.46:0/4197687937 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63cc0433f0 con 0x7f63d80d4680 2023-12-02T00:28:58.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.722+0000 7f63cbfff640 1 -- 172.21.15.46:0/4197687937 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63cc0437a0 con 0x7f63d80d4680 2023-12-02T00:28:58.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.722+0000 7f63e5d1d640 1 -- 172.21.15.46:0/4197687937 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63d804a5c0 con 0x7f63d80d4680 2023-12-02T00:28:58.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.722+0000 7f63e5d1d640 1 -- 172.21.15.46:0/4197687937 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d803a830 con 0x7f63d80d4680 2023-12-02T00:28:58.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.723+0000 7f63e5d1d640 1 -- 172.21.15.46:0/4197687937 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f63c0000fc0 con 0x7f63d80d4680 2023-12-02T00:28:58.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.724+0000 7f9276ffd640 1 --2- 172.21.15.46:0/4190831256 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f926406af10 0x7f926406d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.724+0000 7f9276ffd640 1 -- 172.21.15.46:0/4190831256 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f927c089b50 con 0x7f92800714e0 2023-12-02T00:28:58.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.724+0000 7f9276ffd640 1 --2- 172.21.15.46:0/4190831256 >> [v2:172.21.15.162:6816/3463332362,v1:172.21.15.162:6817/3463332362] conn(0x7f9264073130 0x7f9264075550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.724+0000 7f9276ffd640 1 -- 172.21.15.46:0/4190831256 --> [v2:172.21.15.162:6816/3463332362,v1:172.21.15.162:6817/3463332362] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9264075c00 con 0x7f9264073130 2023-12-02T00:28:58.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.724+0000 7f9276ffd640 1 -- 172.21.15.46:0/4190831256 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f927c089f00 con 0x7f92800714e0 2023-12-02T00:28:58.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.724+0000 7f8235da0640 1 Processor -- start 2023-12-02T00:28:58.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.724+0000 7f9286bc3640 1 --2- 172.21.15.46:0/4190831256 >> [v2:172.21.15.162:6816/3463332362,v1:172.21.15.162:6817/3463332362] conn(0x7f9264073130 0x7f9264075550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.724+0000 7fe331098640 1 --2- 172.21.15.46:0/3201185982 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3180039c0 0x7fe32c148a40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe328031000 tx=0x7fe328001930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:58.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.724+0000 7fb87b7fe640 1 --2- 172.21.15.46:0/1743745081 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb86806af10 0x7fb86806d3d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb87c17a250 tx=0x7fb86c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:58.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.724+0000 7fe321ffb640 1 -- 172.21.15.46:0/3201185982 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe328004bf0 con 0x7fe3180039c0 2023-12-02T00:28:58.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.722+0000 7f70d8ff9640 1 -- 172.21.15.46:0/2535793539 <== mon.2 v2:172.21.15.162:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f70cc004b60 con 0x7f70dc06cf40 2023-12-02T00:28:58.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.725+0000 7f5f7c17c640 1 -- 172.21.15.46:0/3616434915 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5f5c000fc0 con 0x7f5f740b0cf0 2023-12-02T00:28:58.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.724+0000 7f8235da0640 1 -- start start 2023-12-02T00:28:58.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.723+0000 7f70d8ff9640 1 --2- 172.21.15.46:0/2535793539 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f70c806af10 0x7f70c806d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.723+0000 7f70db7fe640 1 --2- 172.21.15.46:0/2535793539 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f70c806af10 0x7f70c806d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.723+0000 7f70d8ff9640 1 -- 172.21.15.46:0/2535793539 <== mon.2 v2:172.21.15.162:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f70cc088960 con 0x7f70dc06cf40 2023-12-02T00:28:58.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.724+0000 7f70e29b7640 1 --2- 172.21.15.46:0/2535793539 >> [v2:172.21.15.162:6808/4132149803,v1:172.21.15.162:6809/4132149803] conn(0x7f70dc1df510 0x7f70dc1df8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.724+0000 7f70e29b7640 1 -- 172.21.15.46:0/2535793539 --> [v2:172.21.15.162:6808/4132149803,v1:172.21.15.162:6809/4132149803] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f70dc0b2a60 con 0x7f70dc1df510 2023-12-02T00:28:58.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.725+0000 7f70e0f54640 1 --2- 172.21.15.46:0/2535793539 >> [v2:172.21.15.162:6808/4132149803,v1:172.21.15.162:6809/4132149803] conn(0x7f70dc1df510 0x7f70dc1df8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.725+0000 7f70db7fe640 1 --2- 172.21.15.46:0/2535793539 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f70c806af10 0x7f70c806d3d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f70dc159fb0 tx=0x7f70d000b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:58.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.724+0000 7fe321ffb640 1 -- 172.21.15.46:0/3201185982 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe32803a070 con 0x7fe3180039c0 2023-12-02T00:28:58.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.725+0000 7f9285bc1640 1 --2- 172.21.15.46:0/4190831256 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f926406af10 0x7f926406d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.724+0000 7fe321ffb640 1 -- 172.21.15.46:0/3201185982 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe328007070 con 0x7fe3180039c0 2023-12-02T00:28:58.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.725+0000 7fe3332fc640 1 -- 172.21.15.46:0/3201185982 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe32c1598a0 con 0x7fe3180039c0 2023-12-02T00:28:58.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.725+0000 7fe3332fc640 1 -- 172.21.15.46:0/3201185982 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe32c159e20 con 0x7fe3180039c0 2023-12-02T00:28:58.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.725+0000 7f5f6a7fc640 1 -- 172.21.15.46:0/3616434915 <== mon.2 v2:172.21.15.162:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5f70013470 con 0x7f5f740b0cf0 2023-12-02T00:28:58.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.725+0000 7f9286bc3640 1 --2- 172.21.15.46:0/4190831256 >> [v2:172.21.15.162:6816/3463332362,v1:172.21.15.162:6817/3463332362] conn(0x7f9264073130 0x7f9264075550 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:58.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.726+0000 7f9285bc1640 1 --2- 172.21.15.46:0/4190831256 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f926406af10 0x7f926406d3d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9280141b20 tx=0x7f927800d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:58.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.726+0000 7f32c5ffb640 1 -- 172.21.15.46:0/2043782315 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f32cc007070 con 0x7f32d015ba30 2023-12-02T00:28:58.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.726+0000 7f8235da0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f823006d0b0 0x7f8230155300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.725+0000 7f5f6a7fc640 1 --2- 172.21.15.46:0/3616434915 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5f5806ae40 0x7f5f5806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.726+0000 7f5f6a7fc640 1 -- 172.21.15.46:0/3616434915 <== mon.2 v2:172.21.15.162:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5f70089780 con 0x7f5f740b0cf0 2023-12-02T00:28:58.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.726+0000 7f5f79717640 1 --2- 172.21.15.46:0/3616434915 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5f5806ae40 0x7f5f5806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.726+0000 7f5f6a7fc640 1 --2- 172.21.15.46:0/3616434915 >> [v2:172.21.15.46:6808/2057169608,v1:172.21.15.46:6809/2057169608] conn(0x7f5f58073060 0x7f5f58075480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.726+0000 7f5f6a7fc640 1 -- 172.21.15.46:0/3616434915 --> [v2:172.21.15.46:6808/2057169608,v1:172.21.15.46:6809/2057169608] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5f58075b30 con 0x7f5f58073060 2023-12-02T00:28:58.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.726+0000 7f5f6a7fc640 1 -- 172.21.15.46:0/3616434915 <== mon.2 v2:172.21.15.162:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f7008bda0 con 0x7f5f740b0cf0 2023-12-02T00:28:58.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.726+0000 7f5f7a719640 1 --2- 172.21.15.46:0/3616434915 >> [v2:172.21.15.46:6808/2057169608,v1:172.21.15.46:6809/2057169608] conn(0x7f5f58073060 0x7f5f58075480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.727+0000 7f5f79717640 1 --2- 172.21.15.46:0/3616434915 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5f5806ae40 0x7f5f5806d300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5f64036000 tx=0x7f5f64008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:58.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.726+0000 7f32c5ffb640 1 --2- 172.21.15.46:0/2043782315 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f32b406af10 0x7f32b406d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.726+0000 7f32c5ffb640 1 -- 172.21.15.46:0/2043782315 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f32cc0891c0 con 0x7f32d015ba30 2023-12-02T00:28:58.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.726+0000 7f9276ffd640 1 -- 172.21.15.46:0/4190831256 <== osd.5 v2:172.21.15.162:6816/3463332362 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f9264075c00 con 0x7f9264073130 2023-12-02T00:28:58.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.727+0000 7fe3332fc640 1 -- 172.21.15.46:0/3201185982 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe32c159690 con 0x7fe3180039c0 2023-12-02T00:28:58.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.727+0000 7f5f7a719640 1 --2- 172.21.15.46:0/3616434915 >> [v2:172.21.15.46:6808/2057169608,v1:172.21.15.46:6809/2057169608] conn(0x7f5f58073060 0x7f5f58075480 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:58.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.727+0000 7f32c5ffb640 1 --2- 172.21.15.46:0/2043782315 >> [v2:172.21.15.46:6816/1799228239,v1:172.21.15.46:6817/1799228239] conn(0x7f32b4073130 0x7f32b4075550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.727+0000 7f32c5ffb640 1 -- 172.21.15.46:0/2043782315 --> [v2:172.21.15.46:6816/1799228239,v1:172.21.15.46:6817/1799228239] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f32b4075c00 con 0x7f32b4073130 2023-12-02T00:28:58.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.727+0000 7f32c5ffb640 1 -- 172.21.15.46:0/2043782315 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32cc089570 con 0x7f32d015ba30 2023-12-02T00:28:58.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.727+0000 7f32d4ed4640 1 --2- 172.21.15.46:0/2043782315 >> [v2:172.21.15.46:6816/1799228239,v1:172.21.15.46:6817/1799228239] conn(0x7f32b4073130 0x7f32b4075550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.727+0000 7f8235da0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82300b45f0 0x7f823009d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.727+0000 7f8235da0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8230154580 0x7f8230146640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.727+0000 7f8235da0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82301520c0 con 0x7f823006d0b0 2023-12-02T00:28:58.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.727+0000 7f8235da0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823009c2d0 con 0x7f82300b45f0 2023-12-02T00:28:58.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.727+0000 7f8235da0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f823009c5b0 con 0x7f8230154580 2023-12-02T00:28:58.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.727+0000 7f32d56d5640 1 --2- 172.21.15.46:0/2043782315 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f32b406af10 0x7f32b406d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.727+0000 7f63cbfff640 1 -- 172.21.15.46:0/4197687937 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f63cc04c3b0 con 0x7f63d80d4680 2023-12-02T00:28:58.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.727+0000 7f63cbfff640 1 --2- 172.21.15.46:0/4197687937 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f63c406af40 0x7f63c406d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.727+0000 7f63cbfff640 1 -- 172.21.15.46:0/4197687937 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f63cc0ac230 con 0x7f63d80d4680 2023-12-02T00:28:58.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.727+0000 7f63cbfff640 1 --2- 172.21.15.46:0/4197687937 >> [v2:172.21.15.162:6824/4107689713,v1:172.21.15.162:6825/4107689713] conn(0x7f63c4073160 0x7f63c4075580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.727+0000 7f63cbfff640 1 -- 172.21.15.46:0/4197687937 --> [v2:172.21.15.162:6824/4107689713,v1:172.21.15.162:6825/4107689713] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f63c4075c30 con 0x7f63c4073160 2023-12-02T00:28:58.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.727+0000 7f63cbfff640 1 -- 172.21.15.46:0/4197687937 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63cc0ac5e0 con 0x7f63d80d4680 2023-12-02T00:28:58.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.728+0000 7f32d4ed4640 1 --2- 172.21.15.46:0/2043782315 >> [v2:172.21.15.46:6816/1799228239,v1:172.21.15.46:6817/1799228239] conn(0x7f32b4073130 0x7f32b4075550 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:58.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.728+0000 7fe321ffb640 1 -- 172.21.15.46:0/3201185982 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe328032070 con 0x7fe3180039c0 2023-12-02T00:28:58.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.728+0000 7f63df7fe640 1 --2- 172.21.15.46:0/4197687937 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f63c406af40 0x7f63c406d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.728+0000 7f63deffd640 1 --2- 172.21.15.46:0/4197687937 >> [v2:172.21.15.162:6824/4107689713,v1:172.21.15.162:6825/4107689713] conn(0x7f63c4073160 0x7f63c4075580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.727+0000 7f8234d9e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f823006d0b0 0x7f8230155300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.727+0000 7f8234d9e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f823006d0b0 0x7f8230155300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37976/0 (socket says 172.21.15.46:37976) 2023-12-02T00:28:58.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.727+0000 7f8234d9e640 1 -- 172.21.15.46:0/3937545204 learned_addr learned my addr 172.21.15.46:0/3937545204 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:58.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.728+0000 7f823559f640 1 --2- 172.21.15.46:0/3937545204 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8230154580 0x7f8230146640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.728+0000 7f822ffff640 1 --2- 172.21.15.46:0/3937545204 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82300b45f0 0x7f823009d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.728+0000 7f8234d9e640 1 -- 172.21.15.46:0/3937545204 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8230154580 msgr2=0x7f8230146640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.728+0000 7f8234d9e640 1 --2- 172.21.15.46:0/3937545204 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8230154580 0x7f8230146640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.728+0000 7f8234d9e640 1 -- 172.21.15.46:0/3937545204 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82300b45f0 msgr2=0x7f823009d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.728+0000 7f8234d9e640 1 --2- 172.21.15.46:0/3937545204 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82300b45f0 0x7f823009d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.728+0000 7f8234d9e640 1 -- 172.21.15.46:0/3937545204 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8224009400 con 0x7f823006d0b0 2023-12-02T00:28:58.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.728+0000 7f63df7fe640 1 --2- 172.21.15.46:0/4197687937 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f63c406af40 0x7f63c406d400 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f63e0069ef0 tx=0x7f63e006d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:58.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.728+0000 7f8234d9e640 1 --2- 172.21.15.46:0/3937545204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f823006d0b0 0x7f8230155300 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f822400e960 tx=0x7f822400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:58.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.728+0000 7f63deffd640 1 --2- 172.21.15.46:0/4197687937 >> [v2:172.21.15.162:6824/4107689713,v1:172.21.15.162:6825/4107689713] conn(0x7f63c4073160 0x7f63c4075580 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:58.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.728+0000 7f32d56d5640 1 --2- 172.21.15.46:0/2043782315 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f32b406af10 0x7f32b406d3d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f32c000b7a0 tx=0x7f32c0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:58.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.728+0000 7fe321ffb640 1 --2- 172.21.15.46:0/3201185982 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe31006af10 0x7fe31006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.728+0000 7fe321ffb640 1 -- 172.21.15.46:0/3201185982 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe3280aaf00 con 0x7fe3180039c0 2023-12-02T00:28:58.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.728+0000 7fe330897640 1 --2- 172.21.15.46:0/3201185982 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe31006af10 0x7fe31006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.728+0000 7fe321ffb640 1 --2- 172.21.15.46:0/3201185982 >> [v2:172.21.15.162:6800/3225626420,v1:172.21.15.162:6801/3225626420] conn(0x7fe310073130 0x7fe310075550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.728+0000 7fe321ffb640 1 -- 172.21.15.46:0/3201185982 --> [v2:172.21.15.162:6800/3225626420,v1:172.21.15.162:6801/3225626420] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe310075c00 con 0x7fe310073130 2023-12-02T00:28:58.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.729+0000 7f5f6a7fc640 1 -- 172.21.15.46:0/3616434915 <== osd.0 v2:172.21.15.46:6808/2057169608 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f5f58075b30 con 0x7f5f58073060 2023-12-02T00:28:58.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.728+0000 7fe321ffb640 1 -- 172.21.15.46:0/3201185982 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3280ad520 con 0x7fe3180039c0 2023-12-02T00:28:58.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.729+0000 7fe331899640 1 --2- 172.21.15.46:0/3201185982 >> [v2:172.21.15.162:6800/3225626420,v1:172.21.15.162:6801/3225626420] conn(0x7fe310073130 0x7fe310075550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.729+0000 7f822d7fa640 1 -- 172.21.15.46:0/3937545204 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822400ca00 con 0x7f823006d0b0 2023-12-02T00:28:58.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.729+0000 7f70e0f54640 1 --2- 172.21.15.46:0/2535793539 >> [v2:172.21.15.162:6808/4132149803,v1:172.21.15.162:6809/4132149803] conn(0x7f70dc1df510 0x7f70dc1df8f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:58.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.729+0000 7f8235da0640 1 -- 172.21.15.46:0/3937545204 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82200094d0 con 0x7f823006d0b0 2023-12-02T00:28:58.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.729+0000 7f32c5ffb640 1 -- 172.21.15.46:0/2043782315 <== osd.3 v2:172.21.15.46:6816/1799228239 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f32b4075c00 con 0x7f32b4073130 2023-12-02T00:28:58.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.729+0000 7f822d7fa640 1 -- 172.21.15.46:0/3937545204 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f822400cb60 con 0x7f823006d0b0 2023-12-02T00:28:58.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.729+0000 7f822d7fa640 1 -- 172.21.15.46:0/3937545204 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8224013470 con 0x7f823006d0b0 2023-12-02T00:28:58.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.729+0000 7f8235da0640 1 -- 172.21.15.46:0/3937545204 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f823009cb70 con 0x7f823006d0b0 2023-12-02T00:28:58.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.729+0000 7fe330897640 1 --2- 172.21.15.46:0/3201185982 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe31006af10 0x7fe31006d3d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe32c070730 tx=0x7fe31c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:58.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.729+0000 7f63cbfff640 1 -- 172.21.15.46:0/4197687937 <== osd.7 v2:172.21.15.162:6824/4107689713 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f63c4075c30 con 0x7f63c4073160 2023-12-02T00:28:58.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.730+0000 7f70d8ff9640 1 -- 172.21.15.46:0/2535793539 <== osd.4 v2:172.21.15.162:6808/4132149803 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f70d402d970 con 0x7f70dc1df510 2023-12-02T00:28:58.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.730+0000 7f8235da0640 1 -- 172.21.15.46:0/3937545204 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f823015a330 con 0x7f823006d0b0 2023-12-02T00:28:58.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.730+0000 7fe331899640 1 --2- 172.21.15.46:0/3201185982 >> [v2:172.21.15.162:6800/3225626420,v1:172.21.15.162:6801/3225626420] conn(0x7fe310073130 0x7fe310075550 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:58.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.731+0000 7fe321ffb640 1 -- 172.21.15.46:0/3201185982 <== osd.6 v2:172.21.15.162:6800/3225626420 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fe310075c00 con 0x7fe310073130 2023-12-02T00:28:58.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.732+0000 7f822d7fa640 1 -- 172.21.15.46:0/3937545204 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f82240135d0 con 0x7f823006d0b0 2023-12-02T00:28:58.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.732+0000 7f822d7fa640 1 --2- 172.21.15.46:0/3937545204 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f821c06ae90 0x7f821c06d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.733+0000 7f822d7fa640 1 -- 172.21.15.46:0/3937545204 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8224058480 con 0x7f823006d0b0 2023-12-02T00:28:58.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.733+0000 7f822d7fa640 1 --2- 172.21.15.46:0/3937545204 >> [v2:172.21.15.46:6800/10535129,v1:172.21.15.46:6801/10535129] conn(0x7f821c0730b0 0x7f821c0754d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:58.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.733+0000 7f822d7fa640 1 -- 172.21.15.46:0/3937545204 --> [v2:172.21.15.46:6800/10535129,v1:172.21.15.46:6801/10535129] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f821c075b80 con 0x7f821c0730b0 2023-12-02T00:28:58.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.733+0000 7f822d7fa640 1 -- 172.21.15.46:0/3937545204 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82240540c0 con 0x7f823006d0b0 2023-12-02T00:28:58.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.733+0000 7f823559f640 1 --2- 172.21.15.46:0/3937545204 >> [v2:172.21.15.46:6800/10535129,v1:172.21.15.46:6801/10535129] conn(0x7f821c0730b0 0x7f821c0754d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.733+0000 7f822ffff640 1 --2- 172.21.15.46:0/3937545204 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f821c06ae90 0x7f821c06d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:58.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.734+0000 7f822ffff640 1 --2- 172.21.15.46:0/3937545204 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f821c06ae90 0x7f821c06d350 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f82301f4430 tx=0x7f8228009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:58.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.734+0000 7f823559f640 1 --2- 172.21.15.46:0/3937545204 >> [v2:172.21.15.46:6800/10535129,v1:172.21.15.46:6801/10535129] conn(0x7f821c0730b0 0x7f821c0754d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:58.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.735+0000 7f822d7fa640 1 -- 172.21.15.46:0/3937545204 <== osd.1 v2:172.21.15.46:6800/10535129 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f821c075b80 con 0x7f821c0730b0 2023-12-02T00:28:58.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.757+0000 7fb88249b640 1 -- 172.21.15.46:0/1743745081 --> [v2:172.21.15.46:6824/878860153,v1:172.21.15.46:6825/878860153] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb87c069400 con 0x7fb868073130 2023-12-02T00:28:58.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.758+0000 7f92873c4640 1 -- 172.21.15.46:0/4190831256 --> [v2:172.21.15.162:6816/3463332362,v1:172.21.15.162:6817/3463332362] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9268002ce0 con 0x7f9264073130 2023-12-02T00:28:58.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.758+0000 7fb878ff9640 1 -- 172.21.15.46:0/1743745081 <== osd.2 v2:172.21.15.46:6824/878860153 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb87c069400 con 0x7fb868073130 2023-12-02T00:28:58.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.758+0000 7f9276ffd640 1 -- 172.21.15.46:0/4190831256 <== osd.5 v2:172.21.15.162:6816/3463332362 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9268002ce0 con 0x7f9264073130 2023-12-02T00:28:58.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.759+0000 7fb8627fc640 1 -- 172.21.15.46:0/1743745081 >> [v2:172.21.15.46:6824/878860153,v1:172.21.15.46:6825/878860153] conn(0x7fb868073130 msgr2=0x7fb868075550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.759+0000 7fb8627fc640 1 --2- 172.21.15.46:0/1743745081 >> [v2:172.21.15.46:6824/878860153,v1:172.21.15.46:6825/878860153] conn(0x7fb868073130 0x7fb868075550 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.759+0000 7f9274ff9640 1 -- 172.21.15.46:0/4190831256 >> [v2:172.21.15.162:6816/3463332362,v1:172.21.15.162:6817/3463332362] conn(0x7f9264073130 msgr2=0x7f9264075550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.759+0000 7f9274ff9640 1 --2- 172.21.15.46:0/4190831256 >> [v2:172.21.15.162:6816/3463332362,v1:172.21.15.162:6817/3463332362] conn(0x7f9264073130 0x7f9264075550 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.759+0000 7fb8627fc640 1 -- 172.21.15.46:0/1743745081 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb86806af10 msgr2=0x7fb86806d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.759+0000 7fb8627fc640 1 --2- 172.21.15.46:0/1743745081 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb86806af10 0x7fb86806d3d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb87c17a250 tx=0x7fb86c008040 comp rx=0 tx=0).stop 2023-12-02T00:28:58.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.759+0000 7fb8627fc640 1 -- 172.21.15.46:0/1743745081 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb87c08a850 msgr2=0x7fb87c16fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.759+0000 7fb8627fc640 1 --2- 172.21.15.46:0/1743745081 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb87c08a850 0x7fb87c16fed0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb87002bc90 tx=0x7fb8700047d0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.759+0000 7f9274ff9640 1 -- 172.21.15.46:0/4190831256 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f926406af10 msgr2=0x7f926406d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.760+0000 7fb880a38640 1 -- 172.21.15.46:0/1743745081 reap_dead start 2023-12-02T00:28:58.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.760+0000 7fb8627fc640 1 -- 172.21.15.46:0/1743745081 shutdown_connections 2023-12-02T00:28:58.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.760+0000 7fb8627fc640 1 -- 172.21.15.46:0/1743745081 >> 172.21.15.46:0/1743745081 conn(0x7fb87c0b0d60 msgr2=0x7fb87c16cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:58.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.759+0000 7f9274ff9640 1 --2- 172.21.15.46:0/4190831256 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f926406af10 0x7f926406d3d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9280141b20 tx=0x7f927800d040 comp rx=0 tx=0).stop 2023-12-02T00:28:58.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.759+0000 7f9274ff9640 1 -- 172.21.15.46:0/4190831256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92800714e0 msgr2=0x7f92800b1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.759+0000 7f9274ff9640 1 --2- 172.21.15.46:0/4190831256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92800714e0 0x7f92800b1540 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f927c00b3e0 tx=0x7f927c00b8a0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.759+0000 7f9286bc3640 1 -- 172.21.15.46:0/4190831256 reap_dead start 2023-12-02T00:28:58.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.759+0000 7f9274ff9640 1 -- 172.21.15.46:0/4190831256 shutdown_connections 2023-12-02T00:28:58.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.759+0000 7f9274ff9640 1 -- 172.21.15.46:0/4190831256 >> 172.21.15.46:0/4190831256 conn(0x7f928006a4d0 msgr2=0x7f928015df00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:58.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.760+0000 7f9274ff9640 1 -- 172.21.15.46:0/4190831256 shutdown_connections 2023-12-02T00:28:58.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.760+0000 7fb8627fc640 1 -- 172.21.15.46:0/1743745081 shutdown_connections 2023-12-02T00:28:58.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.760+0000 7fb8627fc640 1 -- 172.21.15.46:0/1743745081 wait complete. 2023-12-02T00:28:58.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.760+0000 7f9274ff9640 1 -- 172.21.15.46:0/4190831256 wait complete. 2023-12-02T00:28:58.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.761+0000 7f5f7c17c640 1 -- 172.21.15.46:0/3616434915 --> [v2:172.21.15.46:6808/2057169608,v1:172.21.15.46:6809/2057169608] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5f5c002cb0 con 0x7f5f58073060 2023-12-02T00:28:58.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.761+0000 7f5f6a7fc640 1 -- 172.21.15.46:0/3616434915 <== osd.0 v2:172.21.15.46:6808/2057169608 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5f5c002cb0 con 0x7f5f58073060 2023-12-02T00:28:58.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.762+0000 7f32d7939640 1 -- 172.21.15.46:0/2043782315 --> [v2:172.21.15.46:6816/1799228239,v1:172.21.15.46:6817/1799228239] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f32d00cdba0 con 0x7f32b4073130 2023-12-02T00:28:58.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.762+0000 7f5f57fff640 1 -- 172.21.15.46:0/3616434915 >> [v2:172.21.15.46:6808/2057169608,v1:172.21.15.46:6809/2057169608] conn(0x7f5f58073060 msgr2=0x7f5f58075480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.762+0000 7f5f57fff640 1 --2- 172.21.15.46:0/3616434915 >> [v2:172.21.15.46:6808/2057169608,v1:172.21.15.46:6809/2057169608] conn(0x7f5f58073060 0x7f5f58075480 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.762+0000 7f5f57fff640 1 -- 172.21.15.46:0/3616434915 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5f5806ae40 msgr2=0x7f5f5806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.762+0000 7f5f57fff640 1 --2- 172.21.15.46:0/3616434915 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5f5806ae40 0x7f5f5806d300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5f64036000 tx=0x7f5f64008030 comp rx=0 tx=0).stop 2023-12-02T00:28:58.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.762+0000 7f32c5ffb640 1 -- 172.21.15.46:0/2043782315 <== osd.3 v2:172.21.15.46:6816/1799228239 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f32c802cd40 con 0x7f32b4073130 2023-12-02T00:28:58.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.763+0000 7f5f57fff640 1 -- 172.21.15.46:0/3616434915 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f740b0cf0 msgr2=0x7f5f74151fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.763+0000 7f5f57fff640 1 --2- 172.21.15.46:0/3616434915 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f740b0cf0 0x7f5f74151fe0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5f7000e9b0 tx=0x7f5f7000ee70 comp rx=0 tx=0).stop 2023-12-02T00:28:58.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.763+0000 7f5f7a719640 1 -- 172.21.15.46:0/3616434915 reap_dead start 2023-12-02T00:28:58.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.763+0000 7f32b37fe640 1 -- 172.21.15.46:0/2043782315 >> [v2:172.21.15.46:6816/1799228239,v1:172.21.15.46:6817/1799228239] conn(0x7f32b4073130 msgr2=0x7f32b4075550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.763+0000 7f32b37fe640 1 --2- 172.21.15.46:0/2043782315 >> [v2:172.21.15.46:6816/1799228239,v1:172.21.15.46:6817/1799228239] conn(0x7f32b4073130 0x7f32b4075550 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.763+0000 7f32b37fe640 1 -- 172.21.15.46:0/2043782315 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f32b406af10 msgr2=0x7f32b406d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.763+0000 7f32b37fe640 1 --2- 172.21.15.46:0/2043782315 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f32b406af10 0x7f32b406d3d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f32c000b7a0 tx=0x7f32c0008040 comp rx=0 tx=0).stop 2023-12-02T00:28:58.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.764+0000 7f5f57fff640 1 -- 172.21.15.46:0/3616434915 shutdown_connections 2023-12-02T00:28:58.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.764+0000 7f5f57fff640 1 -- 172.21.15.46:0/3616434915 >> 172.21.15.46:0/3616434915 conn(0x7f5f7406a4d0 msgr2=0x7f5f74155280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:58.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.763+0000 7f32b37fe640 1 -- 172.21.15.46:0/2043782315 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32d015ba30 msgr2=0x7f32d00ad990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.763+0000 7f32b37fe640 1 --2- 172.21.15.46:0/2043782315 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32d015ba30 0x7f32d00ad990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f32d00b1c60 tx=0x7f32cc00fb90 comp rx=0 tx=0).stop 2023-12-02T00:28:58.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.764+0000 7f32d5ed6640 1 -- 172.21.15.46:0/2043782315 reap_dead start 2023-12-02T00:28:58.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.764+0000 7f63e5d1d640 1 -- 172.21.15.46:0/4197687937 --> [v2:172.21.15.162:6824/4107689713,v1:172.21.15.162:6825/4107689713] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f63c0002cb0 con 0x7f63c4073160 2023-12-02T00:28:58.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.764+0000 7f32b37fe640 1 -- 172.21.15.46:0/2043782315 shutdown_connections 2023-12-02T00:28:58.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.764+0000 7f32b37fe640 1 -- 172.21.15.46:0/2043782315 >> 172.21.15.46:0/2043782315 conn(0x7f32d00b0d60 msgr2=0x7f32d00af4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:58.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.764+0000 7f5f57fff640 1 -- 172.21.15.46:0/3616434915 shutdown_connections 2023-12-02T00:28:58.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.764+0000 7f5f57fff640 1 -- 172.21.15.46:0/3616434915 wait complete. 2023-12-02T00:28:58.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.764+0000 7f63cbfff640 1 -- 172.21.15.46:0/4197687937 <== osd.7 v2:172.21.15.162:6824/4107689713 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f63c0002cb0 con 0x7f63c4073160 2023-12-02T00:28:58.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.764+0000 7f32b37fe640 1 -- 172.21.15.46:0/2043782315 shutdown_connections 2023-12-02T00:28:58.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.765+0000 7f32b37fe640 1 -- 172.21.15.46:0/2043782315 wait complete. 2023-12-02T00:28:58.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.765+0000 7fe3332fc640 1 -- 172.21.15.46:0/3201185982 --> [v2:172.21.15.162:6800/3225626420,v1:172.21.15.162:6801/3225626420] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe32c069400 con 0x7fe310073130 2023-12-02T00:28:58.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.765+0000 7fe321ffb640 1 -- 172.21.15.46:0/3201185982 <== osd.6 v2:172.21.15.162:6800/3225626420 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe32c069400 con 0x7fe310073130 2023-12-02T00:28:58.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.766+0000 7fe3177fe640 1 -- 172.21.15.46:0/3201185982 >> [v2:172.21.15.162:6800/3225626420,v1:172.21.15.162:6801/3225626420] conn(0x7fe310073130 msgr2=0x7fe310075550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.766+0000 7fe3177fe640 1 --2- 172.21.15.46:0/3201185982 >> [v2:172.21.15.162:6800/3225626420,v1:172.21.15.162:6801/3225626420] conn(0x7fe310073130 0x7fe310075550 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.766+0000 7f8235da0640 1 -- 172.21.15.46:0/3937545204 --> [v2:172.21.15.46:6800/10535129,v1:172.21.15.46:6801/10535129] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f823006d510 con 0x7f821c0730b0 2023-12-02T00:28:58.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.766+0000 7fe3177fe640 1 -- 172.21.15.46:0/3201185982 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe31006af10 msgr2=0x7fe31006d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.766+0000 7fe3177fe640 1 --2- 172.21.15.46:0/3201185982 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe31006af10 0x7fe31006d3d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe32c070730 tx=0x7fe31c009040 comp rx=0 tx=0).stop 2023-12-02T00:28:58.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.766+0000 7fe3177fe640 1 -- 172.21.15.46:0/3201185982 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3180039c0 msgr2=0x7fe32c148a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.766+0000 7fe3177fe640 1 --2- 172.21.15.46:0/3201185982 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3180039c0 0x7fe32c148a40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe328031000 tx=0x7fe328001930 comp rx=0 tx=0).stop 2023-12-02T00:28:58.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.766+0000 7fe331899640 1 -- 172.21.15.46:0/3201185982 reap_dead start 2023-12-02T00:28:58.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.766+0000 7fe3177fe640 1 -- 172.21.15.46:0/3201185982 shutdown_connections 2023-12-02T00:28:58.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.766+0000 7fe3177fe640 1 -- 172.21.15.46:0/3201185982 >> 172.21.15.46:0/3201185982 conn(0x7fe32c0b0d60 msgr2=0x7fe32c06de10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:58.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.767+0000 7fe3177fe640 1 -- 172.21.15.46:0/3201185982 shutdown_connections 2023-12-02T00:28:58.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.767+0000 7f63e5d1d640 1 -- 172.21.15.46:0/4197687937 >> [v2:172.21.15.162:6824/4107689713,v1:172.21.15.162:6825/4107689713] conn(0x7f63c4073160 msgr2=0x7f63c4075580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.767+0000 7f63e5d1d640 1 --2- 172.21.15.46:0/4197687937 >> [v2:172.21.15.162:6824/4107689713,v1:172.21.15.162:6825/4107689713] conn(0x7f63c4073160 0x7f63c4075580 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.767+0000 7f70e29b7640 1 -- 172.21.15.46:0/2535793539 --> [v2:172.21.15.162:6808/4132149803,v1:172.21.15.162:6809/4132149803] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f70dc06e340 con 0x7f70dc1df510 2023-12-02T00:28:58.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.767+0000 7f822d7fa640 1 -- 172.21.15.46:0/3937545204 <== osd.1 v2:172.21.15.46:6800/10535129 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8220031e10 con 0x7f821c0730b0 2023-12-02T00:28:58.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.767+0000 7fe3177fe640 1 -- 172.21.15.46:0/3201185982 wait complete. 2023-12-02T00:28:58.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.767+0000 7f63e5d1d640 1 -- 172.21.15.46:0/4197687937 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f63c406af40 msgr2=0x7f63c406d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.767+0000 7f63e5d1d640 1 --2- 172.21.15.46:0/4197687937 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f63c406af40 0x7f63c406d400 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f63e0069ef0 tx=0x7f63e006d040 comp rx=0 tx=0).stop 2023-12-02T00:28:58.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.767+0000 7f63e5d1d640 1 -- 172.21.15.46:0/4197687937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63d80d4680 msgr2=0x7f63d804d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.767+0000 7f63e5d1d640 1 --2- 172.21.15.46:0/4197687937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63d80d4680 0x7f63d804d470 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f63cc033000 tx=0x7f63cc004730 comp rx=0 tx=0).stop 2023-12-02T00:28:58.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.767+0000 7f63dffff640 1 -- 172.21.15.46:0/4197687937 reap_dead start 2023-12-02T00:28:58.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.767+0000 7f63e5d1d640 1 -- 172.21.15.46:0/4197687937 shutdown_connections 2023-12-02T00:28:58.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.767+0000 7f63e5d1d640 1 -- 172.21.15.46:0/4197687937 >> 172.21.15.46:0/4197687937 conn(0x7f63d8054ed0 msgr2=0x7f63d80e79f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:58.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.767+0000 7f63e5d1d640 1 -- 172.21.15.46:0/4197687937 shutdown_connections 2023-12-02T00:28:58.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.767+0000 7f8235da0640 1 -- 172.21.15.46:0/3937545204 >> [v2:172.21.15.46:6800/10535129,v1:172.21.15.46:6801/10535129] conn(0x7f821c0730b0 msgr2=0x7f821c0754d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.767+0000 7f63e5d1d640 1 -- 172.21.15.46:0/4197687937 wait complete. 2023-12-02T00:28:58.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.767+0000 7f8235da0640 1 --2- 172.21.15.46:0/3937545204 >> [v2:172.21.15.46:6800/10535129,v1:172.21.15.46:6801/10535129] conn(0x7f821c0730b0 0x7f821c0754d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.767+0000 7f70d8ff9640 1 -- 172.21.15.46:0/2535793539 <== osd.4 v2:172.21.15.162:6808/4132149803 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f70d402dae0 con 0x7f70dc1df510 2023-12-02T00:28:58.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.768+0000 7f8235da0640 1 -- 172.21.15.46:0/3937545204 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f821c06ae90 msgr2=0x7f821c06d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.768+0000 7f8235da0640 1 --2- 172.21.15.46:0/3937545204 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f821c06ae90 0x7f821c06d350 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f82301f4430 tx=0x7f8228009040 comp rx=0 tx=0).stop 2023-12-02T00:28:58.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.768+0000 7f8235da0640 1 -- 172.21.15.46:0/3937545204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f823006d0b0 msgr2=0x7f8230155300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.768+0000 7f8235da0640 1 --2- 172.21.15.46:0/3937545204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f823006d0b0 0x7f8230155300 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f822400e960 tx=0x7f822400ee20 comp rx=0 tx=0).stop 2023-12-02T00:28:58.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.768+0000 7f823559f640 1 -- 172.21.15.46:0/3937545204 reap_dead start 2023-12-02T00:28:58.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.768+0000 7f70e29b7640 1 -- 172.21.15.46:0/2535793539 >> [v2:172.21.15.162:6808/4132149803,v1:172.21.15.162:6809/4132149803] conn(0x7f70dc1df510 msgr2=0x7f70dc1df8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.768+0000 7f70e29b7640 1 --2- 172.21.15.46:0/2535793539 >> [v2:172.21.15.162:6808/4132149803,v1:172.21.15.162:6809/4132149803] conn(0x7f70dc1df510 0x7f70dc1df8f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:58.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.768+0000 7f70e29b7640 1 -- 172.21.15.46:0/2535793539 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f70c806af10 msgr2=0x7f70c806d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.768+0000 7f70e29b7640 1 --2- 172.21.15.46:0/2535793539 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f70c806af10 0x7f70c806d3d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f70dc159fb0 tx=0x7f70d000b040 comp rx=0 tx=0).stop 2023-12-02T00:28:58.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.768+0000 7f8235da0640 1 -- 172.21.15.46:0/3937545204 shutdown_connections 2023-12-02T00:28:58.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.768+0000 7f8235da0640 1 -- 172.21.15.46:0/3937545204 >> 172.21.15.46:0/3937545204 conn(0x7f8230071330 msgr2=0x7f823006dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:58.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.768+0000 7f70e29b7640 1 -- 172.21.15.46:0/2535793539 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f70dc06cf40 msgr2=0x7f70dc1eee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:58.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.768+0000 7f70e29b7640 1 --2- 172.21.15.46:0/2535793539 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f70dc06cf40 0x7f70dc1eee40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f70cc00c3c0 tx=0x7f70cc00c880 comp rx=0 tx=0).stop 2023-12-02T00:28:58.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.769+0000 7f70e0f54640 1 -- 172.21.15.46:0/2535793539 reap_dead start 2023-12-02T00:28:58.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.769+0000 7f8235da0640 1 -- 172.21.15.46:0/3937545204 shutdown_connections 2023-12-02T00:28:58.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.769+0000 7f70e29b7640 1 -- 172.21.15.46:0/2535793539 shutdown_connections 2023-12-02T00:28:58.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.769+0000 7f70e29b7640 1 -- 172.21.15.46:0/2535793539 >> 172.21.15.46:0/2535793539 conn(0x7f70dc06a6b0 msgr2=0x7f70dc1419d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:58.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.769+0000 7f8235da0640 1 -- 172.21.15.46:0/3937545204 wait complete. 2023-12-02T00:28:58.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.769+0000 7f70e29b7640 1 -- 172.21.15.46:0/2535793539 shutdown_connections 2023-12-02T00:28:58.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:58.769+0000 7f70e29b7640 1 -- 172.21.15.46:0/2535793539 wait complete. 2023-12-02T00:28:58.830 INFO:teuthology.orchestra.run.smithi046.stdout:55834574856 2023-12-02T00:28:58.830 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-12-02T00:28:58.831 INFO:teuthology.orchestra.run.smithi046.stdout:55834574855 2023-12-02T00:28:58.831 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-12-02T00:28:58.832 INFO:teuthology.orchestra.run.smithi046.stdout:55834574856 2023-12-02T00:28:58.832 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-12-02T00:28:58.833 INFO:teuthology.orchestra.run.smithi046.stdout:55834574854 2023-12-02T00:28:58.833 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-12-02T00:28:58.834 INFO:teuthology.orchestra.run.smithi046.stdout:55834574855 2023-12-02T00:28:58.834 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-12-02T00:28:58.835 INFO:teuthology.orchestra.run.smithi046.stdout:55834574855 2023-12-02T00:28:58.835 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-12-02T00:28:58.836 INFO:teuthology.orchestra.run.smithi046.stdout:55834574854 2023-12-02T00:28:58.836 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-12-02T00:28:58.837 INFO:teuthology.orchestra.run.smithi046.stdout:55834574855 2023-12-02T00:28:58.837 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-12-02T00:28:59.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.053+0000 7f3a4af68640 1 Processor -- start 2023-12-02T00:28:59.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.053+0000 7f3a4af68640 1 -- start start 2023-12-02T00:28:59.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.053+0000 7fed7b7b7640 1 Processor -- start 2023-12-02T00:28:59.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.054+0000 7f3a4af68640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3a44159210 0x7f3a4415a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:59.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.054+0000 7f3a4af68640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a4415a830 con 0x7f3a440b34a0 2023-12-02T00:28:59.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.054+0000 7f3a4af68640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a4415dc80 con 0x7f3a440b40b0 2023-12-02T00:28:59.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.054+0000 7f3a4af68640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a440cd020 con 0x7f3a44159210 2023-12-02T00:28:59.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.054+0000 7fed7b7b7640 1 -- start start 2023-12-02T00:28:59.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.054+0000 7f3a49505640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f3a440b34a0 0x7f3a4415d570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:44932/0 (socket says 172.21.15.46:44932) 2023-12-02T00:28:59.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.055+0000 7f3a49505640 1 -- 172.21.15.46:0/1000354555 learned_addr learned my addr 172.21.15.46:0/1000354555 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:28:59.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.055+0000 7f45c2c9b640 1 Processor -- start 2023-12-02T00:28:59.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.055+0000 7f45c2c9b640 1 -- start start 2023-12-02T00:28:59.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.055+0000 7fed7b7b7640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fed6c0eb020 0x7fed6c0552c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:59.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.055+0000 7f4d3a2da640 1 Processor -- start 2023-12-02T00:28:59.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.055+0000 7fed7b7b7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed6c0eb400 0x7fed6c0eb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:59.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.055+0000 7f3a48d04640 1 --2- 172.21.15.46:0/1000354555 >> v2:172.21.15.46:3300/0 conn(0x7f3a44159210 0x7f3a4415a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:59.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.055+0000 7f724415a640 1 Processor -- start 2023-12-02T00:28:59.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.055+0000 7f4d3a2da640 1 -- start start 2023-12-02T00:28:59.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.055+0000 7fcefcd33640 1 Processor -- start 2023-12-02T00:28:59.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.055+0000 7fcefcd33640 1 -- start start 2023-12-02T00:28:59.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.055+0000 7fed7b7b7640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed6c057df0 con 0x7fed6c0e7a30 2023-12-02T00:28:59.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.055+0000 7fed7b7b7640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed6c057960 con 0x7fed6c0eb020 2023-12-02T00:28:59.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.055+0000 7fed7b7b7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c0387a0 con 0x7fed6c0eb400 2023-12-02T00:28:59.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.055+0000 7fed79d54640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fed6c0e7a30 0x7fed6c0e7ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:44948/0 (socket says 172.21.15.46:44948) 2023-12-02T00:28:59.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.056+0000 7fed79d54640 1 -- 172.21.15.46:0/2482524840 learned_addr learned my addr 172.21.15.46:0/2482524840 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:28:59.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.056+0000 7f724415a640 1 -- start start 2023-12-02T00:28:59.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.056+0000 7f45c2c9b640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f45bc15df30 0x7f45bc167fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:59.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.056+0000 7f45c2c9b640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45bc146f80 con 0x7f45bc072400 2023-12-02T00:28:59.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.056+0000 7f45c2c9b640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45bc0b47e0 con 0x7f45bc146ba0 2023-12-02T00:28:59.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.056+0000 7f45c2c9b640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45bc0ac8e0 con 0x7f45bc15df30 2023-12-02T00:28:59.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.056+0000 7f723a28c640 1 Processor -- start 2023-12-02T00:28:59.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.056+0000 7fed79553640 1 --2- 172.21.15.46:0/2482524840 >> v2:172.21.15.46:3300/0 conn(0x7fed6c0eb020 0x7fed6c0552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:59.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.057+0000 7f45c1238640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f45bc15df30 0x7f45bc167fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:59.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.057+0000 7f45c0a37640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f45bc146ba0 0x7f45bc0b3d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.46:36808/0 (socket says 172.21.15.46:36808) 2023-12-02T00:28:59.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.057+0000 7f45c0a37640 1 -- 172.21.15.46:0/1401713728 learned_addr learned my addr 172.21.15.46:0/1401713728 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:28:59.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.057+0000 7f45bb7fe640 1 -- 172.21.15.46:0/1401713728 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3439203077 0 0) 0x7f45bc0b47e0 con 0x7f45bc146ba0 2023-12-02T00:28:59.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.057+0000 7f45bb7fe640 1 -- 172.21.15.46:0/1401713728 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45ac003520 con 0x7f45bc146ba0 2023-12-02T00:28:59.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.057+0000 7f45c1238640 1 -- 172.21.15.46:0/1401713728 >> v1:172.21.15.162:6789/0 conn(0x7f45bc146ba0 legacy=0x7f45bc0b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:59.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.057+0000 7f45c1238640 1 -- 172.21.15.46:0/1401713728 >> v1:172.21.15.46:6789/0 conn(0x7f45bc072400 legacy=0x7f45bc15d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:59.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.057+0000 7f45c1238640 1 -- 172.21.15.46:0/1401713728 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45bc0b4ac0 con 0x7f45bc15df30 2023-12-02T00:28:59.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.057+0000 7fcefcd33640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fcef806d170 0x7fcef806f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:59.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.057+0000 7fcefcd33640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcef806db30 con 0x7fcef806abe0 2023-12-02T00:28:59.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.057+0000 7fcefcd33640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcef806e340 con 0x7fcef814b2e0 2023-12-02T00:28:59.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.057+0000 7fcefcd33640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcef806ea10 con 0x7fcef806d170 2023-12-02T00:28:59.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.057+0000 7f723a28c640 1 -- start start 2023-12-02T00:28:59.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.057+0000 7fed78d52640 1 --2- 172.21.15.46:0/2482524840 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed6c0eb400 0x7fed6c0eb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:59.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.057+0000 7fed78d52640 1 -- 172.21.15.46:0/2482524840 >> v2:172.21.15.46:3300/0 conn(0x7fed6c0eb020 msgr2=0x7fed6c0552c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:59.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.057+0000 7fed78d52640 1 --2- 172.21.15.46:0/2482524840 >> v2:172.21.15.46:3300/0 conn(0x7fed6c0eb020 0x7fed6c0552c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.057+0000 7fed78d52640 1 -- 172.21.15.46:0/2482524840 >> v1:172.21.15.46:6789/0 conn(0x7fed6c0e7a30 legacy=0x7fed6c0e7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:59.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.058+0000 7fed78d52640 1 -- 172.21.15.46:0/2482524840 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed6c056d50 con 0x7fed6c0eb400 2023-12-02T00:28:59.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.056+0000 7f3a48d04640 1 -- 172.21.15.46:0/1000354555 >> v1:172.21.15.162:6789/0 conn(0x7f3a440b40b0 legacy=0x7f3a4415ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:59.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.056+0000 7f3a48d04640 1 -- 172.21.15.46:0/1000354555 >> v1:172.21.15.46:6789/0 conn(0x7f3a440b34a0 legacy=0x7f3a4415d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:59.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.056+0000 7f3a48d04640 1 -- 172.21.15.46:0/1000354555 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a440cd300 con 0x7f3a44159210 2023-12-02T00:28:59.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.057+0000 7f3a48d04640 1 --2- 172.21.15.46:0/1000354555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a44159210 0x7f3a4415a2f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3a380087a0 tx=0x7f3a3802de50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33964ea853b3ae6c server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:59.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.057+0000 7f4d3a2da640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4d2c0e1270 0x7f4d2c0534b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:59.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.057+0000 7f4d3a2da640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4d2c0e4cf0 0x7f4d2c0539f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:59.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.057+0000 7f4d3a2da640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d2c054fa0 con 0x7f4d2c054b10 2023-12-02T00:28:59.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.057+0000 7f4d3a2da640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d2c055c20 con 0x7f4d2c0e1270 2023-12-02T00:28:59.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.057+0000 7f4d3a2da640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d2c055280 con 0x7f4d2c0e4cf0 2023-12-02T00:28:59.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.057+0000 7f4d38877640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4d2c0e4cf0 0x7f4d2c0539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:59.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.057+0000 7f4d38877640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4d2c0e4cf0 0x7f4d2c0539f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.46:44406/0 (socket says 172.21.15.46:44406) 2023-12-02T00:28:59.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.057+0000 7f4d38877640 1 -- 172.21.15.46:0/657434853 learned_addr learned my addr 172.21.15.46:0/657434853 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:59.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.058+0000 7f4d33fff640 1 --2- 172.21.15.46:0/657434853 >> v2:172.21.15.46:3300/0 conn(0x7f4d2c0e1270 0x7f4d2c0534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:59.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.058+0000 7f724415a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f723c15d3f0 0x7f723c0d0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:59.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.058+0000 7f724415a640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f723c0d2da0 0x7f723c0b6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:59.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.057+0000 7ff270e48640 1 Processor -- start 2023-12-02T00:28:59.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.057+0000 7ff270e48640 1 -- start start 2023-12-02T00:28:59.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.058+0000 7f723a28c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f722c0deae0 0x7f722c033e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:59.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.058+0000 7f723a28c640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f722c04f940 con 0x7f722c0062a0 2023-12-02T00:28:59.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.058+0000 7f723a28c640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f722c0deec0 con 0x7f722c034390 2023-12-02T00:28:59.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.058+0000 7f723a28c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f722c04fc00 con 0x7f722c0deae0 2023-12-02T00:28:59.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.058+0000 7f7239a8b640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f722c0062a0 0x7f722c037d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:44998/0 (socket says 172.21.15.46:44998) 2023-12-02T00:28:59.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.058+0000 7f7239a8b640 1 -- 172.21.15.46:0/3099177029 learned_addr learned my addr 172.21.15.46:0/3099177029 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:28:59.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.059+0000 7f723928a640 1 --2- 172.21.15.46:0/3099177029 >> v2:172.21.15.162:3300/0 conn(0x7f722c0deae0 0x7f722c033e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:59.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.058+0000 7fcef6d9d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fcef806d170 0x7fcef806f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:59.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.058+0000 7fcef659c640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fcef814b2e0 0x7fcef806a4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.46:36818/0 (socket says 172.21.15.46:36818) 2023-12-02T00:28:59.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.058+0000 7fcef659c640 1 -- 172.21.15.46:0/2825144596 learned_addr learned my addr 172.21.15.46:0/2825144596 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:28:59.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.058+0000 7fcef6d9d640 1 --2- 172.21.15.46:0/2825144596 >> v2:172.21.15.162:3300/0 conn(0x7fcef806d170 0x7fcef806f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.46:44420/0 (socket says 172.21.15.46:44420) 2023-12-02T00:28:59.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.059+0000 7f45c1238640 1 --2- 172.21.15.46:0/1401713728 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45bc15df30 0x7f45bc167fa0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f45b4009d90 tx=0x7f45b402beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f7ccaf235ee8c9d server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:59.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.059+0000 7f722bfff640 1 -- 172.21.15.46:0/3099177029 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3290298689 0 0) 0x7f722c04f940 con 0x7f722c0062a0 2023-12-02T00:28:59.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.059+0000 7fed78d52640 1 --2- 172.21.15.46:0/2482524840 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed6c0eb400 0x7fed6c0eb880 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fed640097a0 tx=0x7fed6402b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5da3955f46ba72c1 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:59.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.059+0000 7f3a437fe640 1 -- 172.21.15.46:0/1000354555 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a3802ec00 con 0x7f3a44159210 2023-12-02T00:28:59.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.059+0000 7f722bfff640 1 -- 172.21.15.46:0/3099177029 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7220003520 con 0x7f722c0062a0 2023-12-02T00:28:59.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.059+0000 7f722bfff640 1 -- 172.21.15.46:0/3099177029 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3244821319 0 0) 0x7f722c0deec0 con 0x7f722c034390 2023-12-02T00:28:59.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.058+0000 7f724415a640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f723c0d12f0 con 0x7f723c0b4c60 2023-12-02T00:28:59.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.058+0000 7f4d38877640 1 -- 172.21.15.46:0/657434853 >> v2:172.21.15.46:3300/0 conn(0x7f4d2c0e1270 msgr2=0x7f4d2c0534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:59.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.058+0000 7f4d38877640 1 --2- 172.21.15.46:0/657434853 >> v2:172.21.15.46:3300/0 conn(0x7f4d2c0e1270 0x7f4d2c0534b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.058+0000 7f4d38877640 1 -- 172.21.15.46:0/657434853 >> v1:172.21.15.46:6789/0 conn(0x7f4d2c054b10 legacy=0x7f4d2c0e1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:59.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.058+0000 7f4d38877640 1 -- 172.21.15.46:0/657434853 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d2c055560 con 0x7f4d2c0e4cf0 2023-12-02T00:28:59.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.059+0000 7ff270e48640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff2640e12d0 0x7ff26403a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:59.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.058+0000 7f724415a640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f723c0b61e0 con 0x7f723c0d2da0 2023-12-02T00:28:59.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.058+0000 7f724415a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f723c15d7d0 con 0x7f723c15d3f0 2023-12-02T00:28:59.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.058+0000 7f7241ef6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f723c15d3f0 0x7f723c0d0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:59.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.058+0000 7f7241ef6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f723c15d3f0 0x7f723c0d0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.46:35064/0 (socket says 172.21.15.46:35064) 2023-12-02T00:28:59.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.058+0000 7f7241ef6640 1 -- 172.21.15.46:0/1971018797 learned_addr learned my addr 172.21.15.46:0/1971018797 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:59.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.058+0000 7f7241ef6640 1 -- 172.21.15.46:0/1971018797 >> v2:172.21.15.46:3300/0 conn(0x7f723c0d2da0 msgr2=0x7f723c0b6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:59.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.058+0000 7f72426f7640 1 --2- 172.21.15.46:0/1971018797 >> v2:172.21.15.46:3300/0 conn(0x7f723c0d2da0 0x7f723c0b6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:59.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.059+0000 7f7241ef6640 1 --2- 172.21.15.46:0/1971018797 >> v2:172.21.15.46:3300/0 conn(0x7f723c0d2da0 0x7f723c0b6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.059+0000 7f7241ef6640 1 -- 172.21.15.46:0/1971018797 >> v1:172.21.15.46:6789/0 conn(0x7f723c0b4c60 legacy=0x7f723c0d2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:59.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.059+0000 7f7241ef6640 1 -- 172.21.15.46:0/1971018797 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f723c0d1520 con 0x7f723c15d3f0 2023-12-02T00:28:59.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.059+0000 7fed6bfff640 1 -- 172.21.15.46:0/2482524840 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed64031780 con 0x7fed6c0eb400 2023-12-02T00:28:59.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.059+0000 7ff270e48640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2640558e0 con 0x7ff26400a420 2023-12-02T00:28:59.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.059+0000 7ff270e48640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff26403a670 con 0x7ff26400ab00 2023-12-02T00:28:59.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.059+0000 7ff270e48640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2640e16b0 con 0x7ff2640e12d0 2023-12-02T00:28:59.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.059+0000 7fed6bfff640 1 -- 172.21.15.46:0/2482524840 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed640318e0 con 0x7fed6c0eb400 2023-12-02T00:28:59.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.059+0000 7fed6bfff640 1 -- 172.21.15.46:0/2482524840 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed64031c70 con 0x7fed6c0eb400 2023-12-02T00:28:59.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.059+0000 7f723928a640 1 -- 172.21.15.46:0/3099177029 >> v1:172.21.15.162:6789/0 conn(0x7f722c034390 legacy=0x7f722c0509c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:59.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.059+0000 7f723928a640 1 -- 172.21.15.46:0/3099177029 >> v1:172.21.15.46:6789/0 conn(0x7f722c0062a0 legacy=0x7f722c037d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:59.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.059+0000 7f723928a640 1 -- 172.21.15.46:0/3099177029 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f722c04fee0 con 0x7f722c0deae0 2023-12-02T00:28:59.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.059+0000 7f3a437fe640 1 -- 172.21.15.46:0/1000354555 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a38032070 con 0x7f3a44159210 2023-12-02T00:28:59.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.059+0000 7f4d38877640 1 --2- 172.21.15.46:0/657434853 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d2c0e4cf0 0x7f4d2c0539f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4d34050140 tx=0x7f4d340a5dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=364393a644179c97 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:59.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.059+0000 7fcef6d9d640 1 -- 172.21.15.46:0/2825144596 >> v1:172.21.15.162:6789/0 conn(0x7fcef814b2e0 legacy=0x7fcef806a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:59.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.059+0000 7f3a437fe640 1 -- 172.21.15.46:0/1000354555 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a38038400 con 0x7f3a44159210 2023-12-02T00:28:59.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.059+0000 7ff26ad9d640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff26400a420 0x7ff264055d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45012/0 (socket says 172.21.15.46:45012) 2023-12-02T00:28:59.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.059+0000 7ff26ad9d640 1 -- 172.21.15.46:0/712289493 learned_addr learned my addr 172.21.15.46:0/712289493 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:28:59.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.059+0000 7fcef6d9d640 1 -- 172.21.15.46:0/2825144596 >> v1:172.21.15.46:6789/0 conn(0x7fcef806abe0 legacy=0x7fcef80b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:28:59.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.059+0000 7fcef6d9d640 1 -- 172.21.15.46:0/2825144596 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef806d750 con 0x7fcef806d170 2023-12-02T00:28:59.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.060+0000 7f45bb7fe640 1 -- 172.21.15.46:0/1401713728 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b402cc60 con 0x7f45bc15df30 2023-12-02T00:28:59.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.060+0000 7f45bb7fe640 1 -- 172.21.15.46:0/1401713728 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45b402cdc0 con 0x7f45bc15df30 2023-12-02T00:28:59.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.060+0000 7f45bb7fe640 1 -- 172.21.15.46:0/1401713728 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b402cc60 con 0x7f45bc15df30 2023-12-02T00:28:59.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.060+0000 7f72426f7640 1 --2- 172.21.15.46:0/1971018797 >> v2:172.21.15.46:3300/0 conn(0x7f723c0d2da0 0x7f723c0b6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:28:59.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.060+0000 7f7241ef6640 1 --2- 172.21.15.46:0/1971018797 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f723c15d3f0 0x7f723c0d0db0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7238009870 tx=0x7f723802b9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b485b1fd98bcf223 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:59.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.060+0000 7ff26a59c640 1 --2- 172.21.15.46:0/712289493 >> v2:172.21.15.46:3300/0 conn(0x7ff2640e12d0 0x7ff26403a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:59.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.060+0000 7f3a4af68640 1 -- 172.21.15.46:0/1000354555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a44159210 msgr2=0x7f3a4415a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:59.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.060+0000 7f3a4af68640 1 --2- 172.21.15.46:0/1000354555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a44159210 0x7f3a4415a2f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3a380087a0 tx=0x7f3a3802de50 comp rx=0 tx=0).stop 2023-12-02T00:28:59.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.060+0000 7ff26959a640 1 -- 172.21.15.46:0/712289493 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2170536813 0 0) 0x7ff2640558e0 con 0x7ff26400a420 2023-12-02T00:28:59.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.060+0000 7f723928a640 1 --2- 172.21.15.46:0/3099177029 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f722c0deae0 0x7f722c033e50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f72240097d0 tx=0x7f722402b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa677d49b7098fba server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:59.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.060+0000 7f3a4af68640 1 -- 172.21.15.46:0/1000354555 shutdown_connections 2023-12-02T00:28:59.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.060+0000 7f3a4af68640 1 --2- 172.21.15.46:0/1000354555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a44159210 0x7f3a4415a2f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.060+0000 7f3a4af68640 1 -- 172.21.15.46:0/1000354555 >> 172.21.15.46:0/1000354555 conn(0x7f3a440b0d60 msgr2=0x7f3a440b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:59.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.060+0000 7f7240ef4640 1 -- 172.21.15.46:0/1971018797 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7238030860 con 0x7f723c15d3f0 2023-12-02T00:28:59.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.060+0000 7f7240ef4640 1 -- 172.21.15.46:0/1971018797 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72380309c0 con 0x7f723c15d3f0 2023-12-02T00:28:59.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.060+0000 7f7240ef4640 1 -- 172.21.15.46:0/1971018797 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7238030d30 con 0x7f723c15d3f0 2023-12-02T00:28:59.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.060+0000 7ff26959a640 1 -- 172.21.15.46:0/712289493 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff254003520 con 0x7ff26400a420 2023-12-02T00:28:59.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.060+0000 7ff26959a640 1 -- 172.21.15.46:0/712289493 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2122975787 0 0) 0x7ff26403a670 con 0x7ff26400ab00 2023-12-02T00:28:59.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.060+0000 7ff26959a640 1 -- 172.21.15.46:0/712289493 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2640558e0 con 0x7ff26400ab00 2023-12-02T00:28:59.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.061+0000 7ff26a59c640 1 -- 172.21.15.46:0/712289493 >> v1:172.21.15.162:6789/0 conn(0x7ff26400ab00 legacy=0x7ff264009d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:59.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.061+0000 7ff26a59c640 1 -- 172.21.15.46:0/712289493 >> v1:172.21.15.46:6789/0 conn(0x7ff26400a420 legacy=0x7ff264055d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:59.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.061+0000 7ff26a59c640 1 -- 172.21.15.46:0/712289493 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff264054cd0 con 0x7ff2640e12d0 2023-12-02T00:28:59.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.061+0000 7f4d32ffd640 1 -- 172.21.15.46:0/657434853 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d340a6c40 con 0x7f4d2c0e4cf0 2023-12-02T00:28:59.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.060+0000 7f3a4af68640 1 -- 172.21.15.46:0/1000354555 shutdown_connections 2023-12-02T00:28:59.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.061+0000 7f4d32ffd640 1 -- 172.21.15.46:0/657434853 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d340a6da0 con 0x7f4d2c0e4cf0 2023-12-02T00:28:59.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.061+0000 7f4d32ffd640 1 -- 172.21.15.46:0/657434853 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d340ae640 con 0x7f4d2c0e4cf0 2023-12-02T00:28:59.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.061+0000 7fed7b7b7640 1 -- 172.21.15.46:0/2482524840 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed6c0eb400 msgr2=0x7fed6c0eb880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:59.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.061+0000 7fed7b7b7640 1 --2- 172.21.15.46:0/2482524840 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed6c0eb400 0x7fed6c0eb880 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fed640097a0 tx=0x7fed6402b820 comp rx=0 tx=0).stop 2023-12-02T00:28:59.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.061+0000 7f45c2c9b640 1 -- 172.21.15.46:0/1401713728 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45bc15df30 msgr2=0x7f45bc167fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:59.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.061+0000 7f45c2c9b640 1 --2- 172.21.15.46:0/1401713728 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45bc15df30 0x7f45bc167fa0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f45b4009d90 tx=0x7f45b402beb0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.061+0000 7fed7b7b7640 1 -- 172.21.15.46:0/2482524840 shutdown_connections 2023-12-02T00:28:59.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.061+0000 7f722bfff640 1 -- 172.21.15.46:0/3099177029 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7224030610 con 0x7f722c0deae0 2023-12-02T00:28:59.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.061+0000 7fed7b7b7640 1 --2- 172.21.15.46:0/2482524840 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed6c0eb400 0x7fed6c0eb880 secure :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fed640097a0 tx=0x7fed6402b820 comp rx=0 tx=0).stop 2023-12-02T00:28:59.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.061+0000 7f45c2c9b640 1 -- 172.21.15.46:0/1401713728 shutdown_connections 2023-12-02T00:28:59.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.061+0000 7f45c2c9b640 1 --2- 172.21.15.46:0/1401713728 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45bc15df30 0x7f45bc167fa0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.061+0000 7f45c2c9b640 1 -- 172.21.15.46:0/1401713728 >> 172.21.15.46:0/1401713728 conn(0x7f45bc0b0d60 msgr2=0x7f45bc0b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:59.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.061+0000 7f722bfff640 1 -- 172.21.15.46:0/3099177029 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7224030770 con 0x7f722c0deae0 2023-12-02T00:28:59.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.061+0000 7fed7b7b7640 1 --2- 172.21.15.46:0/2482524840 >> v2:172.21.15.46:3300/0 conn(0x7fed6c0eb020 0x7fed6c0552c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.061+0000 7f45c2c9b640 1 -- 172.21.15.46:0/1401713728 shutdown_connections 2023-12-02T00:28:59.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.061+0000 7f722bfff640 1 -- 172.21.15.46:0/3099177029 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7224030b00 con 0x7f722c0deae0 2023-12-02T00:28:59.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.061+0000 7fed7b7b7640 1 -- 172.21.15.46:0/2482524840 >> 172.21.15.46:0/2482524840 conn(0x7fed6c0559a0 msgr2=0x7fed6c055db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:59.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.061+0000 7f724415a640 1 -- 172.21.15.46:0/1971018797 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f723c15d3f0 msgr2=0x7f723c0d0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:59.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.061+0000 7f724415a640 1 --2- 172.21.15.46:0/1971018797 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f723c15d3f0 0x7f723c0d0db0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7238009870 tx=0x7f723802b9a0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.061+0000 7f45c2c9b640 1 -- 172.21.15.46:0/1401713728 wait complete. 2023-12-02T00:28:59.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.061+0000 7f724415a640 1 -- 172.21.15.46:0/1971018797 shutdown_connections 2023-12-02T00:28:59.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.061+0000 7f3a4af68640 1 -- 172.21.15.46:0/1000354555 wait complete. 2023-12-02T00:28:59.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.061+0000 7f724415a640 1 --2- 172.21.15.46:0/1971018797 >> v2:172.21.15.46:3300/0 conn(0x7f723c0d2da0 0x7f723c0b6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.061+0000 7fed7b7b7640 1 -- 172.21.15.46:0/2482524840 shutdown_connections 2023-12-02T00:28:59.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.061+0000 7fcef6d9d640 1 --2- 172.21.15.46:0/2825144596 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcef806d170 0x7fcef806f7a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcef0009d90 tx=0x7fcef002bd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdb67109b2979aae server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:59.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.061+0000 7fcef559a640 1 -- 172.21.15.46:0/2825144596 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef002cb40 con 0x7fcef806d170 2023-12-02T00:28:59.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.061+0000 7fcef559a640 1 -- 172.21.15.46:0/2825144596 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef002cca0 con 0x7fcef806d170 2023-12-02T00:28:59.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.061+0000 7fcef559a640 1 -- 172.21.15.46:0/2825144596 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef0034660 con 0x7fcef806d170 2023-12-02T00:28:59.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.061+0000 7f724415a640 1 --2- 172.21.15.46:0/1971018797 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f723c15d3f0 0x7f723c0d0db0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.062+0000 7f724415a640 1 -- 172.21.15.46:0/1971018797 >> 172.21.15.46:0/1971018797 conn(0x7f723c072230 msgr2=0x7f723c0cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:59.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.062+0000 7f4d3a2da640 1 -- 172.21.15.46:0/657434853 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d2c0e4cf0 msgr2=0x7f4d2c0539f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:59.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.062+0000 7f4d3a2da640 1 --2- 172.21.15.46:0/657434853 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d2c0e4cf0 0x7f4d2c0539f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4d34050140 tx=0x7f4d340a5dd0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.062+0000 7fed7b7b7640 1 -- 172.21.15.46:0/2482524840 wait complete. 2023-12-02T00:28:59.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.062+0000 7f724415a640 1 -- 172.21.15.46:0/1971018797 shutdown_connections 2023-12-02T00:28:59.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.062+0000 7f724415a640 1 -- 172.21.15.46:0/1971018797 wait complete. 2023-12-02T00:28:59.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.062+0000 7ff26a59c640 1 --2- 172.21.15.46:0/712289493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2640e12d0 0x7ff26403a130 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff25c0097a0 tx=0x7ff25c034190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c90979e3762004f7 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:59.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.062+0000 7f723a28c640 1 -- 172.21.15.46:0/3099177029 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f722c0deae0 msgr2=0x7f722c033e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:59.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.062+0000 7f723a28c640 1 --2- 172.21.15.46:0/3099177029 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f722c0deae0 0x7f722c033e50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f72240097d0 tx=0x7f722402b960 comp rx=0 tx=0).stop 2023-12-02T00:28:59.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.062+0000 7f4d3a2da640 1 -- 172.21.15.46:0/657434853 shutdown_connections 2023-12-02T00:28:59.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.062+0000 7f4d3a2da640 1 --2- 172.21.15.46:0/657434853 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d2c0e4cf0 0x7f4d2c0539f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.062+0000 7f723a28c640 1 -- 172.21.15.46:0/3099177029 shutdown_connections 2023-12-02T00:28:59.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.062+0000 7f4d3a2da640 1 --2- 172.21.15.46:0/657434853 >> v2:172.21.15.46:3300/0 conn(0x7f4d2c0e1270 0x7f4d2c0534b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.062+0000 7f723a28c640 1 --2- 172.21.15.46:0/3099177029 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f722c0deae0 0x7f722c033e50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.062+0000 7f723a28c640 1 -- 172.21.15.46:0/3099177029 >> 172.21.15.46:0/3099177029 conn(0x7f722c054ed0 msgr2=0x7f722c0552e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:59.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.062+0000 7f4d3a2da640 1 -- 172.21.15.46:0/657434853 >> 172.21.15.46:0/657434853 conn(0x7f4d2c020000 msgr2=0x7f4d2c056d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:59.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.062+0000 7f4d3a2da640 1 -- 172.21.15.46:0/657434853 shutdown_connections 2023-12-02T00:28:59.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.062+0000 7f723a28c640 1 -- 172.21.15.46:0/3099177029 shutdown_connections 2023-12-02T00:28:59.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.062+0000 7ff26959a640 1 -- 172.21.15.46:0/712289493 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25c03c440 con 0x7ff2640e12d0 2023-12-02T00:28:59.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.062+0000 7f723a28c640 1 -- 172.21.15.46:0/3099177029 wait complete. 2023-12-02T00:28:59.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.062+0000 7f4d3a2da640 1 -- 172.21.15.46:0/657434853 wait complete. 2023-12-02T00:28:59.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.063+0000 7f45c2c9b640 1 Processor -- start 2023-12-02T00:28:59.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.063+0000 7f45c2c9b640 1 -- start start 2023-12-02T00:28:59.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.063+0000 7f3a4af68640 1 Processor -- start 2023-12-02T00:28:59.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.062+0000 7ff26959a640 1 -- 172.21.15.46:0/712289493 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff25c03c5a0 con 0x7ff2640e12d0 2023-12-02T00:28:59.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.063+0000 7ff270e48640 1 -- 172.21.15.46:0/712289493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2640e12d0 msgr2=0x7ff26403a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:59.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.063+0000 7ff270e48640 1 --2- 172.21.15.46:0/712289493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2640e12d0 0x7ff26403a130 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff25c0097a0 tx=0x7ff25c034190 comp rx=0 tx=0).stop 2023-12-02T00:28:59.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.063+0000 7fcefcd33640 1 -- 172.21.15.46:0/2825144596 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcef806d170 msgr2=0x7fcef806f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:59.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.063+0000 7fcefcd33640 1 --2- 172.21.15.46:0/2825144596 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcef806d170 0x7fcef806f7a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcef0009d90 tx=0x7fcef002bd90 comp rx=0 tx=0).stop 2023-12-02T00:28:59.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.063+0000 7fcefcd33640 1 -- 172.21.15.46:0/2825144596 shutdown_connections 2023-12-02T00:28:59.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.063+0000 7fcefcd33640 1 --2- 172.21.15.46:0/2825144596 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcef806d170 0x7fcef806f7a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.063+0000 7fcefcd33640 1 -- 172.21.15.46:0/2825144596 >> 172.21.15.46:0/2825144596 conn(0x7fcef80cfd80 msgr2=0x7fcef80d0190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:59.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.063+0000 7f3a4af68640 1 -- start start 2023-12-02T00:28:59.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.064+0000 7ff270e48640 1 -- 172.21.15.46:0/712289493 shutdown_connections 2023-12-02T00:28:59.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.064+0000 7f724415a640 1 Processor -- start 2023-12-02T00:28:59.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.064+0000 7ff270e48640 1 --2- 172.21.15.46:0/712289493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2640e12d0 0x7ff26403a130 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.064+0000 7ff270e48640 1 -- 172.21.15.46:0/712289493 >> 172.21.15.46:0/712289493 conn(0x7ff264056c90 msgr2=0x7ff2640570a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:59.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.064+0000 7fcefcd33640 1 -- 172.21.15.46:0/2825144596 shutdown_connections 2023-12-02T00:28:59.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.064+0000 7ff270e48640 1 -- 172.21.15.46:0/712289493 shutdown_connections 2023-12-02T00:28:59.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.064+0000 7f723a28c640 1 Processor -- start 2023-12-02T00:28:59.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.064+0000 7f724415a640 1 -- start start 2023-12-02T00:28:59.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.064+0000 7f723a28c640 1 -- start start 2023-12-02T00:28:59.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.064+0000 7f45c2c9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45bc072400 0x7f45bc141910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:59.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.064+0000 7f45c2c9b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f45bc146ba0 0x7f45bc1f6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:59.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.064+0000 7f45c2c9b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45bc15df30 0x7f45bc0ccf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:59.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.064+0000 7f45c2c9b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45bc0cc7e0 con 0x7f45bc072400 2023-12-02T00:28:59.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.064+0000 7f45c2c9b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45bc141e50 con 0x7f45bc15df30 2023-12-02T00:28:59.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.064+0000 7f45c2c9b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45bc0ab2f0 con 0x7f45bc146ba0 2023-12-02T00:28:59.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.065+0000 7f4d3a2da640 1 Processor -- start 2023-12-02T00:28:59.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.065+0000 7f45c0a37640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45bc072400 0x7f45bc141910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:59.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.065+0000 7f45bbfff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f45bc146ba0 0x7f45bc1f6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:59.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.065+0000 7f45c0a37640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45bc072400 0x7f45bc141910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38036/0 (socket says 172.21.15.46:38036) 2023-12-02T00:28:59.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.065+0000 7f45c0a37640 1 -- 172.21.15.46:0/1056919858 learned_addr learned my addr 172.21.15.46:0/1056919858 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:59.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.065+0000 7f45c1238640 1 --2- 172.21.15.46:0/1056919858 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45bc15df30 0x7f45bc0ccf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:59.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.065+0000 7f4d3a2da640 1 -- start start 2023-12-02T00:28:59.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.065+0000 7f723a28c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f722c0062a0 0x7f722c0daa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:59.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.065+0000 7f723a28c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f722c034390 0x7f722c0e3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:59.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.065+0000 7f723a28c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f722c0e3a90 0x7f722c053c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:59.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.065+0000 7f45c0a37640 1 -- 172.21.15.46:0/1056919858 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f45bc146ba0 msgr2=0x7f45bc1f6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:59.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.065+0000 7f45c0a37640 1 --2- 172.21.15.46:0/1056919858 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f45bc146ba0 0x7f45bc1f6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.065+0000 7f723a28c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f722c181090 con 0x7f722c034390 2023-12-02T00:28:59.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.065+0000 7f45c0a37640 1 -- 172.21.15.46:0/1056919858 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45bc15df30 msgr2=0x7f45bc0ccf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:59.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.065+0000 7f45c0a37640 1 --2- 172.21.15.46:0/1056919858 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45bc15df30 0x7f45bc0ccf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.065+0000 7f45c0a37640 1 -- 172.21.15.46:0/1056919858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b4009a70 con 0x7f45bc072400 2023-12-02T00:28:59.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.065+0000 7fed7b7b7640 1 Processor -- start 2023-12-02T00:28:59.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.065+0000 7f724415a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f723c0b4c60 0x7f723c0c91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:59.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.065+0000 7f723a28c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f722c0e3f40 con 0x7f722c0062a0 2023-12-02T00:28:59.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.065+0000 7f723a28c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f722c180430 con 0x7f722c0e3a90 2023-12-02T00:28:59.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.065+0000 7f7238a89640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f722c034390 0x7f722c0e3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:59.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.065+0000 7f7238a89640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f722c034390 0x7f722c0e3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38052/0 (socket says 172.21.15.46:38052) 2023-12-02T00:28:59.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.065+0000 7f7238a89640 1 -- 172.21.15.46:0/55362548 learned_addr learned my addr 172.21.15.46:0/55362548 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:59.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.065+0000 7fed7b7b7640 1 -- start start 2023-12-02T00:28:59.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.066+0000 7f45c0a37640 1 --2- 172.21.15.46:0/1056919858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45bc072400 0x7f45bc141910 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f45b000c500 tx=0x7f45b000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:59.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.064+0000 7fcefcd33640 1 -- 172.21.15.46:0/2825144596 wait complete. 2023-12-02T00:28:59.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.066+0000 7f724415a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f723c0d2da0 0x7f723c0b16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:59.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.066+0000 7f7238a89640 1 -- 172.21.15.46:0/55362548 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f722c0e3a90 msgr2=0x7f722c053c40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-02T00:28:59.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.066+0000 7f3a4af68640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a440b34a0 0x7f3a441f71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:59.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.066+0000 7f7241ef6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f723c0b4c60 0x7f723c0c91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:59.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.066+0000 7f7241ef6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f723c0b4c60 0x7f723c0c91f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38064/0 (socket says 172.21.15.46:38064) 2023-12-02T00:28:59.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.066+0000 7f7238a89640 1 --2- 172.21.15.46:0/55362548 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f722c0e3a90 0x7f722c053c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.066+0000 7f7238a89640 1 -- 172.21.15.46:0/55362548 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f722c0062a0 msgr2=0x7f722c0daa10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-02T00:28:59.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.066+0000 7f7241ef6640 1 -- 172.21.15.46:0/1772902290 learned_addr learned my addr 172.21.15.46:0/1772902290 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:59.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.066+0000 7f7238a89640 1 --2- 172.21.15.46:0/55362548 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f722c0062a0 0x7f722c0daa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.066+0000 7f7238a89640 1 -- 172.21.15.46:0/55362548 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7224009400 con 0x7f722c034390 2023-12-02T00:28:59.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.066+0000 7f724415a640 1 --2- 172.21.15.46:0/1772902290 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f723c15d3f0 0x7f723c0cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:59.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.066+0000 7f72416f5640 1 --2- 172.21.15.46:0/1772902290 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f723c0d2da0 0x7f723c0b16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:59.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.066+0000 7fcefcd33640 1 Processor -- start 2023-12-02T00:28:59.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.066+0000 7f3a48d04640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a440b34a0 0x7f3a441f71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:59.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.066+0000 7f724415a640 1 -- 172.21.15.46:0/1772902290 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f723c0ccde0 con 0x7f723c0b4c60 2023-12-02T00:28:59.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.066+0000 7ff270e48640 1 -- 172.21.15.46:0/712289493 wait complete. 2023-12-02T00:28:59.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.066+0000 7f72426f7640 1 --2- 172.21.15.46:0/1772902290 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f723c15d3f0 0x7f723c0cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:59.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.066+0000 7f4d3a2da640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d2c0e1270 0x7f4d2c0e26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:59.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.066+0000 7f724415a640 1 -- 172.21.15.46:0/1772902290 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f723c0cc310 con 0x7f723c15d3f0 2023-12-02T00:28:59.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.066+0000 7f724415a640 1 -- 172.21.15.46:0/1772902290 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f723c0cc540 con 0x7f723c0d2da0 2023-12-02T00:28:59.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.066+0000 7f3a48d04640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a440b34a0 0x7f3a441f71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38070/0 (socket says 172.21.15.46:38070) 2023-12-02T00:28:59.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.066+0000 7f7238a89640 1 --2- 172.21.15.46:0/55362548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f722c034390 0x7f722c0e3550 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f723000c3c0 tx=0x7f723000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:59.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.066+0000 7f3a4af68640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a440b40b0 0x7f3a44157da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:59.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.066+0000 7f3a4af68640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a44159210 0x7f3a44150610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:59.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.066+0000 7f3a4af68640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a44150f50 con 0x7f3a440b34a0 2023-12-02T00:28:59.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.066+0000 7f3a4af68640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a44150b50 con 0x7f3a44159210 2023-12-02T00:28:59.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.066+0000 7f3a4af68640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a44158a60 con 0x7f3a440b40b0 2023-12-02T00:28:59.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.067+0000 7f3a43fff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a440b40b0 0x7f3a44157da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:59.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.067+0000 7f3a49505640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a44159210 0x7f3a44150610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:59.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.067+0000 7f3a43fff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a440b40b0 0x7f3a44157da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.46:35094/0 (socket says 172.21.15.46:35094) 2023-12-02T00:28:59.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.067+0000 7f3a43fff640 1 -- 172.21.15.46:0/706164443 learned_addr learned my addr 172.21.15.46:0/706164443 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:59.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.067+0000 7f4d33fff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d2c0e1270 0x7f4d2c0e26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:59.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.067+0000 7f4d33fff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d2c0e1270 0x7f4d2c0e26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.46:44460/0 (socket says 172.21.15.46:44460) 2023-12-02T00:28:59.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.067+0000 7fed7b7b7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed6c0e7a30 0x7fed6c0dab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:59.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.067+0000 7f4d33fff640 1 -- 172.21.15.46:0/2327041625 learned_addr learned my addr 172.21.15.46:0/2327041625 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:59.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.067+0000 7f4d3a2da640 1 --2- 172.21.15.46:0/2327041625 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d2c0e4cf0 0x7f4d2c0d5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:59.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.067+0000 7fed7b7b7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed6c0eb020 0x7fed6c04f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:59.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.067+0000 7f3a43fff640 1 -- 172.21.15.46:0/706164443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a44159210 msgr2=0x7f3a44150610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:59.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.067+0000 7f3a43fff640 1 --2- 172.21.15.46:0/706164443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a44159210 0x7f3a44150610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.067+0000 7f3a43fff640 1 -- 172.21.15.46:0/706164443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a440b34a0 msgr2=0x7f3a441f71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:59.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.067+0000 7f3a43fff640 1 --2- 172.21.15.46:0/706164443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a440b34a0 0x7f3a441f71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.067+0000 7f3a43fff640 1 -- 172.21.15.46:0/706164443 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a38008480 con 0x7f3a440b40b0 2023-12-02T00:28:59.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.067+0000 7f3a49505640 1 --2- 172.21.15.46:0/706164443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a44159210 0x7f3a44150610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:28:59.118 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.067+0000 7fed7b7b7640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed6c0d7be0 0x7fed6c0c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:59.118 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.067+0000 7fed79553640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed6c0e7a30 0x7fed6c0dab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:59.118 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.067+0000 7fed79553640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed6c0e7a30 0x7fed6c0dab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.46:35102/0 (socket says 172.21.15.46:35102) 2023-12-02T00:28:59.118 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.067+0000 7fed78d52640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed6c0eb020 0x7fed6c04f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:59.118 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.067+0000 7fed78d52640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed6c0eb020 0x7fed6c04f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38080/0 (socket says 172.21.15.46:38080) 2023-12-02T00:28:59.118 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.067+0000 7fed79553640 1 -- 172.21.15.46:0/111113773 learned_addr learned my addr 172.21.15.46:0/111113773 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:59.118 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.067+0000 7fed7b7b7640 1 -- 172.21.15.46:0/111113773 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c0d7920 con 0x7fed6c0eb020 2023-12-02T00:28:59.118 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.067+0000 7fed7b7b7640 1 -- 172.21.15.46:0/111113773 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c0d81a0 con 0x7fed6c0d7be0 2023-12-02T00:28:59.118 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.067+0000 7f3a43fff640 1 --2- 172.21.15.46:0/706164443 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a440b40b0 0x7f3a44157da0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3a3400c430 tx=0x7f3a3400c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:59.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.067+0000 7fed7b7b7640 1 -- 172.21.15.46:0/111113773 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c050fb0 con 0x7fed6c0e7a30 2023-12-02T00:28:59.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.067+0000 7f7241ef6640 1 -- 172.21.15.46:0/1772902290 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f723c0d2da0 msgr2=0x7f723c0b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:59.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.067+0000 7f7241ef6640 1 --2- 172.21.15.46:0/1772902290 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f723c0d2da0 0x7f723c0b16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.067+0000 7f7241ef6640 1 -- 172.21.15.46:0/1772902290 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f723c15d3f0 msgr2=0x7f723c0cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:59.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.067+0000 7f7241ef6640 1 --2- 172.21.15.46:0/1772902290 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f723c15d3f0 0x7f723c0cbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.067+0000 7f7241ef6640 1 -- 172.21.15.46:0/1772902290 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72380094d0 con 0x7f723c0b4c60 2023-12-02T00:28:59.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.067+0000 7f72426f7640 1 --2- 172.21.15.46:0/1772902290 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f723c15d3f0 0x7f723c0cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:28:59.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.067+0000 7f7241ef6640 1 --2- 172.21.15.46:0/1772902290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f723c0b4c60 0x7f723c0c91f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f72380090b0 tx=0x7f72380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:59.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.066+0000 7fcefcd33640 1 -- start start 2023-12-02T00:28:59.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.067+0000 7f4d3a2da640 1 --2- 172.21.15.46:0/2327041625 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d2c0e3d20 0x7f4d2c04e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:59.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.067+0000 7fed79d54640 1 --2- 172.21.15.46:0/111113773 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed6c0d7be0 0x7fed6c0c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:59.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.068+0000 7fcefcd33640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcef806abe0 0x7fcef81f6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:59.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.068+0000 7ff270e48640 1 Processor -- start 2023-12-02T00:28:59.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.068+0000 7f3a417fa640 1 -- 172.21.15.46:0/706164443 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a34011790 con 0x7f3a440b40b0 2023-12-02T00:28:59.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.068+0000 7fcefcd33640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcef806d170 0x7fcef8144290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:59.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.068+0000 7fcefcd33640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcef814b2e0 0x7fcef81447d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:59.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.068+0000 7f45b97fa640 1 -- 172.21.15.46:0/1056919858 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b0005ce0 con 0x7f45bc072400 2023-12-02T00:28:59.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.068+0000 7fcef659c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcef806abe0 0x7fcef81f6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:59.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.068+0000 7fcef659c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcef806abe0 0x7fcef81f6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.46:35112/0 (socket says 172.21.15.46:35112) 2023-12-02T00:28:59.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.068+0000 7f3a4af68640 1 -- 172.21.15.46:0/706164443 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a44154d60 con 0x7f3a440b40b0 2023-12-02T00:28:59.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.068+0000 7ff270e48640 1 -- start start 2023-12-02T00:28:59.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.068+0000 7f45b97fa640 1 -- 172.21.15.46:0/1056919858 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45b0005e40 con 0x7f45bc072400 2023-12-02T00:28:59.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.068+0000 7fed78d52640 1 -- 172.21.15.46:0/111113773 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed6c0e7a30 msgr2=0x7fed6c0dab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:59.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.068+0000 7fed78d52640 1 --2- 172.21.15.46:0/111113773 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed6c0e7a30 0x7fed6c0dab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.068+0000 7fed78d52640 1 -- 172.21.15.46:0/111113773 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed6c0d7be0 msgr2=0x7fed6c0c7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:59.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.068+0000 7fed78d52640 1 --2- 172.21.15.46:0/111113773 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed6c0d7be0 0x7fed6c0c7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.068+0000 7fed78d52640 1 -- 172.21.15.46:0/111113773 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed64009400 con 0x7fed6c0eb020 2023-12-02T00:28:59.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.068+0000 7fcefcd33640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef8142960 con 0x7fcef806d170 2023-12-02T00:28:59.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.068+0000 7fcefcd33640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef8142c40 con 0x7fcef814b2e0 2023-12-02T00:28:59.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.068+0000 7fcefcd33640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcef8142f20 con 0x7fcef806abe0 2023-12-02T00:28:59.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.068+0000 7f3a4af68640 1 -- 172.21.15.46:0/706164443 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a44158780 con 0x7f3a440b40b0 2023-12-02T00:28:59.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.068+0000 7f45c2c9b640 1 -- 172.21.15.46:0/1056919858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45bc1f74d0 con 0x7f45bc072400 2023-12-02T00:28:59.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.068+0000 7f7229ffb640 1 -- 172.21.15.46:0/55362548 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72300035f0 con 0x7f722c034390 2023-12-02T00:28:59.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.068+0000 7f45c2c9b640 1 -- 172.21.15.46:0/1056919858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45bc1f7a90 con 0x7f45bc072400 2023-12-02T00:28:59.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.069+0000 7f45b97fa640 1 -- 172.21.15.46:0/1056919858 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b0007330 con 0x7f45bc072400 2023-12-02T00:28:59.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.068+0000 7fcef5d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcef806d170 0x7fcef8144290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:59.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.068+0000 7fcef5d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcef806d170 0x7fcef8144290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38102/0 (socket says 172.21.15.46:38102) 2023-12-02T00:28:59.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.068+0000 7fcef5d9b640 1 -- 172.21.15.46:0/803281621 learned_addr learned my addr 172.21.15.46:0/803281621 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:59.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.068+0000 7f723a28c640 1 -- 172.21.15.46:0/55362548 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f722c1806f0 con 0x7f722c034390 2023-12-02T00:28:59.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.068+0000 7f7229ffb640 1 -- 172.21.15.46:0/55362548 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7230003750 con 0x7f722c034390 2023-12-02T00:28:59.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.068+0000 7f7229ffb640 1 -- 172.21.15.46:0/55362548 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72300035f0 con 0x7f722c034390 2023-12-02T00:28:59.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.068+0000 7f3a417fa640 1 -- 172.21.15.46:0/706164443 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a340118f0 con 0x7f3a440b40b0 2023-12-02T00:28:59.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.069+0000 7f3a417fa640 1 -- 172.21.15.46:0/706164443 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a34011ce0 con 0x7f3a440b40b0 2023-12-02T00:28:59.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.069+0000 7f723a28c640 1 -- 172.21.15.46:0/55362548 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f722c180cb0 con 0x7f722c034390 2023-12-02T00:28:59.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.070+0000 7f45c2c9b640 1 -- 172.21.15.46:0/1056919858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f459c005250 con 0x7f45bc072400 2023-12-02T00:28:59.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.070+0000 7f72327fc640 1 -- 172.21.15.46:0/1772902290 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723803e070 con 0x7f723c0b4c60 2023-12-02T00:28:59.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.070+0000 7f72327fc640 1 -- 172.21.15.46:0/1772902290 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7238003820 con 0x7f723c0b4c60 2023-12-02T00:28:59.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.068+0000 7f4d3a2da640 1 -- 172.21.15.46:0/2327041625 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2c0e3890 con 0x7f4d2c0e3d20 2023-12-02T00:28:59.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.068+0000 7f4d3a2da640 1 -- 172.21.15.46:0/2327041625 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2c04ebf0 con 0x7f4d2c0e1270 2023-12-02T00:28:59.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.068+0000 7f4d3a2da640 1 -- 172.21.15.46:0/2327041625 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2c0e2c80 con 0x7f4d2c0e4cf0 2023-12-02T00:28:59.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.068+0000 7f4d38877640 1 --2- 172.21.15.46:0/2327041625 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d2c0e3d20 0x7f4d2c04e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:59.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.068+0000 7f4d33fff640 1 -- 172.21.15.46:0/2327041625 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d2c0e4cf0 msgr2=0x7f4d2c0d5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:59.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.068+0000 7f4d33fff640 1 --2- 172.21.15.46:0/2327041625 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d2c0e4cf0 0x7f4d2c0d5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.068+0000 7f4d33fff640 1 -- 172.21.15.46:0/2327041625 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d2c0e3d20 msgr2=0x7f4d2c04e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:59.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.068+0000 7f4d33fff640 1 --2- 172.21.15.46:0/2327041625 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d2c0e3d20 0x7f4d2c04e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.070+0000 7f3a2affd640 1 -- 172.21.15.46:0/706164443 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a1c005250 con 0x7f3a440b40b0 2023-12-02T00:28:59.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.068+0000 7f4d33fff640 1 -- 172.21.15.46:0/2327041625 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d340687b0 con 0x7f4d2c0e1270 2023-12-02T00:28:59.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.069+0000 7ff270e48640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff26400a420 0x7ff26403bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:59.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.069+0000 7ff270e48640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff26400ab00 0x7ff264008b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:59.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.069+0000 7ff270e48640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2640e12d0 0x7ff264009090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:59.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.069+0000 7ff270e48640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26404e2d0 con 0x7ff26400ab00 2023-12-02T00:28:59.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.069+0000 7ff270e48640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26404e5b0 con 0x7ff2640e12d0 2023-12-02T00:28:59.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.069+0000 7ff270e48640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff26404e890 con 0x7ff26400a420 2023-12-02T00:28:59.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.069+0000 7ff26a59c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff26400a420 0x7ff26403bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:59.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.070+0000 7ff26a59c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff26400a420 0x7ff26403bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.46:35120/0 (socket says 172.21.15.46:35120) 2023-12-02T00:28:59.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.070+0000 7ff26a59c640 1 -- 172.21.15.46:0/2740033130 learned_addr learned my addr 172.21.15.46:0/2740033130 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:28:59.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.070+0000 7ff26ad9d640 1 --2- 172.21.15.46:0/2740033130 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2640e12d0 0x7ff264009090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:59.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.070+0000 7ff26a59c640 1 -- 172.21.15.46:0/2740033130 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2640e12d0 msgr2=0x7ff264009090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:59.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.070+0000 7ff26a59c640 1 --2- 172.21.15.46:0/2740033130 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2640e12d0 0x7ff264009090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.070+0000 7ff26a59c640 1 -- 172.21.15.46:0/2740033130 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff26400ab00 msgr2=0x7ff264008b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:59.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.070+0000 7ff26a59c640 1 --2- 172.21.15.46:0/2740033130 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff26400ab00 0x7ff264008b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.070+0000 7ff26a59c640 1 -- 172.21.15.46:0/2740033130 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff25c009400 con 0x7ff26400a420 2023-12-02T00:28:59.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.069+0000 7fcef6d9d640 1 --2- 172.21.15.46:0/803281621 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcef814b2e0 0x7fcef81447d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:59.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.070+0000 7fcef659c640 1 -- 172.21.15.46:0/803281621 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcef814b2e0 msgr2=0x7fcef81447d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:59.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.070+0000 7fcef659c640 1 --2- 172.21.15.46:0/803281621 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcef814b2e0 0x7fcef81447d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.070+0000 7fcef659c640 1 -- 172.21.15.46:0/803281621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcef806d170 msgr2=0x7fcef8144290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:59.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.070+0000 7fcef659c640 1 --2- 172.21.15.46:0/803281621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcef806d170 0x7fcef8144290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.070+0000 7fcef659c640 1 -- 172.21.15.46:0/803281621 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef0009a70 con 0x7fcef806abe0 2023-12-02T00:28:59.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.070+0000 7f4d33fff640 1 --2- 172.21.15.46:0/2327041625 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d2c0e1270 0x7f4d2c0e26a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4d2800d5f0 tx=0x7f4d2800dab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:59.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.070+0000 7fcef659c640 1 --2- 172.21.15.46:0/803281621 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcef806abe0 0x7fcef81f6980 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fceec000f60 tx=0x7fceec00c760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:59.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.070+0000 7f721f7fe640 1 -- 172.21.15.46:0/55362548 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f722c1811d0 con 0x7f722c034390 2023-12-02T00:28:59.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.070+0000 7f72327fc640 1 -- 172.21.15.46:0/1772902290 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7238003bf0 con 0x7f723c0b4c60 2023-12-02T00:28:59.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.070+0000 7f724415a640 1 -- 172.21.15.46:0/1772902290 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f723c0cc820 con 0x7f723c0b4c60 2023-12-02T00:28:59.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.071+0000 7f724415a640 1 -- 172.21.15.46:0/1772902290 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f723c0c3a50 con 0x7f723c0b4c60 2023-12-02T00:28:59.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.070+0000 7fed78d52640 1 --2- 172.21.15.46:0/111113773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed6c0eb020 0x7fed6c04f8d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fed64033000 tx=0x7fed640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:59.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.071+0000 7ff26a59c640 1 --2- 172.21.15.46:0/2740033130 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff26400a420 0x7ff26403bd30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff25c0098c0 tx=0x7ff25c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:59.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.071+0000 7f4d30ff9640 1 -- 172.21.15.46:0/2327041625 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d280125f0 con 0x7f4d2c0e1270 2023-12-02T00:28:59.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.071+0000 7f4d30ff9640 1 -- 172.21.15.46:0/2327041625 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d28012750 con 0x7f4d2c0e1270 2023-12-02T00:28:59.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.071+0000 7f4d30ff9640 1 -- 172.21.15.46:0/2327041625 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d28012b00 con 0x7f4d2c0e1270 2023-12-02T00:28:59.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.071+0000 7fcee6ffd640 1 -- 172.21.15.46:0/803281621 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceec005ce0 con 0x7fcef806abe0 2023-12-02T00:28:59.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.071+0000 7fcee6ffd640 1 -- 172.21.15.46:0/803281621 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceec005e40 con 0x7fcef806abe0 2023-12-02T00:28:59.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.071+0000 7fcee6ffd640 1 -- 172.21.15.46:0/803281621 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceec0072d0 con 0x7fcef806abe0 2023-12-02T00:28:59.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.071+0000 7f3a417fa640 1 -- 172.21.15.46:0/706164443 <== mon.2 v2:172.21.15.162:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3a3401c460 con 0x7f3a440b40b0 2023-12-02T00:28:59.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.071+0000 7fcefcd33640 1 -- 172.21.15.46:0/803281621 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcef8143200 con 0x7fcef806abe0 2023-12-02T00:28:59.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.071+0000 7fcefcd33640 1 -- 172.21.15.46:0/803281621 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcef81f6ec0 con 0x7fcef806abe0 2023-12-02T00:28:59.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.071+0000 7ff25affd640 1 -- 172.21.15.46:0/2740033130 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25c03c440 con 0x7ff26400a420 2023-12-02T00:28:59.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.071+0000 7ff25affd640 1 -- 172.21.15.46:0/2740033130 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff25c03add0 con 0x7ff26400a420 2023-12-02T00:28:59.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.071+0000 7f3a417fa640 1 --2- 172.21.15.46:0/706164443 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3a3006af10 0x7f3a3006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:59.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.072+0000 7f3a48d04640 1 --2- 172.21.15.46:0/706164443 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3a3006af10 0x7f3a3006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:59.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.072+0000 7f3a417fa640 1 -- 172.21.15.46:0/706164443 <== mon.2 v2:172.21.15.162:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3a34089310 con 0x7f3a440b40b0 2023-12-02T00:28:59.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.072+0000 7f724415a640 1 -- 172.21.15.46:0/1772902290 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f723c0c38b0 con 0x7f723c0b4c60 2023-12-02T00:28:59.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.072+0000 7fed69ffb640 1 -- 172.21.15.46:0/111113773 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed64032820 con 0x7fed6c0eb020 2023-12-02T00:28:59.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.072+0000 7fed7b7b7640 1 -- 172.21.15.46:0/111113773 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed6c0c8500 con 0x7fed6c0eb020 2023-12-02T00:28:59.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.072+0000 7f4d3a2da640 1 -- 172.21.15.46:0/2327041625 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d2c0e2f60 con 0x7f4d2c0e1270 2023-12-02T00:28:59.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.072+0000 7f4d3a2da640 1 -- 172.21.15.46:0/2327041625 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d2c0e3520 con 0x7f4d2c0e1270 2023-12-02T00:28:59.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.072+0000 7f45b97fa640 1 -- 172.21.15.46:0/1056919858 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f45b0007550 con 0x7f45bc072400 2023-12-02T00:28:59.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.072+0000 7fed7b7b7640 1 -- 172.21.15.46:0/111113773 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed6c0514e0 con 0x7fed6c0eb020 2023-12-02T00:28:59.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.072+0000 7ff270e48640 1 -- 172.21.15.46:0/2740033130 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff26404eb70 con 0x7ff26400a420 2023-12-02T00:28:59.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.072+0000 7ff270e48640 1 -- 172.21.15.46:0/2740033130 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff26404f370 con 0x7ff26400a420 2023-12-02T00:28:59.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.072+0000 7ff25affd640 1 -- 172.21.15.46:0/2740033130 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25c014270 con 0x7ff26400a420 2023-12-02T00:28:59.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.073+0000 7f3a48d04640 1 --2- 172.21.15.46:0/706164443 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3a3006af10 0x7f3a3006d3d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3a38039000 tx=0x7f3a38034000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:59.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.073+0000 7f45b97fa640 1 --2- 172.21.15.46:0/1056919858 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f45a006ae40 0x7f45a006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:59.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.073+0000 7fcefcd33640 1 -- 172.21.15.46:0/803281621 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcef8143720 con 0x7fcef806abe0 2023-12-02T00:28:59.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.073+0000 7f45bbfff640 1 --2- 172.21.15.46:0/1056919858 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f45a006ae40 0x7f45a006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:59.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.073+0000 7f45b97fa640 1 -- 172.21.15.46:0/1056919858 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f45b0088bb0 con 0x7f45bc072400 2023-12-02T00:28:59.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.074+0000 7fed69ffb640 1 -- 172.21.15.46:0/111113773 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed64032980 con 0x7fed6c0eb020 2023-12-02T00:28:59.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.074+0000 7f72327fc640 1 -- 172.21.15.46:0/1772902290 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7238003d50 con 0x7f723c0b4c60 2023-12-02T00:28:59.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.074+0000 7f4d3a2da640 1 -- 172.21.15.46:0/2327041625 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d2c0e36e0 con 0x7f4d2c0e1270 2023-12-02T00:28:59.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.074+0000 7f45bbfff640 1 --2- 172.21.15.46:0/1056919858 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f45a006ae40 0x7f45a006d300 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f45bc142f30 tx=0x7f45a8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:59.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.074+0000 7fed69ffb640 1 -- 172.21.15.46:0/111113773 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed64032cf0 con 0x7fed6c0eb020 2023-12-02T00:28:59.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.074+0000 7f7229ffb640 1 -- 172.21.15.46:0/55362548 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f72300038c0 con 0x7f722c034390 2023-12-02T00:28:59.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.074+0000 7f4d30ff9640 1 -- 172.21.15.46:0/2327041625 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4d28012c60 con 0x7f4d2c0e1270 2023-12-02T00:28:59.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.074+0000 7f72327fc640 1 --2- 172.21.15.46:0/1772902290 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f722006ae90 0x7f722006d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:59.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.075+0000 7f72416f5640 1 --2- 172.21.15.46:0/1772902290 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f722006ae90 0x7f722006d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:59.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.075+0000 7ff270e48640 1 -- 172.21.15.46:0/2740033130 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff26404f090 con 0x7ff26400a420 2023-12-02T00:28:59.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.075+0000 7fed7b7b7640 1 -- 172.21.15.46:0/111113773 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed6c0057a0 con 0x7fed6c0eb020 2023-12-02T00:28:59.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.075+0000 7f7229ffb640 1 --2- 172.21.15.46:0/55362548 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f721806aea0 0x7f721806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:59.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.075+0000 7f7229ffb640 1 -- 172.21.15.46:0/55362548 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7230088b00 con 0x7f722c034390 2023-12-02T00:28:59.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.075+0000 7f723928a640 1 --2- 172.21.15.46:0/55362548 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f721806aea0 0x7f721806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:59.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.075+0000 7f4d30ff9640 1 --2- 172.21.15.46:0/2327041625 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f4d1806af10 0x7f4d1806d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:59.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.075+0000 7f4d30ff9640 1 -- 172.21.15.46:0/2327041625 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4d28089a40 con 0x7f4d2c0e1270 2023-12-02T00:28:59.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.075+0000 7f4d337fe640 1 --2- 172.21.15.46:0/2327041625 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f4d1806af10 0x7f4d1806d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:59.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.075+0000 7f72416f5640 1 --2- 172.21.15.46:0/1772902290 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f722006ae90 0x7f722006d350 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f722c007fd0 tx=0x7f722c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:59.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.076+0000 7f723928a640 1 --2- 172.21.15.46:0/55362548 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f721806aea0 0x7f721806d360 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7224036000 tx=0x7f722403b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:59.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.075+0000 7f4d337fe640 1 --2- 172.21.15.46:0/2327041625 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f4d1806af10 0x7f4d1806d3d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4d20006fd0 tx=0x7f4d20006bc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:59.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.077+0000 7f3a417fa640 1 -- 172.21.15.46:0/706164443 <== mon.2 v2:172.21.15.162:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3a340544e0 con 0x7f3a440b40b0 2023-12-02T00:28:59.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.078+0000 7fcee6ffd640 1 -- 172.21.15.46:0/803281621 <== mon.2 v2:172.21.15.162:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fceec007430 con 0x7fcef806abe0 2023-12-02T00:28:59.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.078+0000 7f72327fc640 1 -- 172.21.15.46:0/1772902290 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f723807b140 con 0x7f723c0b4c60 2023-12-02T00:28:59.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.078+0000 7f7229ffb640 1 -- 172.21.15.46:0/55362548 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7230053bc0 con 0x7f722c034390 2023-12-02T00:28:59.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.079+0000 7fcee6ffd640 1 --2- 172.21.15.46:0/803281621 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fced806ae40 0x7fced806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:59.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.079+0000 7fcee6ffd640 1 -- 172.21.15.46:0/803281621 <== mon.2 v2:172.21.15.162:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fceec088ff0 con 0x7fcef806abe0 2023-12-02T00:28:59.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.079+0000 7fcef5d9b640 1 --2- 172.21.15.46:0/803281621 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fced806ae40 0x7fced806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:59.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.079+0000 7ff25affd640 1 -- 172.21.15.46:0/2740033130 <== mon.2 v2:172.21.15.162:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff25c03c750 con 0x7ff26400a420 2023-12-02T00:28:59.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.079+0000 7ff25affd640 1 --2- 172.21.15.46:0/2740033130 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff24c06aec0 0x7ff24c06d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:59.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.079+0000 7ff25affd640 1 -- 172.21.15.46:0/2740033130 <== mon.2 v2:172.21.15.162:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff25c0ac3a0 con 0x7ff26400a420 2023-12-02T00:28:59.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.079+0000 7fcef5d9b640 1 --2- 172.21.15.46:0/803281621 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fced806ae40 0x7fced806d300 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcef8156730 tx=0x7fcee8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:59.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.080+0000 7fed69ffb640 1 -- 172.21.15.46:0/111113773 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fed64046450 con 0x7fed6c0eb020 2023-12-02T00:28:59.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.080+0000 7ff269d9b640 1 --2- 172.21.15.46:0/2740033130 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff24c06aec0 0x7ff24c06d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:59.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.080+0000 7fed69ffb640 1 --2- 172.21.15.46:0/111113773 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fed5806ae40 0x7fed5806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:28:59.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.081+0000 7ff269d9b640 1 --2- 172.21.15.46:0/2740033130 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff24c06aec0 0x7ff24c06d380 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff26c063e60 tx=0x7ff26c06a220 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:59.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.081+0000 7f4d30ff9640 1 -- 172.21.15.46:0/2327041625 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4d28054ad0 con 0x7f4d2c0e1270 2023-12-02T00:28:59.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.081+0000 7fed79553640 1 --2- 172.21.15.46:0/111113773 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fed5806ae40 0x7fed5806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:28:59.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.081+0000 7fed79553640 1 --2- 172.21.15.46:0/111113773 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fed5806ae40 0x7fed5806d300 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fed70009b60 tx=0x7fed70009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:28:59.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.082+0000 7fcee6ffd640 1 -- 172.21.15.46:0/803281621 <== mon.2 v2:172.21.15.162:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fceec054190 con 0x7fcef806abe0 2023-12-02T00:28:59.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.088+0000 7ff25affd640 1 -- 172.21.15.46:0/2740033130 <== mon.2 v2:172.21.15.162:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff25c0774e0 con 0x7ff26400a420 2023-12-02T00:28:59.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.088+0000 7f45b97fa640 1 -- 172.21.15.46:0/1056919858 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f45b0053dc0 con 0x7f45bc072400 2023-12-02T00:28:59.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.097+0000 7fed69ffb640 1 -- 172.21.15.46:0/111113773 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fed6407ab40 con 0x7fed6c0eb020 2023-12-02T00:28:59.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.097+0000 7f72327fc640 1 -- 172.21.15.46:0/1772902290 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7238076d80 con 0x7f723c0b4c60 2023-12-02T00:28:59.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.106+0000 7fed69ffb640 1 -- 172.21.15.46:0/111113773 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fed64076e80 con 0x7fed6c0eb020 2023-12-02T00:28:59.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.475+0000 7f3a2affd640 1 -- 172.21.15.46:0/706164443 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f3a1c005500 con 0x7f3a440b40b0 2023-12-02T00:28:59.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.476+0000 7f3a417fa640 1 -- 172.21.15.46:0/706164443 <== mon.2 v2:172.21.15.162:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3a34057b00 con 0x7f3a440b40b0 2023-12-02T00:28:59.477 INFO:teuthology.orchestra.run.smithi046.stdout:55834574854 2023-12-02T00:28:59.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.479+0000 7f3a4af68640 1 -- 172.21.15.46:0/706164443 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3a3006af10 msgr2=0x7f3a3006d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:59.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.479+0000 7f3a4af68640 1 --2- 172.21.15.46:0/706164443 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3a3006af10 0x7f3a3006d3d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3a38039000 tx=0x7f3a38034000 comp rx=0 tx=0).stop 2023-12-02T00:28:59.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.479+0000 7f3a4af68640 1 -- 172.21.15.46:0/706164443 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a440b40b0 msgr2=0x7f3a44157da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:59.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.479+0000 7f3a4af68640 1 --2- 172.21.15.46:0/706164443 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a440b40b0 0x7f3a44157da0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3a3400c430 tx=0x7f3a3400c8f0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.480+0000 7f3a4af68640 1 -- 172.21.15.46:0/706164443 shutdown_connections 2023-12-02T00:28:59.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.480+0000 7f3a4af68640 1 --2- 172.21.15.46:0/706164443 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3a3006af10 0x7f3a3006d3d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.480+0000 7f3a4af68640 1 --2- 172.21.15.46:0/706164443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a44159210 0x7f3a44150610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.480+0000 7f3a4af68640 1 --2- 172.21.15.46:0/706164443 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a440b40b0 0x7f3a44157da0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.480+0000 7f3a4af68640 1 --2- 172.21.15.46:0/706164443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a440b34a0 0x7f3a441f71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.480+0000 7f3a4af68640 1 -- 172.21.15.46:0/706164443 >> 172.21.15.46:0/706164443 conn(0x7f3a440b0d60 msgr2=0x7f3a440af820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:59.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.480+0000 7f3a4af68640 1 -- 172.21.15.46:0/706164443 shutdown_connections 2023-12-02T00:28:59.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.480+0000 7f3a4af68640 1 -- 172.21.15.46:0/706164443 wait complete. 2023-12-02T00:28:59.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.487+0000 7f4d3a2da640 1 -- 172.21.15.46:0/2327041625 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4d2c0057a0 con 0x7f4d2c0e1270 2023-12-02T00:28:59.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.488+0000 7f4d30ff9640 1 -- 172.21.15.46:0/2327041625 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4d280580f0 con 0x7f4d2c0e1270 2023-12-02T00:28:59.489 INFO:teuthology.orchestra.run.smithi046.stdout:55834574853 2023-12-02T00:28:59.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.489+0000 7f721f7fe640 1 -- 172.21.15.46:0/55362548 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f722c1811d0 con 0x7f722c034390 2023-12-02T00:28:59.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.490+0000 7f7229ffb640 1 -- 172.21.15.46:0/55362548 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f722c1811d0 con 0x7f722c034390 2023-12-02T00:28:59.491 INFO:teuthology.orchestra.run.smithi046.stdout:55834574853 2023-12-02T00:28:59.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.490+0000 7fcefcd33640 1 -- 172.21.15.46:0/803281621 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fcef8143720 con 0x7fcef806abe0 2023-12-02T00:28:59.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.491+0000 7fcee6ffd640 1 -- 172.21.15.46:0/803281621 <== mon.2 v2:172.21.15.162:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fceec021020 con 0x7fcef806abe0 2023-12-02T00:28:59.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.491+0000 7f4d1e7fc640 1 -- 172.21.15.46:0/2327041625 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f4d1806af10 msgr2=0x7f4d1806d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:59.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.491+0000 7f4d1e7fc640 1 --2- 172.21.15.46:0/2327041625 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f4d1806af10 0x7f4d1806d3d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4d20006fd0 tx=0x7f4d20006bc0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.491+0000 7f4d1e7fc640 1 -- 172.21.15.46:0/2327041625 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d2c0e1270 msgr2=0x7f4d2c0e26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:59.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.491+0000 7f4d1e7fc640 1 --2- 172.21.15.46:0/2327041625 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d2c0e1270 0x7f4d2c0e26a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4d2800d5f0 tx=0x7f4d2800dab0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.493 INFO:teuthology.orchestra.run.smithi046.stdout:55834574855 2023-12-02T00:28:59.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.491+0000 7f4d1e7fc640 1 -- 172.21.15.46:0/2327041625 shutdown_connections 2023-12-02T00:28:59.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.491+0000 7f4d1e7fc640 1 --2- 172.21.15.46:0/2327041625 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f4d1806af10 0x7f4d1806d3d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.491+0000 7f4d1e7fc640 1 --2- 172.21.15.46:0/2327041625 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d2c0e3d20 0x7f4d2c04e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.491+0000 7f4d1e7fc640 1 --2- 172.21.15.46:0/2327041625 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d2c0e4cf0 0x7f4d2c0d5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.491+0000 7f4d1e7fc640 1 --2- 172.21.15.46:0/2327041625 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d2c0e1270 0x7f4d2c0e26a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.491+0000 7f4d1e7fc640 1 -- 172.21.15.46:0/2327041625 >> 172.21.15.46:0/2327041625 conn(0x7f4d2c020000 msgr2=0x7f4d2c053140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:59.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.491+0000 7f4d1e7fc640 1 -- 172.21.15.46:0/2327041625 shutdown_connections 2023-12-02T00:28:59.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.491+0000 7f4d1e7fc640 1 -- 172.21.15.46:0/2327041625 wait complete. 2023-12-02T00:28:59.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.493+0000 7f723a28c640 1 -- 172.21.15.46:0/55362548 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f721806aea0 msgr2=0x7f721806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:59.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.494+0000 7f723a28c640 1 --2- 172.21.15.46:0/55362548 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f721806aea0 0x7f721806d360 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7224036000 tx=0x7f722403b040 comp rx=0 tx=0).stop 2023-12-02T00:28:59.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.494+0000 7f723a28c640 1 -- 172.21.15.46:0/55362548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f722c034390 msgr2=0x7f722c0e3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:59.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.494+0000 7f723a28c640 1 --2- 172.21.15.46:0/55362548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f722c034390 0x7f722c0e3550 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f723000c3c0 tx=0x7f723000c880 comp rx=0 tx=0).stop 2023-12-02T00:28:59.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.494+0000 7f723a28c640 1 -- 172.21.15.46:0/55362548 shutdown_connections 2023-12-02T00:28:59.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.494+0000 7f723a28c640 1 --2- 172.21.15.46:0/55362548 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f721806aea0 0x7f721806d360 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.494+0000 7f723a28c640 1 --2- 172.21.15.46:0/55362548 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f722c0e3a90 0x7f722c053c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.494+0000 7f723a28c640 1 --2- 172.21.15.46:0/55362548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f722c034390 0x7f722c0e3550 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.494+0000 7f723a28c640 1 --2- 172.21.15.46:0/55362548 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f722c0062a0 0x7f722c0daa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.494+0000 7f723a28c640 1 -- 172.21.15.46:0/55362548 >> 172.21.15.46:0/55362548 conn(0x7f722c054ed0 msgr2=0x7f722c034930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:59.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.494+0000 7fcefcd33640 1 -- 172.21.15.46:0/803281621 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fced806ae40 msgr2=0x7fced806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:59.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.494+0000 7fcefcd33640 1 --2- 172.21.15.46:0/803281621 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fced806ae40 0x7fced806d300 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcef8156730 tx=0x7fcee8009040 comp rx=0 tx=0).stop 2023-12-02T00:28:59.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.494+0000 7fcefcd33640 1 -- 172.21.15.46:0/803281621 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcef806abe0 msgr2=0x7fcef81f6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:59.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.494+0000 7fcefcd33640 1 --2- 172.21.15.46:0/803281621 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcef806abe0 0x7fcef81f6980 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fceec000f60 tx=0x7fceec00c760 comp rx=0 tx=0).stop 2023-12-02T00:28:59.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.495+0000 7f723a28c640 1 -- 172.21.15.46:0/55362548 shutdown_connections 2023-12-02T00:28:59.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.495+0000 7fcefcd33640 1 -- 172.21.15.46:0/803281621 shutdown_connections 2023-12-02T00:28:59.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.495+0000 7fcefcd33640 1 --2- 172.21.15.46:0/803281621 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fced806ae40 0x7fced806d300 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.495+0000 7fcefcd33640 1 --2- 172.21.15.46:0/803281621 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcef814b2e0 0x7fcef81447d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.495+0000 7fcefcd33640 1 --2- 172.21.15.46:0/803281621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcef806d170 0x7fcef8144290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.495+0000 7fcefcd33640 1 --2- 172.21.15.46:0/803281621 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcef806abe0 0x7fcef81f6980 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.495+0000 7fcefcd33640 1 -- 172.21.15.46:0/803281621 >> 172.21.15.46:0/803281621 conn(0x7fcef80cfd80 msgr2=0x7fcef80b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:59.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.495+0000 7f723a28c640 1 -- 172.21.15.46:0/55362548 wait complete. 2023-12-02T00:28:59.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.495+0000 7fcefcd33640 1 -- 172.21.15.46:0/803281621 shutdown_connections 2023-12-02T00:28:59.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.495+0000 7fcefcd33640 1 -- 172.21.15.46:0/803281621 wait complete. 2023-12-02T00:28:59.500 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.7 2023-12-02T00:28:59.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.500+0000 7f45c2c9b640 1 -- 172.21.15.46:0/1056919858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f459c005500 con 0x7f45bc072400 2023-12-02T00:28:59.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.501+0000 7f45b97fa640 1 -- 172.21.15.46:0/1056919858 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f45b0017020 con 0x7f45bc072400 2023-12-02T00:28:59.502 INFO:teuthology.orchestra.run.smithi046.stdout:55834574854 2023-12-02T00:28:59.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.501+0000 7ff270e48640 1 -- 172.21.15.46:0/2740033130 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7ff26404f090 con 0x7ff26400a420 2023-12-02T00:28:59.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.501+0000 7ff25affd640 1 -- 172.21.15.46:0/2740033130 <== mon.2 v2:172.21.15.162:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff25c041020 con 0x7ff26400a420 2023-12-02T00:28:59.503 INFO:teuthology.orchestra.run.smithi046.stdout:55834574854 2023-12-02T00:28:59.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.504+0000 7f45c2c9b640 1 -- 172.21.15.46:0/1056919858 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f45a006ae40 msgr2=0x7f45a006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:59.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.504+0000 7f45c2c9b640 1 --2- 172.21.15.46:0/1056919858 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f45a006ae40 0x7f45a006d300 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f45bc142f30 tx=0x7f45a8009040 comp rx=0 tx=0).stop 2023-12-02T00:28:59.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.504+0000 7f45c2c9b640 1 -- 172.21.15.46:0/1056919858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45bc072400 msgr2=0x7f45bc141910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:59.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.504+0000 7f45c2c9b640 1 --2- 172.21.15.46:0/1056919858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45bc072400 0x7f45bc141910 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f45b000c500 tx=0x7f45b000c9c0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.505+0000 7f45c2c9b640 1 -- 172.21.15.46:0/1056919858 shutdown_connections 2023-12-02T00:28:59.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.505+0000 7f45c2c9b640 1 --2- 172.21.15.46:0/1056919858 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f45a006ae40 0x7f45a006d300 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.505+0000 7f45c2c9b640 1 --2- 172.21.15.46:0/1056919858 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45bc15df30 0x7f45bc0ccf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.505+0000 7f45c2c9b640 1 --2- 172.21.15.46:0/1056919858 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f45bc146ba0 0x7f45bc1f6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.505+0000 7f45c2c9b640 1 --2- 172.21.15.46:0/1056919858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45bc072400 0x7f45bc141910 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.505+0000 7f45c2c9b640 1 -- 172.21.15.46:0/1056919858 >> 172.21.15.46:0/1056919858 conn(0x7f45bc0b0d60 msgr2=0x7f45bc0b3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:59.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.505+0000 7f45c2c9b640 1 -- 172.21.15.46:0/1056919858 shutdown_connections 2023-12-02T00:28:59.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.505+0000 7f45c2c9b640 1 -- 172.21.15.46:0/1056919858 wait complete. 2023-12-02T00:28:59.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.506+0000 7ff270e48640 1 -- 172.21.15.46:0/2740033130 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff24c06aec0 msgr2=0x7ff24c06d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:59.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.506+0000 7ff270e48640 1 --2- 172.21.15.46:0/2740033130 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff24c06aec0 0x7ff24c06d380 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff26c063e60 tx=0x7ff26c06a220 comp rx=0 tx=0).stop 2023-12-02T00:28:59.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.506+0000 7ff270e48640 1 -- 172.21.15.46:0/2740033130 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff26400a420 msgr2=0x7ff26403bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:59.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.506+0000 7ff270e48640 1 --2- 172.21.15.46:0/2740033130 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff26400a420 0x7ff26403bd30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff25c0098c0 tx=0x7ff25c0046c0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.506+0000 7ff270e48640 1 -- 172.21.15.46:0/2740033130 shutdown_connections 2023-12-02T00:28:59.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.506+0000 7ff270e48640 1 --2- 172.21.15.46:0/2740033130 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff24c06aec0 0x7ff24c06d380 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.506+0000 7ff270e48640 1 --2- 172.21.15.46:0/2740033130 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2640e12d0 0x7ff264009090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.506+0000 7ff270e48640 1 --2- 172.21.15.46:0/2740033130 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff26400ab00 0x7ff264008b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.506+0000 7ff270e48640 1 --2- 172.21.15.46:0/2740033130 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff26400a420 0x7ff26403bd30 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.506+0000 7ff270e48640 1 -- 172.21.15.46:0/2740033130 >> 172.21.15.46:0/2740033130 conn(0x7ff264056c90 msgr2=0x7ff2640e0140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:59.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.506+0000 7ff270e48640 1 -- 172.21.15.46:0/2740033130 shutdown_connections 2023-12-02T00:28:59.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.506+0000 7ff270e48640 1 -- 172.21.15.46:0/2740033130 wait complete. 2023-12-02T00:28:59.512 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.3 2023-12-02T00:28:59.513 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.1 2023-12-02T00:28:59.514 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.5 2023-12-02T00:28:59.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.513+0000 7f724415a640 1 -- 172.21.15.46:0/1772902290 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f723c0c38b0 con 0x7f723c0b4c60 2023-12-02T00:28:59.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.513+0000 7f72327fc640 1 -- 172.21.15.46:0/1772902290 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f723c0c38b0 con 0x7f723c0b4c60 2023-12-02T00:28:59.515 INFO:teuthology.orchestra.run.smithi046.stdout:55834574855 2023-12-02T00:28:59.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.516+0000 7f724415a640 1 -- 172.21.15.46:0/1772902290 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f722006ae90 msgr2=0x7f722006d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:59.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.516+0000 7f724415a640 1 --2- 172.21.15.46:0/1772902290 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f722006ae90 0x7f722006d350 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f722c007fd0 tx=0x7f722c009040 comp rx=0 tx=0).stop 2023-12-02T00:28:59.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.517+0000 7f724415a640 1 -- 172.21.15.46:0/1772902290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f723c0b4c60 msgr2=0x7f723c0c91f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:59.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.517+0000 7f724415a640 1 --2- 172.21.15.46:0/1772902290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f723c0b4c60 0x7f723c0c91f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f72380090b0 tx=0x7f72380046c0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.517+0000 7f724415a640 1 -- 172.21.15.46:0/1772902290 shutdown_connections 2023-12-02T00:28:59.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.517+0000 7f724415a640 1 --2- 172.21.15.46:0/1772902290 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f722006ae90 0x7f722006d350 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.517+0000 7f724415a640 1 --2- 172.21.15.46:0/1772902290 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f723c15d3f0 0x7f723c0cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.517+0000 7f724415a640 1 --2- 172.21.15.46:0/1772902290 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f723c0d2da0 0x7f723c0b16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.517+0000 7f724415a640 1 --2- 172.21.15.46:0/1772902290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f723c0b4c60 0x7f723c0c91f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.517+0000 7f724415a640 1 -- 172.21.15.46:0/1772902290 >> 172.21.15.46:0/1772902290 conn(0x7f723c072230 msgr2=0x7f723c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:59.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.517+0000 7f724415a640 1 -- 172.21.15.46:0/1772902290 shutdown_connections 2023-12-02T00:28:59.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.517+0000 7f724415a640 1 -- 172.21.15.46:0/1772902290 wait complete. 2023-12-02T00:28:59.521 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.2 2023-12-02T00:28:59.523 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.4 2023-12-02T00:28:59.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.522+0000 7fed7b7b7640 1 -- 172.21.15.46:0/111113773 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fed6c04c7a0 con 0x7fed6c0eb020 2023-12-02T00:28:59.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.523+0000 7fed69ffb640 1 -- 172.21.15.46:0/111113773 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fed6407a4a0 con 0x7fed6c0eb020 2023-12-02T00:28:59.524 INFO:teuthology.orchestra.run.smithi046.stdout:55834574854 2023-12-02T00:28:59.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.526+0000 7fed7b7b7640 1 -- 172.21.15.46:0/111113773 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fed5806ae40 msgr2=0x7fed5806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:59.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.526+0000 7fed7b7b7640 1 --2- 172.21.15.46:0/111113773 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fed5806ae40 0x7fed5806d300 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fed70009b60 tx=0x7fed70009190 comp rx=0 tx=0).stop 2023-12-02T00:28:59.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.526+0000 7fed7b7b7640 1 -- 172.21.15.46:0/111113773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed6c0eb020 msgr2=0x7fed6c04f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:28:59.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.526+0000 7fed7b7b7640 1 --2- 172.21.15.46:0/111113773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed6c0eb020 0x7fed6c04f8d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fed64033000 tx=0x7fed640046c0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.526+0000 7fed7b7b7640 1 -- 172.21.15.46:0/111113773 shutdown_connections 2023-12-02T00:28:59.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.526+0000 7fed7b7b7640 1 --2- 172.21.15.46:0/111113773 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fed5806ae40 0x7fed5806d300 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.526+0000 7fed7b7b7640 1 --2- 172.21.15.46:0/111113773 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed6c0d7be0 0x7fed6c0c7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.526+0000 7fed7b7b7640 1 --2- 172.21.15.46:0/111113773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed6c0eb020 0x7fed6c04f8d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.527+0000 7fed7b7b7640 1 --2- 172.21.15.46:0/111113773 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed6c0e7a30 0x7fed6c0dab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:28:59.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.527+0000 7fed7b7b7640 1 -- 172.21.15.46:0/111113773 >> 172.21.15.46:0/111113773 conn(0x7fed6c0559a0 msgr2=0x7fed6c0e8930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:28:59.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.527+0000 7fed7b7b7640 1 -- 172.21.15.46:0/111113773 shutdown_connections 2023-12-02T00:28:59.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:28:59.527+0000 7fed7b7b7640 1 -- 172.21.15.46:0/111113773 wait complete. 2023-12-02T00:28:59.533 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.0 2023-12-02T00:28:59.539 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.6 2023-12-02T00:29:00.501 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-12-02T00:29:00.514 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-12-02T00:29:00.515 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-12-02T00:29:00.516 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-12-02T00:29:00.522 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-12-02T00:29:00.524 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-12-02T00:29:00.533 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-12-02T00:29:00.540 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-12-02T00:29:00.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.678+0000 7f150ed5a640 1 Processor -- start 2023-12-02T00:29:00.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.678+0000 7f150ed5a640 1 -- start start 2023-12-02T00:29:00.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.681+0000 7f150ed5a640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1508071d10 0x7f1508176e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.681+0000 7f150ed5a640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15080cee20 con 0x7f15080b3e90 2023-12-02T00:29:00.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.681+0000 7f150ed5a640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15080b4460 con 0x7f150808a850 2023-12-02T00:29:00.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.681+0000 7f150ed5a640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15080711e0 con 0x7f1508071d10 2023-12-02T00:29:00.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.681+0000 7f150caf6640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f15080b3e90 0x7f15080cc770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45016/0 (socket says 172.21.15.46:45016) 2023-12-02T00:29:00.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.681+0000 7f150caf6640 1 -- 172.21.15.46:0/1925500193 learned_addr learned my addr 172.21.15.46:0/1925500193 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:00.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.681+0000 7f15077fe640 1 -- 172.21.15.46:0/1925500193 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3649600049 0 0) 0x7f15080cee20 con 0x7f15080b3e90 2023-12-02T00:29:00.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.681+0000 7f15077fe640 1 -- 172.21.15.46:0/1925500193 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14f0003520 con 0x7f15080b3e90 2023-12-02T00:29:00.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.682+0000 7f150d2f7640 1 --2- 172.21.15.46:0/1925500193 >> v2:172.21.15.162:3300/0 conn(0x7f1508071d10 0x7f1508176e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:00.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.682+0000 7f15077fe640 1 -- 172.21.15.46:0/1925500193 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2691459098 0 0) 0x7f14f0003520 con 0x7f15080b3e90 2023-12-02T00:29:00.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.682+0000 7f15077fe640 1 -- 172.21.15.46:0/1925500193 >> v2:172.21.15.162:3300/0 conn(0x7f1508071d10 msgr2=0x7f1508176e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:00.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.682+0000 7f15077fe640 1 --2- 172.21.15.46:0/1925500193 >> v2:172.21.15.162:3300/0 conn(0x7f1508071d10 0x7f1508176e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:00.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.682+0000 7f15077fe640 1 -- 172.21.15.46:0/1925500193 >> v1:172.21.15.162:6789/0 conn(0x7f150808a850 legacy=0x7f1508071600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:00.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.682+0000 7f15077fe640 1 -- 172.21.15.46:0/1925500193 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1508170c70 con 0x7f15080b3e90 2023-12-02T00:29:00.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.682+0000 7f15077fe640 1 -- 172.21.15.46:0/1925500193 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7f14fc001670 con 0x7f15080b3e90 2023-12-02T00:29:00.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.683+0000 7f15077fe640 1 -- 172.21.15.46:0/1925500193 >> v1:172.21.15.46:6789/0 conn(0x7f15080b3e90 legacy=0x7f15080cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:00.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.683+0000 7f15077fe640 1 --2- 172.21.15.46:0/1925500193 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14f00039a0 0x7f14f0003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.683+0000 7f15077fe640 1 --2- 172.21.15.46:0/1925500193 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14f00047c0 0x7f14f0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.683+0000 7f15077fe640 1 --2- 172.21.15.46:0/1925500193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14f0005250 0x7f14f000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.683+0000 7f15077fe640 1 -- 172.21.15.46:0/1925500193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15080711e0 con 0x7f14f0005250 2023-12-02T00:29:00.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.683+0000 7f15077fe640 1 -- 172.21.15.46:0/1925500193 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f000c1a0 con 0x7f14f00047c0 2023-12-02T00:29:00.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.683+0000 7f15077fe640 1 -- 172.21.15.46:0/1925500193 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14f000c360 con 0x7f14f00039a0 2023-12-02T00:29:00.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.683+0000 7f150caf6640 1 --2- 172.21.15.46:0/1925500193 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14f00039a0 0x7f14f0003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:00.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.684+0000 7f150d2f7640 1 --2- 172.21.15.46:0/1925500193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14f0005250 0x7f14f000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:00.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.684+0000 7f150d2f7640 1 -- 172.21.15.46:0/1925500193 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14f00039a0 msgr2=0x7f14f0003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:00.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.684+0000 7f150d2f7640 1 --2- 172.21.15.46:0/1925500193 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14f00039a0 0x7f14f0003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:00.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.684+0000 7f150d2f7640 1 -- 172.21.15.46:0/1925500193 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14f00047c0 msgr2=0x7f14f0004c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-02T00:29:00.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.684+0000 7f150d2f7640 1 --2- 172.21.15.46:0/1925500193 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14f00047c0 0x7f14f0004c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:00.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.684+0000 7f150d2f7640 1 -- 172.21.15.46:0/1925500193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f0003520 con 0x7f14f0005250 2023-12-02T00:29:00.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.686+0000 7f150d2f7640 1 --2- 172.21.15.46:0/1925500193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14f0005250 0x7f14f000b980 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f150806eb30 tx=0x7f14f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b20cc211ff083aa server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:00.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.686+0000 7f150d2f7640 1 -- 172.21.15.46:0/1925500193 reap_dead start 2023-12-02T00:29:00.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.686+0000 7f15077fe640 1 -- 172.21.15.46:0/1925500193 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1500003b90 con 0x7f14f0005250 2023-12-02T00:29:00.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.687+0000 7f15077fe640 1 -- 172.21.15.46:0/1925500193 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f150002f050 con 0x7f14f0005250 2023-12-02T00:29:00.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.688+0000 7f150ed5a640 1 -- 172.21.15.46:0/1925500193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14f0005250 msgr2=0x7f14f000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:00.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.688+0000 7f150ed5a640 1 --2- 172.21.15.46:0/1925500193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14f0005250 0x7f14f000b980 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f150806eb30 tx=0x7f14f00057f0 comp rx=0 tx=0).stop 2023-12-02T00:29:00.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.688+0000 7f150ed5a640 1 -- 172.21.15.46:0/1925500193 shutdown_connections 2023-12-02T00:29:00.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.688+0000 7f150ed5a640 1 --2- 172.21.15.46:0/1925500193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14f0005250 0x7f14f000b980 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:00.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.688+0000 7f150ed5a640 1 -- 172.21.15.46:0/1925500193 >> 172.21.15.46:0/1925500193 conn(0x7f15080b0d60 msgr2=0x7f15080b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:00.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.689+0000 7f150ed5a640 1 -- 172.21.15.46:0/1925500193 shutdown_connections 2023-12-02T00:29:00.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.689+0000 7f150ed5a640 1 -- 172.21.15.46:0/1925500193 wait complete. 2023-12-02T00:29:00.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.691+0000 7f150ed5a640 1 Processor -- start 2023-12-02T00:29:00.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.691+0000 7f150ed5a640 1 -- start start 2023-12-02T00:29:00.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.692+0000 7f150ed5a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14f0005250 0x7f15080aded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.692+0000 7f150ed5a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f15081629c0 0x7f15081675f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.692+0000 7f150ed5a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1508204590 0x7f150816fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.692+0000 7f150ed5a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150816f730 con 0x7f1508204590 2023-12-02T00:29:00.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.692+0000 7f150ed5a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150816edd0 con 0x7f14f0005250 2023-12-02T00:29:00.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.692+0000 7f150ed5a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f150816f070 con 0x7f15081629c0 2023-12-02T00:29:00.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.693+0000 7f1507fff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f15081629c0 0x7f15081675f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:00.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.693+0000 7f1507fff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f15081629c0 0x7f15081675f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.46:35128/0 (socket says 172.21.15.46:35128) 2023-12-02T00:29:00.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.693+0000 7f1507fff640 1 -- 172.21.15.46:0/825243255 learned_addr learned my addr 172.21.15.46:0/825243255 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:00.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.693+0000 7f150d2f7640 1 --2- 172.21.15.46:0/825243255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1508204590 0x7f150816fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:00.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.693+0000 7f1507fff640 1 -- 172.21.15.46:0/825243255 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14f0005250 msgr2=0x7f15080aded0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-02T00:29:00.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.693+0000 7f1507fff640 1 --2- 172.21.15.46:0/825243255 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14f0005250 0x7f15080aded0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:00.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.693+0000 7f1507fff640 1 -- 172.21.15.46:0/825243255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1508204590 msgr2=0x7f150816fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:00.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.693+0000 7f1507fff640 1 --2- 172.21.15.46:0/825243255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1508204590 0x7f150816fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:00.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.693+0000 7f1507fff640 1 -- 172.21.15.46:0/825243255 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f150816f350 con 0x7f15081629c0 2023-12-02T00:29:00.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.693+0000 7f150d2f7640 1 --2- 172.21.15.46:0/825243255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1508204590 0x7f150816fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:29:00.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.694+0000 7f1507fff640 1 --2- 172.21.15.46:0/825243255 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f15081629c0 0x7f15081675f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f14f800c390 tx=0x7f14f800c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:00.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.695+0000 7f15057fa640 1 -- 172.21.15.46:0/825243255 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f8011850 con 0x7f15081629c0 2023-12-02T00:29:00.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.695+0000 7f150ed5a640 1 -- 172.21.15.46:0/825243255 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15080ae560 con 0x7f15081629c0 2023-12-02T00:29:00.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.695+0000 7f150ed5a640 1 -- 172.21.15.46:0/825243255 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1508167b30 con 0x7f15081629c0 2023-12-02T00:29:00.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.697+0000 7f150ed5a640 1 -- 172.21.15.46:0/825243255 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f150816f870 con 0x7f15081629c0 2023-12-02T00:29:00.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.698+0000 7f15057fa640 1 -- 172.21.15.46:0/825243255 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14f80119b0 con 0x7f15081629c0 2023-12-02T00:29:00.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.698+0000 7f15057fa640 1 -- 172.21.15.46:0/825243255 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f8011d60 con 0x7f15081629c0 2023-12-02T00:29:00.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.698+0000 7f15057fa640 1 -- 172.21.15.46:0/825243255 <== mon.2 v2:172.21.15.162:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f14f801c460 con 0x7f15081629c0 2023-12-02T00:29:00.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.699+0000 7f15057fa640 1 --2- 172.21.15.46:0/825243255 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f14f406af90 0x7f14f406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.699+0000 7f15057fa640 1 -- 172.21.15.46:0/825243255 <== mon.2 v2:172.21.15.162:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f14f8089310 con 0x7f15081629c0 2023-12-02T00:29:00.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.700+0000 7f150caf6640 1 --2- 172.21.15.46:0/825243255 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f14f406af90 0x7f14f406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:00.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.701+0000 7f150caf6640 1 --2- 172.21.15.46:0/825243255 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f14f406af90 0x7f14f406d450 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f14fc001050 tx=0x7f14fc000fe0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:00.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.703+0000 7f15057fa640 1 -- 172.21.15.46:0/825243255 <== mon.2 v2:172.21.15.162:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f150816f870 con 0x7f15081629c0 2023-12-02T00:29:00.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.731+0000 7fd152a67640 1 Processor -- start 2023-12-02T00:29:00.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.732+0000 7fd152a67640 1 -- start start 2023-12-02T00:29:00.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.733+0000 7fd152a67640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd14c0b1780 0x7fd14c0b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.733+0000 7fd152a67640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd14c0b3a80 0x7fd14c0b3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.733+0000 7fd152a67640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd14c0cc9f0 con 0x7fd14c0ce1d0 2023-12-02T00:29:00.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.733+0000 7fd152a67640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd14c0b30c0 con 0x7fd14c0b3a80 2023-12-02T00:29:00.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.733+0000 7fd152a67640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c0b4450 con 0x7fd14c0b1780 2023-12-02T00:29:00.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.734+0000 7fd151264640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd14c0ce1d0 0x7fd14c158ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45028/0 (socket says 172.21.15.46:45028) 2023-12-02T00:29:00.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.734+0000 7fd151264640 1 -- 172.21.15.46:0/1579804322 learned_addr learned my addr 172.21.15.46:0/1579804322 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:00.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.734+0000 7fd152266640 1 --2- 172.21.15.46:0/1579804322 >> v2:172.21.15.162:3300/0 conn(0x7fd14c0b3a80 0x7fd14c0b3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:00.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.734+0000 7fd151a65640 1 --2- 172.21.15.46:0/1579804322 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd14c0b1780 0x7fd14c0b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:00.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.734+0000 7fd150a63640 1 -- 172.21.15.46:0/1579804322 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1831467020 0 0) 0x7fd14c0cc9f0 con 0x7fd14c0ce1d0 2023-12-02T00:29:00.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.734+0000 7fd150a63640 1 -- 172.21.15.46:0/1579804322 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd138003520 con 0x7fd14c0ce1d0 2023-12-02T00:29:00.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.734+0000 7fd151a65640 1 -- 172.21.15.46:0/1579804322 >> v2:172.21.15.162:3300/0 conn(0x7fd14c0b3a80 msgr2=0x7fd14c0b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:00.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.735+0000 7fd151a65640 1 --2- 172.21.15.46:0/1579804322 >> v2:172.21.15.162:3300/0 conn(0x7fd14c0b3a80 0x7fd14c0b3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:00.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.735+0000 7fd150a63640 1 -- 172.21.15.46:0/1579804322 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3108268053 0 0) 0x7fd138003520 con 0x7fd14c0ce1d0 2023-12-02T00:29:00.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.735+0000 7fd151a65640 1 -- 172.21.15.46:0/1579804322 >> v1:172.21.15.46:6789/0 conn(0x7fd14c0ce1d0 legacy=0x7fd14c158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:00.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.735+0000 7fd151a65640 1 -- 172.21.15.46:0/1579804322 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd14c0b2540 con 0x7fd14c0b1780 2023-12-02T00:29:00.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.735+0000 7f8dcf5b6640 1 Processor -- start 2023-12-02T00:29:00.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.735+0000 7f8dcf5b6640 1 -- start start 2023-12-02T00:29:00.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.734+0000 7f6d7e954640 1 Processor -- start 2023-12-02T00:29:00.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.735+0000 7f6d7e954640 1 -- start start 2023-12-02T00:29:00.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.736+0000 7fd151a65640 1 --2- 172.21.15.46:0/1579804322 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd14c0b1780 0x7fd14c0b1f60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd1480097a0 tx=0x7fd14802b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8de9735f7b8a25c server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:00.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.736+0000 7fd150a63640 1 -- 172.21.15.46:0/1579804322 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14802f050 con 0x7fd14c0b1780 2023-12-02T00:29:00.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.736+0000 7fd150a63640 1 -- 172.21.15.46:0/1579804322 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd148031500 con 0x7fd14c0b1780 2023-12-02T00:29:00.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.736+0000 7fd150a63640 1 -- 172.21.15.46:0/1579804322 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1440019f0 con 0x7fd14c0b1780 2023-12-02T00:29:00.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.737+0000 7f8dcf5b6640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8dc80b4c40 0x7f8dc8165290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.737+0000 7f8dcf5b6640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8dc81623b0 0x7f8dc806d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.737+0000 7f8dcf5b6640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dc8162db0 con 0x7f8dc81657d0 2023-12-02T00:29:00.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.737+0000 7f8dcf5b6640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dc81650d0 con 0x7f8dc81623b0 2023-12-02T00:29:00.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.737+0000 7f8dcf5b6640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dc8165c50 con 0x7f8dc80b4c40 2023-12-02T00:29:00.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.737+0000 7f8dcdb53640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8dc81623b0 0x7f8dc806d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:00.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.737+0000 7f6d7e954640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6d78150680 0x7f6d780ce570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.737+0000 7f6d7e954640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d78071f70 0x7f6d7806cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.737+0000 7f6d7e954640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d780aea40 con 0x7f6d7806d330 2023-12-02T00:29:00.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.737+0000 7f6d7e954640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d7806d7c0 con 0x7f6d78150680 2023-12-02T00:29:00.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.737+0000 7f6d7e954640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d780ccd90 con 0x7f6d78071f70 2023-12-02T00:29:00.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.737+0000 7f6d7e153640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6d7806d330 0x7f6d7815d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45054/0 (socket says 172.21.15.46:45054) 2023-12-02T00:29:00.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.737+0000 7f6d7e153640 1 -- 172.21.15.46:0/1132256707 learned_addr learned my addr 172.21.15.46:0/1132256707 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:00.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.737+0000 7f8dcdb53640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8dc81623b0 0x7f8dc806d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38160/0 (socket says 172.21.15.46:38160) 2023-12-02T00:29:00.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.737+0000 7f8dcdb53640 1 -- 172.21.15.46:0/2031708634 learned_addr learned my addr 172.21.15.46:0/2031708634 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:00.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.737+0000 7f8dcd352640 1 --2- 172.21.15.46:0/2031708634 >> v2:172.21.15.162:3300/0 conn(0x7f8dc80b4c40 0x7f8dc8165290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:00.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.738+0000 7f6d7c950640 1 -- 172.21.15.46:0/1132256707 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1608183004 0 0) 0x7f6d780aea40 con 0x7f6d7806d330 2023-12-02T00:29:00.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.738+0000 7f6d7c950640 1 -- 172.21.15.46:0/1132256707 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d64003440 con 0x7f6d7806d330 2023-12-02T00:29:00.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.734+0000 7f7f0a661640 1 Processor -- start 2023-12-02T00:29:00.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.735+0000 7f7f0a661640 1 -- start start 2023-12-02T00:29:00.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.736+0000 7f7f03fff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7f04161590 0x7f7f0406a4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45042/0 (socket says 172.21.15.46:45042) 2023-12-02T00:29:00.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.736+0000 7f7f03fff640 1 -- 172.21.15.46:0/2471674682 learned_addr learned my addr 172.21.15.46:0/2471674682 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:00.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.737+0000 7f7f0a661640 1 --2- 172.21.15.46:0/2471674682 >> v2:172.21.15.46:3300/0 conn(0x7f7f0406efa0 0x7f7f0406abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.737+0000 7f7f037fe640 1 --2- 172.21.15.46:0/2471674682 >> v2:172.21.15.46:3300/0 conn(0x7f7f0406efa0 0x7f7f0406abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:00.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.737+0000 7f7f0a661640 1 --2- 172.21.15.46:0/2471674682 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7f040b2f60 0x7f7f040b33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.737+0000 7f7f0a661640 1 -- 172.21.15.46:0/2471674682 --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f0406f420 con 0x7f7f04161590 2023-12-02T00:29:00.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.737+0000 7f7f0a661640 1 -- 172.21.15.46:0/2471674682 --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f040b2b10 con 0x7f7f0406efa0 2023-12-02T00:29:00.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.737+0000 7f7f0a661640 1 -- 172.21.15.46:0/2471674682 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0406b580 con 0x7f7f040b2f60 2023-12-02T00:29:00.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.737+0000 7f7f08bfe640 1 --2- 172.21.15.46:0/2471674682 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7f040b2f60 0x7f7f040b33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:00.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.737+0000 7f7f02ffd640 1 -- 172.21.15.46:0/2471674682 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 254882839 0 0) 0x7f7f0406f420 con 0x7f7f04161590 2023-12-02T00:29:00.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.737+0000 7f7f02ffd640 1 -- 172.21.15.46:0/2471674682 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7eec003520 con 0x7f7f04161590 2023-12-02T00:29:00.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.738+0000 7fd152a67640 1 -- 172.21.15.46:0/1579804322 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd14c0b1780 msgr2=0x7fd14c0b1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:00.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.738+0000 7fd152a67640 1 --2- 172.21.15.46:0/1579804322 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd14c0b1780 0x7fd14c0b1f60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd1480097a0 tx=0x7fd14802b820 comp rx=0 tx=0).stop 2023-12-02T00:29:00.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.738+0000 7f6d7d952640 1 --2- 172.21.15.46:0/1132256707 >> v2:172.21.15.162:3300/0 conn(0x7f6d78150680 0x7f6d780ce570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:00.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.738+0000 7f8dbffff640 1 -- 172.21.15.46:0/2031708634 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 398171277 0 0) 0x7f8dc8162db0 con 0x7f8dc81657d0 2023-12-02T00:29:00.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.738+0000 7f8dbffff640 1 -- 172.21.15.46:0/2031708634 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8db4003520 con 0x7f8dc81657d0 2023-12-02T00:29:00.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.738+0000 7f6d7d952640 1 -- 172.21.15.46:0/1132256707 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d78071f70 msgr2=0x7f6d7806cdf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-02T00:29:00.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.738+0000 7f6d7d952640 1 --2- 172.21.15.46:0/1132256707 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d78071f70 0x7f6d7806cdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:00.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.738+0000 7f6d7d952640 1 -- 172.21.15.46:0/1132256707 >> v1:172.21.15.46:6789/0 conn(0x7f6d7806d330 legacy=0x7f6d7815d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:00.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.738+0000 7f8dbffff640 1 -- 172.21.15.46:0/2031708634 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3637754822 0 0) 0x7f8db4003520 con 0x7f8dc81657d0 2023-12-02T00:29:00.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.738+0000 7f8dbffff640 1 -- 172.21.15.46:0/2031708634 >> v2:172.21.15.162:3300/0 conn(0x7f8dc80b4c40 msgr2=0x7f8dc8165290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:00.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.738+0000 7f8dbffff640 1 --2- 172.21.15.46:0/2031708634 >> v2:172.21.15.162:3300/0 conn(0x7f8dc80b4c40 0x7f8dc8165290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:00.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.738+0000 7f8dbffff640 1 -- 172.21.15.46:0/2031708634 >> v2:172.21.15.46:3300/0 conn(0x7f8dc81623b0 msgr2=0x7f8dc806d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:00.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.739+0000 7fd152a67640 1 -- 172.21.15.46:0/1579804322 shutdown_connections 2023-12-02T00:29:00.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.739+0000 7fd152a67640 1 --2- 172.21.15.46:0/1579804322 >> v2:172.21.15.162:3300/0 conn(0x7fd14c0b3a80 0x7fd14c0b3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:00.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.739+0000 7fd152a67640 1 --2- 172.21.15.46:0/1579804322 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd14c0b1780 0x7fd14c0b1f60 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:00.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.739+0000 7fd152a67640 1 -- 172.21.15.46:0/1579804322 >> 172.21.15.46:0/1579804322 conn(0x7fd14c0b0c60 msgr2=0x7fd14c0b1070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:00.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.739+0000 7fd152a67640 1 -- 172.21.15.46:0/1579804322 shutdown_connections 2023-12-02T00:29:00.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.739+0000 7fd152a67640 1 -- 172.21.15.46:0/1579804322 wait complete. 2023-12-02T00:29:00.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.738+0000 7f6d7d952640 1 -- 172.21.15.46:0/1132256707 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d780cec70 con 0x7f6d78150680 2023-12-02T00:29:00.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.739+0000 7f6d7d952640 1 --2- 172.21.15.46:0/1132256707 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d78150680 0x7f6d780ce570 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6d740097a0 tx=0x7f6d7402b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3071c5a4393fb06 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:00.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.738+0000 7f7f08bfe640 1 -- 172.21.15.46:0/2471674682 >> v2:172.21.15.46:3300/0 conn(0x7f7f0406efa0 msgr2=0x7f7f0406abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:00.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.738+0000 7f7f08bfe640 1 --2- 172.21.15.46:0/2471674682 >> v2:172.21.15.46:3300/0 conn(0x7f7f0406efa0 0x7f7f0406abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:00.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.738+0000 7f7f02ffd640 1 -- 172.21.15.46:0/2471674682 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2175187687 0 0) 0x7f7eec003520 con 0x7f7f04161590 2023-12-02T00:29:00.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.738+0000 7f7f08bfe640 1 -- 172.21.15.46:0/2471674682 >> v1:172.21.15.46:6789/0 conn(0x7f7f04161590 legacy=0x7f7f0406a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:00.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.738+0000 7f7f08bfe640 1 -- 172.21.15.46:0/2471674682 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f0406b860 con 0x7f7f040b2f60 2023-12-02T00:29:00.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.738+0000 7f7f037fe640 1 --2- 172.21.15.46:0/2471674682 >> v2:172.21.15.46:3300/0 conn(0x7f7f0406efa0 0x7f7f0406abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:00.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.738+0000 7f8dbffff640 1 --2- 172.21.15.46:0/2031708634 >> v2:172.21.15.46:3300/0 conn(0x7f8dc81623b0 0x7f8dc806d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:00.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.738+0000 7f8dbffff640 1 -- 172.21.15.46:0/2031708634 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc8163020 con 0x7f8dc81657d0 2023-12-02T00:29:00.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.738+0000 7f8dbffff640 1 -- 172.21.15.46:0/2031708634 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7f8db8001670 con 0x7f8dc81657d0 2023-12-02T00:29:00.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.738+0000 7f8dbffff640 1 -- 172.21.15.46:0/2031708634 >> v1:172.21.15.162:6789/0 conn(0x7f8dc81657d0 legacy=0x7f8dc8161ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:00.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.738+0000 7f8dbffff640 1 --2- 172.21.15.46:0/2031708634 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8db4003970 0x7f8db4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.738+0000 7f8dbffff640 1 --2- 172.21.15.46:0/2031708634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8db40047c0 0x7f8db4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.738+0000 7f8dbffff640 1 --2- 172.21.15.46:0/2031708634 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8db4005180 0x7f8db400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.738+0000 7f8dbffff640 1 -- 172.21.15.46:0/2031708634 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc8165c50 con 0x7f8db40047c0 2023-12-02T00:29:00.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.738+0000 7f8dbffff640 1 -- 172.21.15.46:0/2031708634 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db400c020 con 0x7f8db4005180 2023-12-02T00:29:00.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.738+0000 7f8dbffff640 1 -- 172.21.15.46:0/2031708634 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8db400c1e0 con 0x7f8db4003970 2023-12-02T00:29:00.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.739+0000 7f8dccb51640 1 --2- 172.21.15.46:0/2031708634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8db40047c0 0x7f8db4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:00.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.739+0000 7f8dcd352640 1 --2- 172.21.15.46:0/2031708634 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8db4003970 0x7f8db4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:00.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.739+0000 7f8dccb51640 1 -- 172.21.15.46:0/2031708634 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8db4003970 msgr2=0x7f8db4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:00.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.739+0000 7f8dccb51640 1 --2- 172.21.15.46:0/2031708634 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8db4003970 0x7f8db4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:00.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.739+0000 7f8dccb51640 1 -- 172.21.15.46:0/2031708634 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8db4005180 msgr2=0x7f8db400b980 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-02T00:29:00.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.739+0000 7f8dccb51640 1 --2- 172.21.15.46:0/2031708634 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8db4005180 0x7f8db400b980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:00.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.739+0000 7f8dccb51640 1 -- 172.21.15.46:0/2031708634 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc8163020 con 0x7f8db40047c0 2023-12-02T00:29:00.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.739+0000 7f8dcd352640 1 --2- 172.21.15.46:0/2031708634 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8db4003970 0x7f8db4003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:29:00.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.740+0000 7f8dcdb53640 1 --2- 172.21.15.46:0/2031708634 >> v2:172.21.15.46:3300/0 conn(0x7f8dc81623b0 0x7f8dc806d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:29:00.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.740+0000 7f8dcdb53640 1 -- 172.21.15.46:0/2031708634 reap_dead start 2023-12-02T00:29:00.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.740+0000 7f6d7c950640 1 -- 172.21.15.46:0/1132256707 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7402e070 con 0x7f6d78150680 2023-12-02T00:29:00.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.740+0000 7f6d7c950640 1 -- 172.21.15.46:0/1132256707 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d740315a0 con 0x7f6d78150680 2023-12-02T00:29:00.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.740+0000 7f7f08bfe640 1 --2- 172.21.15.46:0/2471674682 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7f040b2f60 0x7f7f040b33f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7efc009d90 tx=0x7f7efc02bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=434a101559a4b0b0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:00.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.740+0000 7f7f02ffd640 1 -- 172.21.15.46:0/2471674682 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef4001670 con 0x7f7f040b2f60 2023-12-02T00:29:00.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.740+0000 7f7f02ffd640 1 -- 172.21.15.46:0/2471674682 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7efc02c810 con 0x7f7f040b2f60 2023-12-02T00:29:00.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.740+0000 7f7f02ffd640 1 -- 172.21.15.46:0/2471674682 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7efc02cba0 con 0x7f7f040b2f60 2023-12-02T00:29:00.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.740+0000 7f6d7c950640 1 -- 172.21.15.46:0/1132256707 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d74031910 con 0x7f6d78150680 2023-12-02T00:29:00.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.740+0000 7f40c7fff640 1 Processor -- start 2023-12-02T00:29:00.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.740+0000 7f04e277c640 1 Processor -- start 2023-12-02T00:29:00.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.741+0000 7fd152a67640 1 Processor -- start 2023-12-02T00:29:00.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.741+0000 7f6d7e954640 1 -- 172.21.15.46:0/1132256707 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d78150680 msgr2=0x7f6d780ce570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:00.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.741+0000 7f6d7e954640 1 --2- 172.21.15.46:0/1132256707 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d78150680 0x7f6d780ce570 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6d740097a0 tx=0x7f6d7402b840 comp rx=0 tx=0).stop 2023-12-02T00:29:00.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.740+0000 7f04e277c640 1 -- start start 2023-12-02T00:29:00.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.741+0000 7fd152a67640 1 -- start start 2023-12-02T00:29:00.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.741+0000 7f6d7e954640 1 -- 172.21.15.46:0/1132256707 shutdown_connections 2023-12-02T00:29:00.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.741+0000 7f6d7e954640 1 --2- 172.21.15.46:0/1132256707 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d78071f70 0x7f6d7806cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:00.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.741+0000 7f6d7e954640 1 --2- 172.21.15.46:0/1132256707 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d78150680 0x7f6d780ce570 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:00.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.741+0000 7f6d7e954640 1 -- 172.21.15.46:0/1132256707 >> 172.21.15.46:0/1132256707 conn(0x7f6d7806a4d0 msgr2=0x7f6d7806a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:00.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.741+0000 7f7f0a661640 1 -- 172.21.15.46:0/2471674682 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7f040b2f60 msgr2=0x7f7f040b33f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:00.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.741+0000 7f7f0a661640 1 --2- 172.21.15.46:0/2471674682 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7f040b2f60 0x7f7f040b33f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7efc009d90 tx=0x7f7efc02bce0 comp rx=0 tx=0).stop 2023-12-02T00:29:00.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.741+0000 7f8dccb51640 1 --2- 172.21.15.46:0/2031708634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8db40047c0 0x7f8db4004c40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8db801d270 tx=0x7f8db40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f954ec69878a24e2 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:00.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.741+0000 7f7f0a661640 1 -- 172.21.15.46:0/2471674682 shutdown_connections 2023-12-02T00:29:00.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.741+0000 7f7f0a661640 1 --2- 172.21.15.46:0/2471674682 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7f040b2f60 0x7f7f040b33f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:00.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.741+0000 7f7f0a661640 1 --2- 172.21.15.46:0/2471674682 >> v2:172.21.15.46:3300/0 conn(0x7f7f0406efa0 0x7f7f0406abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:00.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.741+0000 7f7f0a661640 1 -- 172.21.15.46:0/2471674682 >> 172.21.15.46:0/2471674682 conn(0x7f7f040cf410 msgr2=0x7f7f040cf820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:00.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.740+0000 7f40c7fff640 1 -- start start 2023-12-02T00:29:00.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.742+0000 7f8dbffff640 1 -- 172.21.15.46:0/2031708634 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db803db50 con 0x7f8db40047c0 2023-12-02T00:29:00.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.742+0000 7f8dbffff640 1 -- 172.21.15.46:0/2031708634 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8db803dcb0 con 0x7f8db40047c0 2023-12-02T00:29:00.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.742+0000 7f8dbffff640 1 -- 172.21.15.46:0/2031708634 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db8007270 con 0x7f8db40047c0 2023-12-02T00:29:00.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.742+0000 7f7f0a661640 1 -- 172.21.15.46:0/2471674682 shutdown_connections 2023-12-02T00:29:00.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.742+0000 7f04e277c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f04dc15f2c0 0x7f04dc0b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.742+0000 7f7f0a661640 1 -- 172.21.15.46:0/2471674682 wait complete. 2023-12-02T00:29:00.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.742+0000 7f40c7fff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40c00eacd0 0x7f40c004fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.742+0000 7f40c7fff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f40c004ffa0 0x7f40c0037ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.742+0000 7f40c7fff640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40c0054300 con 0x7f40c00380e0 2023-12-02T00:29:00.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.742+0000 7f40c7fff640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40c00536f0 con 0x7f40c004ffa0 2023-12-02T00:29:00.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.742+0000 7f40c7fff640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c00539d0 con 0x7f40c00eacd0 2023-12-02T00:29:00.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.742+0000 7f04dbfff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f04dc15f2c0 0x7f04dc0b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:00.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.742+0000 7f04dbfff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f04dc15f2c0 0x7f04dc0b4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38178/0 (socket says 172.21.15.46:38178) 2023-12-02T00:29:00.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.742+0000 7f04dbfff640 1 -- 172.21.15.46:0/2059184422 learned_addr learned my addr 172.21.15.46:0/2059184422 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:00.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.742+0000 7fd152a67640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd14c0b1780 0x7fd14c157210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.742+0000 7f04e277c640 1 --2- 172.21.15.46:0/2059184422 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f04dc0b2310 0x7f04dc0b27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.742+0000 7fd152a67640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd14c0b3a80 0x7fd14c152a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.742+0000 7f04e277c640 1 -- 172.21.15.46:0/2059184422 --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04dc161da0 con 0x7f04dc162af0 2023-12-02T00:29:00.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.743+0000 7f8dcf5b6640 1 -- 172.21.15.46:0/2031708634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8db40047c0 msgr2=0x7f8db4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:00.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.743+0000 7f8dcf5b6640 1 --2- 172.21.15.46:0/2031708634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8db40047c0 0x7f8db4004c40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8db801d270 tx=0x7f8db40057f0 comp rx=0 tx=0).stop 2023-12-02T00:29:00.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.743+0000 7fd152a67640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd14c0ce1d0 0x7fd14c152f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.742+0000 7f40c6ffd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40c00eacd0 0x7f40c004fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:00.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.743+0000 7f04e0d19640 1 --2- 172.21.15.46:0/2059184422 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f04dc0b2310 0x7f04dc0b27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:00.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.743+0000 7fd151264640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd14c0b3a80 0x7fd14c152a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:00.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.743+0000 7f8dcf5b6640 1 -- 172.21.15.46:0/2031708634 shutdown_connections 2023-12-02T00:29:00.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.743+0000 7f8dcf5b6640 1 --2- 172.21.15.46:0/2031708634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8db40047c0 0x7f8db4004c40 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:00.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.743+0000 7f8dcf5b6640 1 -- 172.21.15.46:0/2031708634 >> 172.21.15.46:0/2031708634 conn(0x7f8dc8070900 msgr2=0x7f8dc8070d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:00.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.743+0000 7f04e277c640 1 -- 172.21.15.46:0/2059184422 --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04dc0b2d00 con 0x7f04dc15f2c0 2023-12-02T00:29:00.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.743+0000 7f04e277c640 1 -- 172.21.15.46:0/2059184422 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04dc0721e0 con 0x7f04dc0b2310 2023-12-02T00:29:00.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.743+0000 7f6d7e954640 1 -- 172.21.15.46:0/1132256707 shutdown_connections 2023-12-02T00:29:00.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.742+0000 7f40c67fc640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f40c004ffa0 0x7f40c0037ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:00.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.742+0000 7f40c67fc640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f40c004ffa0 0x7f40c0037ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.46:44538/0 (socket says 172.21.15.46:44538) 2023-12-02T00:29:00.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.742+0000 7f40c67fc640 1 -- 172.21.15.46:0/2564172216 learned_addr learned my addr 172.21.15.46:0/2564172216 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:00.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.742+0000 7f40c6ffd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40c00eacd0 0x7f40c004fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.46:35190/0 (socket says 172.21.15.46:35190) 2023-12-02T00:29:00.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.743+0000 7f40c67fc640 1 -- 172.21.15.46:0/2564172216 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40c00eacd0 msgr2=0x7f40c004fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:00.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.743+0000 7f40c67fc640 1 --2- 172.21.15.46:0/2564172216 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40c00eacd0 0x7f40c004fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:00.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.743+0000 7f40c67fc640 1 -- 172.21.15.46:0/2564172216 >> v1:172.21.15.162:6789/0 conn(0x7f40c00380e0 legacy=0x7f40c0038570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:00.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.743+0000 7f40c67fc640 1 -- 172.21.15.46:0/2564172216 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c0053cb0 con 0x7f40c004ffa0 2023-12-02T00:29:00.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.743+0000 7fd151264640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd14c0b3a80 0x7fd14c152a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.46:35208/0 (socket says 172.21.15.46:35208) 2023-12-02T00:29:00.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.743+0000 7fd151264640 1 -- 172.21.15.46:0/946350441 learned_addr learned my addr 172.21.15.46:0/946350441 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:00.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.743+0000 7f04daffd640 1 -- 172.21.15.46:0/2059184422 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4192262722 0 0) 0x7f04dc161da0 con 0x7f04dc162af0 2023-12-02T00:29:00.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.743+0000 7f04daffd640 1 -- 172.21.15.46:0/2059184422 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04c4003520 con 0x7f04dc162af0 2023-12-02T00:29:00.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.743+0000 7fd152266640 1 --2- 172.21.15.46:0/946350441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd14c0ce1d0 0x7fd14c152f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:00.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.743+0000 7f8dcf5b6640 1 -- 172.21.15.46:0/2031708634 shutdown_connections 2023-12-02T00:29:00.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.743+0000 7fd152a67640 1 -- 172.21.15.46:0/946350441 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c14ebf0 con 0x7fd14c0ce1d0 2023-12-02T00:29:00.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.743+0000 7fd152a67640 1 -- 172.21.15.46:0/946350441 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c14df70 con 0x7fd14c0b1780 2023-12-02T00:29:00.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.743+0000 7fd152a67640 1 -- 172.21.15.46:0/946350441 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c14e250 con 0x7fd14c0b3a80 2023-12-02T00:29:00.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.743+0000 7f8dcf5b6640 1 -- 172.21.15.46:0/2031708634 wait complete. 2023-12-02T00:29:00.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.744+0000 7f8b6b64f640 1 Processor -- start 2023-12-02T00:29:00.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.744+0000 7fd151a65640 1 --2- 172.21.15.46:0/946350441 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd14c0b1780 0x7fd14c157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:00.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.743+0000 7f04dbfff640 1 -- 172.21.15.46:0/2059184422 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f04dc0b2310 msgr2=0x7f04dc0b27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:00.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.743+0000 7f04dbfff640 1 --2- 172.21.15.46:0/2059184422 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f04dc0b2310 0x7f04dc0b27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:00.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.743+0000 7f04dbfff640 1 -- 172.21.15.46:0/2059184422 >> v1:172.21.15.162:6789/0 conn(0x7f04dc162af0 legacy=0x7f04dc071a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:00.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.744+0000 7f7f0a661640 1 Processor -- start 2023-12-02T00:29:00.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.743+0000 7f04dbfff640 1 -- 172.21.15.46:0/2059184422 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04dc162010 con 0x7f04dc15f2c0 2023-12-02T00:29:00.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.744+0000 7f8b6b64f640 1 -- start start 2023-12-02T00:29:00.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.744+0000 7fd151a65640 1 -- 172.21.15.46:0/946350441 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd14c0b3a80 msgr2=0x7fd14c152a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:00.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.744+0000 7fd151a65640 1 --2- 172.21.15.46:0/946350441 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd14c0b3a80 0x7fd14c152a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:00.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.744+0000 7fd151a65640 1 -- 172.21.15.46:0/946350441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd14c0ce1d0 msgr2=0x7fd14c152f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:00.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.744+0000 7fd151a65640 1 --2- 172.21.15.46:0/946350441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd14c0ce1d0 0x7fd14c152f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:00.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.744+0000 7f7f0a661640 1 -- start start 2023-12-02T00:29:00.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.744+0000 7fd151a65640 1 -- 172.21.15.46:0/946350441 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd148009400 con 0x7fd14c0b1780 2023-12-02T00:29:00.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.744+0000 7fd151264640 1 --2- 172.21.15.46:0/946350441 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd14c0b3a80 0x7fd14c152a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:29:00.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.744+0000 7fd152266640 1 --2- 172.21.15.46:0/946350441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd14c0ce1d0 0x7fd14c152f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:29:00.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.744+0000 7f40c67fc640 1 --2- 172.21.15.46:0/2564172216 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40c004ffa0 0x7f40c0037ba0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f40b8009870 tx=0x7f40b802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17d835a287f20295 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:00.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.745+0000 7f04dbfff640 1 --2- 172.21.15.46:0/2059184422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04dc15f2c0 0x7f04dc0b4a30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f04cc0098a0 tx=0x7f04cc02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe0863f9b999aafa server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:00.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.744+0000 7f6d7e954640 1 -- 172.21.15.46:0/1132256707 wait complete. 2023-12-02T00:29:00.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.745+0000 7f04daffd640 1 -- 172.21.15.46:0/2059184422 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04cc030840 con 0x7f04dc15f2c0 2023-12-02T00:29:00.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.745+0000 7f40c5ffb640 1 -- 172.21.15.46:0/2564172216 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b8030840 con 0x7f40c004ffa0 2023-12-02T00:29:00.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.745+0000 7f04daffd640 1 -- 172.21.15.46:0/2059184422 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04cc0309a0 con 0x7f04dc15f2c0 2023-12-02T00:29:00.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.745+0000 7f04daffd640 1 -- 172.21.15.46:0/2059184422 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04cc030d10 con 0x7f04dc15f2c0 2023-12-02T00:29:00.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.745+0000 7f8dcf5b6640 1 Processor -- start 2023-12-02T00:29:00.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.745+0000 7f40c5ffb640 1 -- 172.21.15.46:0/2564172216 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b80309a0 con 0x7f40c004ffa0 2023-12-02T00:29:00.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.745+0000 7f40c5ffb640 1 -- 172.21.15.46:0/2564172216 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b8030d30 con 0x7f40c004ffa0 2023-12-02T00:29:00.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.745+0000 7f8dcf5b6640 1 -- start start 2023-12-02T00:29:00.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.746+0000 7f7f0a661640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f0406efa0 0x7f7f04164200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.746+0000 7fd151a65640 1 --2- 172.21.15.46:0/946350441 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd14c0b1780 0x7fd14c157210 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd148033000 tx=0x7fd1480046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:00.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.746+0000 7f8b6b64f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b640b40b0 0x7f8b640b4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.746+0000 7f8b6b64f640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8b640b2490 0x7f8b640b2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.746+0000 7f8b6b64f640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b640cd020 con 0x7f8b64159210 2023-12-02T00:29:00.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.746+0000 7f8b6b64f640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b640b4a70 con 0x7f8b640b2490 2023-12-02T00:29:00.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.746+0000 7f8b6b64f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b640b36f0 con 0x7f8b640b40b0 2023-12-02T00:29:00.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.746+0000 7f7f0a661640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f040b2f60 0x7f7f040cd650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.746+0000 7f40c7fff640 1 -- 172.21.15.46:0/2564172216 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40c004ffa0 msgr2=0x7f40c0037ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:00.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.746+0000 7f40c7fff640 1 --2- 172.21.15.46:0/2564172216 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40c004ffa0 0x7f40c0037ba0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f40b8009870 tx=0x7f40b802b980 comp rx=0 tx=0).stop 2023-12-02T00:29:00.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.746+0000 7f7f0a661640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7f04161590 0x7f7f0414e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.746+0000 7f6d7e954640 1 Processor -- start 2023-12-02T00:29:00.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.746+0000 7f40c7fff640 1 -- 172.21.15.46:0/2564172216 shutdown_connections 2023-12-02T00:29:00.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.746+0000 7f40c7fff640 1 --2- 172.21.15.46:0/2564172216 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40c004ffa0 0x7f40c0037ba0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:00.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.746+0000 7f40c7fff640 1 --2- 172.21.15.46:0/2564172216 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40c00eacd0 0x7f40c004fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:00.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.746+0000 7f40c7fff640 1 -- 172.21.15.46:0/2564172216 >> 172.21.15.46:0/2564172216 conn(0x7f40c0009f80 msgr2=0x7f40c000a360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:00.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.746+0000 7fd1427fc640 1 -- 172.21.15.46:0/946350441 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd148033940 con 0x7fd14c0b1780 2023-12-02T00:29:00.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.746+0000 7f8b693eb640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8b64159210 0x7f8b6415a2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45064/0 (socket says 172.21.15.46:45064) 2023-12-02T00:29:00.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.746+0000 7f8b693eb640 1 -- 172.21.15.46:0/3023452565 learned_addr learned my addr 172.21.15.46:0/3023452565 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:00.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.746+0000 7fd1427fc640 1 -- 172.21.15.46:0/946350441 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd148033ac0 con 0x7fd14c0b1780 2023-12-02T00:29:00.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.746+0000 7f8dcf5b6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8db40047c0 0x7f8dc813d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.746+0000 7fd1427fc640 1 -- 172.21.15.46:0/946350441 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd148033e70 con 0x7fd14c0b1780 2023-12-02T00:29:00.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.747+0000 7fd152a67640 1 -- 172.21.15.46:0/946350441 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd14c14e530 con 0x7fd14c0b1780 2023-12-02T00:29:00.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.746+0000 7f8b68bea640 1 --2- 172.21.15.46:0/3023452565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b640b40b0 0x7f8b640b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:00.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.747+0000 7fd152a67640 1 -- 172.21.15.46:0/946350441 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd14c157750 con 0x7fd14c0b1780 2023-12-02T00:29:00.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.747+0000 7f8dcf5b6640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8dc813f570 0x7f8dc813da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.747+0000 7f8b69bec640 1 --2- 172.21.15.46:0/3023452565 >> v2:172.21.15.46:3300/0 conn(0x7f8b640b2490 0x7f8b640b2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:00.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.746+0000 7f7f03fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f0406efa0 0x7f7f04164200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:00.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.746+0000 7f7f03fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f0406efa0 0x7f7f04164200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38202/0 (socket says 172.21.15.46:38202) 2023-12-02T00:29:00.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.747+0000 7f7f037fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f040b2f60 0x7f7f040cd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:00.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.747+0000 7f7f037fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f040b2f60 0x7f7f040cd650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.46:44550/0 (socket says 172.21.15.46:44550) 2023-12-02T00:29:00.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.747+0000 7f8dcf5b6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8dc813dfc0 0x7f8dc80b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.747+0000 7f8b5bfff640 1 -- 172.21.15.46:0/3023452565 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 350978318 0 0) 0x7f8b640cd020 con 0x7f8b64159210 2023-12-02T00:29:00.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.747+0000 7f7f0a661640 1 -- 172.21.15.46:0/4171928624 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f041474f0 con 0x7f7f0406efa0 2023-12-02T00:29:00.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.747+0000 7f7f0a661640 1 -- 172.21.15.46:0/4171928624 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f04147210 con 0x7f7f040b2f60 2023-12-02T00:29:00.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.747+0000 7f7f0a661640 1 -- 172.21.15.46:0/4171928624 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f04163740 con 0x7f7f04161590 2023-12-02T00:29:00.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.747+0000 7f6d7e954640 1 -- start start 2023-12-02T00:29:00.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.747+0000 7f8dcd352640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8db40047c0 0x7f8dc813d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:00.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.747+0000 7f8b5bfff640 1 -- 172.21.15.46:0/3023452565 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b50003520 con 0x7f8b64159210 2023-12-02T00:29:00.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.747+0000 7f8b68bea640 1 -- 172.21.15.46:0/3023452565 >> v2:172.21.15.46:3300/0 conn(0x7f8b640b2490 msgr2=0x7f8b640b2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:00.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.747+0000 7f8b68bea640 1 --2- 172.21.15.46:0/3023452565 >> v2:172.21.15.46:3300/0 conn(0x7f8b640b2490 0x7f8b640b2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:00.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.747+0000 7f8b68bea640 1 -- 172.21.15.46:0/3023452565 >> v1:172.21.15.46:6789/0 conn(0x7f8b64159210 legacy=0x7f8b6415a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:00.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.747+0000 7f8b68bea640 1 -- 172.21.15.46:0/3023452565 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b640cd300 con 0x7f8b640b40b0 2023-12-02T00:29:00.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.747+0000 7f7f08bfe640 1 --2- 172.21.15.46:0/4171928624 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7f04161590 0x7f7f0414e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:00.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.747+0000 7f8dcd352640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8db40047c0 0x7f8dc813d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38212/0 (socket says 172.21.15.46:38212) 2023-12-02T00:29:00.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.747+0000 7f7f03fff640 1 -- 172.21.15.46:0/4171928624 learned_addr learned my addr 172.21.15.46:0/4171928624 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:00.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.748+0000 7f8dcd352640 1 -- 172.21.15.46:0/3673836823 learned_addr learned my addr 172.21.15.46:0/3673836823 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:00.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.748+0000 7f8dcdb53640 1 --2- 172.21.15.46:0/3673836823 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8dc813dfc0 0x7f8dc80b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:00.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.748+0000 7f8dccb51640 1 --2- 172.21.15.46:0/3673836823 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8dc813f570 0x7f8dc813da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:00.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.748+0000 7f8dcf5b6640 1 -- 172.21.15.46:0/3673836823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc815e1d0 con 0x7f8db40047c0 2023-12-02T00:29:00.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.748+0000 7f7f03fff640 1 -- 172.21.15.46:0/4171928624 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7f04161590 msgr2=0x7f7f0414e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:00.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.748+0000 7f7f03fff640 1 --2- 172.21.15.46:0/4171928624 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7f04161590 0x7f7f0414e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:00.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.748+0000 7f7f03fff640 1 -- 172.21.15.46:0/4171928624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f040b2f60 msgr2=0x7f7f040cd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:00.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.748+0000 7f7f03fff640 1 --2- 172.21.15.46:0/4171928624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f040b2f60 0x7f7f040cd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:00.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.748+0000 7f7f03fff640 1 -- 172.21.15.46:0/4171928624 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7efc009a70 con 0x7f7f0406efa0 2023-12-02T00:29:00.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.748+0000 7f8dcf5b6640 1 -- 172.21.15.46:0/3673836823 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc813e570 con 0x7f8dc813f570 2023-12-02T00:29:00.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.748+0000 7f8dcf5b6640 1 -- 172.21.15.46:0/3673836823 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc815db80 con 0x7f8dc813dfc0 2023-12-02T00:29:00.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.746+0000 7f04e277c640 1 -- 172.21.15.46:0/2059184422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04dc15f2c0 msgr2=0x7f04dc0b4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:00.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.746+0000 7f04e277c640 1 --2- 172.21.15.46:0/2059184422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04dc15f2c0 0x7f04dc0b4a30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f04cc0098a0 tx=0x7f04cc02b980 comp rx=0 tx=0).stop 2023-12-02T00:29:00.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.748+0000 7f6d7e954640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d7806d330 0x7f6d7814bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.748+0000 7f7f03fff640 1 --2- 172.21.15.46:0/4171928624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f0406efa0 0x7f7f04164200 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7ef40011e0 tx=0x7f7ef400cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:00.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.748+0000 7f8b68bea640 1 --2- 172.21.15.46:0/3023452565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b640b40b0 0x7f8b640b4530 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8b5400ba40 tx=0x7f8b5402ffd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=287c1f6eb7f74e44 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:00.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.747+0000 7f04e277c640 1 -- 172.21.15.46:0/2059184422 shutdown_connections 2023-12-02T00:29:00.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.747+0000 7f04e277c640 1 --2- 172.21.15.46:0/2059184422 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f04dc0b2310 0x7f04dc0b27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:00.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.747+0000 7f04e277c640 1 --2- 172.21.15.46:0/2059184422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04dc15f2c0 0x7f04dc0b4a30 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:00.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.747+0000 7f04e277c640 1 -- 172.21.15.46:0/2059184422 >> 172.21.15.46:0/2059184422 conn(0x7f04dc066270 msgr2=0x7f04dc066680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:00.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.747+0000 7f04e277c640 1 -- 172.21.15.46:0/2059184422 shutdown_connections 2023-12-02T00:29:00.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.747+0000 7f04e277c640 1 -- 172.21.15.46:0/2059184422 wait complete. 2023-12-02T00:29:00.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.748+0000 7f6d7e954640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d78071f70 0x7f6d78158070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.748+0000 7f6d7e954640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d78150680 0x7f6d7815bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.749+0000 7f6d7e954640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7814c530 con 0x7f6d78071f70 2023-12-02T00:29:00.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.749+0000 7f6d7e954640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d78158970 con 0x7f6d7806d330 2023-12-02T00:29:00.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.749+0000 7f6d7e954640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d78158c50 con 0x7f6d78150680 2023-12-02T00:29:00.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.749+0000 7f6d7d952640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d7806d330 0x7f6d7814bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:00.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.749+0000 7f6d7d952640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d7806d330 0x7f6d7814bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.46:44576/0 (socket says 172.21.15.46:44576) 2023-12-02T00:29:00.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.749+0000 7f6d7d952640 1 -- 172.21.15.46:0/2886581194 learned_addr learned my addr 172.21.15.46:0/2886581194 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:00.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.749+0000 7f04e277c640 1 Processor -- start 2023-12-02T00:29:00.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.749+0000 7f6d7e153640 1 --2- 172.21.15.46:0/2886581194 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d78150680 0x7f6d7815bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:00.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.749+0000 7f6d7d151640 1 --2- 172.21.15.46:0/2886581194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d78071f70 0x7f6d78158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:00.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.749+0000 7f04e277c640 1 -- start start 2023-12-02T00:29:00.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.749+0000 7fd1427fc640 1 -- 172.21.15.46:0/946350441 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd14803a050 con 0x7fd14c0b1780 2023-12-02T00:29:00.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.749+0000 7f8b5bfff640 1 -- 172.21.15.46:0/3023452565 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b54030d80 con 0x7f8b640b40b0 2023-12-02T00:29:00.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.747+0000 7f40c7fff640 1 -- 172.21.15.46:0/2564172216 shutdown_connections 2023-12-02T00:29:00.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.749+0000 7f8b5bfff640 1 -- 172.21.15.46:0/3023452565 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b54034070 con 0x7f8b640b40b0 2023-12-02T00:29:00.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.749+0000 7f8b5bfff640 1 -- 172.21.15.46:0/3023452565 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5403a620 con 0x7f8b640b40b0 2023-12-02T00:29:00.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.749+0000 7f40c7fff640 1 -- 172.21.15.46:0/2564172216 wait complete. 2023-12-02T00:29:00.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.749+0000 7f7f00ff9640 1 -- 172.21.15.46:0/4171928624 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef4005ad0 con 0x7f7f0406efa0 2023-12-02T00:29:00.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.749+0000 7f8dcd352640 1 -- 172.21.15.46:0/3673836823 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8dc813dfc0 msgr2=0x7f8dc80b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:00.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.749+0000 7f8dcd352640 1 --2- 172.21.15.46:0/3673836823 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8dc813dfc0 0x7f8dc80b21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:00.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.749+0000 7f8dcd352640 1 -- 172.21.15.46:0/3673836823 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8dc813f570 msgr2=0x7f8dc813da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:00.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.749+0000 7f8dcd352640 1 --2- 172.21.15.46:0/3673836823 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8dc813f570 0x7f8dc813da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:00.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.749+0000 7f8dcd352640 1 -- 172.21.15.46:0/3673836823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc815de60 con 0x7f8db40047c0 2023-12-02T00:29:00.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.750+0000 7f8dcd352640 1 --2- 172.21.15.46:0/3673836823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8db40047c0 0x7f8dc813d540 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8dc400d860 tx=0x7f8dc400dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:00.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.750+0000 7f8dcdb53640 1 --2- 172.21.15.46:0/3673836823 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8dc813dfc0 0x7f8dc80b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:29:00.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.750+0000 7f7f00ff9640 1 -- 172.21.15.46:0/4171928624 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef4005c50 con 0x7f7f0406efa0 2023-12-02T00:29:00.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.750+0000 7f7f00ff9640 1 -- 172.21.15.46:0/4171928624 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef4007070 con 0x7f7f0406efa0 2023-12-02T00:29:00.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.750+0000 7f7f0a661640 1 -- 172.21.15.46:0/4171928624 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f04163a20 con 0x7f7f0406efa0 2023-12-02T00:29:00.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.750+0000 7f7f0a661640 1 -- 172.21.15.46:0/4171928624 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f04164740 con 0x7f7f0406efa0 2023-12-02T00:29:00.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.750+0000 7f8dbdffb640 1 -- 172.21.15.46:0/3673836823 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc4004440 con 0x7f8db40047c0 2023-12-02T00:29:00.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.750+0000 7f8b6b64f640 1 -- 172.21.15.46:0/3023452565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b640b40b0 msgr2=0x7f8b640b4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:00.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.750+0000 7f8b6b64f640 1 --2- 172.21.15.46:0/3023452565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b640b40b0 0x7f8b640b4530 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8b5400ba40 tx=0x7f8b5402ffd0 comp rx=0 tx=0).stop 2023-12-02T00:29:00.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.750+0000 7f04e277c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04dc0b2310 0x7f04dc152110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.749+0000 7f6d7e153640 1 -- 172.21.15.46:0/2886581194 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d7806d330 msgr2=0x7f6d7814bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:00.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.750+0000 7f6d7e153640 1 --2- 172.21.15.46:0/2886581194 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d7806d330 0x7f6d7814bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:00.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.750+0000 7f6d7e153640 1 -- 172.21.15.46:0/2886581194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d78071f70 msgr2=0x7f6d78158070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:00.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.750+0000 7f6d7e153640 1 --2- 172.21.15.46:0/2886581194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d78071f70 0x7f6d78158070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:00.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.750+0000 7f6d7e153640 1 -- 172.21.15.46:0/2886581194 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d74009400 con 0x7f6d78150680 2023-12-02T00:29:00.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.750+0000 7f6d7d952640 1 --2- 172.21.15.46:0/2886581194 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d7806d330 0x7f6d7814bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:29:00.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.750+0000 7f8b6b64f640 1 -- 172.21.15.46:0/3023452565 shutdown_connections 2023-12-02T00:29:00.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.750+0000 7f8b6b64f640 1 --2- 172.21.15.46:0/3023452565 >> v2:172.21.15.46:3300/0 conn(0x7f8b640b2490 0x7f8b640b2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:00.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.750+0000 7f8b6b64f640 1 --2- 172.21.15.46:0/3023452565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b640b40b0 0x7f8b640b4530 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:00.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.750+0000 7f8b6b64f640 1 -- 172.21.15.46:0/3023452565 >> 172.21.15.46:0/3023452565 conn(0x7f8b640b0d60 msgr2=0x7f8b640b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:00.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.750+0000 7f04e277c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f04dc15f2c0 0x7f04dc0c9a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.750+0000 7f6d7e153640 1 --2- 172.21.15.46:0/2886581194 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d78150680 0x7f6d7815bfb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6d7000caa0 tx=0x7f6d7000cea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:00.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.751+0000 7f04e277c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f04dc162af0 0x7f04dc0c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.750+0000 7f8dcf5b6640 1 -- 172.21.15.46:0/3673836823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dc813ea10 con 0x7f8db40047c0 2023-12-02T00:29:00.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.751+0000 7f8dcf5b6640 1 -- 172.21.15.46:0/3673836823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dc813ee10 con 0x7f8db40047c0 2023-12-02T00:29:00.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.751+0000 7f8dbdffb640 1 -- 172.21.15.46:0/3673836823 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dc400c040 con 0x7f8db40047c0 2023-12-02T00:29:00.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.751+0000 7f8dbdffb640 1 -- 172.21.15.46:0/3673836823 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc4008890 con 0x7f8db40047c0 2023-12-02T00:29:00.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.751+0000 7f04e277c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04dc1f72e0 con 0x7f04dc0b2310 2023-12-02T00:29:00.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.751+0000 7f6d6e7fc640 1 -- 172.21.15.46:0/2886581194 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7000ab40 con 0x7f6d78150680 2023-12-02T00:29:00.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.751+0000 7f04e277c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04dc0c68e0 con 0x7f04dc162af0 2023-12-02T00:29:00.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.751+0000 7f04e277c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04dc0c7580 con 0x7f04dc15f2c0 2023-12-02T00:29:00.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.751+0000 7f04dbfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04dc0b2310 0x7f04dc152110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:00.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.751+0000 7f6d7e954640 1 -- 172.21.15.46:0/2886581194 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d78158f30 con 0x7f6d78150680 2023-12-02T00:29:00.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.751+0000 7f04dbfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04dc0b2310 0x7f04dc152110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38228/0 (socket says 172.21.15.46:38228) 2023-12-02T00:29:00.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.751+0000 7f04dbfff640 1 -- 172.21.15.46:0/762498469 learned_addr learned my addr 172.21.15.46:0/762498469 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:00.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.751+0000 7f04e0d19640 1 --2- 172.21.15.46:0/762498469 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f04dc162af0 0x7f04dc0c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:00.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.751+0000 7f6d7e954640 1 -- 172.21.15.46:0/2886581194 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d781585b0 con 0x7f6d78150680 2023-12-02T00:29:00.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.751+0000 7f6d6e7fc640 1 -- 172.21.15.46:0/2886581194 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d7000aca0 con 0x7f6d78150680 2023-12-02T00:29:00.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.751+0000 7f6d6e7fc640 1 -- 172.21.15.46:0/2886581194 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d70012600 con 0x7f6d78150680 2023-12-02T00:29:00.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.751+0000 7f8dccb51640 1 --2- 172.21.15.46:0/3673836823 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8dc813f570 0x7f8dc813da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:29:00.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.750+0000 7fd1427fc640 1 --2- 172.21.15.46:0/946350441 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd13006aec0 0x7fd13006d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.750+0000 7fd1427fc640 1 -- 172.21.15.46:0/946350441 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd1480aaf70 con 0x7fd14c0b1780 2023-12-02T00:29:00.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.750+0000 7fd151264640 1 --2- 172.21.15.46:0/946350441 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd13006aec0 0x7fd13006d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:00.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.750+0000 7fd12ffff640 1 -- 172.21.15.46:0/946350441 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd128005250 con 0x7fd14c0b1780 2023-12-02T00:29:00.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.752+0000 7f04db7fe640 1 --2- 172.21.15.46:0/762498469 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f04dc15f2c0 0x7f04dc0c9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:00.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.752+0000 7f04e0d19640 1 -- 172.21.15.46:0/762498469 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f04dc15f2c0 msgr2=0x7f04dc0c9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:00.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.752+0000 7f04e0d19640 1 --2- 172.21.15.46:0/762498469 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f04dc15f2c0 0x7f04dc0c9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:00.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.752+0000 7f04e0d19640 1 -- 172.21.15.46:0/762498469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04dc0b2310 msgr2=0x7f04dc152110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:00.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.752+0000 7f04e0d19640 1 --2- 172.21.15.46:0/762498469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04dc0b2310 0x7f04dc152110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:00.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.752+0000 7f04e0d19640 1 -- 172.21.15.46:0/762498469 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04cc0094d0 con 0x7f04dc162af0 2023-12-02T00:29:00.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.752+0000 7f04db7fe640 1 --2- 172.21.15.46:0/762498469 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f04dc15f2c0 0x7f04dc0c9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:29:00.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.752+0000 7f04e0d19640 1 --2- 172.21.15.46:0/762498469 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f04dc162af0 0x7f04dc0c9f60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f04d400efc0 tx=0x7f04d400c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:00.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.752+0000 7f40c7fff640 1 Processor -- start 2023-12-02T00:29:00.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.753+0000 7f8b6b64f640 1 -- 172.21.15.46:0/3023452565 shutdown_connections 2023-12-02T00:29:00.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.753+0000 7f7eea7fc640 1 -- 172.21.15.46:0/4171928624 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ee4005250 con 0x7f7f0406efa0 2023-12-02T00:29:00.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.753+0000 7fd151264640 1 --2- 172.21.15.46:0/946350441 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd13006aec0 0x7fd13006d380 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd14400d280 tx=0x7fd144009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:00.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.753+0000 7f40c7fff640 1 -- start start 2023-12-02T00:29:00.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.753+0000 7f6d7e954640 1 -- 172.21.15.46:0/2886581194 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d60005250 con 0x7f6d78150680 2023-12-02T00:29:00.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.753+0000 7f6d6e7fc640 1 -- 172.21.15.46:0/2886581194 <== mon.2 v2:172.21.15.162:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6d70012760 con 0x7f6d78150680 2023-12-02T00:29:00.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.753+0000 7f8b6b64f640 1 -- 172.21.15.46:0/3023452565 wait complete. 2023-12-02T00:29:00.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.753+0000 7f8dcf5b6640 1 -- 172.21.15.46:0/3673836823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8da8005250 con 0x7f8db40047c0 2023-12-02T00:29:00.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.753+0000 7f8dbdffb640 1 -- 172.21.15.46:0/3673836823 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8dc40089f0 con 0x7f8db40047c0 2023-12-02T00:29:00.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.754+0000 7f6d6e7fc640 1 --2- 172.21.15.46:0/2886581194 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f6d5c06af10 0x7f6d5c06d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.754+0000 7f6d6e7fc640 1 -- 172.21.15.46:0/2886581194 <== mon.2 v2:172.21.15.162:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6d70089360 con 0x7f6d78150680 2023-12-02T00:29:00.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.754+0000 7f6d7d952640 1 --2- 172.21.15.46:0/2886581194 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f6d5c06af10 0x7f6d5c06d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:00.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.754+0000 7f8dbdffb640 1 --2- 172.21.15.46:0/3673836823 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8dac06aec0 0x7f8dac06d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.754+0000 7f8dbdffb640 1 -- 172.21.15.46:0/3673836823 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8dc40898c0 con 0x7f8db40047c0 2023-12-02T00:29:00.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.754+0000 7f8dccb51640 1 --2- 172.21.15.46:0/3673836823 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8dac06aec0 0x7f8dac06d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:00.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.754+0000 7f7f00ff9640 1 -- 172.21.15.46:0/4171928624 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7ef40071d0 con 0x7f7f0406efa0 2023-12-02T00:29:00.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.754+0000 7f40c7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40c00380e0 0x7f40c00d9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.755+0000 7f40c7fff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40c004ffa0 0x7f40c00dcd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.755+0000 7f40c7fff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40c00eacd0 0x7f40c0043770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.755+0000 7f8dccb51640 1 --2- 172.21.15.46:0/3673836823 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8dac06aec0 0x7f8dac06d380 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8db8046000 tx=0x7f8db8041000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:00.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.755+0000 7f40c6ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40c00380e0 0x7f40c00d9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:00.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.755+0000 7f40c6ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40c00380e0 0x7f40c00d9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38240/0 (socket says 172.21.15.46:38240) 2023-12-02T00:29:00.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.755+0000 7f7f00ff9640 1 --2- 172.21.15.46:0/4171928624 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7ef006ae90 0x7f7ef006d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.755+0000 7f8b6b64f640 1 Processor -- start 2023-12-02T00:29:00.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.755+0000 7f40c67fc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40c004ffa0 0x7f40c00dcd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:00.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.755+0000 7f8b6b64f640 1 -- start start 2023-12-02T00:29:00.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.755+0000 7f40c67fc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40c004ffa0 0x7f40c00dcd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.46:44598/0 (socket says 172.21.15.46:44598) 2023-12-02T00:29:00.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.755+0000 7f7f00ff9640 1 -- 172.21.15.46:0/4171928624 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7ef4089130 con 0x7f7f0406efa0 2023-12-02T00:29:00.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.755+0000 7f6d7d952640 1 --2- 172.21.15.46:0/2886581194 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f6d5c06af10 0x7f6d5c06d3d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6d740097a0 tx=0x7f6d74008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:00.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.755+0000 7f40c6ffd640 1 -- 172.21.15.46:0/2410708867 learned_addr learned my addr 172.21.15.46:0/2410708867 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:00.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.755+0000 7f7f037fe640 1 --2- 172.21.15.46:0/4171928624 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7ef006ae90 0x7f7ef006d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:00.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.755+0000 7f40c7fff640 1 -- 172.21.15.46:0/2410708867 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c00da070 con 0x7f40c00380e0 2023-12-02T00:29:00.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.755+0000 7f40c77fe640 1 --2- 172.21.15.46:0/2410708867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40c00eacd0 0x7f40c0043770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:00.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.756+0000 7f40c7fff640 1 -- 172.21.15.46:0/2410708867 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c00db340 con 0x7f40c004ffa0 2023-12-02T00:29:00.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.756+0000 7f40c7fff640 1 -- 172.21.15.46:0/2410708867 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c00e9730 con 0x7f40c00eacd0 2023-12-02T00:29:00.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.756+0000 7f7f037fe640 1 --2- 172.21.15.46:0/4171928624 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7ef006ae90 0x7f7ef006d350 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7f040ce4e0 tx=0x7f7ef8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:00.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.756+0000 7f40c6ffd640 1 -- 172.21.15.46:0/2410708867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40c00eacd0 msgr2=0x7f40c0043770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:00.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.756+0000 7f40c6ffd640 1 --2- 172.21.15.46:0/2410708867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40c00eacd0 0x7f40c0043770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:00.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.756+0000 7f40c6ffd640 1 -- 172.21.15.46:0/2410708867 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40c004ffa0 msgr2=0x7f40c00dcd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:00.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.756+0000 7f40c6ffd640 1 --2- 172.21.15.46:0/2410708867 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40c004ffa0 0x7f40c00dcd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:00.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.756+0000 7f40c6ffd640 1 -- 172.21.15.46:0/2410708867 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b80094d0 con 0x7f40c00380e0 2023-12-02T00:29:00.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.756+0000 7f40c77fe640 1 --2- 172.21.15.46:0/2410708867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40c00eacd0 0x7f40c0043770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:29:00.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.756+0000 7f40c6ffd640 1 --2- 172.21.15.46:0/2410708867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40c00380e0 0x7f40c00d9b30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f40bc00e9b0 tx=0x7f40bc00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:00.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.756+0000 7f8b6b64f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b640b2490 0x7f8b640a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.756+0000 7f8b6b64f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b640b40b0 0x7f8b640a90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.757+0000 7f8b6b64f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b64159210 0x7f8b64152010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.757+0000 7f8b68bea640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b640b40b0 0x7f8b640a90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:00.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.757+0000 7f8b6b64f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b640a95e0 con 0x7f8b640b40b0 2023-12-02T00:29:00.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.757+0000 7f8b68bea640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b640b40b0 0x7f8b640a90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38254/0 (socket says 172.21.15.46:38254) 2023-12-02T00:29:00.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.757+0000 7f8b68bea640 1 -- 172.21.15.46:0/1064729604 learned_addr learned my addr 172.21.15.46:0/1064729604 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:00.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.757+0000 7f8b693eb640 1 --2- 172.21.15.46:0/1064729604 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b640b2490 0x7f8b640a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:00.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.757+0000 7f8b6b64f640 1 -- 172.21.15.46:0/1064729604 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b64152550 con 0x7f8b640b2490 2023-12-02T00:29:00.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.757+0000 7f8b6b64f640 1 -- 172.21.15.46:0/1064729604 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b640a9dd0 con 0x7f8b64159210 2023-12-02T00:29:00.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.757+0000 7f8b69bec640 1 --2- 172.21.15.46:0/1064729604 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b64159210 0x7f8b64152010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:00.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.757+0000 7f8b68bea640 1 -- 172.21.15.46:0/1064729604 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b64159210 msgr2=0x7f8b64152010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:00.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.757+0000 7f8b68bea640 1 --2- 172.21.15.46:0/1064729604 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b64159210 0x7f8b64152010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:00.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.757+0000 7f8b68bea640 1 -- 172.21.15.46:0/1064729604 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b640b2490 msgr2=0x7f8b640a84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:00.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.757+0000 7f8b68bea640 1 --2- 172.21.15.46:0/1064729604 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b640b2490 0x7f8b640a84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:00.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.757+0000 7f8b68bea640 1 -- 172.21.15.46:0/1064729604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b5400b6a0 con 0x7f8b640b40b0 2023-12-02T00:29:00.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.757+0000 7f8b69bec640 1 --2- 172.21.15.46:0/1064729604 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b64159210 0x7f8b64152010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:29:00.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.757+0000 7f8b68bea640 1 --2- 172.21.15.46:0/1064729604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b640b40b0 0x7f8b640a90a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8b5403c040 tx=0x7f8b540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:00.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.760+0000 7f8dbdffb640 1 -- 172.21.15.46:0/3673836823 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8dc4054ae0 con 0x7f8db40047c0 2023-12-02T00:29:00.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.760+0000 7fd1427fc640 1 -- 172.21.15.46:0/946350441 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd148038080 con 0x7fd14c0b1780 2023-12-02T00:29:00.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.760+0000 7f04d8ff9640 1 -- 172.21.15.46:0/762498469 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d4009e90 con 0x7f04dc162af0 2023-12-02T00:29:00.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.760+0000 7f04d8ff9640 1 -- 172.21.15.46:0/762498469 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04d400f040 con 0x7f04dc162af0 2023-12-02T00:29:00.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.760+0000 7f04e277c640 1 -- 172.21.15.46:0/762498469 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04dc0c7860 con 0x7f04dc162af0 2023-12-02T00:29:00.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.760+0000 7f04d8ff9640 1 -- 172.21.15.46:0/762498469 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d40146c0 con 0x7f04dc162af0 2023-12-02T00:29:00.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.760+0000 7f04e277c640 1 -- 172.21.15.46:0/762498469 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04dc152650 con 0x7f04dc162af0 2023-12-02T00:29:00.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.762+0000 7f04e277c640 1 -- 172.21.15.46:0/762498469 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04dc0c73e0 con 0x7f04dc162af0 2023-12-02T00:29:00.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.762+0000 7f6d6e7fc640 1 -- 172.21.15.46:0/2886581194 <== mon.2 v2:172.21.15.162:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6d700544e0 con 0x7f6d78150680 2023-12-02T00:29:00.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.762+0000 7f04d8ff9640 1 -- 172.21.15.46:0/762498469 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f04d4014820 con 0x7f04dc162af0 2023-12-02T00:29:00.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.763+0000 7f04d8ff9640 1 --2- 172.21.15.46:0/762498469 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f04c806af10 0x7f04c806d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.763+0000 7f04d8ff9640 1 -- 172.21.15.46:0/762498469 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f04d408a460 con 0x7f04dc162af0 2023-12-02T00:29:00.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.764+0000 7f04dbfff640 1 --2- 172.21.15.46:0/762498469 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f04c806af10 0x7f04c806d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:00.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.764+0000 7f04dbfff640 1 --2- 172.21.15.46:0/762498469 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f04c806af10 0x7f04c806d3d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f04cc0098a0 tx=0x7f04cc03b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:00.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.765+0000 7f7f00ff9640 1 -- 172.21.15.46:0/4171928624 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7ef4054350 con 0x7f7f0406efa0 2023-12-02T00:29:00.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.765+0000 7f40affff640 1 -- 172.21.15.46:0/2410708867 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40bc009670 con 0x7f40c00380e0 2023-12-02T00:29:00.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.765+0000 7f40affff640 1 -- 172.21.15.46:0/2410708867 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40bc00c9a0 con 0x7f40c00380e0 2023-12-02T00:29:00.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.765+0000 7f40c7fff640 1 -- 172.21.15.46:0/2410708867 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40c00dd2c0 con 0x7f40c00380e0 2023-12-02T00:29:00.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.766+0000 7f40c7fff640 1 -- 172.21.15.46:0/2410708867 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40c00da730 con 0x7f40c00380e0 2023-12-02T00:29:00.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.767+0000 7f40affff640 1 -- 172.21.15.46:0/2410708867 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40bc00cd50 con 0x7f40c00380e0 2023-12-02T00:29:00.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.767+0000 7f40adffb640 1 -- 172.21.15.46:0/2410708867 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40c0005360 con 0x7f40c00380e0 2023-12-02T00:29:00.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.767+0000 7f8b59ffb640 1 -- 172.21.15.46:0/1064729604 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5403ad80 con 0x7f8b640b40b0 2023-12-02T00:29:00.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.767+0000 7f8b59ffb640 1 -- 172.21.15.46:0/1064729604 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b54045470 con 0x7f8b640b40b0 2023-12-02T00:29:00.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.767+0000 7f8b59ffb640 1 -- 172.21.15.46:0/1064729604 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b54045820 con 0x7f8b640b40b0 2023-12-02T00:29:00.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.768+0000 7f8b6b64f640 1 -- 172.21.15.46:0/1064729604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b640a9810 con 0x7f8b640b40b0 2023-12-02T00:29:00.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.768+0000 7f8b6b64f640 1 -- 172.21.15.46:0/1064729604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b6415ae60 con 0x7f8b640b40b0 2023-12-02T00:29:00.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.770+0000 7f8b6b64f640 1 -- 172.21.15.46:0/1064729604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b6415acc0 con 0x7f8b640b40b0 2023-12-02T00:29:00.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.771+0000 7f40affff640 1 -- 172.21.15.46:0/2410708867 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f40bc013470 con 0x7f40c00380e0 2023-12-02T00:29:00.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.772+0000 7f40affff640 1 --2- 172.21.15.46:0/2410708867 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f40b406ae90 0x7f40b406d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.772+0000 7f8b59ffb640 1 -- 172.21.15.46:0/1064729604 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8b54045980 con 0x7f8b640b40b0 2023-12-02T00:29:00.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.772+0000 7f40affff640 1 -- 172.21.15.46:0/2410708867 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f40bc089fc0 con 0x7f40c00380e0 2023-12-02T00:29:00.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.772+0000 7f8b59ffb640 1 --2- 172.21.15.46:0/1064729604 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8b4806ae90 0x7f8b4806d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:00.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.773+0000 7f04d8ff9640 1 -- 172.21.15.46:0/762498469 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f04dc0c73e0 con 0x7f04dc162af0 2023-12-02T00:29:00.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.773+0000 7f8b693eb640 1 --2- 172.21.15.46:0/1064729604 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8b4806ae90 0x7f8b4806d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:00.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.773+0000 7f40c67fc640 1 --2- 172.21.15.46:0/2410708867 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f40b406ae90 0x7f40b406d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:00.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.774+0000 7f40c67fc640 1 --2- 172.21.15.46:0/2410708867 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f40b406ae90 0x7f40b406d350 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f40b8036000 tx=0x7f40b8008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:00.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.775+0000 7f8b693eb640 1 --2- 172.21.15.46:0/1064729604 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8b4806ae90 0x7f8b4806d350 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c0013b0 tx=0x7f8b5c001630 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:00.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.779+0000 7f40affff640 1 -- 172.21.15.46:0/2410708867 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f40bc055100 con 0x7f40c00380e0 2023-12-02T00:29:00.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.783+0000 7f8b59ffb640 1 -- 172.21.15.46:0/1064729604 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b5407f6d0 con 0x7f8b640b40b0 2023-12-02T00:29:00.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:00.788+0000 7f8b59ffb640 1 -- 172.21.15.46:0/1064729604 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8b6415acc0 con 0x7f8b640b40b0 2023-12-02T00:29:01.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.103+0000 7f150ed5a640 1 -- 172.21.15.46:0/825243255 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f150816f870 con 0x7f15081629c0 2023-12-02T00:29:01.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.103+0000 7f15057fa640 1 -- 172.21.15.46:0/825243255 <== mon.2 v2:172.21.15.162:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f150816f870 con 0x7f15081629c0 2023-12-02T00:29:01.105 INFO:teuthology.orchestra.run.smithi046.stdout:55834574855 2023-12-02T00:29:01.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.107+0000 7f14eeffd640 1 -- 172.21.15.46:0/825243255 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f14f406af90 msgr2=0x7f14f406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:01.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.107+0000 7f14eeffd640 1 --2- 172.21.15.46:0/825243255 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f14f406af90 0x7f14f406d450 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f14fc001050 tx=0x7f14fc000fe0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.107+0000 7f14eeffd640 1 -- 172.21.15.46:0/825243255 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f15081629c0 msgr2=0x7f15081675f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:01.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.107+0000 7f14eeffd640 1 --2- 172.21.15.46:0/825243255 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f15081629c0 0x7f15081675f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f14f800c390 tx=0x7f14f800c850 comp rx=0 tx=0).stop 2023-12-02T00:29:01.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.107+0000 7f14eeffd640 1 -- 172.21.15.46:0/825243255 shutdown_connections 2023-12-02T00:29:01.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.108+0000 7f14eeffd640 1 --2- 172.21.15.46:0/825243255 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f14f406af90 0x7f14f406d450 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.108+0000 7f14eeffd640 1 --2- 172.21.15.46:0/825243255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1508204590 0x7f150816fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.108+0000 7f14eeffd640 1 --2- 172.21.15.46:0/825243255 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f15081629c0 0x7f15081675f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.108+0000 7f14eeffd640 1 --2- 172.21.15.46:0/825243255 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14f0005250 0x7f15080aded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.108+0000 7f14eeffd640 1 -- 172.21.15.46:0/825243255 >> 172.21.15.46:0/825243255 conn(0x7f15080b0d60 msgr2=0x7f15080aef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:01.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.108+0000 7f14eeffd640 1 -- 172.21.15.46:0/825243255 shutdown_connections 2023-12-02T00:29:01.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.108+0000 7f14eeffd640 1 -- 172.21.15.46:0/825243255 wait complete. 2023-12-02T00:29:01.126 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.7 2023-12-02T00:29:01.126 DEBUG:teuthology.parallel:result is None 2023-12-02T00:29:01.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.154+0000 7f8b6b64f640 1 -- 172.21.15.46:0/1064729604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f8b6415acc0 con 0x7f8b640b40b0 2023-12-02T00:29:01.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.155+0000 7f8b59ffb640 1 -- 172.21.15.46:0/1064729604 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8b6415acc0 con 0x7f8b640b40b0 2023-12-02T00:29:01.156 INFO:teuthology.orchestra.run.smithi046.stdout:55834574856 2023-12-02T00:29:01.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.157+0000 7f8b6b64f640 1 -- 172.21.15.46:0/1064729604 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8b4806ae90 msgr2=0x7f8b4806d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:01.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.157+0000 7f8b6b64f640 1 --2- 172.21.15.46:0/1064729604 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8b4806ae90 0x7f8b4806d350 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c0013b0 tx=0x7f8b5c001630 comp rx=0 tx=0).stop 2023-12-02T00:29:01.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.157+0000 7f8b6b64f640 1 -- 172.21.15.46:0/1064729604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b640b40b0 msgr2=0x7f8b640a90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:01.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.157+0000 7f8b6b64f640 1 --2- 172.21.15.46:0/1064729604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b640b40b0 0x7f8b640a90a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8b5403c040 tx=0x7f8b540046c0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.157+0000 7f8b6b64f640 1 -- 172.21.15.46:0/1064729604 shutdown_connections 2023-12-02T00:29:01.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.157+0000 7f8b6b64f640 1 --2- 172.21.15.46:0/1064729604 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8b4806ae90 0x7f8b4806d350 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.157+0000 7f8b6b64f640 1 --2- 172.21.15.46:0/1064729604 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b64159210 0x7f8b64152010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.157+0000 7f8b6b64f640 1 --2- 172.21.15.46:0/1064729604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b640b40b0 0x7f8b640a90a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.157+0000 7f8b6b64f640 1 --2- 172.21.15.46:0/1064729604 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b640b2490 0x7f8b640a84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.157+0000 7f8b6b64f640 1 -- 172.21.15.46:0/1064729604 >> 172.21.15.46:0/1064729604 conn(0x7f8b640b0d60 msgr2=0x7f8b640717d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:01.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.158+0000 7f8b6b64f640 1 -- 172.21.15.46:0/1064729604 shutdown_connections 2023-12-02T00:29:01.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.158+0000 7f8b6b64f640 1 -- 172.21.15.46:0/1064729604 wait complete. 2023-12-02T00:29:01.170 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.5 2023-12-02T00:29:01.170 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.169+0000 7fd12ffff640 1 -- 172.21.15.46:0/946350441 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd1280057d0 con 0x7fd14c0b1780 2023-12-02T00:29:01.170 DEBUG:teuthology.parallel:result is None 2023-12-02T00:29:01.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.170+0000 7fd1427fc640 1 -- 172.21.15.46:0/946350441 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd148076190 con 0x7fd14c0b1780 2023-12-02T00:29:01.171 INFO:teuthology.orchestra.run.smithi046.stdout:55834574854 2023-12-02T00:29:01.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.173+0000 7fd12ffff640 1 -- 172.21.15.46:0/946350441 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd13006aec0 msgr2=0x7fd13006d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:01.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.173+0000 7fd12ffff640 1 --2- 172.21.15.46:0/946350441 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd13006aec0 0x7fd13006d380 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd14400d280 tx=0x7fd144009040 comp rx=0 tx=0).stop 2023-12-02T00:29:01.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.173+0000 7fd12ffff640 1 -- 172.21.15.46:0/946350441 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd14c0b1780 msgr2=0x7fd14c157210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:01.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.173+0000 7fd12ffff640 1 --2- 172.21.15.46:0/946350441 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd14c0b1780 0x7fd14c157210 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd148033000 tx=0x7fd1480046c0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.173+0000 7fd12ffff640 1 -- 172.21.15.46:0/946350441 shutdown_connections 2023-12-02T00:29:01.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.173+0000 7fd12ffff640 1 --2- 172.21.15.46:0/946350441 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd13006aec0 0x7fd13006d380 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.173+0000 7fd12ffff640 1 --2- 172.21.15.46:0/946350441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd14c0ce1d0 0x7fd14c152f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.173+0000 7fd12ffff640 1 --2- 172.21.15.46:0/946350441 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd14c0b3a80 0x7fd14c152a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.173+0000 7fd12ffff640 1 --2- 172.21.15.46:0/946350441 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd14c0b1780 0x7fd14c157210 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.173+0000 7fd12ffff640 1 -- 172.21.15.46:0/946350441 >> 172.21.15.46:0/946350441 conn(0x7fd14c0b0c60 msgr2=0x7fd14c0af740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:01.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.173+0000 7fd12ffff640 1 -- 172.21.15.46:0/946350441 shutdown_connections 2023-12-02T00:29:01.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.173+0000 7fd12ffff640 1 -- 172.21.15.46:0/946350441 wait complete. 2023-12-02T00:29:01.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.177+0000 7f8dcf5b6640 1 -- 172.21.15.46:0/3673836823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8da80057d0 con 0x7f8db40047c0 2023-12-02T00:29:01.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.178+0000 7f8dbdffb640 1 -- 172.21.15.46:0/3673836823 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8dc4058100 con 0x7f8db40047c0 2023-12-02T00:29:01.182 INFO:teuthology.orchestra.run.smithi046.stdout:55834574854 2023-12-02T00:29:01.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.181+0000 7f6d7e954640 1 -- 172.21.15.46:0/2886581194 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f6d60005500 con 0x7f6d78150680 2023-12-02T00:29:01.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.181+0000 7f8dcf5b6640 1 -- 172.21.15.46:0/3673836823 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8dac06aec0 msgr2=0x7f8dac06d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:01.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.181+0000 7f8dcf5b6640 1 --2- 172.21.15.46:0/3673836823 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8dac06aec0 0x7f8dac06d380 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8db8046000 tx=0x7f8db8041000 comp rx=0 tx=0).stop 2023-12-02T00:29:01.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.181+0000 7f8dcf5b6640 1 -- 172.21.15.46:0/3673836823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8db40047c0 msgr2=0x7f8dc813d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:01.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.181+0000 7f8dcf5b6640 1 --2- 172.21.15.46:0/3673836823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8db40047c0 0x7f8dc813d540 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8dc400d860 tx=0x7f8dc400dd20 comp rx=0 tx=0).stop 2023-12-02T00:29:01.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.181+0000 7f8dcf5b6640 1 -- 172.21.15.46:0/3673836823 shutdown_connections 2023-12-02T00:29:01.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.181+0000 7f8dcf5b6640 1 --2- 172.21.15.46:0/3673836823 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8dac06aec0 0x7f8dac06d380 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.181+0000 7f8dcf5b6640 1 --2- 172.21.15.46:0/3673836823 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8dc813dfc0 0x7f8dc80b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.181+0000 7f8dcf5b6640 1 --2- 172.21.15.46:0/3673836823 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8dc813f570 0x7f8dc813da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.181+0000 7f8dcf5b6640 1 --2- 172.21.15.46:0/3673836823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8db40047c0 0x7f8dc813d540 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.181+0000 7f8dcf5b6640 1 -- 172.21.15.46:0/3673836823 >> 172.21.15.46:0/3673836823 conn(0x7f8dc8070900 msgr2=0x7f8dc815f850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:01.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.182+0000 7f8dcf5b6640 1 -- 172.21.15.46:0/3673836823 shutdown_connections 2023-12-02T00:29:01.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.182+0000 7f6d6e7fc640 1 -- 172.21.15.46:0/2886581194 <== mon.2 v2:172.21.15.162:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6d70057b00 con 0x7f6d78150680 2023-12-02T00:29:01.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.182+0000 7f7eea7fc640 1 -- 172.21.15.46:0/4171928624 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f7ee40057d0 con 0x7f7f0406efa0 2023-12-02T00:29:01.184 INFO:teuthology.orchestra.run.smithi046.stdout:55834574855 2023-12-02T00:29:01.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.182+0000 7f8dcf5b6640 1 -- 172.21.15.46:0/3673836823 wait complete. 2023-12-02T00:29:01.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.182+0000 7f7f00ff9640 1 -- 172.21.15.46:0/4171928624 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7ef4057970 con 0x7f7f0406efa0 2023-12-02T00:29:01.185 INFO:teuthology.orchestra.run.smithi046.stdout:55834574856 2023-12-02T00:29:01.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.184+0000 7f7eea7fc640 1 -- 172.21.15.46:0/4171928624 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7ef006ae90 msgr2=0x7f7ef006d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:01.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.184+0000 7f7eea7fc640 1 --2- 172.21.15.46:0/4171928624 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7ef006ae90 0x7f7ef006d350 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7f040ce4e0 tx=0x7f7ef8008040 comp rx=0 tx=0).stop 2023-12-02T00:29:01.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.184+0000 7f7eea7fc640 1 -- 172.21.15.46:0/4171928624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f0406efa0 msgr2=0x7f7f04164200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:01.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.184+0000 7f7eea7fc640 1 --2- 172.21.15.46:0/4171928624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f0406efa0 0x7f7f04164200 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7ef40011e0 tx=0x7f7ef400cbe0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.185+0000 7f6d7e954640 1 -- 172.21.15.46:0/2886581194 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f6d5c06af10 msgr2=0x7f6d5c06d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:01.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.185+0000 7f6d7e954640 1 --2- 172.21.15.46:0/2886581194 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f6d5c06af10 0x7f6d5c06d3d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6d740097a0 tx=0x7f6d74008030 comp rx=0 tx=0).stop 2023-12-02T00:29:01.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.185+0000 7f7eea7fc640 1 -- 172.21.15.46:0/4171928624 shutdown_connections 2023-12-02T00:29:01.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.185+0000 7f6d7e954640 1 -- 172.21.15.46:0/2886581194 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d78150680 msgr2=0x7f6d7815bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:01.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.185+0000 7f6d7e954640 1 --2- 172.21.15.46:0/2886581194 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d78150680 0x7f6d7815bfb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6d7000caa0 tx=0x7f6d7000cea0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.185+0000 7f7eea7fc640 1 --2- 172.21.15.46:0/4171928624 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7ef006ae90 0x7f7ef006d350 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.185+0000 7f7eea7fc640 1 --2- 172.21.15.46:0/4171928624 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7f04161590 0x7f7f0414e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.185+0000 7f7eea7fc640 1 --2- 172.21.15.46:0/4171928624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f040b2f60 0x7f7f040cd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.185+0000 7f7eea7fc640 1 --2- 172.21.15.46:0/4171928624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f0406efa0 0x7f7f04164200 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.185+0000 7f7eea7fc640 1 -- 172.21.15.46:0/4171928624 >> 172.21.15.46:0/4171928624 conn(0x7f7f040cf410 msgr2=0x7f7f0406cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:01.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.185+0000 7f04e277c640 1 -- 172.21.15.46:0/762498469 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f04dc0c73e0 con 0x7f04dc162af0 2023-12-02T00:29:01.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.185+0000 7f6d7e954640 1 -- 172.21.15.46:0/2886581194 shutdown_connections 2023-12-02T00:29:01.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.185+0000 7f6d7e954640 1 --2- 172.21.15.46:0/2886581194 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f6d5c06af10 0x7f6d5c06d3d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.185+0000 7f6d7e954640 1 --2- 172.21.15.46:0/2886581194 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d78150680 0x7f6d7815bfb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.185+0000 7f6d7e954640 1 --2- 172.21.15.46:0/2886581194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d78071f70 0x7f6d78158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.185+0000 7f6d7e954640 1 --2- 172.21.15.46:0/2886581194 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d7806d330 0x7f6d7814bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.185+0000 7f6d7e954640 1 -- 172.21.15.46:0/2886581194 >> 172.21.15.46:0/2886581194 conn(0x7f6d7806a4d0 msgr2=0x7f6d780b0750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:01.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.185+0000 7f7eea7fc640 1 -- 172.21.15.46:0/4171928624 shutdown_connections 2023-12-02T00:29:01.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.185+0000 7f7eea7fc640 1 -- 172.21.15.46:0/4171928624 wait complete. 2023-12-02T00:29:01.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.185+0000 7f6d7e954640 1 -- 172.21.15.46:0/2886581194 shutdown_connections 2023-12-02T00:29:01.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.185+0000 7f6d7e954640 1 -- 172.21.15.46:0/2886581194 wait complete. 2023-12-02T00:29:01.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.185+0000 7f04d8ff9640 1 -- 172.21.15.46:0/762498469 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f04dc0c73e0 con 0x7f04dc162af0 2023-12-02T00:29:01.196 INFO:teuthology.orchestra.run.smithi046.stdout:55834574855 2023-12-02T00:29:01.197 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2023-12-02T00:29:01.197 DEBUG:teuthology.parallel:result is None 2023-12-02T00:29:01.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.188+0000 7f04e277c640 1 -- 172.21.15.46:0/762498469 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f04c806af10 msgr2=0x7f04c806d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:01.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.188+0000 7f04e277c640 1 --2- 172.21.15.46:0/762498469 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f04c806af10 0x7f04c806d3d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f04cc0098a0 tx=0x7f04cc03b040 comp rx=0 tx=0).stop 2023-12-02T00:29:01.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.189+0000 7f04e277c640 1 -- 172.21.15.46:0/762498469 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f04dc162af0 msgr2=0x7f04dc0c9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:01.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.189+0000 7f04e277c640 1 --2- 172.21.15.46:0/762498469 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f04dc162af0 0x7f04dc0c9f60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f04d400efc0 tx=0x7f04d400c490 comp rx=0 tx=0).stop 2023-12-02T00:29:01.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.189+0000 7f04e277c640 1 -- 172.21.15.46:0/762498469 shutdown_connections 2023-12-02T00:29:01.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.189+0000 7f04e277c640 1 --2- 172.21.15.46:0/762498469 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f04c806af10 0x7f04c806d3d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.189+0000 7f04e277c640 1 --2- 172.21.15.46:0/762498469 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f04dc162af0 0x7f04dc0c9f60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.189+0000 7f04e277c640 1 --2- 172.21.15.46:0/762498469 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f04dc15f2c0 0x7f04dc0c9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.189+0000 7f04e277c640 1 --2- 172.21.15.46:0/762498469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04dc0b2310 0x7f04dc152110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.189+0000 7f04e277c640 1 -- 172.21.15.46:0/762498469 >> 172.21.15.46:0/762498469 conn(0x7f04dc066270 msgr2=0x7f04dc15e560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:01.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.189+0000 7f40adffb640 1 -- 172.21.15.46:0/2410708867 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f40c004f5e0 con 0x7f40c00380e0 2023-12-02T00:29:01.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.189+0000 7f04e277c640 1 -- 172.21.15.46:0/762498469 shutdown_connections 2023-12-02T00:29:01.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.189+0000 7f04e277c640 1 -- 172.21.15.46:0/762498469 wait complete. 2023-12-02T00:29:01.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.189+0000 7f40affff640 1 -- 172.21.15.46:0/2410708867 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f40bc058720 con 0x7f40c00380e0 2023-12-02T00:29:01.199 INFO:teuthology.orchestra.run.smithi046.stdout:55834574855 2023-12-02T00:29:01.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.192+0000 7f40adffb640 1 -- 172.21.15.46:0/2410708867 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f40b406ae90 msgr2=0x7f40b406d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:01.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.192+0000 7f40adffb640 1 --2- 172.21.15.46:0/2410708867 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f40b406ae90 0x7f40b406d350 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f40b8036000 tx=0x7f40b8008030 comp rx=0 tx=0).stop 2023-12-02T00:29:01.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.192+0000 7f40adffb640 1 -- 172.21.15.46:0/2410708867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40c00380e0 msgr2=0x7f40c00d9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:01.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.192+0000 7f40adffb640 1 --2- 172.21.15.46:0/2410708867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40c00380e0 0x7f40c00d9b30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f40bc00e9b0 tx=0x7f40bc00ee70 comp rx=0 tx=0).stop 2023-12-02T00:29:01.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.192+0000 7f40adffb640 1 -- 172.21.15.46:0/2410708867 shutdown_connections 2023-12-02T00:29:01.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.192+0000 7f40adffb640 1 --2- 172.21.15.46:0/2410708867 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f40b406ae90 0x7f40b406d350 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.192+0000 7f40adffb640 1 --2- 172.21.15.46:0/2410708867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40c00eacd0 0x7f40c0043770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.192+0000 7f40adffb640 1 --2- 172.21.15.46:0/2410708867 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40c004ffa0 0x7f40c00dcd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.192+0000 7f40adffb640 1 --2- 172.21.15.46:0/2410708867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40c00380e0 0x7f40c00d9b30 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.192+0000 7f40adffb640 1 -- 172.21.15.46:0/2410708867 >> 172.21.15.46:0/2410708867 conn(0x7f40c0009f80 msgr2=0x7f40c0052ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:01.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.192+0000 7f40adffb640 1 -- 172.21.15.46:0/2410708867 shutdown_connections 2023-12-02T00:29:01.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.192+0000 7f40adffb640 1 -- 172.21.15.46:0/2410708867 wait complete. 2023-12-02T00:29:01.203 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2023-12-02T00:29:01.203 DEBUG:teuthology.parallel:result is None 2023-12-02T00:29:01.207 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.6 2023-12-02T00:29:01.208 DEBUG:teuthology.parallel:result is None 2023-12-02T00:29:01.208 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.0 2023-12-02T00:29:01.208 DEBUG:teuthology.parallel:result is None 2023-12-02T00:29:01.209 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.4 2023-12-02T00:29:01.209 DEBUG:teuthology.parallel:result is None 2023-12-02T00:29:01.210 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2023-12-02T00:29:01.210 DEBUG:teuthology.parallel:result is None 2023-12-02T00:29:01.210 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-12-02T00:29:01.210 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-02T00:29:01.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.353+0000 7f7ad0810640 1 Processor -- start 2023-12-02T00:29:01.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.353+0000 7f7ad0810640 1 -- start start 2023-12-02T00:29:01.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.354+0000 7f7ad0810640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7acc143490 0x7f7acc0b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:01.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.354+0000 7f7ad0810640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7acc15e730 0x7f7acc0b1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:01.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.354+0000 7f7ad0810640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7acc15a4e0 0x7f7acc15a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:01.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.354+0000 7f7ad0810640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7acc15b520 con 0x7f7acc143490 2023-12-02T00:29:01.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.354+0000 7f7ad0810640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7acc143870 con 0x7f7acc15e730 2023-12-02T00:29:01.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.354+0000 7f7ad0810640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7acc15e2a0 con 0x7f7acc15a4e0 2023-12-02T00:29:01.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.354+0000 7f7aca59c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7acc143490 0x7f7acc0b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:01.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.354+0000 7f7aca59c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7acc143490 0x7f7acc0b0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38256/0 (socket says 172.21.15.46:38256) 2023-12-02T00:29:01.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.354+0000 7f7aca59c640 1 -- 172.21.15.46:0/41174743 learned_addr learned my addr 172.21.15.46:0/41174743 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:01.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.354+0000 7f7ac9d9b640 1 --2- 172.21.15.46:0/41174743 >> v2:172.21.15.162:3300/0 conn(0x7f7acc15e730 0x7f7acc0b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:01.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.354+0000 7f7aca59c640 1 -- 172.21.15.46:0/41174743 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7acc15a4e0 msgr2=0x7f7acc15a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:01.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.354+0000 7f7aca59c640 1 --2- 172.21.15.46:0/41174743 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7acc15a4e0 0x7f7acc15a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.354+0000 7f7aca59c640 1 -- 172.21.15.46:0/41174743 >> v2:172.21.15.162:3300/0 conn(0x7f7acc15e730 msgr2=0x7f7acc0b1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:01.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.354+0000 7f7aca59c640 1 --2- 172.21.15.46:0/41174743 >> v2:172.21.15.162:3300/0 conn(0x7f7acc15e730 0x7f7acc0b1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.354+0000 7f7aca59c640 1 -- 172.21.15.46:0/41174743 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7acc15b240 con 0x7f7acc143490 2023-12-02T00:29:01.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.355+0000 7f7aca59c640 1 --2- 172.21.15.46:0/41174743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7acc143490 0x7f7acc0b0cf0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7acc0aeca0 tx=0x7f7ac402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=264db3aa556697ba server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:01.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.355+0000 7f7ac959a640 1 -- 172.21.15.46:0/41174743 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac4031870 con 0x7f7acc143490 2023-12-02T00:29:01.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.355+0000 7f7ac959a640 1 -- 172.21.15.46:0/41174743 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ac40319d0 con 0x7f7acc143490 2023-12-02T00:29:01.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.355+0000 7f7ac959a640 1 -- 172.21.15.46:0/41174743 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac4031d60 con 0x7f7acc143490 2023-12-02T00:29:01.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.356+0000 7f7ad0810640 1 -- 172.21.15.46:0/41174743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7acc143490 msgr2=0x7f7acc0b0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:01.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.356+0000 7f7ad0810640 1 --2- 172.21.15.46:0/41174743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7acc143490 0x7f7acc0b0cf0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7acc0aeca0 tx=0x7f7ac402b910 comp rx=0 tx=0).stop 2023-12-02T00:29:01.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.356+0000 7f7ad0810640 1 -- 172.21.15.46:0/41174743 shutdown_connections 2023-12-02T00:29:01.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.356+0000 7f7ad0810640 1 --2- 172.21.15.46:0/41174743 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7acc15a4e0 0x7f7acc15a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.356+0000 7f7ad0810640 1 --2- 172.21.15.46:0/41174743 >> v2:172.21.15.162:3300/0 conn(0x7f7acc15e730 0x7f7acc0b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.356+0000 7f7ad0810640 1 --2- 172.21.15.46:0/41174743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7acc143490 0x7f7acc0b0cf0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.356+0000 7f7ad0810640 1 -- 172.21.15.46:0/41174743 >> 172.21.15.46:0/41174743 conn(0x7f7acc06a4d0 msgr2=0x7f7acc06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:01.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.356+0000 7f7ad0810640 1 -- 172.21.15.46:0/41174743 shutdown_connections 2023-12-02T00:29:01.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.356+0000 7f7ad0810640 1 -- 172.21.15.46:0/41174743 wait complete. 2023-12-02T00:29:01.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.357+0000 7f7ad0810640 1 Processor -- start 2023-12-02T00:29:01.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.357+0000 7f7ad0810640 1 -- start start 2023-12-02T00:29:01.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.358+0000 7f7ad0810640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7acc143490 0x7f7acc0abda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:01.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.358+0000 7f7ad0810640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7acc15a4e0 0x7f7acc0ada70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:01.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.358+0000 7f7ad0810640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7acc15e730 0x7f7acc0adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:01.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.358+0000 7f7ad0810640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7acc0ac2e0 con 0x7f7acc143490 2023-12-02T00:29:01.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.358+0000 7f7ad0810640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7acc130b20 con 0x7f7acc15e730 2023-12-02T00:29:01.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.358+0000 7f7ad0810640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7acc130e00 con 0x7f7acc15a4e0 2023-12-02T00:29:01.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.358+0000 7f7aca59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7acc143490 0x7f7acc0abda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:01.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.358+0000 7f7aca59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7acc143490 0x7f7acc0abda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38266/0 (socket says 172.21.15.46:38266) 2023-12-02T00:29:01.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.358+0000 7f7aca59c640 1 -- 172.21.15.46:0/2470389576 learned_addr learned my addr 172.21.15.46:0/2470389576 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:01.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.358+0000 7f7aca59c640 1 -- 172.21.15.46:0/2470389576 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7acc15a4e0 msgr2=0x7f7acc0ada70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:01.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.358+0000 7f7aca59c640 1 --2- 172.21.15.46:0/2470389576 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7acc15a4e0 0x7f7acc0ada70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.358+0000 7f7aca59c640 1 -- 172.21.15.46:0/2470389576 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7acc15e730 msgr2=0x7f7acc0adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:01.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.358+0000 7f7aca59c640 1 --2- 172.21.15.46:0/2470389576 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7acc15e730 0x7f7acc0adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.358+0000 7f7aca59c640 1 -- 172.21.15.46:0/2470389576 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ac40094d0 con 0x7f7acc143490 2023-12-02T00:29:01.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.359+0000 7f7aca59c640 1 --2- 172.21.15.46:0/2470389576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7acc143490 0x7f7acc0abda0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7ac402b950 tx=0x7f7ac40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:01.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.359+0000 7f7abaffd640 1 -- 172.21.15.46:0/2470389576 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac4031860 con 0x7f7acc143490 2023-12-02T00:29:01.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.359+0000 7f7abaffd640 1 -- 172.21.15.46:0/2470389576 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ac40443f0 con 0x7f7acc143490 2023-12-02T00:29:01.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.359+0000 7f7abaffd640 1 -- 172.21.15.46:0/2470389576 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac40447a0 con 0x7f7acc143490 2023-12-02T00:29:01.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.359+0000 7f7ad0810640 1 -- 172.21.15.46:0/2470389576 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7acc1310e0 con 0x7f7acc143490 2023-12-02T00:29:01.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.359+0000 7f7ad0810640 1 -- 172.21.15.46:0/2470389576 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7acc1316a0 con 0x7f7acc143490 2023-12-02T00:29:01.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.360+0000 7f7ad0810640 1 -- 172.21.15.46:0/2470389576 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7acc0b4380 con 0x7f7acc143490 2023-12-02T00:29:01.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.360+0000 7f7abaffd640 1 -- 172.21.15.46:0/2470389576 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7ac404d450 con 0x7f7acc143490 2023-12-02T00:29:01.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.361+0000 7f7abaffd640 1 --2- 172.21.15.46:0/2470389576 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7aac06ae40 0x7f7aac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:01.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.361+0000 7f7abaffd640 1 -- 172.21.15.46:0/2470389576 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7ac402d040 con 0x7f7acc143490 2023-12-02T00:29:01.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.361+0000 7f7ac9d9b640 1 --2- 172.21.15.46:0/2470389576 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7aac06ae40 0x7f7aac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:01.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.361+0000 7f7ac9d9b640 1 --2- 172.21.15.46:0/2470389576 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7aac06ae40 0x7f7aac06d300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7acc0a8750 tx=0x7f7ac0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:01.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.365+0000 7f7abaffd640 1 -- 172.21.15.46:0/2470389576 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7ac4076e90 con 0x7f7acc143490 2023-12-02T00:29:01.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.539+0000 7f7ad0810640 1 -- 172.21.15.46:0/2470389576 --> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7acc0b2670 con 0x7f7aac06ae40 2023-12-02T00:29:01.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.546+0000 7f7abaffd640 1 -- 172.21.15.46:0/2470389576 <== mgr.4108 v2:172.21.15.162:6832/3248644722 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264578 (secure 0 0 0) 0x7f7acc0b2670 con 0x7f7aac06ae40 2023-12-02T00:29:01.550 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:29:01.551 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-12-02T00:29:01.552 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.549+0000 7f7ad0810640 1 -- 172.21.15.46:0/2470389576 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7aac06ae40 msgr2=0x7f7aac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:01.552 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.549+0000 7f7ad0810640 1 --2- 172.21.15.46:0/2470389576 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7aac06ae40 0x7f7aac06d300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7acc0a8750 tx=0x7f7ac0008040 comp rx=0 tx=0).stop 2023-12-02T00:29:01.552 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.549+0000 7f7ad0810640 1 -- 172.21.15.46:0/2470389576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7acc143490 msgr2=0x7f7acc0abda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:01.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.549+0000 7f7ad0810640 1 --2- 172.21.15.46:0/2470389576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7acc143490 0x7f7acc0abda0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7ac402b950 tx=0x7f7ac40046c0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.549+0000 7f7ad0810640 1 -- 172.21.15.46:0/2470389576 shutdown_connections 2023-12-02T00:29:01.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.549+0000 7f7ad0810640 1 --2- 172.21.15.46:0/2470389576 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7aac06ae40 0x7f7aac06d300 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.549+0000 7f7ad0810640 1 --2- 172.21.15.46:0/2470389576 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7acc15e730 0x7f7acc0adfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.549+0000 7f7ad0810640 1 --2- 172.21.15.46:0/2470389576 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7acc15a4e0 0x7f7acc0ada70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.549+0000 7f7ad0810640 1 --2- 172.21.15.46:0/2470389576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7acc143490 0x7f7acc0abda0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.549+0000 7f7ad0810640 1 -- 172.21.15.46:0/2470389576 >> 172.21.15.46:0/2470389576 conn(0x7f7acc06a4d0 msgr2=0x7f7acc0b1010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:01.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.549+0000 7f7ad0810640 1 -- 172.21.15.46:0/2470389576 shutdown_connections 2023-12-02T00:29:01.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.549+0000 7f7ad0810640 1 -- 172.21.15.46:0/2470389576 wait complete. 2023-12-02T00:29:01.561 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-12-02T00:29:01.161031+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235744,"kb_used_data":2912,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738976,"statfs":{"total":773094113280,"available":772852711424,"internally_reserved":0,"allocated":2981888,"data_stored":1788966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.457779"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021839+0000","last_change":"2023-12-02T00:28:49.717914+0000","last_active":"2023-12-02T00:28:51.021839+0000","last_peered":"2023-12-02T00:28:51.021839+0000","last_clean":"2023-12-02T00:28:51.021839+0000","last_became_active":"2023-12-02T00:28:49.717594+0000","last_became_peered":"2023-12-02T00:28:49.717594+0000","last_unstale":"2023-12-02T00:28:51.021839+0000","last_undegraded":"2023-12-02T00:28:51.021839+0000","last_fullsized":"2023-12-02T00:28:51.021839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:48:16.850673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703744+0000","last_change":"2023-12-02T00:28:49.713605+0000","last_active":"2023-12-02T00:28:50.703744+0000","last_peered":"2023-12-02T00:28:50.703744+0000","last_clean":"2023-12-02T00:28:50.703744+0000","last_became_active":"2023-12-02T00:28:49.712497+0000","last_became_peered":"2023-12-02T00:28:49.712497+0000","last_unstale":"2023-12-02T00:28:50.703744+0000","last_undegraded":"2023-12-02T00:28:50.703744+0000","last_fullsized":"2023-12-02T00:28:50.703744+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:15:51.747629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021917+0000","last_change":"2023-12-02T00:28:49.713065+0000","last_active":"2023-12-02T00:28:51.021917+0000","last_peered":"2023-12-02T00:28:51.021917+0000","last_clean":"2023-12-02T00:28:51.021917+0000","last_became_active":"2023-12-02T00:28:49.712715+0000","last_became_peered":"2023-12-02T00:28:49.712715+0000","last_unstale":"2023-12-02T00:28:51.021917+0000","last_undegraded":"2023-12-02T00:28:51.021917+0000","last_fullsized":"2023-12-02T00:28:51.021917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:52:22.600655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706337+0000","last_change":"2023-12-02T00:28:49.727831+0000","last_active":"2023-12-02T00:28:50.706337+0000","last_peered":"2023-12-02T00:28:50.706337+0000","last_clean":"2023-12-02T00:28:50.706337+0000","last_became_active":"2023-12-02T00:28:49.725804+0000","last_became_peered":"2023-12-02T00:28:49.725804+0000","last_unstale":"2023-12-02T00:28:50.706337+0000","last_undegraded":"2023-12-02T00:28:50.706337+0000","last_fullsized":"2023-12-02T00:28:50.706337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:50:45.793304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021332+0000","last_change":"2023-12-02T00:28:49.721391+0000","last_active":"2023-12-02T00:28:51.021332+0000","last_peered":"2023-12-02T00:28:51.021332+0000","last_clean":"2023-12-02T00:28:51.021332+0000","last_became_active":"2023-12-02T00:28:49.720314+0000","last_became_peered":"2023-12-02T00:28:49.720314+0000","last_unstale":"2023-12-02T00:28:51.021332+0000","last_undegraded":"2023-12-02T00:28:51.021332+0000","last_fullsized":"2023-12-02T00:28:51.021332+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:32:13.515968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705990+0000","last_change":"2023-12-02T00:28:49.725516+0000","last_active":"2023-12-02T00:28:50.705990+0000","last_peered":"2023-12-02T00:28:50.705990+0000","last_clean":"2023-12-02T00:28:50.705990+0000","last_became_active":"2023-12-02T00:28:49.725149+0000","last_became_peered":"2023-12-02T00:28:49.725149+0000","last_unstale":"2023-12-02T00:28:50.705990+0000","last_undegraded":"2023-12-02T00:28:50.705990+0000","last_fullsized":"2023-12-02T00:28:50.705990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T10:27:29.244181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021273+0000","last_change":"2023-12-02T00:28:49.719153+0000","last_active":"2023-12-02T00:28:51.021273+0000","last_peered":"2023-12-02T00:28:51.021273+0000","last_clean":"2023-12-02T00:28:51.021273+0000","last_became_active":"2023-12-02T00:28:49.718567+0000","last_became_peered":"2023-12-02T00:28:49.718567+0000","last_unstale":"2023-12-02T00:28:51.021273+0000","last_undegraded":"2023-12-02T00:28:51.021273+0000","last_fullsized":"2023-12-02T00:28:51.021273+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T10:27:29.244181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020683+0000","last_change":"2023-12-02T00:28:49.726986+0000","last_active":"2023-12-02T00:28:51.020683+0000","last_peered":"2023-12-02T00:28:51.020683+0000","last_clean":"2023-12-02T00:28:51.020683+0000","last_became_active":"2023-12-02T00:28:49.726646+0000","last_became_peered":"2023-12-02T00:28:49.726646+0000","last_unstale":"2023-12-02T00:28:51.020683+0000","last_undegraded":"2023-12-02T00:28:51.020683+0000","last_fullsized":"2023-12-02T00:28:51.020683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:35:09.231704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021827+0000","last_change":"2023-12-02T00:28:49.725402+0000","last_active":"2023-12-02T00:28:51.021827+0000","last_peered":"2023-12-02T00:28:51.021827+0000","last_clean":"2023-12-02T00:28:51.021827+0000","last_became_active":"2023-12-02T00:28:49.724978+0000","last_became_peered":"2023-12-02T00:28:49.724978+0000","last_unstale":"2023-12-02T00:28:51.021827+0000","last_undegraded":"2023-12-02T00:28:51.021827+0000","last_fullsized":"2023-12-02T00:28:51.021827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:18:31.615464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021500+0000","last_change":"2023-12-02T00:28:49.718464+0000","last_active":"2023-12-02T00:28:51.021500+0000","last_peered":"2023-12-02T00:28:51.021500+0000","last_clean":"2023-12-02T00:28:51.021500+0000","last_became_active":"2023-12-02T00:28:49.718060+0000","last_became_peered":"2023-12-02T00:28:49.718060+0000","last_unstale":"2023-12-02T00:28:51.021500+0000","last_undegraded":"2023-12-02T00:28:51.021500+0000","last_fullsized":"2023-12-02T00:28:51.021500+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:36:22.587928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021177+0000","last_change":"2023-12-02T00:28:49.720030+0000","last_active":"2023-12-02T00:28:51.021177+0000","last_peered":"2023-12-02T00:28:51.021177+0000","last_clean":"2023-12-02T00:28:51.021177+0000","last_became_active":"2023-12-02T00:28:49.719715+0000","last_became_peered":"2023-12-02T00:28:49.719715+0000","last_unstale":"2023-12-02T00:28:51.021177+0000","last_undegraded":"2023-12-02T00:28:51.021177+0000","last_fullsized":"2023-12-02T00:28:51.021177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:07:52.802590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705569+0000","last_change":"2023-12-02T00:28:49.713687+0000","last_active":"2023-12-02T00:28:50.705569+0000","last_peered":"2023-12-02T00:28:50.705569+0000","last_clean":"2023-12-02T00:28:50.705569+0000","last_became_active":"2023-12-02T00:28:49.713359+0000","last_became_peered":"2023-12-02T00:28:49.713359+0000","last_unstale":"2023-12-02T00:28:50.705569+0000","last_undegraded":"2023-12-02T00:28:50.705569+0000","last_fullsized":"2023-12-02T00:28:50.705569+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:11:06.388890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021465+0000","last_change":"2023-12-02T00:28:49.720866+0000","last_active":"2023-12-02T00:28:51.021465+0000","last_peered":"2023-12-02T00:28:51.021465+0000","last_clean":"2023-12-02T00:28:51.021465+0000","last_became_active":"2023-12-02T00:28:49.720524+0000","last_became_peered":"2023-12-02T00:28:49.720524+0000","last_unstale":"2023-12-02T00:28:51.021465+0000","last_undegraded":"2023-12-02T00:28:51.021465+0000","last_fullsized":"2023-12-02T00:28:51.021465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:32:13.515968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021720+0000","last_change":"2023-12-02T00:28:49.711387+0000","last_active":"2023-12-02T00:28:51.021720+0000","last_peered":"2023-12-02T00:28:51.021720+0000","last_clean":"2023-12-02T00:28:51.021720+0000","last_became_active":"2023-12-02T00:28:49.711036+0000","last_became_peered":"2023-12-02T00:28:49.711036+0000","last_unstale":"2023-12-02T00:28:51.021720+0000","last_undegraded":"2023-12-02T00:28:51.021720+0000","last_fullsized":"2023-12-02T00:28:51.021720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:13:36.512466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703511+0000","last_change":"2023-12-02T00:28:49.706444+0000","last_active":"2023-12-02T00:28:50.703511+0000","last_peered":"2023-12-02T00:28:50.703511+0000","last_clean":"2023-12-02T00:28:50.703511+0000","last_became_active":"2023-12-02T00:28:49.706132+0000","last_became_peered":"2023-12-02T00:28:49.706132+0000","last_unstale":"2023-12-02T00:28:50.703511+0000","last_undegraded":"2023-12-02T00:28:50.703511+0000","last_fullsized":"2023-12-02T00:28:50.703511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:52:22.600655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705451+0000","last_change":"2023-12-02T00:28:49.725045+0000","last_active":"2023-12-02T00:28:50.705451+0000","last_peered":"2023-12-02T00:28:50.705451+0000","last_clean":"2023-12-02T00:28:50.705451+0000","last_became_active":"2023-12-02T00:28:49.722884+0000","last_became_peered":"2023-12-02T00:28:49.722884+0000","last_unstale":"2023-12-02T00:28:50.705451+0000","last_undegraded":"2023-12-02T00:28:50.705451+0000","last_fullsized":"2023-12-02T00:28:50.705451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:36:22.587928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706621+0000","last_change":"2023-12-02T00:28:49.727232+0000","last_active":"2023-12-02T00:28:50.706621+0000","last_peered":"2023-12-02T00:28:50.706621+0000","last_clean":"2023-12-02T00:28:50.706621+0000","last_became_active":"2023-12-02T00:28:49.726881+0000","last_became_peered":"2023-12-02T00:28:49.726881+0000","last_unstale":"2023-12-02T00:28:50.706621+0000","last_undegraded":"2023-12-02T00:28:50.706621+0000","last_fullsized":"2023-12-02T00:28:50.706621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:48:16.850673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021328+0000","last_change":"2023-12-02T00:28:49.711953+0000","last_active":"2023-12-02T00:28:51.021328+0000","last_peered":"2023-12-02T00:28:51.021328+0000","last_clean":"2023-12-02T00:28:51.021328+0000","last_became_active":"2023-12-02T00:28:49.711653+0000","last_became_peered":"2023-12-02T00:28:49.711653+0000","last_unstale":"2023-12-02T00:28:51.021328+0000","last_undegraded":"2023-12-02T00:28:51.021328+0000","last_fullsized":"2023-12-02T00:28:51.021328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:52:08.689376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021886+0000","last_change":"2023-12-02T00:28:49.717365+0000","last_active":"2023-12-02T00:28:51.021886+0000","last_peered":"2023-12-02T00:28:51.021886+0000","last_clean":"2023-12-02T00:28:51.021886+0000","last_became_active":"2023-12-02T00:28:49.717071+0000","last_became_peered":"2023-12-02T00:28:49.717071+0000","last_unstale":"2023-12-02T00:28:51.021886+0000","last_undegraded":"2023-12-02T00:28:51.021886+0000","last_fullsized":"2023-12-02T00:28:51.021886+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:10:26.775049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021416+0000","last_change":"2023-12-02T00:28:49.716297+0000","last_active":"2023-12-02T00:28:51.021416+0000","last_peered":"2023-12-02T00:28:51.021416+0000","last_clean":"2023-12-02T00:28:51.021416+0000","last_became_active":"2023-12-02T00:28:49.715963+0000","last_became_peered":"2023-12-02T00:28:49.715963+0000","last_unstale":"2023-12-02T00:28:51.021416+0000","last_undegraded":"2023-12-02T00:28:51.021416+0000","last_fullsized":"2023-12-02T00:28:51.021416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:58:11.596095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021739+0000","last_change":"2023-12-02T00:28:49.722259+0000","last_active":"2023-12-02T00:28:51.021739+0000","last_peered":"2023-12-02T00:28:51.021739+0000","last_clean":"2023-12-02T00:28:51.021739+0000","last_became_active":"2023-12-02T00:28:49.721989+0000","last_became_peered":"2023-12-02T00:28:49.721989+0000","last_unstale":"2023-12-02T00:28:51.021739+0000","last_undegraded":"2023-12-02T00:28:51.021739+0000","last_fullsized":"2023-12-02T00:28:51.021739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:55:55.141283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705987+0000","last_change":"2023-12-02T00:28:49.729526+0000","last_active":"2023-12-02T00:28:50.705987+0000","last_peered":"2023-12-02T00:28:50.705987+0000","last_clean":"2023-12-02T00:28:50.705987+0000","last_became_active":"2023-12-02T00:28:49.728999+0000","last_became_peered":"2023-12-02T00:28:49.728999+0000","last_unstale":"2023-12-02T00:28:50.705987+0000","last_undegraded":"2023-12-02T00:28:50.705987+0000","last_fullsized":"2023-12-02T00:28:50.705987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:58:15.388237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021070+0000","last_change":"2023-12-02T00:28:49.717936+0000","last_active":"2023-12-02T00:28:51.021070+0000","last_peered":"2023-12-02T00:28:51.021070+0000","last_clean":"2023-12-02T00:28:51.021070+0000","last_became_active":"2023-12-02T00:28:49.716770+0000","last_became_peered":"2023-12-02T00:28:49.716770+0000","last_unstale":"2023-12-02T00:28:51.021070+0000","last_undegraded":"2023-12-02T00:28:51.021070+0000","last_fullsized":"2023-12-02T00:28:51.021070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:36:22.587928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705591+0000","last_change":"2023-12-02T00:28:49.724863+0000","last_active":"2023-12-02T00:28:50.705591+0000","last_peered":"2023-12-02T00:28:50.705591+0000","last_clean":"2023-12-02T00:28:50.705591+0000","last_became_active":"2023-12-02T00:28:49.724592+0000","last_became_peered":"2023-12-02T00:28:49.724592+0000","last_unstale":"2023-12-02T00:28:50.705591+0000","last_undegraded":"2023-12-02T00:28:50.705591+0000","last_fullsized":"2023-12-02T00:28:50.705591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:15:51.747629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020786+0000","last_change":"2023-12-02T00:28:49.724283+0000","last_active":"2023-12-02T00:28:51.020786+0000","last_peered":"2023-12-02T00:28:51.020786+0000","last_clean":"2023-12-02T00:28:51.020786+0000","last_became_active":"2023-12-02T00:28:49.723940+0000","last_became_peered":"2023-12-02T00:28:49.723940+0000","last_unstale":"2023-12-02T00:28:51.020786+0000","last_undegraded":"2023-12-02T00:28:51.020786+0000","last_fullsized":"2023-12-02T00:28:51.020786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:52:22.600655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021632+0000","last_change":"2023-12-02T00:28:49.717015+0000","last_active":"2023-12-02T00:28:51.021632+0000","last_peered":"2023-12-02T00:28:51.021632+0000","last_clean":"2023-12-02T00:28:51.021632+0000","last_became_active":"2023-12-02T00:28:49.716394+0000","last_became_peered":"2023-12-02T00:28:49.716394+0000","last_unstale":"2023-12-02T00:28:51.021632+0000","last_undegraded":"2023-12-02T00:28:51.021632+0000","last_fullsized":"2023-12-02T00:28:51.021632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:15:51.747629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703687+0000","last_change":"2023-12-02T00:28:49.716707+0000","last_active":"2023-12-02T00:28:50.703687+0000","last_peered":"2023-12-02T00:28:50.703687+0000","last_clean":"2023-12-02T00:28:50.703687+0000","last_became_active":"2023-12-02T00:28:49.716392+0000","last_became_peered":"2023-12-02T00:28:49.716392+0000","last_unstale":"2023-12-02T00:28:50.703687+0000","last_undegraded":"2023-12-02T00:28:50.703687+0000","last_fullsized":"2023-12-02T00:28:50.703687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:36:22.587928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021477+0000","last_change":"2023-12-02T00:28:49.724290+0000","last_active":"2023-12-02T00:28:51.021477+0000","last_peered":"2023-12-02T00:28:51.021477+0000","last_clean":"2023-12-02T00:28:51.021477+0000","last_became_active":"2023-12-02T00:28:49.723952+0000","last_became_peered":"2023-12-02T00:28:49.723952+0000","last_unstale":"2023-12-02T00:28:51.021477+0000","last_undegraded":"2023-12-02T00:28:51.021477+0000","last_fullsized":"2023-12-02T00:28:51.021477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:15:02.988933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021881+0000","last_change":"2023-12-02T00:28:49.721218+0000","last_active":"2023-12-02T00:28:51.021881+0000","last_peered":"2023-12-02T00:28:51.021881+0000","last_clean":"2023-12-02T00:28:51.021881+0000","last_became_active":"2023-12-02T00:28:49.720894+0000","last_became_peered":"2023-12-02T00:28:49.720894+0000","last_unstale":"2023-12-02T00:28:51.021881+0000","last_undegraded":"2023-12-02T00:28:51.021881+0000","last_fullsized":"2023-12-02T00:28:51.021881+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:35:09.231704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706145+0000","last_change":"2023-12-02T00:28:49.725424+0000","last_active":"2023-12-02T00:28:50.706145+0000","last_peered":"2023-12-02T00:28:50.706145+0000","last_clean":"2023-12-02T00:28:50.706145+0000","last_became_active":"2023-12-02T00:28:49.724671+0000","last_became_peered":"2023-12-02T00:28:49.724671+0000","last_unstale":"2023-12-02T00:28:50.706145+0000","last_undegraded":"2023-12-02T00:28:50.706145+0000","last_fullsized":"2023-12-02T00:28:50.706145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:32:13.515968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021246+0000","last_change":"2023-12-02T00:28:49.722395+0000","last_active":"2023-12-02T00:28:51.021246+0000","last_peered":"2023-12-02T00:28:51.021246+0000","last_clean":"2023-12-02T00:28:51.021246+0000","last_became_active":"2023-12-02T00:28:49.722052+0000","last_became_peered":"2023-12-02T00:28:49.722052+0000","last_unstale":"2023-12-02T00:28:51.021246+0000","last_undegraded":"2023-12-02T00:28:51.021246+0000","last_fullsized":"2023-12-02T00:28:51.021246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:11:06.388890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020869+0000","last_change":"2023-12-02T00:28:49.713618+0000","last_active":"2023-12-02T00:28:51.020869+0000","last_peered":"2023-12-02T00:28:51.020869+0000","last_clean":"2023-12-02T00:28:51.020869+0000","last_became_active":"2023-12-02T00:28:49.713325+0000","last_became_peered":"2023-12-02T00:28:49.713325+0000","last_unstale":"2023-12-02T00:28:51.020869+0000","last_undegraded":"2023-12-02T00:28:51.020869+0000","last_fullsized":"2023-12-02T00:28:51.020869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:35:09.231704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705601+0000","last_change":"2023-12-02T00:28:49.710604+0000","last_active":"2023-12-02T00:28:50.705601+0000","last_peered":"2023-12-02T00:28:50.705601+0000","last_clean":"2023-12-02T00:28:50.705601+0000","last_became_active":"2023-12-02T00:28:49.710316+0000","last_became_peered":"2023-12-02T00:28:49.710316+0000","last_unstale":"2023-12-02T00:28:50.705601+0000","last_undegraded":"2023-12-02T00:28:50.705601+0000","last_fullsized":"2023-12-02T00:28:50.705601+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:52:08.689376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021549+0000","last_change":"2023-12-02T00:28:48.735813+0000","last_active":"2023-12-02T00:28:51.021549+0000","last_peered":"2023-12-02T00:28:51.021549+0000","last_clean":"2023-12-02T00:28:51.021549+0000","last_became_active":"2023-12-02T00:28:48.735462+0000","last_became_peered":"2023-12-02T00:28:48.735462+0000","last_unstale":"2023-12-02T00:28:51.021549+0000","last_undegraded":"2023-12-02T00:28:51.021549+0000","last_fullsized":"2023-12-02T00:28:51.021549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:55:54.133908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.022018+0000","last_change":"2023-12-02T00:28:49.715318+0000","last_active":"2023-12-02T00:28:51.022018+0000","last_peered":"2023-12-02T00:28:51.022018+0000","last_clean":"2023-12-02T00:28:51.022018+0000","last_became_active":"2023-12-02T00:28:49.715020+0000","last_became_peered":"2023-12-02T00:28:49.715020+0000","last_unstale":"2023-12-02T00:28:51.022018+0000","last_undegraded":"2023-12-02T00:28:51.022018+0000","last_fullsized":"2023-12-02T00:28:51.022018+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:36:22.587928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021150+0000","last_change":"2023-12-02T00:28:48.730625+0000","last_active":"2023-12-02T00:28:51.021150+0000","last_peered":"2023-12-02T00:28:51.021150+0000","last_clean":"2023-12-02T00:28:51.021150+0000","last_became_active":"2023-12-02T00:28:48.730306+0000","last_became_peered":"2023-12-02T00:28:48.730306+0000","last_unstale":"2023-12-02T00:28:51.021150+0000","last_undegraded":"2023-12-02T00:28:51.021150+0000","last_fullsized":"2023-12-02T00:28:51.021150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:11:05.381516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020857+0000","last_change":"2023-12-02T00:28:48.735160+0000","last_active":"2023-12-02T00:28:51.020857+0000","last_peered":"2023-12-02T00:28:51.020857+0000","last_clean":"2023-12-02T00:28:51.020857+0000","last_became_active":"2023-12-02T00:28:48.734860+0000","last_became_peered":"2023-12-02T00:28:48.734860+0000","last_unstale":"2023-12-02T00:28:51.020857+0000","last_undegraded":"2023-12-02T00:28:51.020857+0000","last_fullsized":"2023-12-02T00:28:51.020857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T09:17:51.234416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020811+0000","last_change":"2023-12-02T00:28:49.716640+0000","last_active":"2023-12-02T00:28:51.020811+0000","last_peered":"2023-12-02T00:28:51.020811+0000","last_clean":"2023-12-02T00:28:51.020811+0000","last_became_active":"2023-12-02T00:28:49.716325+0000","last_became_peered":"2023-12-02T00:28:49.716325+0000","last_unstale":"2023-12-02T00:28:51.020811+0000","last_undegraded":"2023-12-02T00:28:51.020811+0000","last_fullsized":"2023-12-02T00:28:51.020811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:07:52.802590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705691+0000","last_change":"2023-12-02T00:28:48.730403+0000","last_active":"2023-12-02T00:28:50.705691+0000","last_peered":"2023-12-02T00:28:50.705691+0000","last_clean":"2023-12-02T00:28:50.705691+0000","last_became_active":"2023-12-02T00:28:48.730053+0000","last_became_peered":"2023-12-02T00:28:48.730053+0000","last_unstale":"2023-12-02T00:28:50.705691+0000","last_undegraded":"2023-12-02T00:28:50.705691+0000","last_fullsized":"2023-12-02T00:28:50.705691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:13:35.505091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703512+0000","last_change":"2023-12-02T00:28:49.716925+0000","last_active":"2023-12-02T00:28:50.703512+0000","last_peered":"2023-12-02T00:28:50.703512+0000","last_clean":"2023-12-02T00:28:50.703512+0000","last_became_active":"2023-12-02T00:28:49.716530+0000","last_became_peered":"2023-12-02T00:28:49.716530+0000","last_unstale":"2023-12-02T00:28:50.703512+0000","last_undegraded":"2023-12-02T00:28:50.703512+0000","last_fullsized":"2023-12-02T00:28:50.703512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:35:09.231704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021002+0000","last_change":"2023-12-02T00:28:48.732302+0000","last_active":"2023-12-02T00:28:51.021002+0000","last_peered":"2023-12-02T00:28:51.021002+0000","last_clean":"2023-12-02T00:28:51.021002+0000","last_became_active":"2023-12-02T00:28:48.731891+0000","last_became_peered":"2023-12-02T00:28:48.731891+0000","last_unstale":"2023-12-02T00:28:51.021002+0000","last_undegraded":"2023-12-02T00:28:51.021002+0000","last_fullsized":"2023-12-02T00:28:51.021002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:44:45.468077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706568+0000","last_change":"2023-12-02T00:28:49.719085+0000","last_active":"2023-12-02T00:28:50.706568+0000","last_peered":"2023-12-02T00:28:50.706568+0000","last_clean":"2023-12-02T00:28:50.706568+0000","last_became_active":"2023-12-02T00:28:49.718606+0000","last_became_peered":"2023-12-02T00:28:49.718606+0000","last_unstale":"2023-12-02T00:28:50.706568+0000","last_undegraded":"2023-12-02T00:28:50.706568+0000","last_fullsized":"2023-12-02T00:28:50.706568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:18:31.615464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706430+0000","last_change":"2023-12-02T00:28:48.732243+0000","last_active":"2023-12-02T00:28:50.706430+0000","last_peered":"2023-12-02T00:28:50.706430+0000","last_clean":"2023-12-02T00:28:50.706430+0000","last_became_active":"2023-12-02T00:28:48.731895+0000","last_became_peered":"2023-12-02T00:28:48.731895+0000","last_unstale":"2023-12-02T00:28:50.706430+0000","last_undegraded":"2023-12-02T00:28:50.706430+0000","last_fullsized":"2023-12-02T00:28:50.706430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:55:54.133908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021998+0000","last_change":"2023-12-02T00:28:49.718319+0000","last_active":"2023-12-02T00:28:51.021998+0000","last_peered":"2023-12-02T00:28:51.021998+0000","last_clean":"2023-12-02T00:28:51.021998+0000","last_became_active":"2023-12-02T00:28:49.718045+0000","last_became_peered":"2023-12-02T00:28:49.718045+0000","last_unstale":"2023-12-02T00:28:51.021998+0000","last_undegraded":"2023-12-02T00:28:51.021998+0000","last_fullsized":"2023-12-02T00:28:51.021998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:58:11.596095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703791+0000","last_change":"2023-12-02T00:28:48.728008+0000","last_active":"2023-12-02T00:28:50.703791+0000","last_peered":"2023-12-02T00:28:50.703791+0000","last_clean":"2023-12-02T00:28:50.703791+0000","last_became_active":"2023-12-02T00:28:48.727662+0000","last_became_peered":"2023-12-02T00:28:48.727662+0000","last_unstale":"2023-12-02T00:28:50.703791+0000","last_undegraded":"2023-12-02T00:28:50.703791+0000","last_fullsized":"2023-12-02T00:28:50.703791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:13:35.505091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705692+0000","last_change":"2023-12-02T00:28:49.719391+0000","last_active":"2023-12-02T00:28:50.705692+0000","last_peered":"2023-12-02T00:28:50.705692+0000","last_clean":"2023-12-02T00:28:50.705692+0000","last_became_active":"2023-12-02T00:28:49.719126+0000","last_became_peered":"2023-12-02T00:28:49.719126+0000","last_unstale":"2023-12-02T00:28:50.705692+0000","last_undegraded":"2023-12-02T00:28:50.705692+0000","last_fullsized":"2023-12-02T00:28:50.705692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:47:09.825514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021521+0000","last_change":"2023-12-02T00:28:48.736811+0000","last_active":"2023-12-02T00:28:51.021521+0000","last_peered":"2023-12-02T00:28:51.021521+0000","last_clean":"2023-12-02T00:28:51.021521+0000","last_became_active":"2023-12-02T00:28:48.736576+0000","last_became_peered":"2023-12-02T00:28:48.736576+0000","last_unstale":"2023-12-02T00:28:51.021521+0000","last_undegraded":"2023-12-02T00:28:51.021521+0000","last_fullsized":"2023-12-02T00:28:51.021521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:52:07.682002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021600+0000","last_change":"2023-12-02T00:28:49.717541+0000","last_active":"2023-12-02T00:28:51.021600+0000","last_peered":"2023-12-02T00:28:51.021600+0000","last_clean":"2023-12-02T00:28:51.021600+0000","last_became_active":"2023-12-02T00:28:49.717225+0000","last_became_peered":"2023-12-02T00:28:49.717225+0000","last_unstale":"2023-12-02T00:28:51.021600+0000","last_undegraded":"2023-12-02T00:28:51.021600+0000","last_fullsized":"2023-12-02T00:28:51.021600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:29:16.078935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.022091+0000","last_change":"2023-12-02T00:28:48.725581+0000","last_active":"2023-12-02T00:28:51.022091+0000","last_peered":"2023-12-02T00:28:51.022091+0000","last_clean":"2023-12-02T00:28:51.022091+0000","last_became_active":"2023-12-02T00:28:48.725230+0000","last_became_peered":"2023-12-02T00:28:48.725230+0000","last_unstale":"2023-12-02T00:28:51.022091+0000","last_undegraded":"2023-12-02T00:28:51.022091+0000","last_fullsized":"2023-12-02T00:28:51.022091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:39:30.211229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021761+0000","last_change":"2023-12-02T00:28:49.716591+0000","last_active":"2023-12-02T00:28:51.021761+0000","last_peered":"2023-12-02T00:28:51.021761+0000","last_clean":"2023-12-02T00:28:51.021761+0000","last_became_active":"2023-12-02T00:28:49.716237+0000","last_became_peered":"2023-12-02T00:28:49.716237+0000","last_unstale":"2023-12-02T00:28:51.021761+0000","last_undegraded":"2023-12-02T00:28:51.021761+0000","last_fullsized":"2023-12-02T00:28:51.021761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T12:06:34.965686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706099+0000","last_change":"2023-12-02T00:28:48.734778+0000","last_active":"2023-12-02T00:28:50.706099+0000","last_peered":"2023-12-02T00:28:50.706099+0000","last_clean":"2023-12-02T00:28:50.706099+0000","last_became_active":"2023-12-02T00:28:48.734474+0000","last_became_peered":"2023-12-02T00:28:48.734474+0000","last_unstale":"2023-12-02T00:28:50.706099+0000","last_undegraded":"2023-12-02T00:28:50.706099+0000","last_fullsized":"2023-12-02T00:28:50.706099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:07:51.795216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021342+0000","last_change":"2023-12-02T00:28:49.721274+0000","last_active":"2023-12-02T00:28:51.021342+0000","last_peered":"2023-12-02T00:28:51.021342+0000","last_clean":"2023-12-02T00:28:51.021342+0000","last_became_active":"2023-12-02T00:28:49.719979+0000","last_became_peered":"2023-12-02T00:28:49.719979+0000","last_unstale":"2023-12-02T00:28:51.021342+0000","last_undegraded":"2023-12-02T00:28:51.021342+0000","last_fullsized":"2023-12-02T00:28:51.021342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:50:45.793304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020850+0000","last_change":"2023-12-02T00:28:48.732958+0000","last_active":"2023-12-02T00:28:51.020850+0000","last_peered":"2023-12-02T00:28:51.020850+0000","last_clean":"2023-12-02T00:28:51.020850+0000","last_became_active":"2023-12-02T00:28:48.732651+0000","last_became_peered":"2023-12-02T00:28:48.732651+0000","last_unstale":"2023-12-02T00:28:51.020850+0000","last_undegraded":"2023-12-02T00:28:51.020850+0000","last_fullsized":"2023-12-02T00:28:51.020850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:29:26.381170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020869+0000","last_change":"2023-12-02T00:28:49.712859+0000","last_active":"2023-12-02T00:28:51.020869+0000","last_peered":"2023-12-02T00:28:51.020869+0000","last_clean":"2023-12-02T00:28:51.020869+0000","last_became_active":"2023-12-02T00:28:49.712585+0000","last_became_peered":"2023-12-02T00:28:49.712585+0000","last_unstale":"2023-12-02T00:28:51.020869+0000","last_undegraded":"2023-12-02T00:28:51.020869+0000","last_fullsized":"2023-12-02T00:28:51.020869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T12:06:34.965686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020660+0000","last_change":"2023-12-02T00:28:48.734113+0000","last_active":"2023-12-02T00:28:51.020660+0000","last_peered":"2023-12-02T00:28:51.020660+0000","last_clean":"2023-12-02T00:28:51.020660+0000","last_became_active":"2023-12-02T00:28:48.733749+0000","last_became_peered":"2023-12-02T00:28:48.733749+0000","last_unstale":"2023-12-02T00:28:51.020660+0000","last_undegraded":"2023-12-02T00:28:51.020660+0000","last_fullsized":"2023-12-02T00:28:51.020660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T10:56:02.550120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021416+0000","last_change":"2023-12-02T00:28:49.719477+0000","last_active":"2023-12-02T00:28:51.021416+0000","last_peered":"2023-12-02T00:28:51.021416+0000","last_clean":"2023-12-02T00:28:51.021416+0000","last_became_active":"2023-12-02T00:28:49.718537+0000","last_became_peered":"2023-12-02T00:28:49.718537+0000","last_unstale":"2023-12-02T00:28:51.021416+0000","last_undegraded":"2023-12-02T00:28:51.021416+0000","last_fullsized":"2023-12-02T00:28:51.021416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:15:02.988933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705801+0000","last_change":"2023-12-02T00:28:48.730372+0000","last_active":"2023-12-02T00:28:50.705801+0000","last_peered":"2023-12-02T00:28:50.705801+0000","last_clean":"2023-12-02T00:28:50.705801+0000","last_became_active":"2023-12-02T00:28:48.730051+0000","last_became_peered":"2023-12-02T00:28:48.730051+0000","last_unstale":"2023-12-02T00:28:50.705801+0000","last_undegraded":"2023-12-02T00:28:50.705801+0000","last_fullsized":"2023-12-02T00:28:50.705801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:29:10.550270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021515+0000","last_change":"2023-12-02T00:28:49.714467+0000","last_active":"2023-12-02T00:28:51.021515+0000","last_peered":"2023-12-02T00:28:51.021515+0000","last_clean":"2023-12-02T00:28:51.021515+0000","last_became_active":"2023-12-02T00:28:49.713964+0000","last_became_peered":"2023-12-02T00:28:49.713964+0000","last_unstale":"2023-12-02T00:28:51.021515+0000","last_undegraded":"2023-12-02T00:28:51.021515+0000","last_fullsized":"2023-12-02T00:28:51.021515+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T12:22:41.666941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021286+0000","last_change":"2023-12-02T00:28:48.736447+0000","last_active":"2023-12-02T00:28:51.021286+0000","last_peered":"2023-12-02T00:28:51.021286+0000","last_clean":"2023-12-02T00:28:51.021286+0000","last_became_active":"2023-12-02T00:28:48.736169+0000","last_became_peered":"2023-12-02T00:28:48.736169+0000","last_unstale":"2023-12-02T00:28:51.021286+0000","last_undegraded":"2023-12-02T00:28:51.021286+0000","last_fullsized":"2023-12-02T00:28:51.021286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T09:17:51.234416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021954+0000","last_change":"2023-12-02T00:28:49.719154+0000","last_active":"2023-12-02T00:28:51.021954+0000","last_peered":"2023-12-02T00:28:51.021954+0000","last_clean":"2023-12-02T00:28:51.021954+0000","last_became_active":"2023-12-02T00:28:49.718713+0000","last_became_peered":"2023-12-02T00:28:49.718713+0000","last_unstale":"2023-12-02T00:28:51.021954+0000","last_undegraded":"2023-12-02T00:28:51.021954+0000","last_fullsized":"2023-12-02T00:28:51.021954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:50:17.285633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703845+0000","last_change":"2023-12-02T00:28:48.731504+0000","last_active":"2023-12-02T00:28:50.703845+0000","last_peered":"2023-12-02T00:28:50.703845+0000","last_clean":"2023-12-02T00:28:50.703845+0000","last_became_active":"2023-12-02T00:28:48.731224+0000","last_became_peered":"2023-12-02T00:28:48.731224+0000","last_unstale":"2023-12-02T00:28:50.703845+0000","last_undegraded":"2023-12-02T00:28:50.703845+0000","last_fullsized":"2023-12-02T00:28:50.703845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:55:54.133908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706503+0000","last_change":"2023-12-02T00:28:49.726089+0000","last_active":"2023-12-02T00:28:50.706503+0000","last_peered":"2023-12-02T00:28:50.706503+0000","last_clean":"2023-12-02T00:28:50.706503+0000","last_became_active":"2023-12-02T00:28:49.725482+0000","last_became_peered":"2023-12-02T00:28:49.725482+0000","last_unstale":"2023-12-02T00:28:50.706503+0000","last_undegraded":"2023-12-02T00:28:50.706503+0000","last_fullsized":"2023-12-02T00:28:50.706503+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T12:22:41.666941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020880+0000","last_change":"2023-12-02T00:28:48.729936+0000","last_active":"2023-12-02T00:28:51.020880+0000","last_peered":"2023-12-02T00:28:51.020880+0000","last_clean":"2023-12-02T00:28:51.020880+0000","last_became_active":"2023-12-02T00:28:48.729617+0000","last_became_peered":"2023-12-02T00:28:48.729617+0000","last_unstale":"2023-12-02T00:28:51.020880+0000","last_undegraded":"2023-12-02T00:28:51.020880+0000","last_fullsized":"2023-12-02T00:28:51.020880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:29:26.381170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021803+0000","last_change":"2023-12-02T00:28:49.725211+0000","last_active":"2023-12-02T00:28:51.021803+0000","last_peered":"2023-12-02T00:28:51.021803+0000","last_clean":"2023-12-02T00:28:51.021803+0000","last_became_active":"2023-12-02T00:28:49.724928+0000","last_became_peered":"2023-12-02T00:28:49.724928+0000","last_unstale":"2023-12-02T00:28:51.021803+0000","last_undegraded":"2023-12-02T00:28:51.021803+0000","last_fullsized":"2023-12-02T00:28:51.021803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:33:43.085260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":165,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-02T00:28:49.708269+0000","last_change":"2023-12-02T00:28:39.379380+0000","last_active":"2023-12-02T00:28:49.708269+0000","last_peered":"2023-12-02T00:28:49.708269+0000","last_clean":"2023-12-02T00:28:49.708269+0000","last_became_active":"2023-12-02T00:28:39.378255+0000","last_became_peered":"2023-12-02T00:28:39.378255+0000","last_unstale":"2023-12-02T00:28:49.708269+0000","last_undegraded":"2023-12-02T00:28:49.708269+0000","last_fullsized":"2023-12-02T00:28:49.708269+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:38.177345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:38.177345+0000","last_clean_scrub_stamp":"2023-12-02T00:28:38.177345+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:21:08.582412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706107+0000","last_change":"2023-12-02T00:28:48.731326+0000","last_active":"2023-12-02T00:28:50.706107+0000","last_peered":"2023-12-02T00:28:50.706107+0000","last_clean":"2023-12-02T00:28:50.706107+0000","last_became_active":"2023-12-02T00:28:48.730970+0000","last_became_peered":"2023-12-02T00:28:48.730970+0000","last_unstale":"2023-12-02T00:28:50.706107+0000","last_undegraded":"2023-12-02T00:28:50.706107+0000","last_fullsized":"2023-12-02T00:28:50.706107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:44:45.468077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021403+0000","last_change":"2023-12-02T00:28:49.727341+0000","last_active":"2023-12-02T00:28:51.021403+0000","last_peered":"2023-12-02T00:28:51.021403+0000","last_clean":"2023-12-02T00:28:51.021403+0000","last_became_active":"2023-12-02T00:28:49.727045+0000","last_became_peered":"2023-12-02T00:28:49.727045+0000","last_unstale":"2023-12-02T00:28:51.021403+0000","last_undegraded":"2023-12-02T00:28:51.021403+0000","last_fullsized":"2023-12-02T00:28:51.021403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:58:15.388237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020921+0000","last_change":"2023-12-02T00:28:48.732719+0000","last_active":"2023-12-02T00:28:51.020921+0000","last_peered":"2023-12-02T00:28:51.020921+0000","last_clean":"2023-12-02T00:28:51.020921+0000","last_became_active":"2023-12-02T00:28:48.732466+0000","last_became_peered":"2023-12-02T00:28:48.732466+0000","last_unstale":"2023-12-02T00:28:51.020921+0000","last_undegraded":"2023-12-02T00:28:51.020921+0000","last_fullsized":"2023-12-02T00:28:51.020921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:52:21.593280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021611+0000","last_change":"2023-12-02T00:28:49.716043+0000","last_active":"2023-12-02T00:28:51.021611+0000","last_peered":"2023-12-02T00:28:51.021611+0000","last_clean":"2023-12-02T00:28:51.021611+0000","last_became_active":"2023-12-02T00:28:49.715747+0000","last_became_peered":"2023-12-02T00:28:49.715747+0000","last_unstale":"2023-12-02T00:28:51.021611+0000","last_undegraded":"2023-12-02T00:28:51.021611+0000","last_fullsized":"2023-12-02T00:28:51.021611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:44:46.475452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021208+0000","last_change":"2023-12-02T00:28:48.736024+0000","last_active":"2023-12-02T00:28:51.021208+0000","last_peered":"2023-12-02T00:28:51.021208+0000","last_clean":"2023-12-02T00:28:51.021208+0000","last_became_active":"2023-12-02T00:28:48.735736+0000","last_became_peered":"2023-12-02T00:28:48.735736+0000","last_unstale":"2023-12-02T00:28:51.021208+0000","last_undegraded":"2023-12-02T00:28:51.021208+0000","last_fullsized":"2023-12-02T00:28:51.021208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:55:48.025090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703534+0000","last_change":"2023-12-02T00:28:49.707755+0000","last_active":"2023-12-02T00:28:50.703534+0000","last_peered":"2023-12-02T00:28:50.703534+0000","last_clean":"2023-12-02T00:28:50.703534+0000","last_became_active":"2023-12-02T00:28:49.707449+0000","last_became_peered":"2023-12-02T00:28:49.707449+0000","last_unstale":"2023-12-02T00:28:50.703534+0000","last_undegraded":"2023-12-02T00:28:50.703534+0000","last_fullsized":"2023-12-02T00:28:50.703534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:07:52.802590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706217+0000","last_change":"2023-12-02T00:28:48.729078+0000","last_active":"2023-12-02T00:28:50.706217+0000","last_peered":"2023-12-02T00:28:50.706217+0000","last_clean":"2023-12-02T00:28:50.706217+0000","last_became_active":"2023-12-02T00:28:48.728567+0000","last_became_peered":"2023-12-02T00:28:48.728567+0000","last_unstale":"2023-12-02T00:28:50.706217+0000","last_undegraded":"2023-12-02T00:28:50.706217+0000","last_fullsized":"2023-12-02T00:28:50.706217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:58:10.588720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705483+0000","last_change":"2023-12-02T00:28:49.712801+0000","last_active":"2023-12-02T00:28:50.705483+0000","last_peered":"2023-12-02T00:28:50.705483+0000","last_clean":"2023-12-02T00:28:50.705483+0000","last_became_active":"2023-12-02T00:28:49.712511+0000","last_became_peered":"2023-12-02T00:28:49.712511+0000","last_unstale":"2023-12-02T00:28:50.705483+0000","last_undegraded":"2023-12-02T00:28:50.705483+0000","last_fullsized":"2023-12-02T00:28:50.705483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:55:49.032465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703749+0000","last_change":"2023-12-02T00:28:48.730774+0000","last_active":"2023-12-02T00:28:50.703749+0000","last_peered":"2023-12-02T00:28:50.703749+0000","last_clean":"2023-12-02T00:28:50.703749+0000","last_became_active":"2023-12-02T00:28:48.730498+0000","last_became_peered":"2023-12-02T00:28:48.730498+0000","last_unstale":"2023-12-02T00:28:50.703749+0000","last_undegraded":"2023-12-02T00:28:50.703749+0000","last_fullsized":"2023-12-02T00:28:50.703749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:29:10.550270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706537+0000","last_change":"2023-12-02T00:28:49.726928+0000","last_active":"2023-12-02T00:28:50.706537+0000","last_peered":"2023-12-02T00:28:50.706537+0000","last_clean":"2023-12-02T00:28:50.706537+0000","last_became_active":"2023-12-02T00:28:49.725033+0000","last_became_peered":"2023-12-02T00:28:49.725033+0000","last_unstale":"2023-12-02T00:28:50.706537+0000","last_undegraded":"2023-12-02T00:28:50.706537+0000","last_fullsized":"2023-12-02T00:28:50.706537+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:33:43.085260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021906+0000","last_change":"2023-12-02T00:28:48.735559+0000","last_active":"2023-12-02T00:28:51.021906+0000","last_peered":"2023-12-02T00:28:51.021906+0000","last_clean":"2023-12-02T00:28:51.021906+0000","last_became_active":"2023-12-02T00:28:48.735243+0000","last_became_peered":"2023-12-02T00:28:48.735243+0000","last_unstale":"2023-12-02T00:28:51.021906+0000","last_undegraded":"2023-12-02T00:28:51.021906+0000","last_fullsized":"2023-12-02T00:28:51.021906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:07:51.795216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703595+0000","last_change":"2023-12-02T00:28:49.710546+0000","last_active":"2023-12-02T00:28:50.703595+0000","last_peered":"2023-12-02T00:28:50.703595+0000","last_clean":"2023-12-02T00:28:50.703595+0000","last_became_active":"2023-12-02T00:28:49.710010+0000","last_became_peered":"2023-12-02T00:28:49.710010+0000","last_unstale":"2023-12-02T00:28:50.703595+0000","last_undegraded":"2023-12-02T00:28:50.703595+0000","last_fullsized":"2023-12-02T00:28:50.703595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:58:11.596095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.172997+0000","last_change":"2023-12-02T00:28:48.731556+0000","last_active":"2023-12-02T00:28:51.172997+0000","last_peered":"2023-12-02T00:28:51.172997+0000","last_clean":"2023-12-02T00:28:51.172997+0000","last_became_active":"2023-12-02T00:28:48.731291+0000","last_became_peered":"2023-12-02T00:28:48.731291+0000","last_unstale":"2023-12-02T00:28:51.172997+0000","last_undegraded":"2023-12-02T00:28:51.172997+0000","last_fullsized":"2023-12-02T00:28:51.172997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:39:30.211229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020929+0000","last_change":"2023-12-02T00:28:49.723678+0000","last_active":"2023-12-02T00:28:51.020929+0000","last_peered":"2023-12-02T00:28:51.020929+0000","last_clean":"2023-12-02T00:28:51.020929+0000","last_became_active":"2023-12-02T00:28:49.723299+0000","last_became_peered":"2023-12-02T00:28:49.723299+0000","last_unstale":"2023-12-02T00:28:51.020929+0000","last_undegraded":"2023-12-02T00:28:51.020929+0000","last_fullsized":"2023-12-02T00:28:51.020929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:58:11.596095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021044+0000","last_change":"2023-12-02T00:28:48.735741+0000","last_active":"2023-12-02T00:28:51.021044+0000","last_peered":"2023-12-02T00:28:51.021044+0000","last_clean":"2023-12-02T00:28:51.021044+0000","last_became_active":"2023-12-02T00:28:48.735400+0000","last_became_peered":"2023-12-02T00:28:48.735400+0000","last_unstale":"2023-12-02T00:28:51.021044+0000","last_undegraded":"2023-12-02T00:28:51.021044+0000","last_fullsized":"2023-12-02T00:28:51.021044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:13:35.505091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.022028+0000","last_change":"2023-12-02T00:28:49.711592+0000","last_active":"2023-12-02T00:28:51.022028+0000","last_peered":"2023-12-02T00:28:51.022028+0000","last_clean":"2023-12-02T00:28:51.022028+0000","last_became_active":"2023-12-02T00:28:49.711211+0000","last_became_peered":"2023-12-02T00:28:49.711211+0000","last_unstale":"2023-12-02T00:28:51.022028+0000","last_undegraded":"2023-12-02T00:28:51.022028+0000","last_fullsized":"2023-12-02T00:28:51.022028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:15:51.747629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020816+0000","last_change":"2023-12-02T00:28:48.735722+0000","last_active":"2023-12-02T00:28:51.020816+0000","last_peered":"2023-12-02T00:28:51.020816+0000","last_clean":"2023-12-02T00:28:51.020816+0000","last_became_active":"2023-12-02T00:28:48.735212+0000","last_became_peered":"2023-12-02T00:28:48.735212+0000","last_unstale":"2023-12-02T00:28:51.020816+0000","last_undegraded":"2023-12-02T00:28:51.020816+0000","last_fullsized":"2023-12-02T00:28:51.020816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:47:08.818139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705309+0000","last_change":"2023-12-02T00:28:49.711824+0000","last_active":"2023-12-02T00:28:50.705309+0000","last_peered":"2023-12-02T00:28:50.705309+0000","last_clean":"2023-12-02T00:28:50.705309+0000","last_became_active":"2023-12-02T00:28:49.710946+0000","last_became_peered":"2023-12-02T00:28:49.710946+0000","last_unstale":"2023-12-02T00:28:50.705309+0000","last_undegraded":"2023-12-02T00:28:50.705309+0000","last_fullsized":"2023-12-02T00:28:50.705309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T09:17:52.241790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703622+0000","last_change":"2023-12-02T00:28:48.727127+0000","last_active":"2023-12-02T00:28:50.703622+0000","last_peered":"2023-12-02T00:28:50.703622+0000","last_clean":"2023-12-02T00:28:50.703622+0000","last_became_active":"2023-12-02T00:28:48.726750+0000","last_became_peered":"2023-12-02T00:28:48.726750+0000","last_unstale":"2023-12-02T00:28:50.703622+0000","last_undegraded":"2023-12-02T00:28:50.703622+0000","last_fullsized":"2023-12-02T00:28:50.703622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:44:45.468077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021415+0000","last_change":"2023-12-02T00:28:49.718158+0000","last_active":"2023-12-02T00:28:51.021415+0000","last_peered":"2023-12-02T00:28:51.021415+0000","last_clean":"2023-12-02T00:28:51.021415+0000","last_became_active":"2023-12-02T00:28:49.717721+0000","last_became_peered":"2023-12-02T00:28:49.717721+0000","last_unstale":"2023-12-02T00:28:51.021415+0000","last_undegraded":"2023-12-02T00:28:51.021415+0000","last_fullsized":"2023-12-02T00:28:51.021415+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T10:27:29.244181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703765+0000","last_change":"2023-12-02T00:28:48.735894+0000","last_active":"2023-12-02T00:28:50.703765+0000","last_peered":"2023-12-02T00:28:50.703765+0000","last_clean":"2023-12-02T00:28:50.703765+0000","last_became_active":"2023-12-02T00:28:48.735613+0000","last_became_peered":"2023-12-02T00:28:48.735613+0000","last_unstale":"2023-12-02T00:28:50.703765+0000","last_undegraded":"2023-12-02T00:28:50.703765+0000","last_fullsized":"2023-12-02T00:28:50.703765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:52:07.682002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021464+0000","last_change":"2023-12-02T00:28:49.719314+0000","last_active":"2023-12-02T00:28:51.021464+0000","last_peered":"2023-12-02T00:28:51.021464+0000","last_clean":"2023-12-02T00:28:51.021464+0000","last_became_active":"2023-12-02T00:28:49.718940+0000","last_became_peered":"2023-12-02T00:28:49.718940+0000","last_unstale":"2023-12-02T00:28:51.021464+0000","last_undegraded":"2023-12-02T00:28:51.021464+0000","last_fullsized":"2023-12-02T00:28:51.021464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:50:45.793304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021457+0000","last_change":"2023-12-02T00:28:48.731948+0000","last_active":"2023-12-02T00:28:51.021457+0000","last_peered":"2023-12-02T00:28:51.021457+0000","last_clean":"2023-12-02T00:28:51.021457+0000","last_became_active":"2023-12-02T00:28:48.731641+0000","last_became_peered":"2023-12-02T00:28:48.731641+0000","last_unstale":"2023-12-02T00:28:51.021457+0000","last_undegraded":"2023-12-02T00:28:51.021457+0000","last_fullsized":"2023-12-02T00:28:51.021457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:36:21.580553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021037+0000","last_change":"2023-12-02T00:28:49.715513+0000","last_active":"2023-12-02T00:28:51.021037+0000","last_peered":"2023-12-02T00:28:51.021037+0000","last_clean":"2023-12-02T00:28:51.021037+0000","last_became_active":"2023-12-02T00:28:49.715240+0000","last_became_peered":"2023-12-02T00:28:49.715240+0000","last_unstale":"2023-12-02T00:28:51.021037+0000","last_undegraded":"2023-12-02T00:28:51.021037+0000","last_fullsized":"2023-12-02T00:28:51.021037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:52:22.600655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706521+0000","last_change":"2023-12-02T00:28:49.717785+0000","last_active":"2023-12-02T00:28:50.706521+0000","last_peered":"2023-12-02T00:28:50.706521+0000","last_clean":"2023-12-02T00:28:50.706521+0000","last_became_active":"2023-12-02T00:28:49.717408+0000","last_became_peered":"2023-12-02T00:28:49.717408+0000","last_unstale":"2023-12-02T00:28:50.706521+0000","last_undegraded":"2023-12-02T00:28:50.706521+0000","last_fullsized":"2023-12-02T00:28:50.706521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:29:16.078935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021429+0000","last_change":"2023-12-02T00:28:48.729768+0000","last_active":"2023-12-02T00:28:51.021429+0000","last_peered":"2023-12-02T00:28:51.021429+0000","last_clean":"2023-12-02T00:28:51.021429+0000","last_became_active":"2023-12-02T00:28:48.729466+0000","last_became_peered":"2023-12-02T00:28:48.729466+0000","last_unstale":"2023-12-02T00:28:51.021429+0000","last_undegraded":"2023-12-02T00:28:51.021429+0000","last_fullsized":"2023-12-02T00:28:51.021429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T12:06:33.958311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706378+0000","last_change":"2023-12-02T00:28:49.727325+0000","last_active":"2023-12-02T00:28:50.706378+0000","last_peered":"2023-12-02T00:28:50.706378+0000","last_clean":"2023-12-02T00:28:50.706378+0000","last_became_active":"2023-12-02T00:28:49.727062+0000","last_became_peered":"2023-12-02T00:28:49.727062+0000","last_unstale":"2023-12-02T00:28:50.706378+0000","last_undegraded":"2023-12-02T00:28:50.706378+0000","last_fullsized":"2023-12-02T00:28:50.706378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:15:02.988933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021734+0000","last_change":"2023-12-02T00:28:48.732989+0000","last_active":"2023-12-02T00:28:51.021734+0000","last_peered":"2023-12-02T00:28:51.021734+0000","last_clean":"2023-12-02T00:28:51.021734+0000","last_became_active":"2023-12-02T00:28:48.731645+0000","last_became_peered":"2023-12-02T00:28:48.731645+0000","last_unstale":"2023-12-02T00:28:51.021734+0000","last_undegraded":"2023-12-02T00:28:51.021734+0000","last_fullsized":"2023-12-02T00:28:51.021734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:44:45.468077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021923+0000","last_change":"2023-12-02T00:28:49.725372+0000","last_active":"2023-12-02T00:28:51.021923+0000","last_peered":"2023-12-02T00:28:51.021923+0000","last_clean":"2023-12-02T00:28:51.021923+0000","last_became_active":"2023-12-02T00:28:49.724926+0000","last_became_peered":"2023-12-02T00:28:49.724926+0000","last_unstale":"2023-12-02T00:28:51.021923+0000","last_undegraded":"2023-12-02T00:28:51.021923+0000","last_fullsized":"2023-12-02T00:28:51.021923+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:07:52.802590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021204+0000","last_change":"2023-12-02T00:28:48.731302+0000","last_active":"2023-12-02T00:28:51.021204+0000","last_peered":"2023-12-02T00:28:51.021204+0000","last_clean":"2023-12-02T00:28:51.021204+0000","last_became_active":"2023-12-02T00:28:48.731002+0000","last_became_peered":"2023-12-02T00:28:48.731002+0000","last_unstale":"2023-12-02T00:28:51.021204+0000","last_undegraded":"2023-12-02T00:28:51.021204+0000","last_fullsized":"2023-12-02T00:28:51.021204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:29:26.381170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021221+0000","last_change":"2023-12-02T00:28:49.721605+0000","last_active":"2023-12-02T00:28:51.021221+0000","last_peered":"2023-12-02T00:28:51.021221+0000","last_clean":"2023-12-02T00:28:51.021221+0000","last_became_active":"2023-12-02T00:28:49.721035+0000","last_became_peered":"2023-12-02T00:28:49.721035+0000","last_unstale":"2023-12-02T00:28:51.021221+0000","last_undegraded":"2023-12-02T00:28:51.021221+0000","last_fullsized":"2023-12-02T00:28:51.021221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:15:51.747629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.022018+0000","last_change":"2023-12-02T00:28:48.729331+0000","last_active":"2023-12-02T00:28:51.022018+0000","last_peered":"2023-12-02T00:28:51.022018+0000","last_clean":"2023-12-02T00:28:51.022018+0000","last_became_active":"2023-12-02T00:28:48.728469+0000","last_became_peered":"2023-12-02T00:28:48.728469+0000","last_unstale":"2023-12-02T00:28:51.022018+0000","last_undegraded":"2023-12-02T00:28:51.022018+0000","last_fullsized":"2023-12-02T00:28:51.022018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:47:08.818139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":815104,"data_stored":666531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95199999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":77193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":75671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":75707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":77193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28299999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":212992,"data_stored":74962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":225280,"data_stored":75741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":819200,"data_stored":665968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.377}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-02T00:29:01.565 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-02T00:29:01.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.706+0000 7ffbba13c640 1 Processor -- start 2023-12-02T00:29:01.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.706+0000 7ffbba13c640 1 -- start start 2023-12-02T00:29:01.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.707+0000 7ffbba13c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffbb4143490 0x7ffbb40b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:01.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.707+0000 7ffbba13c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffbb415e730 0x7ffbb40b1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:01.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.707+0000 7ffbba13c640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbb415e2a0 con 0x7ffbb415a4e0 2023-12-02T00:29:01.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.707+0000 7ffbba13c640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbb415b520 con 0x7ffbb415e730 2023-12-02T00:29:01.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.707+0000 7ffbba13c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb4143870 con 0x7ffbb4143490 2023-12-02T00:29:01.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.707+0000 7ffbb2ffd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffbb415e730 0x7ffbb40b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:01.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.707+0000 7ffbb2ffd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffbb415e730 0x7ffbb40b1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38268/0 (socket says 172.21.15.46:38268) 2023-12-02T00:29:01.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.707+0000 7ffbb2ffd640 1 -- 172.21.15.46:0/1589321008 learned_addr learned my addr 172.21.15.46:0/1589321008 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:01.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.707+0000 7ffbb37fe640 1 --2- 172.21.15.46:0/1589321008 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffbb4143490 0x7ffbb40b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:01.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.708+0000 7ffbb2ffd640 1 -- 172.21.15.46:0/1589321008 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffbb4143490 msgr2=0x7ffbb40b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:01.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.708+0000 7ffbb2ffd640 1 --2- 172.21.15.46:0/1589321008 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffbb4143490 0x7ffbb40b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.708+0000 7ffbb2ffd640 1 -- 172.21.15.46:0/1589321008 >> v1:172.21.15.162:6789/0 conn(0x7ffbb415a4e0 legacy=0x7ffbb415a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:01.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.708+0000 7ffbb2ffd640 1 -- 172.21.15.46:0/1589321008 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb415b240 con 0x7ffbb415e730 2023-12-02T00:29:01.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.708+0000 7ffbb37fe640 1 --2- 172.21.15.46:0/1589321008 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffbb4143490 0x7ffbb40b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:29:01.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.708+0000 7ffbb2ffd640 1 --2- 172.21.15.46:0/1589321008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbb415e730 0x7ffbb40b1230 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ffba8009870 tx=0x7ffba802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c09c383a1df144d3 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:01.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.708+0000 7ffbb27fc640 1 -- 172.21.15.46:0/1589321008 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba8031640 con 0x7ffbb415e730 2023-12-02T00:29:01.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.708+0000 7ffbb27fc640 1 -- 172.21.15.46:0/1589321008 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba80317a0 con 0x7ffbb415e730 2023-12-02T00:29:01.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.708+0000 7ffbb27fc640 1 -- 172.21.15.46:0/1589321008 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba8031b30 con 0x7ffbb415e730 2023-12-02T00:29:01.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.709+0000 7ffbba13c640 1 -- 172.21.15.46:0/1589321008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbb415e730 msgr2=0x7ffbb40b1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:01.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.709+0000 7ffbba13c640 1 --2- 172.21.15.46:0/1589321008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbb415e730 0x7ffbb40b1230 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ffba8009870 tx=0x7ffba802b8f0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.709+0000 7ffbba13c640 1 -- 172.21.15.46:0/1589321008 shutdown_connections 2023-12-02T00:29:01.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.709+0000 7ffbba13c640 1 --2- 172.21.15.46:0/1589321008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbb415e730 0x7ffbb40b1230 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.709+0000 7ffbba13c640 1 --2- 172.21.15.46:0/1589321008 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffbb4143490 0x7ffbb40b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.709+0000 7ffbba13c640 1 -- 172.21.15.46:0/1589321008 >> 172.21.15.46:0/1589321008 conn(0x7ffbb406a4d0 msgr2=0x7ffbb406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:01.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.709+0000 7ffbba13c640 1 -- 172.21.15.46:0/1589321008 shutdown_connections 2023-12-02T00:29:01.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.709+0000 7ffbba13c640 1 -- 172.21.15.46:0/1589321008 wait complete. 2023-12-02T00:29:01.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.710+0000 7ffbba13c640 1 Processor -- start 2023-12-02T00:29:01.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.710+0000 7ffbba13c640 1 -- start start 2023-12-02T00:29:01.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.711+0000 7ffbba13c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbb4143490 0x7ffbb4159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:01.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.711+0000 7ffbba13c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffbb415a4e0 0x7ffbb40a9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:01.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.711+0000 7ffbba13c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffbb415e730 0x7ffbb4070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:01.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.711+0000 7ffbba13c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb40ca420 con 0x7ffbb4143490 2023-12-02T00:29:01.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.711+0000 7ffbba13c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb40ca700 con 0x7ffbb415a4e0 2023-12-02T00:29:01.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.711+0000 7ffbba13c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb40ca9e0 con 0x7ffbb415e730 2023-12-02T00:29:01.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.711+0000 7ffbb37fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbb4143490 0x7ffbb4159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:01.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.711+0000 7ffbb37fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbb4143490 0x7ffbb4159b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38284/0 (socket says 172.21.15.46:38284) 2023-12-02T00:29:01.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.711+0000 7ffbb37fe640 1 -- 172.21.15.46:0/20436403 learned_addr learned my addr 172.21.15.46:0/20436403 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:01.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.711+0000 7ffbb2ffd640 1 --2- 172.21.15.46:0/20436403 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffbb415a4e0 0x7ffbb40a9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:01.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.711+0000 7ffbb37fe640 1 -- 172.21.15.46:0/20436403 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffbb415e730 msgr2=0x7ffbb4070390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:01.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.711+0000 7ffbb3fff640 1 --2- 172.21.15.46:0/20436403 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffbb415e730 0x7ffbb4070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:01.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.711+0000 7ffbb37fe640 1 --2- 172.21.15.46:0/20436403 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffbb415e730 0x7ffbb4070390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.711+0000 7ffbb37fe640 1 -- 172.21.15.46:0/20436403 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffbb415a4e0 msgr2=0x7ffbb40a9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:01.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.711+0000 7ffbb37fe640 1 --2- 172.21.15.46:0/20436403 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffbb415a4e0 0x7ffbb40a9f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.712+0000 7ffbb37fe640 1 -- 172.21.15.46:0/20436403 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba80094d0 con 0x7ffbb4143490 2023-12-02T00:29:01.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.712+0000 7ffbb37fe640 1 --2- 172.21.15.46:0/20436403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbb4143490 0x7ffbb4159b10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ffba400c7f0 tx=0x7ffba400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:01.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.712+0000 7ffb9bfff640 1 -- 172.21.15.46:0/20436403 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba4007a30 con 0x7ffbb4143490 2023-12-02T00:29:01.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.712+0000 7ffb9bfff640 1 -- 172.21.15.46:0/20436403 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba4007b90 con 0x7ffbb4143490 2023-12-02T00:29:01.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.712+0000 7ffbba13c640 1 -- 172.21.15.46:0/20436403 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbb40cacc0 con 0x7ffbb4143490 2023-12-02T00:29:01.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.712+0000 7ffb9bfff640 1 -- 172.21.15.46:0/20436403 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba400ce40 con 0x7ffbb4143490 2023-12-02T00:29:01.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.712+0000 7ffbba13c640 1 -- 172.21.15.46:0/20436403 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbb40ca060 con 0x7ffbb4143490 2023-12-02T00:29:01.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.713+0000 7ffbba13c640 1 -- 172.21.15.46:0/20436403 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbb40b4380 con 0x7ffbb4143490 2023-12-02T00:29:01.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.713+0000 7ffb9bfff640 1 -- 172.21.15.46:0/20436403 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ffba40113f0 con 0x7ffbb4143490 2023-12-02T00:29:01.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.714+0000 7ffb9bfff640 1 --2- 172.21.15.46:0/20436403 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ffba006ae40 0x7ffba006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:01.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.714+0000 7ffb9bfff640 1 -- 172.21.15.46:0/20436403 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffba4089800 con 0x7ffbb4143490 2023-12-02T00:29:01.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.714+0000 7ffbb2ffd640 1 --2- 172.21.15.46:0/20436403 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ffba006ae40 0x7ffba006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:01.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.714+0000 7ffbb2ffd640 1 --2- 172.21.15.46:0/20436403 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ffba006ae40 0x7ffba006d300 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ffba8036000 tx=0x7ffba803c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:01.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.718+0000 7ffb9bfff640 1 -- 172.21.15.46:0/20436403 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ffba40548b0 con 0x7ffbb4143490 2023-12-02T00:29:01.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.889+0000 7ffbba13c640 1 -- 172.21.15.46:0/20436403 --> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffbb415df30 con 0x7ffba006ae40 2023-12-02T00:29:01.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.896+0000 7ffb9bfff640 1 -- 172.21.15.46:0/20436403 <== mgr.4108 v2:172.21.15.162:6832/3248644722 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264578 (secure 0 0 0) 0x7ffba8031a90 con 0x7ffba006ae40 2023-12-02T00:29:01.900 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:29:01.901 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-12-02T00:29:01.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.899+0000 7ffbba13c640 1 -- 172.21.15.46:0/20436403 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ffba006ae40 msgr2=0x7ffba006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:01.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.899+0000 7ffbba13c640 1 --2- 172.21.15.46:0/20436403 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ffba006ae40 0x7ffba006d300 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ffba8036000 tx=0x7ffba803c040 comp rx=0 tx=0).stop 2023-12-02T00:29:01.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.899+0000 7ffbba13c640 1 -- 172.21.15.46:0/20436403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbb4143490 msgr2=0x7ffbb4159b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:01.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.899+0000 7ffbba13c640 1 --2- 172.21.15.46:0/20436403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbb4143490 0x7ffbb4159b10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ffba400c7f0 tx=0x7ffba400cbf0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.899+0000 7ffbba13c640 1 -- 172.21.15.46:0/20436403 shutdown_connections 2023-12-02T00:29:01.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.899+0000 7ffbba13c640 1 --2- 172.21.15.46:0/20436403 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ffba006ae40 0x7ffba006d300 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.899+0000 7ffbba13c640 1 --2- 172.21.15.46:0/20436403 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffbb415e730 0x7ffbb4070390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.899+0000 7ffbba13c640 1 --2- 172.21.15.46:0/20436403 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffbb415a4e0 0x7ffbb40a9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.899+0000 7ffbba13c640 1 --2- 172.21.15.46:0/20436403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbb4143490 0x7ffbb4159b10 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:01.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.899+0000 7ffbba13c640 1 -- 172.21.15.46:0/20436403 >> 172.21.15.46:0/20436403 conn(0x7ffbb406a4d0 msgr2=0x7ffbb415c070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:01.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.900+0000 7ffbba13c640 1 -- 172.21.15.46:0/20436403 shutdown_connections 2023-12-02T00:29:01.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:01.900+0000 7ffbba13c640 1 -- 172.21.15.46:0/20436403 wait complete. 2023-12-02T00:29:01.912 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-12-02T00:29:01.161031+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235744,"kb_used_data":2912,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738976,"statfs":{"total":773094113280,"available":772852711424,"internally_reserved":0,"allocated":2981888,"data_stored":1788966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.457779"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021839+0000","last_change":"2023-12-02T00:28:49.717914+0000","last_active":"2023-12-02T00:28:51.021839+0000","last_peered":"2023-12-02T00:28:51.021839+0000","last_clean":"2023-12-02T00:28:51.021839+0000","last_became_active":"2023-12-02T00:28:49.717594+0000","last_became_peered":"2023-12-02T00:28:49.717594+0000","last_unstale":"2023-12-02T00:28:51.021839+0000","last_undegraded":"2023-12-02T00:28:51.021839+0000","last_fullsized":"2023-12-02T00:28:51.021839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:48:16.850673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703744+0000","last_change":"2023-12-02T00:28:49.713605+0000","last_active":"2023-12-02T00:28:50.703744+0000","last_peered":"2023-12-02T00:28:50.703744+0000","last_clean":"2023-12-02T00:28:50.703744+0000","last_became_active":"2023-12-02T00:28:49.712497+0000","last_became_peered":"2023-12-02T00:28:49.712497+0000","last_unstale":"2023-12-02T00:28:50.703744+0000","last_undegraded":"2023-12-02T00:28:50.703744+0000","last_fullsized":"2023-12-02T00:28:50.703744+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:15:51.747629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021917+0000","last_change":"2023-12-02T00:28:49.713065+0000","last_active":"2023-12-02T00:28:51.021917+0000","last_peered":"2023-12-02T00:28:51.021917+0000","last_clean":"2023-12-02T00:28:51.021917+0000","last_became_active":"2023-12-02T00:28:49.712715+0000","last_became_peered":"2023-12-02T00:28:49.712715+0000","last_unstale":"2023-12-02T00:28:51.021917+0000","last_undegraded":"2023-12-02T00:28:51.021917+0000","last_fullsized":"2023-12-02T00:28:51.021917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:52:22.600655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706337+0000","last_change":"2023-12-02T00:28:49.727831+0000","last_active":"2023-12-02T00:28:50.706337+0000","last_peered":"2023-12-02T00:28:50.706337+0000","last_clean":"2023-12-02T00:28:50.706337+0000","last_became_active":"2023-12-02T00:28:49.725804+0000","last_became_peered":"2023-12-02T00:28:49.725804+0000","last_unstale":"2023-12-02T00:28:50.706337+0000","last_undegraded":"2023-12-02T00:28:50.706337+0000","last_fullsized":"2023-12-02T00:28:50.706337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:50:45.793304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021332+0000","last_change":"2023-12-02T00:28:49.721391+0000","last_active":"2023-12-02T00:28:51.021332+0000","last_peered":"2023-12-02T00:28:51.021332+0000","last_clean":"2023-12-02T00:28:51.021332+0000","last_became_active":"2023-12-02T00:28:49.720314+0000","last_became_peered":"2023-12-02T00:28:49.720314+0000","last_unstale":"2023-12-02T00:28:51.021332+0000","last_undegraded":"2023-12-02T00:28:51.021332+0000","last_fullsized":"2023-12-02T00:28:51.021332+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:32:13.515968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705990+0000","last_change":"2023-12-02T00:28:49.725516+0000","last_active":"2023-12-02T00:28:50.705990+0000","last_peered":"2023-12-02T00:28:50.705990+0000","last_clean":"2023-12-02T00:28:50.705990+0000","last_became_active":"2023-12-02T00:28:49.725149+0000","last_became_peered":"2023-12-02T00:28:49.725149+0000","last_unstale":"2023-12-02T00:28:50.705990+0000","last_undegraded":"2023-12-02T00:28:50.705990+0000","last_fullsized":"2023-12-02T00:28:50.705990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T10:27:29.244181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021273+0000","last_change":"2023-12-02T00:28:49.719153+0000","last_active":"2023-12-02T00:28:51.021273+0000","last_peered":"2023-12-02T00:28:51.021273+0000","last_clean":"2023-12-02T00:28:51.021273+0000","last_became_active":"2023-12-02T00:28:49.718567+0000","last_became_peered":"2023-12-02T00:28:49.718567+0000","last_unstale":"2023-12-02T00:28:51.021273+0000","last_undegraded":"2023-12-02T00:28:51.021273+0000","last_fullsized":"2023-12-02T00:28:51.021273+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T10:27:29.244181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020683+0000","last_change":"2023-12-02T00:28:49.726986+0000","last_active":"2023-12-02T00:28:51.020683+0000","last_peered":"2023-12-02T00:28:51.020683+0000","last_clean":"2023-12-02T00:28:51.020683+0000","last_became_active":"2023-12-02T00:28:49.726646+0000","last_became_peered":"2023-12-02T00:28:49.726646+0000","last_unstale":"2023-12-02T00:28:51.020683+0000","last_undegraded":"2023-12-02T00:28:51.020683+0000","last_fullsized":"2023-12-02T00:28:51.020683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:35:09.231704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021827+0000","last_change":"2023-12-02T00:28:49.725402+0000","last_active":"2023-12-02T00:28:51.021827+0000","last_peered":"2023-12-02T00:28:51.021827+0000","last_clean":"2023-12-02T00:28:51.021827+0000","last_became_active":"2023-12-02T00:28:49.724978+0000","last_became_peered":"2023-12-02T00:28:49.724978+0000","last_unstale":"2023-12-02T00:28:51.021827+0000","last_undegraded":"2023-12-02T00:28:51.021827+0000","last_fullsized":"2023-12-02T00:28:51.021827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:18:31.615464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021500+0000","last_change":"2023-12-02T00:28:49.718464+0000","last_active":"2023-12-02T00:28:51.021500+0000","last_peered":"2023-12-02T00:28:51.021500+0000","last_clean":"2023-12-02T00:28:51.021500+0000","last_became_active":"2023-12-02T00:28:49.718060+0000","last_became_peered":"2023-12-02T00:28:49.718060+0000","last_unstale":"2023-12-02T00:28:51.021500+0000","last_undegraded":"2023-12-02T00:28:51.021500+0000","last_fullsized":"2023-12-02T00:28:51.021500+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:36:22.587928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021177+0000","last_change":"2023-12-02T00:28:49.720030+0000","last_active":"2023-12-02T00:28:51.021177+0000","last_peered":"2023-12-02T00:28:51.021177+0000","last_clean":"2023-12-02T00:28:51.021177+0000","last_became_active":"2023-12-02T00:28:49.719715+0000","last_became_peered":"2023-12-02T00:28:49.719715+0000","last_unstale":"2023-12-02T00:28:51.021177+0000","last_undegraded":"2023-12-02T00:28:51.021177+0000","last_fullsized":"2023-12-02T00:28:51.021177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:07:52.802590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705569+0000","last_change":"2023-12-02T00:28:49.713687+0000","last_active":"2023-12-02T00:28:50.705569+0000","last_peered":"2023-12-02T00:28:50.705569+0000","last_clean":"2023-12-02T00:28:50.705569+0000","last_became_active":"2023-12-02T00:28:49.713359+0000","last_became_peered":"2023-12-02T00:28:49.713359+0000","last_unstale":"2023-12-02T00:28:50.705569+0000","last_undegraded":"2023-12-02T00:28:50.705569+0000","last_fullsized":"2023-12-02T00:28:50.705569+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:11:06.388890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021465+0000","last_change":"2023-12-02T00:28:49.720866+0000","last_active":"2023-12-02T00:28:51.021465+0000","last_peered":"2023-12-02T00:28:51.021465+0000","last_clean":"2023-12-02T00:28:51.021465+0000","last_became_active":"2023-12-02T00:28:49.720524+0000","last_became_peered":"2023-12-02T00:28:49.720524+0000","last_unstale":"2023-12-02T00:28:51.021465+0000","last_undegraded":"2023-12-02T00:28:51.021465+0000","last_fullsized":"2023-12-02T00:28:51.021465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:32:13.515968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021720+0000","last_change":"2023-12-02T00:28:49.711387+0000","last_active":"2023-12-02T00:28:51.021720+0000","last_peered":"2023-12-02T00:28:51.021720+0000","last_clean":"2023-12-02T00:28:51.021720+0000","last_became_active":"2023-12-02T00:28:49.711036+0000","last_became_peered":"2023-12-02T00:28:49.711036+0000","last_unstale":"2023-12-02T00:28:51.021720+0000","last_undegraded":"2023-12-02T00:28:51.021720+0000","last_fullsized":"2023-12-02T00:28:51.021720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:13:36.512466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703511+0000","last_change":"2023-12-02T00:28:49.706444+0000","last_active":"2023-12-02T00:28:50.703511+0000","last_peered":"2023-12-02T00:28:50.703511+0000","last_clean":"2023-12-02T00:28:50.703511+0000","last_became_active":"2023-12-02T00:28:49.706132+0000","last_became_peered":"2023-12-02T00:28:49.706132+0000","last_unstale":"2023-12-02T00:28:50.703511+0000","last_undegraded":"2023-12-02T00:28:50.703511+0000","last_fullsized":"2023-12-02T00:28:50.703511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:52:22.600655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705451+0000","last_change":"2023-12-02T00:28:49.725045+0000","last_active":"2023-12-02T00:28:50.705451+0000","last_peered":"2023-12-02T00:28:50.705451+0000","last_clean":"2023-12-02T00:28:50.705451+0000","last_became_active":"2023-12-02T00:28:49.722884+0000","last_became_peered":"2023-12-02T00:28:49.722884+0000","last_unstale":"2023-12-02T00:28:50.705451+0000","last_undegraded":"2023-12-02T00:28:50.705451+0000","last_fullsized":"2023-12-02T00:28:50.705451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:36:22.587928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706621+0000","last_change":"2023-12-02T00:28:49.727232+0000","last_active":"2023-12-02T00:28:50.706621+0000","last_peered":"2023-12-02T00:28:50.706621+0000","last_clean":"2023-12-02T00:28:50.706621+0000","last_became_active":"2023-12-02T00:28:49.726881+0000","last_became_peered":"2023-12-02T00:28:49.726881+0000","last_unstale":"2023-12-02T00:28:50.706621+0000","last_undegraded":"2023-12-02T00:28:50.706621+0000","last_fullsized":"2023-12-02T00:28:50.706621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:48:16.850673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021328+0000","last_change":"2023-12-02T00:28:49.711953+0000","last_active":"2023-12-02T00:28:51.021328+0000","last_peered":"2023-12-02T00:28:51.021328+0000","last_clean":"2023-12-02T00:28:51.021328+0000","last_became_active":"2023-12-02T00:28:49.711653+0000","last_became_peered":"2023-12-02T00:28:49.711653+0000","last_unstale":"2023-12-02T00:28:51.021328+0000","last_undegraded":"2023-12-02T00:28:51.021328+0000","last_fullsized":"2023-12-02T00:28:51.021328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:52:08.689376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021886+0000","last_change":"2023-12-02T00:28:49.717365+0000","last_active":"2023-12-02T00:28:51.021886+0000","last_peered":"2023-12-02T00:28:51.021886+0000","last_clean":"2023-12-02T00:28:51.021886+0000","last_became_active":"2023-12-02T00:28:49.717071+0000","last_became_peered":"2023-12-02T00:28:49.717071+0000","last_unstale":"2023-12-02T00:28:51.021886+0000","last_undegraded":"2023-12-02T00:28:51.021886+0000","last_fullsized":"2023-12-02T00:28:51.021886+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:10:26.775049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021416+0000","last_change":"2023-12-02T00:28:49.716297+0000","last_active":"2023-12-02T00:28:51.021416+0000","last_peered":"2023-12-02T00:28:51.021416+0000","last_clean":"2023-12-02T00:28:51.021416+0000","last_became_active":"2023-12-02T00:28:49.715963+0000","last_became_peered":"2023-12-02T00:28:49.715963+0000","last_unstale":"2023-12-02T00:28:51.021416+0000","last_undegraded":"2023-12-02T00:28:51.021416+0000","last_fullsized":"2023-12-02T00:28:51.021416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:58:11.596095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021739+0000","last_change":"2023-12-02T00:28:49.722259+0000","last_active":"2023-12-02T00:28:51.021739+0000","last_peered":"2023-12-02T00:28:51.021739+0000","last_clean":"2023-12-02T00:28:51.021739+0000","last_became_active":"2023-12-02T00:28:49.721989+0000","last_became_peered":"2023-12-02T00:28:49.721989+0000","last_unstale":"2023-12-02T00:28:51.021739+0000","last_undegraded":"2023-12-02T00:28:51.021739+0000","last_fullsized":"2023-12-02T00:28:51.021739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:55:55.141283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705987+0000","last_change":"2023-12-02T00:28:49.729526+0000","last_active":"2023-12-02T00:28:50.705987+0000","last_peered":"2023-12-02T00:28:50.705987+0000","last_clean":"2023-12-02T00:28:50.705987+0000","last_became_active":"2023-12-02T00:28:49.728999+0000","last_became_peered":"2023-12-02T00:28:49.728999+0000","last_unstale":"2023-12-02T00:28:50.705987+0000","last_undegraded":"2023-12-02T00:28:50.705987+0000","last_fullsized":"2023-12-02T00:28:50.705987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:58:15.388237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021070+0000","last_change":"2023-12-02T00:28:49.717936+0000","last_active":"2023-12-02T00:28:51.021070+0000","last_peered":"2023-12-02T00:28:51.021070+0000","last_clean":"2023-12-02T00:28:51.021070+0000","last_became_active":"2023-12-02T00:28:49.716770+0000","last_became_peered":"2023-12-02T00:28:49.716770+0000","last_unstale":"2023-12-02T00:28:51.021070+0000","last_undegraded":"2023-12-02T00:28:51.021070+0000","last_fullsized":"2023-12-02T00:28:51.021070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:36:22.587928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705591+0000","last_change":"2023-12-02T00:28:49.724863+0000","last_active":"2023-12-02T00:28:50.705591+0000","last_peered":"2023-12-02T00:28:50.705591+0000","last_clean":"2023-12-02T00:28:50.705591+0000","last_became_active":"2023-12-02T00:28:49.724592+0000","last_became_peered":"2023-12-02T00:28:49.724592+0000","last_unstale":"2023-12-02T00:28:50.705591+0000","last_undegraded":"2023-12-02T00:28:50.705591+0000","last_fullsized":"2023-12-02T00:28:50.705591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:15:51.747629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020786+0000","last_change":"2023-12-02T00:28:49.724283+0000","last_active":"2023-12-02T00:28:51.020786+0000","last_peered":"2023-12-02T00:28:51.020786+0000","last_clean":"2023-12-02T00:28:51.020786+0000","last_became_active":"2023-12-02T00:28:49.723940+0000","last_became_peered":"2023-12-02T00:28:49.723940+0000","last_unstale":"2023-12-02T00:28:51.020786+0000","last_undegraded":"2023-12-02T00:28:51.020786+0000","last_fullsized":"2023-12-02T00:28:51.020786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:52:22.600655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021632+0000","last_change":"2023-12-02T00:28:49.717015+0000","last_active":"2023-12-02T00:28:51.021632+0000","last_peered":"2023-12-02T00:28:51.021632+0000","last_clean":"2023-12-02T00:28:51.021632+0000","last_became_active":"2023-12-02T00:28:49.716394+0000","last_became_peered":"2023-12-02T00:28:49.716394+0000","last_unstale":"2023-12-02T00:28:51.021632+0000","last_undegraded":"2023-12-02T00:28:51.021632+0000","last_fullsized":"2023-12-02T00:28:51.021632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:15:51.747629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703687+0000","last_change":"2023-12-02T00:28:49.716707+0000","last_active":"2023-12-02T00:28:50.703687+0000","last_peered":"2023-12-02T00:28:50.703687+0000","last_clean":"2023-12-02T00:28:50.703687+0000","last_became_active":"2023-12-02T00:28:49.716392+0000","last_became_peered":"2023-12-02T00:28:49.716392+0000","last_unstale":"2023-12-02T00:28:50.703687+0000","last_undegraded":"2023-12-02T00:28:50.703687+0000","last_fullsized":"2023-12-02T00:28:50.703687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:36:22.587928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021477+0000","last_change":"2023-12-02T00:28:49.724290+0000","last_active":"2023-12-02T00:28:51.021477+0000","last_peered":"2023-12-02T00:28:51.021477+0000","last_clean":"2023-12-02T00:28:51.021477+0000","last_became_active":"2023-12-02T00:28:49.723952+0000","last_became_peered":"2023-12-02T00:28:49.723952+0000","last_unstale":"2023-12-02T00:28:51.021477+0000","last_undegraded":"2023-12-02T00:28:51.021477+0000","last_fullsized":"2023-12-02T00:28:51.021477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:15:02.988933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021881+0000","last_change":"2023-12-02T00:28:49.721218+0000","last_active":"2023-12-02T00:28:51.021881+0000","last_peered":"2023-12-02T00:28:51.021881+0000","last_clean":"2023-12-02T00:28:51.021881+0000","last_became_active":"2023-12-02T00:28:49.720894+0000","last_became_peered":"2023-12-02T00:28:49.720894+0000","last_unstale":"2023-12-02T00:28:51.021881+0000","last_undegraded":"2023-12-02T00:28:51.021881+0000","last_fullsized":"2023-12-02T00:28:51.021881+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:35:09.231704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706145+0000","last_change":"2023-12-02T00:28:49.725424+0000","last_active":"2023-12-02T00:28:50.706145+0000","last_peered":"2023-12-02T00:28:50.706145+0000","last_clean":"2023-12-02T00:28:50.706145+0000","last_became_active":"2023-12-02T00:28:49.724671+0000","last_became_peered":"2023-12-02T00:28:49.724671+0000","last_unstale":"2023-12-02T00:28:50.706145+0000","last_undegraded":"2023-12-02T00:28:50.706145+0000","last_fullsized":"2023-12-02T00:28:50.706145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:32:13.515968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021246+0000","last_change":"2023-12-02T00:28:49.722395+0000","last_active":"2023-12-02T00:28:51.021246+0000","last_peered":"2023-12-02T00:28:51.021246+0000","last_clean":"2023-12-02T00:28:51.021246+0000","last_became_active":"2023-12-02T00:28:49.722052+0000","last_became_peered":"2023-12-02T00:28:49.722052+0000","last_unstale":"2023-12-02T00:28:51.021246+0000","last_undegraded":"2023-12-02T00:28:51.021246+0000","last_fullsized":"2023-12-02T00:28:51.021246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:11:06.388890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020869+0000","last_change":"2023-12-02T00:28:49.713618+0000","last_active":"2023-12-02T00:28:51.020869+0000","last_peered":"2023-12-02T00:28:51.020869+0000","last_clean":"2023-12-02T00:28:51.020869+0000","last_became_active":"2023-12-02T00:28:49.713325+0000","last_became_peered":"2023-12-02T00:28:49.713325+0000","last_unstale":"2023-12-02T00:28:51.020869+0000","last_undegraded":"2023-12-02T00:28:51.020869+0000","last_fullsized":"2023-12-02T00:28:51.020869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:35:09.231704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705601+0000","last_change":"2023-12-02T00:28:49.710604+0000","last_active":"2023-12-02T00:28:50.705601+0000","last_peered":"2023-12-02T00:28:50.705601+0000","last_clean":"2023-12-02T00:28:50.705601+0000","last_became_active":"2023-12-02T00:28:49.710316+0000","last_became_peered":"2023-12-02T00:28:49.710316+0000","last_unstale":"2023-12-02T00:28:50.705601+0000","last_undegraded":"2023-12-02T00:28:50.705601+0000","last_fullsized":"2023-12-02T00:28:50.705601+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:52:08.689376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021549+0000","last_change":"2023-12-02T00:28:48.735813+0000","last_active":"2023-12-02T00:28:51.021549+0000","last_peered":"2023-12-02T00:28:51.021549+0000","last_clean":"2023-12-02T00:28:51.021549+0000","last_became_active":"2023-12-02T00:28:48.735462+0000","last_became_peered":"2023-12-02T00:28:48.735462+0000","last_unstale":"2023-12-02T00:28:51.021549+0000","last_undegraded":"2023-12-02T00:28:51.021549+0000","last_fullsized":"2023-12-02T00:28:51.021549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:55:54.133908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.022018+0000","last_change":"2023-12-02T00:28:49.715318+0000","last_active":"2023-12-02T00:28:51.022018+0000","last_peered":"2023-12-02T00:28:51.022018+0000","last_clean":"2023-12-02T00:28:51.022018+0000","last_became_active":"2023-12-02T00:28:49.715020+0000","last_became_peered":"2023-12-02T00:28:49.715020+0000","last_unstale":"2023-12-02T00:28:51.022018+0000","last_undegraded":"2023-12-02T00:28:51.022018+0000","last_fullsized":"2023-12-02T00:28:51.022018+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:36:22.587928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021150+0000","last_change":"2023-12-02T00:28:48.730625+0000","last_active":"2023-12-02T00:28:51.021150+0000","last_peered":"2023-12-02T00:28:51.021150+0000","last_clean":"2023-12-02T00:28:51.021150+0000","last_became_active":"2023-12-02T00:28:48.730306+0000","last_became_peered":"2023-12-02T00:28:48.730306+0000","last_unstale":"2023-12-02T00:28:51.021150+0000","last_undegraded":"2023-12-02T00:28:51.021150+0000","last_fullsized":"2023-12-02T00:28:51.021150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:11:05.381516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020857+0000","last_change":"2023-12-02T00:28:48.735160+0000","last_active":"2023-12-02T00:28:51.020857+0000","last_peered":"2023-12-02T00:28:51.020857+0000","last_clean":"2023-12-02T00:28:51.020857+0000","last_became_active":"2023-12-02T00:28:48.734860+0000","last_became_peered":"2023-12-02T00:28:48.734860+0000","last_unstale":"2023-12-02T00:28:51.020857+0000","last_undegraded":"2023-12-02T00:28:51.020857+0000","last_fullsized":"2023-12-02T00:28:51.020857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T09:17:51.234416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020811+0000","last_change":"2023-12-02T00:28:49.716640+0000","last_active":"2023-12-02T00:28:51.020811+0000","last_peered":"2023-12-02T00:28:51.020811+0000","last_clean":"2023-12-02T00:28:51.020811+0000","last_became_active":"2023-12-02T00:28:49.716325+0000","last_became_peered":"2023-12-02T00:28:49.716325+0000","last_unstale":"2023-12-02T00:28:51.020811+0000","last_undegraded":"2023-12-02T00:28:51.020811+0000","last_fullsized":"2023-12-02T00:28:51.020811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:07:52.802590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705691+0000","last_change":"2023-12-02T00:28:48.730403+0000","last_active":"2023-12-02T00:28:50.705691+0000","last_peered":"2023-12-02T00:28:50.705691+0000","last_clean":"2023-12-02T00:28:50.705691+0000","last_became_active":"2023-12-02T00:28:48.730053+0000","last_became_peered":"2023-12-02T00:28:48.730053+0000","last_unstale":"2023-12-02T00:28:50.705691+0000","last_undegraded":"2023-12-02T00:28:50.705691+0000","last_fullsized":"2023-12-02T00:28:50.705691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:13:35.505091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703512+0000","last_change":"2023-12-02T00:28:49.716925+0000","last_active":"2023-12-02T00:28:50.703512+0000","last_peered":"2023-12-02T00:28:50.703512+0000","last_clean":"2023-12-02T00:28:50.703512+0000","last_became_active":"2023-12-02T00:28:49.716530+0000","last_became_peered":"2023-12-02T00:28:49.716530+0000","last_unstale":"2023-12-02T00:28:50.703512+0000","last_undegraded":"2023-12-02T00:28:50.703512+0000","last_fullsized":"2023-12-02T00:28:50.703512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:35:09.231704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021002+0000","last_change":"2023-12-02T00:28:48.732302+0000","last_active":"2023-12-02T00:28:51.021002+0000","last_peered":"2023-12-02T00:28:51.021002+0000","last_clean":"2023-12-02T00:28:51.021002+0000","last_became_active":"2023-12-02T00:28:48.731891+0000","last_became_peered":"2023-12-02T00:28:48.731891+0000","last_unstale":"2023-12-02T00:28:51.021002+0000","last_undegraded":"2023-12-02T00:28:51.021002+0000","last_fullsized":"2023-12-02T00:28:51.021002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:44:45.468077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706568+0000","last_change":"2023-12-02T00:28:49.719085+0000","last_active":"2023-12-02T00:28:50.706568+0000","last_peered":"2023-12-02T00:28:50.706568+0000","last_clean":"2023-12-02T00:28:50.706568+0000","last_became_active":"2023-12-02T00:28:49.718606+0000","last_became_peered":"2023-12-02T00:28:49.718606+0000","last_unstale":"2023-12-02T00:28:50.706568+0000","last_undegraded":"2023-12-02T00:28:50.706568+0000","last_fullsized":"2023-12-02T00:28:50.706568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:18:31.615464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706430+0000","last_change":"2023-12-02T00:28:48.732243+0000","last_active":"2023-12-02T00:28:50.706430+0000","last_peered":"2023-12-02T00:28:50.706430+0000","last_clean":"2023-12-02T00:28:50.706430+0000","last_became_active":"2023-12-02T00:28:48.731895+0000","last_became_peered":"2023-12-02T00:28:48.731895+0000","last_unstale":"2023-12-02T00:28:50.706430+0000","last_undegraded":"2023-12-02T00:28:50.706430+0000","last_fullsized":"2023-12-02T00:28:50.706430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:55:54.133908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021998+0000","last_change":"2023-12-02T00:28:49.718319+0000","last_active":"2023-12-02T00:28:51.021998+0000","last_peered":"2023-12-02T00:28:51.021998+0000","last_clean":"2023-12-02T00:28:51.021998+0000","last_became_active":"2023-12-02T00:28:49.718045+0000","last_became_peered":"2023-12-02T00:28:49.718045+0000","last_unstale":"2023-12-02T00:28:51.021998+0000","last_undegraded":"2023-12-02T00:28:51.021998+0000","last_fullsized":"2023-12-02T00:28:51.021998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:58:11.596095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703791+0000","last_change":"2023-12-02T00:28:48.728008+0000","last_active":"2023-12-02T00:28:50.703791+0000","last_peered":"2023-12-02T00:28:50.703791+0000","last_clean":"2023-12-02T00:28:50.703791+0000","last_became_active":"2023-12-02T00:28:48.727662+0000","last_became_peered":"2023-12-02T00:28:48.727662+0000","last_unstale":"2023-12-02T00:28:50.703791+0000","last_undegraded":"2023-12-02T00:28:50.703791+0000","last_fullsized":"2023-12-02T00:28:50.703791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:13:35.505091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705692+0000","last_change":"2023-12-02T00:28:49.719391+0000","last_active":"2023-12-02T00:28:50.705692+0000","last_peered":"2023-12-02T00:28:50.705692+0000","last_clean":"2023-12-02T00:28:50.705692+0000","last_became_active":"2023-12-02T00:28:49.719126+0000","last_became_peered":"2023-12-02T00:28:49.719126+0000","last_unstale":"2023-12-02T00:28:50.705692+0000","last_undegraded":"2023-12-02T00:28:50.705692+0000","last_fullsized":"2023-12-02T00:28:50.705692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:47:09.825514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021521+0000","last_change":"2023-12-02T00:28:48.736811+0000","last_active":"2023-12-02T00:28:51.021521+0000","last_peered":"2023-12-02T00:28:51.021521+0000","last_clean":"2023-12-02T00:28:51.021521+0000","last_became_active":"2023-12-02T00:28:48.736576+0000","last_became_peered":"2023-12-02T00:28:48.736576+0000","last_unstale":"2023-12-02T00:28:51.021521+0000","last_undegraded":"2023-12-02T00:28:51.021521+0000","last_fullsized":"2023-12-02T00:28:51.021521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:52:07.682002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021600+0000","last_change":"2023-12-02T00:28:49.717541+0000","last_active":"2023-12-02T00:28:51.021600+0000","last_peered":"2023-12-02T00:28:51.021600+0000","last_clean":"2023-12-02T00:28:51.021600+0000","last_became_active":"2023-12-02T00:28:49.717225+0000","last_became_peered":"2023-12-02T00:28:49.717225+0000","last_unstale":"2023-12-02T00:28:51.021600+0000","last_undegraded":"2023-12-02T00:28:51.021600+0000","last_fullsized":"2023-12-02T00:28:51.021600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:29:16.078935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.022091+0000","last_change":"2023-12-02T00:28:48.725581+0000","last_active":"2023-12-02T00:28:51.022091+0000","last_peered":"2023-12-02T00:28:51.022091+0000","last_clean":"2023-12-02T00:28:51.022091+0000","last_became_active":"2023-12-02T00:28:48.725230+0000","last_became_peered":"2023-12-02T00:28:48.725230+0000","last_unstale":"2023-12-02T00:28:51.022091+0000","last_undegraded":"2023-12-02T00:28:51.022091+0000","last_fullsized":"2023-12-02T00:28:51.022091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:39:30.211229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021761+0000","last_change":"2023-12-02T00:28:49.716591+0000","last_active":"2023-12-02T00:28:51.021761+0000","last_peered":"2023-12-02T00:28:51.021761+0000","last_clean":"2023-12-02T00:28:51.021761+0000","last_became_active":"2023-12-02T00:28:49.716237+0000","last_became_peered":"2023-12-02T00:28:49.716237+0000","last_unstale":"2023-12-02T00:28:51.021761+0000","last_undegraded":"2023-12-02T00:28:51.021761+0000","last_fullsized":"2023-12-02T00:28:51.021761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T12:06:34.965686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706099+0000","last_change":"2023-12-02T00:28:48.734778+0000","last_active":"2023-12-02T00:28:50.706099+0000","last_peered":"2023-12-02T00:28:50.706099+0000","last_clean":"2023-12-02T00:28:50.706099+0000","last_became_active":"2023-12-02T00:28:48.734474+0000","last_became_peered":"2023-12-02T00:28:48.734474+0000","last_unstale":"2023-12-02T00:28:50.706099+0000","last_undegraded":"2023-12-02T00:28:50.706099+0000","last_fullsized":"2023-12-02T00:28:50.706099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:07:51.795216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021342+0000","last_change":"2023-12-02T00:28:49.721274+0000","last_active":"2023-12-02T00:28:51.021342+0000","last_peered":"2023-12-02T00:28:51.021342+0000","last_clean":"2023-12-02T00:28:51.021342+0000","last_became_active":"2023-12-02T00:28:49.719979+0000","last_became_peered":"2023-12-02T00:28:49.719979+0000","last_unstale":"2023-12-02T00:28:51.021342+0000","last_undegraded":"2023-12-02T00:28:51.021342+0000","last_fullsized":"2023-12-02T00:28:51.021342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:50:45.793304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020850+0000","last_change":"2023-12-02T00:28:48.732958+0000","last_active":"2023-12-02T00:28:51.020850+0000","last_peered":"2023-12-02T00:28:51.020850+0000","last_clean":"2023-12-02T00:28:51.020850+0000","last_became_active":"2023-12-02T00:28:48.732651+0000","last_became_peered":"2023-12-02T00:28:48.732651+0000","last_unstale":"2023-12-02T00:28:51.020850+0000","last_undegraded":"2023-12-02T00:28:51.020850+0000","last_fullsized":"2023-12-02T00:28:51.020850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:29:26.381170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020869+0000","last_change":"2023-12-02T00:28:49.712859+0000","last_active":"2023-12-02T00:28:51.020869+0000","last_peered":"2023-12-02T00:28:51.020869+0000","last_clean":"2023-12-02T00:28:51.020869+0000","last_became_active":"2023-12-02T00:28:49.712585+0000","last_became_peered":"2023-12-02T00:28:49.712585+0000","last_unstale":"2023-12-02T00:28:51.020869+0000","last_undegraded":"2023-12-02T00:28:51.020869+0000","last_fullsized":"2023-12-02T00:28:51.020869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T12:06:34.965686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020660+0000","last_change":"2023-12-02T00:28:48.734113+0000","last_active":"2023-12-02T00:28:51.020660+0000","last_peered":"2023-12-02T00:28:51.020660+0000","last_clean":"2023-12-02T00:28:51.020660+0000","last_became_active":"2023-12-02T00:28:48.733749+0000","last_became_peered":"2023-12-02T00:28:48.733749+0000","last_unstale":"2023-12-02T00:28:51.020660+0000","last_undegraded":"2023-12-02T00:28:51.020660+0000","last_fullsized":"2023-12-02T00:28:51.020660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T10:56:02.550120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021416+0000","last_change":"2023-12-02T00:28:49.719477+0000","last_active":"2023-12-02T00:28:51.021416+0000","last_peered":"2023-12-02T00:28:51.021416+0000","last_clean":"2023-12-02T00:28:51.021416+0000","last_became_active":"2023-12-02T00:28:49.718537+0000","last_became_peered":"2023-12-02T00:28:49.718537+0000","last_unstale":"2023-12-02T00:28:51.021416+0000","last_undegraded":"2023-12-02T00:28:51.021416+0000","last_fullsized":"2023-12-02T00:28:51.021416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:15:02.988933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705801+0000","last_change":"2023-12-02T00:28:48.730372+0000","last_active":"2023-12-02T00:28:50.705801+0000","last_peered":"2023-12-02T00:28:50.705801+0000","last_clean":"2023-12-02T00:28:50.705801+0000","last_became_active":"2023-12-02T00:28:48.730051+0000","last_became_peered":"2023-12-02T00:28:48.730051+0000","last_unstale":"2023-12-02T00:28:50.705801+0000","last_undegraded":"2023-12-02T00:28:50.705801+0000","last_fullsized":"2023-12-02T00:28:50.705801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:29:10.550270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021515+0000","last_change":"2023-12-02T00:28:49.714467+0000","last_active":"2023-12-02T00:28:51.021515+0000","last_peered":"2023-12-02T00:28:51.021515+0000","last_clean":"2023-12-02T00:28:51.021515+0000","last_became_active":"2023-12-02T00:28:49.713964+0000","last_became_peered":"2023-12-02T00:28:49.713964+0000","last_unstale":"2023-12-02T00:28:51.021515+0000","last_undegraded":"2023-12-02T00:28:51.021515+0000","last_fullsized":"2023-12-02T00:28:51.021515+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T12:22:41.666941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021286+0000","last_change":"2023-12-02T00:28:48.736447+0000","last_active":"2023-12-02T00:28:51.021286+0000","last_peered":"2023-12-02T00:28:51.021286+0000","last_clean":"2023-12-02T00:28:51.021286+0000","last_became_active":"2023-12-02T00:28:48.736169+0000","last_became_peered":"2023-12-02T00:28:48.736169+0000","last_unstale":"2023-12-02T00:28:51.021286+0000","last_undegraded":"2023-12-02T00:28:51.021286+0000","last_fullsized":"2023-12-02T00:28:51.021286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T09:17:51.234416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021954+0000","last_change":"2023-12-02T00:28:49.719154+0000","last_active":"2023-12-02T00:28:51.021954+0000","last_peered":"2023-12-02T00:28:51.021954+0000","last_clean":"2023-12-02T00:28:51.021954+0000","last_became_active":"2023-12-02T00:28:49.718713+0000","last_became_peered":"2023-12-02T00:28:49.718713+0000","last_unstale":"2023-12-02T00:28:51.021954+0000","last_undegraded":"2023-12-02T00:28:51.021954+0000","last_fullsized":"2023-12-02T00:28:51.021954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:50:17.285633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703845+0000","last_change":"2023-12-02T00:28:48.731504+0000","last_active":"2023-12-02T00:28:50.703845+0000","last_peered":"2023-12-02T00:28:50.703845+0000","last_clean":"2023-12-02T00:28:50.703845+0000","last_became_active":"2023-12-02T00:28:48.731224+0000","last_became_peered":"2023-12-02T00:28:48.731224+0000","last_unstale":"2023-12-02T00:28:50.703845+0000","last_undegraded":"2023-12-02T00:28:50.703845+0000","last_fullsized":"2023-12-02T00:28:50.703845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:55:54.133908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706503+0000","last_change":"2023-12-02T00:28:49.726089+0000","last_active":"2023-12-02T00:28:50.706503+0000","last_peered":"2023-12-02T00:28:50.706503+0000","last_clean":"2023-12-02T00:28:50.706503+0000","last_became_active":"2023-12-02T00:28:49.725482+0000","last_became_peered":"2023-12-02T00:28:49.725482+0000","last_unstale":"2023-12-02T00:28:50.706503+0000","last_undegraded":"2023-12-02T00:28:50.706503+0000","last_fullsized":"2023-12-02T00:28:50.706503+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T12:22:41.666941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020880+0000","last_change":"2023-12-02T00:28:48.729936+0000","last_active":"2023-12-02T00:28:51.020880+0000","last_peered":"2023-12-02T00:28:51.020880+0000","last_clean":"2023-12-02T00:28:51.020880+0000","last_became_active":"2023-12-02T00:28:48.729617+0000","last_became_peered":"2023-12-02T00:28:48.729617+0000","last_unstale":"2023-12-02T00:28:51.020880+0000","last_undegraded":"2023-12-02T00:28:51.020880+0000","last_fullsized":"2023-12-02T00:28:51.020880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:29:26.381170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021803+0000","last_change":"2023-12-02T00:28:49.725211+0000","last_active":"2023-12-02T00:28:51.021803+0000","last_peered":"2023-12-02T00:28:51.021803+0000","last_clean":"2023-12-02T00:28:51.021803+0000","last_became_active":"2023-12-02T00:28:49.724928+0000","last_became_peered":"2023-12-02T00:28:49.724928+0000","last_unstale":"2023-12-02T00:28:51.021803+0000","last_undegraded":"2023-12-02T00:28:51.021803+0000","last_fullsized":"2023-12-02T00:28:51.021803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:33:43.085260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":165,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-02T00:28:49.708269+0000","last_change":"2023-12-02T00:28:39.379380+0000","last_active":"2023-12-02T00:28:49.708269+0000","last_peered":"2023-12-02T00:28:49.708269+0000","last_clean":"2023-12-02T00:28:49.708269+0000","last_became_active":"2023-12-02T00:28:39.378255+0000","last_became_peered":"2023-12-02T00:28:39.378255+0000","last_unstale":"2023-12-02T00:28:49.708269+0000","last_undegraded":"2023-12-02T00:28:49.708269+0000","last_fullsized":"2023-12-02T00:28:49.708269+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:38.177345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:38.177345+0000","last_clean_scrub_stamp":"2023-12-02T00:28:38.177345+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:21:08.582412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706107+0000","last_change":"2023-12-02T00:28:48.731326+0000","last_active":"2023-12-02T00:28:50.706107+0000","last_peered":"2023-12-02T00:28:50.706107+0000","last_clean":"2023-12-02T00:28:50.706107+0000","last_became_active":"2023-12-02T00:28:48.730970+0000","last_became_peered":"2023-12-02T00:28:48.730970+0000","last_unstale":"2023-12-02T00:28:50.706107+0000","last_undegraded":"2023-12-02T00:28:50.706107+0000","last_fullsized":"2023-12-02T00:28:50.706107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:44:45.468077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021403+0000","last_change":"2023-12-02T00:28:49.727341+0000","last_active":"2023-12-02T00:28:51.021403+0000","last_peered":"2023-12-02T00:28:51.021403+0000","last_clean":"2023-12-02T00:28:51.021403+0000","last_became_active":"2023-12-02T00:28:49.727045+0000","last_became_peered":"2023-12-02T00:28:49.727045+0000","last_unstale":"2023-12-02T00:28:51.021403+0000","last_undegraded":"2023-12-02T00:28:51.021403+0000","last_fullsized":"2023-12-02T00:28:51.021403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:58:15.388237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020921+0000","last_change":"2023-12-02T00:28:48.732719+0000","last_active":"2023-12-02T00:28:51.020921+0000","last_peered":"2023-12-02T00:28:51.020921+0000","last_clean":"2023-12-02T00:28:51.020921+0000","last_became_active":"2023-12-02T00:28:48.732466+0000","last_became_peered":"2023-12-02T00:28:48.732466+0000","last_unstale":"2023-12-02T00:28:51.020921+0000","last_undegraded":"2023-12-02T00:28:51.020921+0000","last_fullsized":"2023-12-02T00:28:51.020921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:52:21.593280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021611+0000","last_change":"2023-12-02T00:28:49.716043+0000","last_active":"2023-12-02T00:28:51.021611+0000","last_peered":"2023-12-02T00:28:51.021611+0000","last_clean":"2023-12-02T00:28:51.021611+0000","last_became_active":"2023-12-02T00:28:49.715747+0000","last_became_peered":"2023-12-02T00:28:49.715747+0000","last_unstale":"2023-12-02T00:28:51.021611+0000","last_undegraded":"2023-12-02T00:28:51.021611+0000","last_fullsized":"2023-12-02T00:28:51.021611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:44:46.475452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021208+0000","last_change":"2023-12-02T00:28:48.736024+0000","last_active":"2023-12-02T00:28:51.021208+0000","last_peered":"2023-12-02T00:28:51.021208+0000","last_clean":"2023-12-02T00:28:51.021208+0000","last_became_active":"2023-12-02T00:28:48.735736+0000","last_became_peered":"2023-12-02T00:28:48.735736+0000","last_unstale":"2023-12-02T00:28:51.021208+0000","last_undegraded":"2023-12-02T00:28:51.021208+0000","last_fullsized":"2023-12-02T00:28:51.021208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:55:48.025090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703534+0000","last_change":"2023-12-02T00:28:49.707755+0000","last_active":"2023-12-02T00:28:50.703534+0000","last_peered":"2023-12-02T00:28:50.703534+0000","last_clean":"2023-12-02T00:28:50.703534+0000","last_became_active":"2023-12-02T00:28:49.707449+0000","last_became_peered":"2023-12-02T00:28:49.707449+0000","last_unstale":"2023-12-02T00:28:50.703534+0000","last_undegraded":"2023-12-02T00:28:50.703534+0000","last_fullsized":"2023-12-02T00:28:50.703534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:07:52.802590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706217+0000","last_change":"2023-12-02T00:28:48.729078+0000","last_active":"2023-12-02T00:28:50.706217+0000","last_peered":"2023-12-02T00:28:50.706217+0000","last_clean":"2023-12-02T00:28:50.706217+0000","last_became_active":"2023-12-02T00:28:48.728567+0000","last_became_peered":"2023-12-02T00:28:48.728567+0000","last_unstale":"2023-12-02T00:28:50.706217+0000","last_undegraded":"2023-12-02T00:28:50.706217+0000","last_fullsized":"2023-12-02T00:28:50.706217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:58:10.588720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705483+0000","last_change":"2023-12-02T00:28:49.712801+0000","last_active":"2023-12-02T00:28:50.705483+0000","last_peered":"2023-12-02T00:28:50.705483+0000","last_clean":"2023-12-02T00:28:50.705483+0000","last_became_active":"2023-12-02T00:28:49.712511+0000","last_became_peered":"2023-12-02T00:28:49.712511+0000","last_unstale":"2023-12-02T00:28:50.705483+0000","last_undegraded":"2023-12-02T00:28:50.705483+0000","last_fullsized":"2023-12-02T00:28:50.705483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:55:49.032465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703749+0000","last_change":"2023-12-02T00:28:48.730774+0000","last_active":"2023-12-02T00:28:50.703749+0000","last_peered":"2023-12-02T00:28:50.703749+0000","last_clean":"2023-12-02T00:28:50.703749+0000","last_became_active":"2023-12-02T00:28:48.730498+0000","last_became_peered":"2023-12-02T00:28:48.730498+0000","last_unstale":"2023-12-02T00:28:50.703749+0000","last_undegraded":"2023-12-02T00:28:50.703749+0000","last_fullsized":"2023-12-02T00:28:50.703749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:29:10.550270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706537+0000","last_change":"2023-12-02T00:28:49.726928+0000","last_active":"2023-12-02T00:28:50.706537+0000","last_peered":"2023-12-02T00:28:50.706537+0000","last_clean":"2023-12-02T00:28:50.706537+0000","last_became_active":"2023-12-02T00:28:49.725033+0000","last_became_peered":"2023-12-02T00:28:49.725033+0000","last_unstale":"2023-12-02T00:28:50.706537+0000","last_undegraded":"2023-12-02T00:28:50.706537+0000","last_fullsized":"2023-12-02T00:28:50.706537+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:33:43.085260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021906+0000","last_change":"2023-12-02T00:28:48.735559+0000","last_active":"2023-12-02T00:28:51.021906+0000","last_peered":"2023-12-02T00:28:51.021906+0000","last_clean":"2023-12-02T00:28:51.021906+0000","last_became_active":"2023-12-02T00:28:48.735243+0000","last_became_peered":"2023-12-02T00:28:48.735243+0000","last_unstale":"2023-12-02T00:28:51.021906+0000","last_undegraded":"2023-12-02T00:28:51.021906+0000","last_fullsized":"2023-12-02T00:28:51.021906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:07:51.795216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703595+0000","last_change":"2023-12-02T00:28:49.710546+0000","last_active":"2023-12-02T00:28:50.703595+0000","last_peered":"2023-12-02T00:28:50.703595+0000","last_clean":"2023-12-02T00:28:50.703595+0000","last_became_active":"2023-12-02T00:28:49.710010+0000","last_became_peered":"2023-12-02T00:28:49.710010+0000","last_unstale":"2023-12-02T00:28:50.703595+0000","last_undegraded":"2023-12-02T00:28:50.703595+0000","last_fullsized":"2023-12-02T00:28:50.703595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:58:11.596095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.172997+0000","last_change":"2023-12-02T00:28:48.731556+0000","last_active":"2023-12-02T00:28:51.172997+0000","last_peered":"2023-12-02T00:28:51.172997+0000","last_clean":"2023-12-02T00:28:51.172997+0000","last_became_active":"2023-12-02T00:28:48.731291+0000","last_became_peered":"2023-12-02T00:28:48.731291+0000","last_unstale":"2023-12-02T00:28:51.172997+0000","last_undegraded":"2023-12-02T00:28:51.172997+0000","last_fullsized":"2023-12-02T00:28:51.172997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:39:30.211229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020929+0000","last_change":"2023-12-02T00:28:49.723678+0000","last_active":"2023-12-02T00:28:51.020929+0000","last_peered":"2023-12-02T00:28:51.020929+0000","last_clean":"2023-12-02T00:28:51.020929+0000","last_became_active":"2023-12-02T00:28:49.723299+0000","last_became_peered":"2023-12-02T00:28:49.723299+0000","last_unstale":"2023-12-02T00:28:51.020929+0000","last_undegraded":"2023-12-02T00:28:51.020929+0000","last_fullsized":"2023-12-02T00:28:51.020929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:58:11.596095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021044+0000","last_change":"2023-12-02T00:28:48.735741+0000","last_active":"2023-12-02T00:28:51.021044+0000","last_peered":"2023-12-02T00:28:51.021044+0000","last_clean":"2023-12-02T00:28:51.021044+0000","last_became_active":"2023-12-02T00:28:48.735400+0000","last_became_peered":"2023-12-02T00:28:48.735400+0000","last_unstale":"2023-12-02T00:28:51.021044+0000","last_undegraded":"2023-12-02T00:28:51.021044+0000","last_fullsized":"2023-12-02T00:28:51.021044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:13:35.505091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.022028+0000","last_change":"2023-12-02T00:28:49.711592+0000","last_active":"2023-12-02T00:28:51.022028+0000","last_peered":"2023-12-02T00:28:51.022028+0000","last_clean":"2023-12-02T00:28:51.022028+0000","last_became_active":"2023-12-02T00:28:49.711211+0000","last_became_peered":"2023-12-02T00:28:49.711211+0000","last_unstale":"2023-12-02T00:28:51.022028+0000","last_undegraded":"2023-12-02T00:28:51.022028+0000","last_fullsized":"2023-12-02T00:28:51.022028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:15:51.747629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.020816+0000","last_change":"2023-12-02T00:28:48.735722+0000","last_active":"2023-12-02T00:28:51.020816+0000","last_peered":"2023-12-02T00:28:51.020816+0000","last_clean":"2023-12-02T00:28:51.020816+0000","last_became_active":"2023-12-02T00:28:48.735212+0000","last_became_peered":"2023-12-02T00:28:48.735212+0000","last_unstale":"2023-12-02T00:28:51.020816+0000","last_undegraded":"2023-12-02T00:28:51.020816+0000","last_fullsized":"2023-12-02T00:28:51.020816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:47:08.818139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.705309+0000","last_change":"2023-12-02T00:28:49.711824+0000","last_active":"2023-12-02T00:28:50.705309+0000","last_peered":"2023-12-02T00:28:50.705309+0000","last_clean":"2023-12-02T00:28:50.705309+0000","last_became_active":"2023-12-02T00:28:49.710946+0000","last_became_peered":"2023-12-02T00:28:49.710946+0000","last_unstale":"2023-12-02T00:28:50.705309+0000","last_undegraded":"2023-12-02T00:28:50.705309+0000","last_fullsized":"2023-12-02T00:28:50.705309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T09:17:52.241790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703622+0000","last_change":"2023-12-02T00:28:48.727127+0000","last_active":"2023-12-02T00:28:50.703622+0000","last_peered":"2023-12-02T00:28:50.703622+0000","last_clean":"2023-12-02T00:28:50.703622+0000","last_became_active":"2023-12-02T00:28:48.726750+0000","last_became_peered":"2023-12-02T00:28:48.726750+0000","last_unstale":"2023-12-02T00:28:50.703622+0000","last_undegraded":"2023-12-02T00:28:50.703622+0000","last_fullsized":"2023-12-02T00:28:50.703622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:44:45.468077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021415+0000","last_change":"2023-12-02T00:28:49.718158+0000","last_active":"2023-12-02T00:28:51.021415+0000","last_peered":"2023-12-02T00:28:51.021415+0000","last_clean":"2023-12-02T00:28:51.021415+0000","last_became_active":"2023-12-02T00:28:49.717721+0000","last_became_peered":"2023-12-02T00:28:49.717721+0000","last_unstale":"2023-12-02T00:28:51.021415+0000","last_undegraded":"2023-12-02T00:28:51.021415+0000","last_fullsized":"2023-12-02T00:28:51.021415+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T10:27:29.244181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.703765+0000","last_change":"2023-12-02T00:28:48.735894+0000","last_active":"2023-12-02T00:28:50.703765+0000","last_peered":"2023-12-02T00:28:50.703765+0000","last_clean":"2023-12-02T00:28:50.703765+0000","last_became_active":"2023-12-02T00:28:48.735613+0000","last_became_peered":"2023-12-02T00:28:48.735613+0000","last_unstale":"2023-12-02T00:28:50.703765+0000","last_undegraded":"2023-12-02T00:28:50.703765+0000","last_fullsized":"2023-12-02T00:28:50.703765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:52:07.682002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021464+0000","last_change":"2023-12-02T00:28:49.719314+0000","last_active":"2023-12-02T00:28:51.021464+0000","last_peered":"2023-12-02T00:28:51.021464+0000","last_clean":"2023-12-02T00:28:51.021464+0000","last_became_active":"2023-12-02T00:28:49.718940+0000","last_became_peered":"2023-12-02T00:28:49.718940+0000","last_unstale":"2023-12-02T00:28:51.021464+0000","last_undegraded":"2023-12-02T00:28:51.021464+0000","last_fullsized":"2023-12-02T00:28:51.021464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:50:45.793304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021457+0000","last_change":"2023-12-02T00:28:48.731948+0000","last_active":"2023-12-02T00:28:51.021457+0000","last_peered":"2023-12-02T00:28:51.021457+0000","last_clean":"2023-12-02T00:28:51.021457+0000","last_became_active":"2023-12-02T00:28:48.731641+0000","last_became_peered":"2023-12-02T00:28:48.731641+0000","last_unstale":"2023-12-02T00:28:51.021457+0000","last_undegraded":"2023-12-02T00:28:51.021457+0000","last_fullsized":"2023-12-02T00:28:51.021457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:36:21.580553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021037+0000","last_change":"2023-12-02T00:28:49.715513+0000","last_active":"2023-12-02T00:28:51.021037+0000","last_peered":"2023-12-02T00:28:51.021037+0000","last_clean":"2023-12-02T00:28:51.021037+0000","last_became_active":"2023-12-02T00:28:49.715240+0000","last_became_peered":"2023-12-02T00:28:49.715240+0000","last_unstale":"2023-12-02T00:28:51.021037+0000","last_undegraded":"2023-12-02T00:28:51.021037+0000","last_fullsized":"2023-12-02T00:28:51.021037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:52:22.600655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706521+0000","last_change":"2023-12-02T00:28:49.717785+0000","last_active":"2023-12-02T00:28:50.706521+0000","last_peered":"2023-12-02T00:28:50.706521+0000","last_clean":"2023-12-02T00:28:50.706521+0000","last_became_active":"2023-12-02T00:28:49.717408+0000","last_became_peered":"2023-12-02T00:28:49.717408+0000","last_unstale":"2023-12-02T00:28:50.706521+0000","last_undegraded":"2023-12-02T00:28:50.706521+0000","last_fullsized":"2023-12-02T00:28:50.706521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:29:16.078935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021429+0000","last_change":"2023-12-02T00:28:48.729768+0000","last_active":"2023-12-02T00:28:51.021429+0000","last_peered":"2023-12-02T00:28:51.021429+0000","last_clean":"2023-12-02T00:28:51.021429+0000","last_became_active":"2023-12-02T00:28:48.729466+0000","last_became_peered":"2023-12-02T00:28:48.729466+0000","last_unstale":"2023-12-02T00:28:51.021429+0000","last_undegraded":"2023-12-02T00:28:51.021429+0000","last_fullsized":"2023-12-02T00:28:51.021429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T12:06:33.958311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:50.706378+0000","last_change":"2023-12-02T00:28:49.727325+0000","last_active":"2023-12-02T00:28:50.706378+0000","last_peered":"2023-12-02T00:28:50.706378+0000","last_clean":"2023-12-02T00:28:50.706378+0000","last_became_active":"2023-12-02T00:28:49.727062+0000","last_became_peered":"2023-12-02T00:28:49.727062+0000","last_unstale":"2023-12-02T00:28:50.706378+0000","last_undegraded":"2023-12-02T00:28:50.706378+0000","last_fullsized":"2023-12-02T00:28:50.706378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:15:02.988933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021734+0000","last_change":"2023-12-02T00:28:48.732989+0000","last_active":"2023-12-02T00:28:51.021734+0000","last_peered":"2023-12-02T00:28:51.021734+0000","last_clean":"2023-12-02T00:28:51.021734+0000","last_became_active":"2023-12-02T00:28:48.731645+0000","last_became_peered":"2023-12-02T00:28:48.731645+0000","last_unstale":"2023-12-02T00:28:51.021734+0000","last_undegraded":"2023-12-02T00:28:51.021734+0000","last_fullsized":"2023-12-02T00:28:51.021734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:44:45.468077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021923+0000","last_change":"2023-12-02T00:28:49.725372+0000","last_active":"2023-12-02T00:28:51.021923+0000","last_peered":"2023-12-02T00:28:51.021923+0000","last_clean":"2023-12-02T00:28:51.021923+0000","last_became_active":"2023-12-02T00:28:49.724926+0000","last_became_peered":"2023-12-02T00:28:49.724926+0000","last_unstale":"2023-12-02T00:28:51.021923+0000","last_undegraded":"2023-12-02T00:28:51.021923+0000","last_fullsized":"2023-12-02T00:28:51.021923+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:07:52.802590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021204+0000","last_change":"2023-12-02T00:28:48.731302+0000","last_active":"2023-12-02T00:28:51.021204+0000","last_peered":"2023-12-02T00:28:51.021204+0000","last_clean":"2023-12-02T00:28:51.021204+0000","last_became_active":"2023-12-02T00:28:48.731002+0000","last_became_peered":"2023-12-02T00:28:48.731002+0000","last_unstale":"2023-12-02T00:28:51.021204+0000","last_undegraded":"2023-12-02T00:28:51.021204+0000","last_fullsized":"2023-12-02T00:28:51.021204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:29:26.381170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.021221+0000","last_change":"2023-12-02T00:28:49.721605+0000","last_active":"2023-12-02T00:28:51.021221+0000","last_peered":"2023-12-02T00:28:51.021221+0000","last_clean":"2023-12-02T00:28:51.021221+0000","last_became_active":"2023-12-02T00:28:49.721035+0000","last_became_peered":"2023-12-02T00:28:49.721035+0000","last_unstale":"2023-12-02T00:28:51.021221+0000","last_undegraded":"2023-12-02T00:28:51.021221+0000","last_fullsized":"2023-12-02T00:28:51.021221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:48.709287+0000","last_clean_scrub_stamp":"2023-12-02T00:28:48.709287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:15:51.747629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-02T00:28:51.022018+0000","last_change":"2023-12-02T00:28:48.729331+0000","last_active":"2023-12-02T00:28:51.022018+0000","last_peered":"2023-12-02T00:28:51.022018+0000","last_clean":"2023-12-02T00:28:51.022018+0000","last_became_active":"2023-12-02T00:28:48.728469+0000","last_became_peered":"2023-12-02T00:28:48.728469+0000","last_unstale":"2023-12-02T00:28:51.022018+0000","last_undegraded":"2023-12-02T00:28:51.022018+0000","last_fullsized":"2023-12-02T00:28:51.022018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:47.701913+0000","last_clean_scrub_stamp":"2023-12-02T00:28:47.701913+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:47:08.818139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":815104,"data_stored":666531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95199999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":77193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":75671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":75707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":77193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28299999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":212992,"data_stored":74962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":225280,"data_stored":75741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":819200,"data_stored":665968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.377}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-02T00:29:01.918 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-12-02T00:29:01.918 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-12-02T00:29:01.918 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-12-02T00:29:01.919 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-12-02T00:29:02.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.056+0000 7fde7e43d640 1 Processor -- start 2023-12-02T00:29:02.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.057+0000 7fde7e43d640 1 -- start start 2023-12-02T00:29:02.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.057+0000 7fde7e43d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fde7815d330 0x7fde780b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:02.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.057+0000 7fde7e43d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fde780d3900 0x7fde780b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:02.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.057+0000 7fde7e43d640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde780b2b10 con 0x7fde780d2910 2023-12-02T00:29:02.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.057+0000 7fde7e43d640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde780d1590 con 0x7fde7815d330 2023-12-02T00:29:02.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.057+0000 7fde7e43d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde780d1870 con 0x7fde780d3900 2023-12-02T00:29:02.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.058+0000 7fde7c9da640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fde780d3900 0x7fde780b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:02.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.058+0000 7fde77fff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fde7815d330 0x7fde780b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:02.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.058+0000 7fde7c9da640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fde780d3900 0x7fde780b6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.46:35326/0 (socket says 172.21.15.46:35326) 2023-12-02T00:29:02.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.058+0000 7fde777fe640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fde780d2910 0x7fde780d31f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.46:36904/0 (socket says 172.21.15.46:36904) 2023-12-02T00:29:02.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.058+0000 7fde7c9da640 1 -- 172.21.15.46:0/3808677513 learned_addr learned my addr 172.21.15.46:0/3808677513 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:02.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.058+0000 7fde7c9da640 1 -- 172.21.15.46:0/3808677513 >> v2:172.21.15.162:3300/0 conn(0x7fde7815d330 msgr2=0x7fde780b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:02.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.058+0000 7fde7c9da640 1 --2- 172.21.15.46:0/3808677513 >> v2:172.21.15.162:3300/0 conn(0x7fde7815d330 0x7fde780b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:02.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.058+0000 7fde7c9da640 1 -- 172.21.15.46:0/3808677513 >> v1:172.21.15.162:6789/0 conn(0x7fde780d2910 legacy=0x7fde780d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:02.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.058+0000 7fde7c9da640 1 -- 172.21.15.46:0/3808677513 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde780d1b50 con 0x7fde780d3900 2023-12-02T00:29:02.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.058+0000 7fde77fff640 1 --2- 172.21.15.46:0/3808677513 >> v2:172.21.15.162:3300/0 conn(0x7fde7815d330 0x7fde780b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:29:02.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.059+0000 7fde7c9da640 1 --2- 172.21.15.46:0/3808677513 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fde780d3900 0x7fde780b6390 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fde70009e50 tx=0x7fde7002bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d65169f41cc3e8db server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:02.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.059+0000 7fde76ffd640 1 -- 172.21.15.46:0/3808677513 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7002ca70 con 0x7fde780d3900 2023-12-02T00:29:02.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.059+0000 7fde76ffd640 1 -- 172.21.15.46:0/3808677513 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde7002cbd0 con 0x7fde780d3900 2023-12-02T00:29:02.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.059+0000 7fde76ffd640 1 -- 172.21.15.46:0/3808677513 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde700343f0 con 0x7fde780d3900 2023-12-02T00:29:02.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.060+0000 7fde7e43d640 1 -- 172.21.15.46:0/3808677513 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fde780d3900 msgr2=0x7fde780b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:02.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.060+0000 7fde7e43d640 1 --2- 172.21.15.46:0/3808677513 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fde780d3900 0x7fde780b6390 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fde70009e50 tx=0x7fde7002bed0 comp rx=0 tx=0).stop 2023-12-02T00:29:02.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.060+0000 7fde7e43d640 1 -- 172.21.15.46:0/3808677513 shutdown_connections 2023-12-02T00:29:02.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.060+0000 7fde7e43d640 1 --2- 172.21.15.46:0/3808677513 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fde780d3900 0x7fde780b6390 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:02.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.060+0000 7fde7e43d640 1 --2- 172.21.15.46:0/3808677513 >> v2:172.21.15.162:3300/0 conn(0x7fde7815d330 0x7fde780b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:02.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.060+0000 7fde7e43d640 1 -- 172.21.15.46:0/3808677513 >> 172.21.15.46:0/3808677513 conn(0x7fde78072020 msgr2=0x7fde78072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:02.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.060+0000 7fde7e43d640 1 -- 172.21.15.46:0/3808677513 shutdown_connections 2023-12-02T00:29:02.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.060+0000 7fde7e43d640 1 -- 172.21.15.46:0/3808677513 wait complete. 2023-12-02T00:29:02.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.061+0000 7fde7e43d640 1 Processor -- start 2023-12-02T00:29:02.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.061+0000 7fde7e43d640 1 -- start start 2023-12-02T00:29:02.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.062+0000 7fde7e43d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde780d2910 0x7fde78149510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:02.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.062+0000 7fde7e43d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde780d3900 0x7fde7815ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:02.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.062+0000 7fde7e43d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fde7815d330 0x7fde781604b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:02.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.062+0000 7fde7e43d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde7814a070 con 0x7fde780d2910 2023-12-02T00:29:02.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.062+0000 7fde7e43d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde780cd590 con 0x7fde780d3900 2023-12-02T00:29:02.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.062+0000 7fde7e43d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde780ccfd0 con 0x7fde7815d330 2023-12-02T00:29:02.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.062+0000 7fde77fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde780d2910 0x7fde78149510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:02.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.062+0000 7fde77fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde780d2910 0x7fde78149510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38288/0 (socket says 172.21.15.46:38288) 2023-12-02T00:29:02.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.062+0000 7fde77fff640 1 -- 172.21.15.46:0/553315697 learned_addr learned my addr 172.21.15.46:0/553315697 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:02.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.062+0000 7fde777fe640 1 --2- 172.21.15.46:0/553315697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde780d3900 0x7fde7815ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:02.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.062+0000 7fde7c9da640 1 --2- 172.21.15.46:0/553315697 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fde7815d330 0x7fde781604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:02.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.062+0000 7fde77fff640 1 -- 172.21.15.46:0/553315697 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fde7815d330 msgr2=0x7fde781604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:02.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.062+0000 7fde77fff640 1 --2- 172.21.15.46:0/553315697 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fde7815d330 0x7fde781604b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:02.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.062+0000 7fde77fff640 1 -- 172.21.15.46:0/553315697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde780d3900 msgr2=0x7fde7815ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:02.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.062+0000 7fde77fff640 1 --2- 172.21.15.46:0/553315697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde780d3900 0x7fde7815ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:02.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.062+0000 7fde77fff640 1 -- 172.21.15.46:0/553315697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde6c009400 con 0x7fde780d2910 2023-12-02T00:29:02.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.062+0000 7fde777fe640 1 --2- 172.21.15.46:0/553315697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde780d3900 0x7fde7815ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:02.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.062+0000 7fde77fff640 1 --2- 172.21.15.46:0/553315697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde780d2910 0x7fde78149510 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fde6c00e960 tx=0x7fde6c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:02.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.063+0000 7fde74ff9640 1 -- 172.21.15.46:0/553315697 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6c00ca00 con 0x7fde780d2910 2023-12-02T00:29:02.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.063+0000 7fde74ff9640 1 -- 172.21.15.46:0/553315697 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde6c00cb60 con 0x7fde780d2910 2023-12-02T00:29:02.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.063+0000 7fde74ff9640 1 -- 172.21.15.46:0/553315697 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6c013470 con 0x7fde780d2910 2023-12-02T00:29:02.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.063+0000 7fde7e43d640 1 -- 172.21.15.46:0/553315697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde70009b30 con 0x7fde780d2910 2023-12-02T00:29:02.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.063+0000 7fde7e43d640 1 -- 172.21.15.46:0/553315697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde78149c90 con 0x7fde780d2910 2023-12-02T00:29:02.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.064+0000 7fde7e43d640 1 -- 172.21.15.46:0/553315697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde78069400 con 0x7fde780d2910 2023-12-02T00:29:02.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.064+0000 7fde74ff9640 1 -- 172.21.15.46:0/553315697 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fde6c0135d0 con 0x7fde780d2910 2023-12-02T00:29:02.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.064+0000 7fde74ff9640 1 --2- 172.21.15.46:0/553315697 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fde6406ae40 0x7fde6406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:02.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.064+0000 7fde74ff9640 1 -- 172.21.15.46:0/553315697 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fde6c0898a0 con 0x7fde780d2910 2023-12-02T00:29:02.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.065+0000 7fde777fe640 1 --2- 172.21.15.46:0/553315697 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fde6406ae40 0x7fde6406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:02.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.065+0000 7fde777fe640 1 --2- 172.21.15.46:0/553315697 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fde6406ae40 0x7fde6406d300 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fde68007fd0 tx=0x7fde68007d40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:02.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.069+0000 7fde74ff9640 1 -- 172.21.15.46:0/553315697 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fde6c054960 con 0x7fde780d2910 2023-12-02T00:29:02.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.284+0000 7fde7e43d640 1 -- 172.21.15.46:0/553315697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fde780d07d0 con 0x7fde780d2910 2023-12-02T00:29:02.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.285+0000 7fde74ff9640 1 -- 172.21.15.46:0/553315697 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fde6c057f80 con 0x7fde780d2910 2023-12-02T00:29:02.286 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:29:02.286 INFO:teuthology.orchestra.run.smithi046.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-12-02T00:29:02.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.286+0000 7fde7e43d640 1 -- 172.21.15.46:0/553315697 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fde6406ae40 msgr2=0x7fde6406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:02.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.286+0000 7fde7e43d640 1 --2- 172.21.15.46:0/553315697 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fde6406ae40 0x7fde6406d300 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fde68007fd0 tx=0x7fde68007d40 comp rx=0 tx=0).stop 2023-12-02T00:29:02.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.286+0000 7fde7e43d640 1 -- 172.21.15.46:0/553315697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde780d2910 msgr2=0x7fde78149510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:02.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.286+0000 7fde7e43d640 1 --2- 172.21.15.46:0/553315697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde780d2910 0x7fde78149510 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fde6c00e960 tx=0x7fde6c00ee20 comp rx=0 tx=0).stop 2023-12-02T00:29:02.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.287+0000 7fde7e43d640 1 -- 172.21.15.46:0/553315697 shutdown_connections 2023-12-02T00:29:02.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.287+0000 7fde7e43d640 1 --2- 172.21.15.46:0/553315697 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fde6406ae40 0x7fde6406d300 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:02.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.287+0000 7fde7e43d640 1 --2- 172.21.15.46:0/553315697 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fde7815d330 0x7fde781604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:02.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.287+0000 7fde7e43d640 1 --2- 172.21.15.46:0/553315697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde780d3900 0x7fde7815ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:02.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.287+0000 7fde7e43d640 1 --2- 172.21.15.46:0/553315697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde780d2910 0x7fde78149510 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:02.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.287+0000 7fde7e43d640 1 -- 172.21.15.46:0/553315697 >> 172.21.15.46:0/553315697 conn(0x7fde78072020 msgr2=0x7fde780d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:02.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.287+0000 7fde7e43d640 1 -- 172.21.15.46:0/553315697 shutdown_connections 2023-12-02T00:29:02.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.287+0000 7fde7e43d640 1 -- 172.21.15.46:0/553315697 wait complete. 2023-12-02T00:29:02.299 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-12-02T00:29:02.299 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:29:02.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.408+0000 7f98d0fa9640 1 Processor -- start 2023-12-02T00:29:02.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.408+0000 7f98d0fa9640 1 -- start start 2023-12-02T00:29:02.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.409+0000 7f98d0fa9640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f98cc143490 0x7f98cc0b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:02.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.409+0000 7f98d0fa9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f98cc15e730 0x7f98cc0b1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:02.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.409+0000 7f98d0fa9640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98cc15e2a0 con 0x7f98cc15a4e0 2023-12-02T00:29:02.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.409+0000 7f98d0fa9640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98cc15b520 con 0x7f98cc143490 2023-12-02T00:29:02.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.409+0000 7f98d0fa9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc143870 con 0x7f98cc15e730 2023-12-02T00:29:02.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.409+0000 7f98ca59c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f98cc143490 0x7f98cc0b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:02.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.409+0000 7f98cad9d640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f98cc15a4e0 0x7f98cc15a970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45070/0 (socket says 172.21.15.46:45070) 2023-12-02T00:29:02.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.409+0000 7f98cad9d640 1 -- 172.21.15.46:0/2278430927 learned_addr learned my addr 172.21.15.46:0/2278430927 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:02.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.409+0000 7f98c9d9b640 1 --2- 172.21.15.46:0/2278430927 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f98cc15e730 0x7f98cc0b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:02.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.409+0000 7f98ca59c640 1 -- 172.21.15.46:0/2278430927 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f98cc15e730 msgr2=0x7f98cc0b1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:02.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.409+0000 7f98ca59c640 1 --2- 172.21.15.46:0/2278430927 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f98cc15e730 0x7f98cc0b1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:02.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.409+0000 7f98ca59c640 1 -- 172.21.15.46:0/2278430927 >> v1:172.21.15.46:6789/0 conn(0x7f98cc15a4e0 legacy=0x7f98cc15a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:02.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.409+0000 7f98ca59c640 1 -- 172.21.15.46:0/2278430927 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98cc15b240 con 0x7f98cc143490 2023-12-02T00:29:02.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.409+0000 7f98c959a640 1 -- 172.21.15.46:0/2278430927 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2974856254 0 0) 0x7f98cc15e2a0 con 0x7f98cc15a4e0 2023-12-02T00:29:02.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.410+0000 7f98ca59c640 1 --2- 172.21.15.46:0/2278430927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98cc143490 0x7f98cc0b0cf0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f98bc0097a0 tx=0x7f98bc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4de74b31e057a8f6 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:02.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.410+0000 7f98c959a640 1 -- 172.21.15.46:0/2278430927 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98bc031830 con 0x7f98cc143490 2023-12-02T00:29:02.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.410+0000 7f98c959a640 1 -- 172.21.15.46:0/2278430927 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98bc031990 con 0x7f98cc143490 2023-12-02T00:29:02.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.410+0000 7f98c959a640 1 -- 172.21.15.46:0/2278430927 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98bc031d20 con 0x7f98cc143490 2023-12-02T00:29:02.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.411+0000 7f98d0fa9640 1 -- 172.21.15.46:0/2278430927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98cc143490 msgr2=0x7f98cc0b0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:02.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.411+0000 7f98d0fa9640 1 --2- 172.21.15.46:0/2278430927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98cc143490 0x7f98cc0b0cf0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f98bc0097a0 tx=0x7f98bc02b8d0 comp rx=0 tx=0).stop 2023-12-02T00:29:02.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.411+0000 7f98d0fa9640 1 -- 172.21.15.46:0/2278430927 shutdown_connections 2023-12-02T00:29:02.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.411+0000 7f98d0fa9640 1 --2- 172.21.15.46:0/2278430927 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f98cc15e730 0x7f98cc0b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:02.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.411+0000 7f98d0fa9640 1 --2- 172.21.15.46:0/2278430927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98cc143490 0x7f98cc0b0cf0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:02.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.411+0000 7f98d0fa9640 1 -- 172.21.15.46:0/2278430927 >> 172.21.15.46:0/2278430927 conn(0x7f98cc06a4d0 msgr2=0x7f98cc06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:02.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.411+0000 7f98d0fa9640 1 -- 172.21.15.46:0/2278430927 shutdown_connections 2023-12-02T00:29:02.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.411+0000 7f98d0fa9640 1 -- 172.21.15.46:0/2278430927 wait complete. 2023-12-02T00:29:02.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.412+0000 7f98d0fa9640 1 Processor -- start 2023-12-02T00:29:02.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.412+0000 7f98d0fa9640 1 -- start start 2023-12-02T00:29:02.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.413+0000 7f98d0fa9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f98cc143490 0x7f98cc1598d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:02.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.413+0000 7f98d0fa9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98cc15a4e0 0x7f98cc1e5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:02.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.413+0000 7f98d0fa9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98cc15e730 0x7f98cc1e5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:02.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.413+0000 7f98d0fa9640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc0acdf0 con 0x7f98cc15e730 2023-12-02T00:29:02.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.413+0000 7f98d0fa9640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc1e4a10 con 0x7f98cc15a4e0 2023-12-02T00:29:02.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.413+0000 7f98d0fa9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc1e4cf0 con 0x7f98cc143490 2023-12-02T00:29:02.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.413+0000 7f98cad9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98cc15e730 0x7f98cc1e5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:02.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.413+0000 7f98cad9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98cc15e730 0x7f98cc1e5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38314/0 (socket says 172.21.15.46:38314) 2023-12-02T00:29:02.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.413+0000 7f98cad9d640 1 -- 172.21.15.46:0/2279521025 learned_addr learned my addr 172.21.15.46:0/2279521025 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:02.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.413+0000 7f98ca59c640 1 --2- 172.21.15.46:0/2279521025 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f98cc143490 0x7f98cc1598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:02.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.413+0000 7f98c9d9b640 1 --2- 172.21.15.46:0/2279521025 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98cc15a4e0 0x7f98cc1e5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:02.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.413+0000 7f98cad9d640 1 -- 172.21.15.46:0/2279521025 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f98cc143490 msgr2=0x7f98cc1598d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:02.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.413+0000 7f98cad9d640 1 --2- 172.21.15.46:0/2279521025 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f98cc143490 0x7f98cc1598d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:02.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.413+0000 7f98cad9d640 1 -- 172.21.15.46:0/2279521025 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98cc15a4e0 msgr2=0x7f98cc1e5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:02.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.413+0000 7f98cad9d640 1 --2- 172.21.15.46:0/2279521025 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98cc15a4e0 0x7f98cc1e5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:02.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.413+0000 7f98cad9d640 1 -- 172.21.15.46:0/2279521025 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98bc009400 con 0x7f98cc15e730 2023-12-02T00:29:02.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.413+0000 7f98c9d9b640 1 --2- 172.21.15.46:0/2279521025 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98cc15a4e0 0x7f98cc1e5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:02.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.413+0000 7f98cad9d640 1 --2- 172.21.15.46:0/2279521025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98cc15e730 0x7f98cc1e5680 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f98c400c7e0 tx=0x7f98c400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:02.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.414+0000 7f98baffd640 1 -- 172.21.15.46:0/2279521025 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c400a640 con 0x7f98cc15e730 2023-12-02T00:29:02.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.414+0000 7f98baffd640 1 -- 172.21.15.46:0/2279521025 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c400a7c0 con 0x7f98cc15e730 2023-12-02T00:29:02.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.414+0000 7f98d0fa9640 1 -- 172.21.15.46:0/2279521025 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98cc0aca80 con 0x7f98cc15e730 2023-12-02T00:29:02.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.414+0000 7f98baffd640 1 -- 172.21.15.46:0/2279521025 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c400ab70 con 0x7f98cc15e730 2023-12-02T00:29:02.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.414+0000 7f98d0fa9640 1 -- 172.21.15.46:0/2279521025 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98cc131fe0 con 0x7f98cc15e730 2023-12-02T00:29:02.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.415+0000 7f98d0fa9640 1 -- 172.21.15.46:0/2279521025 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98cc0b4380 con 0x7f98cc15e730 2023-12-02T00:29:02.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.415+0000 7f98baffd640 1 -- 172.21.15.46:0/2279521025 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f98c400acd0 con 0x7f98cc15e730 2023-12-02T00:29:02.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.416+0000 7f98baffd640 1 --2- 172.21.15.46:0/2279521025 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f98ac06ae40 0x7f98ac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:02.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.416+0000 7f98baffd640 1 -- 172.21.15.46:0/2279521025 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f98c4088d40 con 0x7f98cc15e730 2023-12-02T00:29:02.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.416+0000 7f98ca59c640 1 --2- 172.21.15.46:0/2279521025 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f98ac06ae40 0x7f98ac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:02.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.416+0000 7f98ca59c640 1 --2- 172.21.15.46:0/2279521025 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f98ac06ae40 0x7f98ac06d300 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f98bc036000 tx=0x7f98bc03c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:02.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.420+0000 7f98baffd640 1 -- 172.21.15.46:0/2279521025 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f98c4053f30 con 0x7f98cc15e730 2023-12-02T00:29:02.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.599+0000 7f98d0fa9640 1 -- 172.21.15.46:0/2279521025 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f98cc0ac8e0 con 0x7f98cc15e730 2023-12-02T00:29:02.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.601+0000 7f98baffd640 1 -- 172.21.15.46:0/2279521025 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f98cc0ac8e0 con 0x7f98cc15e730 2023-12-02T00:29:02.602 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:29:02.602 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":21,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:28:50.700112+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-02T00:28:47.273337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-02T00:28:48.060115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:29:02.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.603+0000 7f98d0fa9640 1 -- 172.21.15.46:0/2279521025 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f98ac06ae40 msgr2=0x7f98ac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:02.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.603+0000 7f98d0fa9640 1 --2- 172.21.15.46:0/2279521025 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f98ac06ae40 0x7f98ac06d300 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f98bc036000 tx=0x7f98bc03c040 comp rx=0 tx=0).stop 2023-12-02T00:29:02.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.603+0000 7f98d0fa9640 1 -- 172.21.15.46:0/2279521025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98cc15e730 msgr2=0x7f98cc1e5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:02.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.603+0000 7f98d0fa9640 1 --2- 172.21.15.46:0/2279521025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98cc15e730 0x7f98cc1e5680 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f98c400c7e0 tx=0x7f98c400cca0 comp rx=0 tx=0).stop 2023-12-02T00:29:02.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.603+0000 7f98d0fa9640 1 -- 172.21.15.46:0/2279521025 shutdown_connections 2023-12-02T00:29:02.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.603+0000 7f98d0fa9640 1 --2- 172.21.15.46:0/2279521025 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f98ac06ae40 0x7f98ac06d300 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:02.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.603+0000 7f98d0fa9640 1 --2- 172.21.15.46:0/2279521025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98cc15e730 0x7f98cc1e5680 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:02.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.603+0000 7f98d0fa9640 1 --2- 172.21.15.46:0/2279521025 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98cc15a4e0 0x7f98cc1e5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:02.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.603+0000 7f98d0fa9640 1 --2- 172.21.15.46:0/2279521025 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f98cc143490 0x7f98cc1598d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:02.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.603+0000 7f98d0fa9640 1 -- 172.21.15.46:0/2279521025 >> 172.21.15.46:0/2279521025 conn(0x7f98cc06a4d0 msgr2=0x7f98cc0b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:02.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.603+0000 7f98d0fa9640 1 -- 172.21.15.46:0/2279521025 shutdown_connections 2023-12-02T00:29:02.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.603+0000 7f98d0fa9640 1 -- 172.21.15.46:0/2279521025 wait complete. 2023-12-02T00:29:02.618 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-02T00:28:37.213645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-02T00:28:47.273337+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-02T00:28:48.060115+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-02T00:29:02.619 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-02T00:29:02.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.733+0000 7f602d44f640 1 Processor -- start 2023-12-02T00:29:02.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.733+0000 7f602d44f640 1 -- start start 2023-12-02T00:29:02.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.733+0000 7f602d44f640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f602806abe0 0x7f60281618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:02.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.733+0000 7f602d44f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60281409d0 0x7f602806efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:02.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.733+0000 7f602d44f640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60281406f0 con 0x7f6028140310 2023-12-02T00:29:02.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.733+0000 7f602d44f640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6028161e20 con 0x7f602806abe0 2023-12-02T00:29:02.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.733+0000 7f602d44f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6028143960 con 0x7f60281409d0 2023-12-02T00:29:02.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.734+0000 7f601ffff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f602806abe0 0x7f60281618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:02.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.734+0000 7f601ffff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f602806abe0 0x7f60281618e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37844/0 (socket says 172.21.15.46:37844) 2023-12-02T00:29:02.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.734+0000 7f601ffff640 1 -- 172.21.15.46:0/3427203174 learned_addr learned my addr 172.21.15.46:0/3427203174 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:02.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.734+0000 7f601ffff640 1 -- 172.21.15.46:0/3427203174 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60281409d0 msgr2=0x7f602806efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:02.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.734+0000 7f60277fe640 1 --2- 172.21.15.46:0/3427203174 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60281409d0 0x7f602806efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:02.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.734+0000 7f601ffff640 1 --2- 172.21.15.46:0/3427203174 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60281409d0 0x7f602806efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:02.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.734+0000 7f601ffff640 1 -- 172.21.15.46:0/3427203174 >> v1:172.21.15.162:6789/0 conn(0x7f6028140310 legacy=0x7f602806a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:02.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.734+0000 7f601ffff640 1 -- 172.21.15.46:0/3427203174 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6028142de0 con 0x7f602806abe0 2023-12-02T00:29:02.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.735+0000 7f601ffff640 1 --2- 172.21.15.46:0/3427203174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f602806abe0 0x7f60281618e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6010009870 tx=0x7f601002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddc236b77c272b80 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:02.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.735+0000 7f60267fc640 1 -- 172.21.15.46:0/3427203174 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60100317c0 con 0x7f602806abe0 2023-12-02T00:29:02.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.735+0000 7f60267fc640 1 -- 172.21.15.46:0/3427203174 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6010031920 con 0x7f602806abe0 2023-12-02T00:29:02.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.735+0000 7f60267fc640 1 -- 172.21.15.46:0/3427203174 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6010031cb0 con 0x7f602806abe0 2023-12-02T00:29:02.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.735+0000 7f602d44f640 1 -- 172.21.15.46:0/3427203174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f602806abe0 msgr2=0x7f60281618e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:02.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.735+0000 7f602d44f640 1 --2- 172.21.15.46:0/3427203174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f602806abe0 0x7f60281618e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6010009870 tx=0x7f601002b860 comp rx=0 tx=0).stop 2023-12-02T00:29:02.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.735+0000 7f602d44f640 1 -- 172.21.15.46:0/3427203174 shutdown_connections 2023-12-02T00:29:02.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.735+0000 7f602d44f640 1 --2- 172.21.15.46:0/3427203174 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60281409d0 0x7f602806efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:02.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.735+0000 7f602d44f640 1 --2- 172.21.15.46:0/3427203174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f602806abe0 0x7f60281618e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:02.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.735+0000 7f602d44f640 1 -- 172.21.15.46:0/3427203174 >> 172.21.15.46:0/3427203174 conn(0x7f60280b1e50 msgr2=0x7f60280b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:02.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.735+0000 7f602d44f640 1 -- 172.21.15.46:0/3427203174 shutdown_connections 2023-12-02T00:29:02.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.735+0000 7f602d44f640 1 -- 172.21.15.46:0/3427203174 wait complete. 2023-12-02T00:29:02.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.737+0000 7f602d44f640 1 Processor -- start 2023-12-02T00:29:02.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.737+0000 7f602d44f640 1 -- start start 2023-12-02T00:29:02.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.737+0000 7f602d44f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f602806abe0 0x7f60281f5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:02.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.737+0000 7f602d44f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6028140310 0x7f60280b10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:02.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.737+0000 7f602d44f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60281409d0 0x7f6028154950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:02.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.737+0000 7f602d44f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6028157a30 con 0x7f602806abe0 2023-12-02T00:29:02.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.737+0000 7f602d44f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60281575a0 con 0x7f60281409d0 2023-12-02T00:29:02.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.737+0000 7f602d44f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6028156a00 con 0x7f6028140310 2023-12-02T00:29:02.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.738+0000 7f6026ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f602806abe0 0x7f60281f5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:02.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.738+0000 7f6026ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f602806abe0 0x7f60281f5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37856/0 (socket says 172.21.15.46:37856) 2023-12-02T00:29:02.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.738+0000 7f6026ffd640 1 -- 172.21.15.46:0/1064484540 learned_addr learned my addr 172.21.15.46:0/1064484540 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:02.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.738+0000 7f60277fe640 1 --2- 172.21.15.46:0/1064484540 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60281409d0 0x7f6028154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:02.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.738+0000 7f601ffff640 1 --2- 172.21.15.46:0/1064484540 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6028140310 0x7f60280b10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:02.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.738+0000 7f6026ffd640 1 -- 172.21.15.46:0/1064484540 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6028140310 msgr2=0x7f60280b10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:02.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.738+0000 7f6026ffd640 1 --2- 172.21.15.46:0/1064484540 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6028140310 0x7f60280b10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:02.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.738+0000 7f6026ffd640 1 -- 172.21.15.46:0/1064484540 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60281409d0 msgr2=0x7f6028154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:02.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.738+0000 7f6026ffd640 1 --2- 172.21.15.46:0/1064484540 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60281409d0 0x7f6028154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:02.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.738+0000 7f6026ffd640 1 -- 172.21.15.46:0/1064484540 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60100094d0 con 0x7f602806abe0 2023-12-02T00:29:02.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.738+0000 7f6026ffd640 1 --2- 172.21.15.46:0/1064484540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f602806abe0 0x7f60281f5ac0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f601800c460 tx=0x7f601800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:02.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.738+0000 7f601f7fe640 1 -- 172.21.15.46:0/1064484540 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6018012830 con 0x7f602806abe0 2023-12-02T00:29:02.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.738+0000 7f601f7fe640 1 -- 172.21.15.46:0/1064484540 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60180129d0 con 0x7f602806abe0 2023-12-02T00:29:02.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.738+0000 7f601f7fe640 1 -- 172.21.15.46:0/1064484540 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6018012d80 con 0x7f602806abe0 2023-12-02T00:29:02.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.738+0000 7f602d44f640 1 -- 172.21.15.46:0/1064484540 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6028156c70 con 0x7f602806abe0 2023-12-02T00:29:02.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.738+0000 7f602d44f640 1 -- 172.21.15.46:0/1064484540 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6028157230 con 0x7f602806abe0 2023-12-02T00:29:02.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.739+0000 7f602d44f640 1 -- 172.21.15.46:0/1064484540 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6028069400 con 0x7f602806abe0 2023-12-02T00:29:02.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.740+0000 7f601f7fe640 1 -- 172.21.15.46:0/1064484540 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f601801a3b0 con 0x7f602806abe0 2023-12-02T00:29:02.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.740+0000 7f601f7fe640 1 --2- 172.21.15.46:0/1064484540 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f600806ae70 0x7f600806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:02.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.740+0000 7f601f7fe640 1 -- 172.21.15.46:0/1064484540 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6018089500 con 0x7f602806abe0 2023-12-02T00:29:02.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.740+0000 7f601ffff640 1 --2- 172.21.15.46:0/1064484540 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f600806ae70 0x7f600806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:02.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.741+0000 7f601ffff640 1 --2- 172.21.15.46:0/1064484540 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f600806ae70 0x7f600806d330 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6010036000 tx=0x7f601003d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:02.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.744+0000 7f601f7fe640 1 -- 172.21.15.46:0/1064484540 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f60180545b0 con 0x7f602806abe0 2023-12-02T00:29:02.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.918+0000 7f602d44f640 1 -- 172.21.15.46:0/1064484540 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f602806ea60 con 0x7f602806abe0 2023-12-02T00:29:02.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.919+0000 7f601f7fe640 1 -- 172.21.15.46:0/1064484540 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6018057bd0 con 0x7f602806abe0 2023-12-02T00:29:02.920 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-12-02T00:29:02.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.921+0000 7f602d44f640 1 -- 172.21.15.46:0/1064484540 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f600806ae70 msgr2=0x7f600806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:02.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.921+0000 7f602d44f640 1 --2- 172.21.15.46:0/1064484540 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f600806ae70 0x7f600806d330 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6010036000 tx=0x7f601003d040 comp rx=0 tx=0).stop 2023-12-02T00:29:02.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.921+0000 7f602d44f640 1 -- 172.21.15.46:0/1064484540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f602806abe0 msgr2=0x7f60281f5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:02.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.921+0000 7f602d44f640 1 --2- 172.21.15.46:0/1064484540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f602806abe0 0x7f60281f5ac0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f601800c460 tx=0x7f601800c920 comp rx=0 tx=0).stop 2023-12-02T00:29:02.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.921+0000 7f602d44f640 1 -- 172.21.15.46:0/1064484540 shutdown_connections 2023-12-02T00:29:02.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.921+0000 7f602d44f640 1 --2- 172.21.15.46:0/1064484540 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f600806ae70 0x7f600806d330 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:02.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.921+0000 7f602d44f640 1 --2- 172.21.15.46:0/1064484540 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60281409d0 0x7f6028154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:02.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.921+0000 7f602d44f640 1 --2- 172.21.15.46:0/1064484540 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6028140310 0x7f60280b10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:02.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.921+0000 7f602d44f640 1 --2- 172.21.15.46:0/1064484540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f602806abe0 0x7f60281f5ac0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:02.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.921+0000 7f602d44f640 1 -- 172.21.15.46:0/1064484540 >> 172.21.15.46:0/1064484540 conn(0x7f60280b1e50 msgr2=0x7f602813e540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:02.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.921+0000 7f602d44f640 1 -- 172.21.15.46:0/1064484540 shutdown_connections 2023-12-02T00:29:02.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:02.921+0000 7f602d44f640 1 -- 172.21.15.46:0/1064484540 wait complete. 2023-12-02T00:29:02.933 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-02T00:29:03.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.044+0000 7f5efa190640 1 Processor -- start 2023-12-02T00:29:03.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.044+0000 7f5efa190640 1 -- start start 2023-12-02T00:29:03.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.044+0000 7f5efa190640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5ef415d330 0x7f5ef40b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:03.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.044+0000 7f5efa190640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5ef40d2910 0x7f5ef40d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:03.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.044+0000 7f5efa190640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ef40b2b10 con 0x7f5ef40d3730 2023-12-02T00:29:03.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.044+0000 7f5efa190640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ef40d3b50 con 0x7f5ef415d330 2023-12-02T00:29:03.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.044+0000 7f5efa190640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef40d1590 con 0x7f5ef40d2910 2023-12-02T00:29:03.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.045+0000 7f5ef3fff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5ef40d3730 0x7f5ef40b6390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58250/0 (socket says 172.21.15.46:58250) 2023-12-02T00:29:03.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.045+0000 7f5ef3fff640 1 -- 172.21.15.46:0/2165096131 learned_addr learned my addr 172.21.15.46:0/2165096131 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:03.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.045+0000 7f5ef2ffd640 1 --2- 172.21.15.46:0/2165096131 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5ef40d2910 0x7f5ef40d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:03.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.045+0000 7f5ef37fe640 1 --2- 172.21.15.46:0/2165096131 >> v2:172.21.15.162:3300/0 conn(0x7f5ef415d330 0x7f5ef40b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:03.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.045+0000 7f5ef27fc640 1 -- 172.21.15.46:0/2165096131 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3873936147 0 0) 0x7f5ef40b2b10 con 0x7f5ef40d3730 2023-12-02T00:29:03.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.045+0000 7f5ef27fc640 1 -- 172.21.15.46:0/2165096131 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5edc003520 con 0x7f5ef40d3730 2023-12-02T00:29:03.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.045+0000 7f5ef27fc640 1 -- 172.21.15.46:0/2165096131 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1764459187 0 0) 0x7f5edc003520 con 0x7f5ef40d3730 2023-12-02T00:29:03.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.045+0000 7f5ef27fc640 1 -- 172.21.15.46:0/2165096131 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5ef40d2910 msgr2=0x7f5ef40d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:03.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.045+0000 7f5ef27fc640 1 --2- 172.21.15.46:0/2165096131 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5ef40d2910 0x7f5ef40d31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:03.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.045+0000 7f5ef27fc640 1 -- 172.21.15.46:0/2165096131 >> v2:172.21.15.162:3300/0 conn(0x7f5ef415d330 msgr2=0x7f5ef40b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:03.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.045+0000 7f5ef27fc640 1 --2- 172.21.15.46:0/2165096131 >> v2:172.21.15.162:3300/0 conn(0x7f5ef415d330 0x7f5ef40b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:03.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.045+0000 7f5ef27fc640 1 -- 172.21.15.46:0/2165096131 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef40d1870 con 0x7f5ef40d3730 2023-12-02T00:29:03.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.045+0000 7f5ef27fc640 1 -- 172.21.15.46:0/2165096131 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7f5eec0019f0 con 0x7f5ef40d3730 2023-12-02T00:29:03.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.045+0000 7f5ef37fe640 1 --2- 172.21.15.46:0/2165096131 >> v2:172.21.15.162:3300/0 conn(0x7f5ef415d330 0x7f5ef40b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:29:03.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.045+0000 7f5ef2ffd640 1 --2- 172.21.15.46:0/2165096131 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5ef40d2910 0x7f5ef40d31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:29:03.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.045+0000 7f5ef27fc640 1 -- 172.21.15.46:0/2165096131 >> v1:172.21.15.46:6789/0 conn(0x7f5ef40d3730 legacy=0x7f5ef40b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:03.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.045+0000 7f5ef27fc640 1 --2- 172.21.15.46:0/2165096131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5edc003970 0x7f5edc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:03.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.045+0000 7f5ef27fc640 1 --2- 172.21.15.46:0/2165096131 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5edc0047c0 0x7f5edc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:03.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.045+0000 7f5ef27fc640 1 --2- 172.21.15.46:0/2165096131 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5ef40d2910 0x7f5edc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:03.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.045+0000 7f5ef27fc640 1 -- 172.21.15.46:0/2165096131 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef40d3b50 con 0x7f5edc003970 2023-12-02T00:29:03.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.045+0000 7f5ef27fc640 1 -- 172.21.15.46:0/2165096131 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef40d1590 con 0x7f5edc0047c0 2023-12-02T00:29:03.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.045+0000 7f5ef27fc640 1 -- 172.21.15.46:0/2165096131 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5edc00b980 con 0x7f5ef40d2910 2023-12-02T00:29:03.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.045+0000 7f5ef37fe640 1 --2- 172.21.15.46:0/2165096131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5edc003970 0x7f5edc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:03.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.045+0000 7f5ef2ffd640 1 --2- 172.21.15.46:0/2165096131 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5edc0047c0 0x7f5edc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:03.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.045+0000 7f5ef3fff640 1 --2- 172.21.15.46:0/2165096131 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5ef40d2910 0x7f5edc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:03.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.046+0000 7f5ef37fe640 1 -- 172.21.15.46:0/2165096131 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5ef40d2910 msgr2=0x7f5edc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:03.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.046+0000 7f5ef37fe640 1 --2- 172.21.15.46:0/2165096131 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5ef40d2910 0x7f5edc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:03.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.046+0000 7f5ef37fe640 1 -- 172.21.15.46:0/2165096131 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5edc0047c0 msgr2=0x7f5edc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:03.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.046+0000 7f5ef37fe640 1 --2- 172.21.15.46:0/2165096131 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5edc0047c0 0x7f5edc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:03.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.046+0000 7f5ef37fe640 1 -- 172.21.15.46:0/2165096131 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee8009400 con 0x7f5edc003970 2023-12-02T00:29:03.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.046+0000 7f5ef2ffd640 1 --2- 172.21.15.46:0/2165096131 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5edc0047c0 0x7f5edc004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:03.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.046+0000 7f5ef37fe640 1 --2- 172.21.15.46:0/2165096131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5edc003970 0x7f5edc003d90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4009c90 tx=0x7f5edc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=986b74760baa73ff server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:03.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.046+0000 7f5ef27fc640 1 -- 172.21.15.46:0/2165096131 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee402c980 con 0x7f5edc003970 2023-12-02T00:29:03.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.046+0000 7f5ef27fc640 1 -- 172.21.15.46:0/2165096131 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee402cae0 con 0x7f5edc003970 2023-12-02T00:29:03.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.046+0000 7f5ef27fc640 1 -- 172.21.15.46:0/2165096131 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee402ce50 con 0x7f5edc003970 2023-12-02T00:29:03.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.047+0000 7f5efa190640 1 -- 172.21.15.46:0/2165096131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5edc003970 msgr2=0x7f5edc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:03.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.047+0000 7f5efa190640 1 --2- 172.21.15.46:0/2165096131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5edc003970 0x7f5edc003d90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4009c90 tx=0x7f5edc0057f0 comp rx=0 tx=0).stop 2023-12-02T00:29:03.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.047+0000 7f5ef3fff640 1 -- 172.21.15.46:0/2165096131 reap_dead start 2023-12-02T00:29:03.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.047+0000 7f5efa190640 1 -- 172.21.15.46:0/2165096131 shutdown_connections 2023-12-02T00:29:03.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.047+0000 7f5efa190640 1 -- 172.21.15.46:0/2165096131 >> 172.21.15.46:0/2165096131 conn(0x7f5ef4072020 msgr2=0x7f5ef4072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:03.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.047+0000 7f5efa190640 1 -- 172.21.15.46:0/2165096131 shutdown_connections 2023-12-02T00:29:03.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.047+0000 7f5efa190640 1 -- 172.21.15.46:0/2165096131 wait complete. 2023-12-02T00:29:03.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.048+0000 7f5efa190640 1 Processor -- start 2023-12-02T00:29:03.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.048+0000 7f5efa190640 1 -- start start 2023-12-02T00:29:03.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.049+0000 7f5efa190640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5edc003970 0x7f5ef40c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:03.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.049+0000 7f5efa190640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ef41536d0 0x7f5ef4145cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:03.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.049+0000 7f5efa190640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ef4156190 0x7f5ef40cf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:03.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.049+0000 7f5efa190640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef4154e70 con 0x7f5ef4156190 2023-12-02T00:29:03.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.049+0000 7f5efa190640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef40c6270 con 0x7f5ef41536d0 2023-12-02T00:29:03.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.049+0000 7f5efa190640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef4146210 con 0x7f5edc003970 2023-12-02T00:29:03.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.049+0000 7f5ef3fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ef4156190 0x7f5ef40cf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:03.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.049+0000 7f5ef3fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ef4156190 0x7f5ef40cf7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37874/0 (socket says 172.21.15.46:37874) 2023-12-02T00:29:03.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.049+0000 7f5ef3fff640 1 -- 172.21.15.46:0/292625192 learned_addr learned my addr 172.21.15.46:0/292625192 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:03.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.049+0000 7f5ef37fe640 1 --2- 172.21.15.46:0/292625192 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5edc003970 0x7f5ef40c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:03.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.049+0000 7f5ef2ffd640 1 --2- 172.21.15.46:0/292625192 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ef41536d0 0x7f5ef4145cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:03.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.049+0000 7f5ef3fff640 1 -- 172.21.15.46:0/292625192 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5edc003970 msgr2=0x7f5ef40c5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:03.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.049+0000 7f5ef3fff640 1 --2- 172.21.15.46:0/292625192 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5edc003970 0x7f5ef40c5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:03.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.049+0000 7f5ef3fff640 1 -- 172.21.15.46:0/292625192 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ef41536d0 msgr2=0x7f5ef4145cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:03.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.050+0000 7f5ef3fff640 1 --2- 172.21.15.46:0/292625192 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ef41536d0 0x7f5ef4145cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:03.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.050+0000 7f5ef3fff640 1 -- 172.21.15.46:0/292625192 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef41550f0 con 0x7f5ef4156190 2023-12-02T00:29:03.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.050+0000 7f5ef2ffd640 1 --2- 172.21.15.46:0/292625192 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ef41536d0 0x7f5ef4145cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:29:03.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.050+0000 7f5ef3fff640 1 --2- 172.21.15.46:0/292625192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ef4156190 0x7f5ef40cf7a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5ef408c510 tx=0x7f5eec007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:03.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.050+0000 7f5edbfff640 1 -- 172.21.15.46:0/292625192 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eec00ad40 con 0x7f5ef4156190 2023-12-02T00:29:03.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.050+0000 7f5edbfff640 1 -- 172.21.15.46:0/292625192 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eec0133f0 con 0x7f5ef4156190 2023-12-02T00:29:03.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.050+0000 7f5efa190640 1 -- 172.21.15.46:0/292625192 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ef41553d0 con 0x7f5ef4156190 2023-12-02T00:29:03.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.050+0000 7f5edbfff640 1 -- 172.21.15.46:0/292625192 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eec0137a0 con 0x7f5ef4156190 2023-12-02T00:29:03.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.050+0000 7f5efa190640 1 -- 172.21.15.46:0/292625192 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ef4155990 con 0x7f5ef4156190 2023-12-02T00:29:03.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.051+0000 7f5efa190640 1 -- 172.21.15.46:0/292625192 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ef40693e0 con 0x7f5ef4156190 2023-12-02T00:29:03.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.052+0000 7f5edbfff640 1 -- 172.21.15.46:0/292625192 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5eec013900 con 0x7f5ef4156190 2023-12-02T00:29:03.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.052+0000 7f5edbfff640 1 --2- 172.21.15.46:0/292625192 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5ee006ae40 0x7f5ee006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:03.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.052+0000 7f5edbfff640 1 -- 172.21.15.46:0/292625192 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5eec08a070 con 0x7f5ef4156190 2023-12-02T00:29:03.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.052+0000 7f5ef37fe640 1 --2- 172.21.15.46:0/292625192 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5ee006ae40 0x7f5ee006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:03.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.053+0000 7f5ef37fe640 1 --2- 172.21.15.46:0/292625192 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5ee006ae40 0x7f5ee006d300 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4036000 tx=0x7f5ee4030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:03.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.056+0000 7f5edbfff640 1 -- 172.21.15.46:0/292625192 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5eec0551b0 con 0x7f5ef4156190 2023-12-02T00:29:03.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.231+0000 7f5efa190640 1 -- 172.21.15.46:0/292625192 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5ef415cce0 con 0x7f5ef4156190 2023-12-02T00:29:03.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.231+0000 7f5edbfff640 1 -- 172.21.15.46:0/292625192 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5eec0587d0 con 0x7f5ef4156190 2023-12-02T00:29:03.234 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-12-02T00:29:03.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.233+0000 7f5efa190640 1 -- 172.21.15.46:0/292625192 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5ee006ae40 msgr2=0x7f5ee006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:03.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.233+0000 7f5efa190640 1 --2- 172.21.15.46:0/292625192 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5ee006ae40 0x7f5ee006d300 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4036000 tx=0x7f5ee4030000 comp rx=0 tx=0).stop 2023-12-02T00:29:03.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.233+0000 7f5efa190640 1 -- 172.21.15.46:0/292625192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ef4156190 msgr2=0x7f5ef40cf7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:03.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.233+0000 7f5efa190640 1 --2- 172.21.15.46:0/292625192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ef4156190 0x7f5ef40cf7a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5ef408c510 tx=0x7f5eec007bd0 comp rx=0 tx=0).stop 2023-12-02T00:29:03.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.233+0000 7f5efa190640 1 -- 172.21.15.46:0/292625192 shutdown_connections 2023-12-02T00:29:03.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.233+0000 7f5efa190640 1 --2- 172.21.15.46:0/292625192 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5ee006ae40 0x7f5ee006d300 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:03.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.233+0000 7f5efa190640 1 --2- 172.21.15.46:0/292625192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ef4156190 0x7f5ef40cf7a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:03.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.233+0000 7f5efa190640 1 --2- 172.21.15.46:0/292625192 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ef41536d0 0x7f5ef4145cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:03.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.233+0000 7f5efa190640 1 --2- 172.21.15.46:0/292625192 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5edc003970 0x7f5ef40c5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:03.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.233+0000 7f5efa190640 1 -- 172.21.15.46:0/292625192 >> 172.21.15.46:0/292625192 conn(0x7f5ef4072020 msgr2=0x7f5ef40c7320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:03.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.233+0000 7f5efa190640 1 -- 172.21.15.46:0/292625192 shutdown_connections 2023-12-02T00:29:03.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.233+0000 7f5efa190640 1 -- 172.21.15.46:0/292625192 wait complete. 2023-12-02T00:29:03.245 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-02T00:29:03.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.355+0000 7f4aab3be640 1 Processor -- start 2023-12-02T00:29:03.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.356+0000 7f4aab3be640 1 -- start start 2023-12-02T00:29:03.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.356+0000 7f4aab3be640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4aa40d2910 0x7f4aa40d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:03.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.356+0000 7f4aab3be640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4aa40d3730 0x7f4aa40b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:03.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.356+0000 7f4aab3be640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4aa40b2b10 con 0x7f4aa415d330 2023-12-02T00:29:03.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.356+0000 7f4aab3be640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4aa40d1590 con 0x7f4aa40d2910 2023-12-02T00:29:03.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.356+0000 7f4aab3be640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa40d1870 con 0x7f4aa40d3730 2023-12-02T00:29:03.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.356+0000 7f4aa8959640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4aa40d2910 0x7f4aa40d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:03.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.356+0000 7f4aa915a640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4aa415d330 0x7f4aa40b8b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58262/0 (socket says 172.21.15.46:58262) 2023-12-02T00:29:03.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.356+0000 7f4aa915a640 1 -- 172.21.15.46:0/3477485090 learned_addr learned my addr 172.21.15.46:0/3477485090 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:03.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.357+0000 7f4aa995b640 1 --2- 172.21.15.46:0/3477485090 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4aa40d3730 0x7f4aa40b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:03.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.357+0000 7f4aa8959640 1 -- 172.21.15.46:0/3477485090 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4aa40d3730 msgr2=0x7f4aa40b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:03.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.357+0000 7f4aa8959640 1 --2- 172.21.15.46:0/3477485090 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4aa40d3730 0x7f4aa40b6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:03.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.357+0000 7f4aa8959640 1 -- 172.21.15.46:0/3477485090 >> v1:172.21.15.46:6789/0 conn(0x7f4aa415d330 legacy=0x7f4aa40b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:03.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.357+0000 7f4aa8959640 1 -- 172.21.15.46:0/3477485090 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aa40d3b90 con 0x7f4aa40d2910 2023-12-02T00:29:03.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.357+0000 7f4aa8959640 1 --2- 172.21.15.46:0/3477485090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4aa40d2910 0x7f4aa40d31f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4a940097a0 tx=0x7f4a9402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=247c94c6cc253d9a server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:03.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.358+0000 7f4a9bfff640 1 -- 172.21.15.46:0/3477485090 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9402e070 con 0x7f4aa40d2910 2023-12-02T00:29:03.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.358+0000 7f4a9bfff640 1 -- 172.21.15.46:0/3477485090 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a940315a0 con 0x7f4aa40d2910 2023-12-02T00:29:03.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.358+0000 7f4a9bfff640 1 -- 172.21.15.46:0/3477485090 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a94031930 con 0x7f4aa40d2910 2023-12-02T00:29:03.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.358+0000 7f4aab3be640 1 -- 172.21.15.46:0/3477485090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4aa40d2910 msgr2=0x7f4aa40d31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:03.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.358+0000 7f4aab3be640 1 --2- 172.21.15.46:0/3477485090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4aa40d2910 0x7f4aa40d31f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4a940097a0 tx=0x7f4a9402b840 comp rx=0 tx=0).stop 2023-12-02T00:29:03.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.358+0000 7f4aab3be640 1 -- 172.21.15.46:0/3477485090 shutdown_connections 2023-12-02T00:29:03.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.358+0000 7f4aab3be640 1 --2- 172.21.15.46:0/3477485090 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4aa40d3730 0x7f4aa40b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:03.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.358+0000 7f4aab3be640 1 --2- 172.21.15.46:0/3477485090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4aa40d2910 0x7f4aa40d31f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:03.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.358+0000 7f4aab3be640 1 -- 172.21.15.46:0/3477485090 >> 172.21.15.46:0/3477485090 conn(0x7f4aa4072020 msgr2=0x7f4aa4072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:03.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.358+0000 7f4aab3be640 1 -- 172.21.15.46:0/3477485090 shutdown_connections 2023-12-02T00:29:03.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.358+0000 7f4aab3be640 1 -- 172.21.15.46:0/3477485090 wait complete. 2023-12-02T00:29:03.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.359+0000 7f4aab3be640 1 Processor -- start 2023-12-02T00:29:03.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.359+0000 7f4aab3be640 1 -- start start 2023-12-02T00:29:03.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.360+0000 7f4aab3be640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4aa40d2910 0x7f4aa41497d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:03.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.360+0000 7f4aab3be640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4aa40d3730 0x7f4aa40cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:03.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.360+0000 7f4aab3be640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4aa415d330 0x7f4aa415d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:03.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.360+0000 7f4aab3be640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa40c8330 con 0x7f4aa415d330 2023-12-02T00:29:03.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.360+0000 7f4aab3be640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa40c76b0 con 0x7f4aa40d2910 2023-12-02T00:29:03.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.360+0000 7f4aab3be640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa40c7990 con 0x7f4aa40d3730 2023-12-02T00:29:03.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.360+0000 7f4aa995b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4aa415d330 0x7f4aa415d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:03.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.360+0000 7f4aa995b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4aa415d330 0x7f4aa415d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37898/0 (socket says 172.21.15.46:37898) 2023-12-02T00:29:03.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.360+0000 7f4aa995b640 1 -- 172.21.15.46:0/2747489748 learned_addr learned my addr 172.21.15.46:0/2747489748 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:03.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.360+0000 7f4aa915a640 1 --2- 172.21.15.46:0/2747489748 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4aa40d2910 0x7f4aa41497d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:03.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.360+0000 7f4aa8959640 1 --2- 172.21.15.46:0/2747489748 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4aa40d3730 0x7f4aa40cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:03.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.361+0000 7f4aa995b640 1 -- 172.21.15.46:0/2747489748 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4aa40d3730 msgr2=0x7f4aa40cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:03.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.361+0000 7f4aa995b640 1 --2- 172.21.15.46:0/2747489748 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4aa40d3730 0x7f4aa40cc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:03.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.361+0000 7f4aa995b640 1 -- 172.21.15.46:0/2747489748 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4aa40d2910 msgr2=0x7f4aa41497d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:03.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.361+0000 7f4aa995b640 1 --2- 172.21.15.46:0/2747489748 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4aa40d2910 0x7f4aa41497d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:03.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.361+0000 7f4aa995b640 1 -- 172.21.15.46:0/2747489748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a94009400 con 0x7f4aa415d330 2023-12-02T00:29:03.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.361+0000 7f4aa8959640 1 --2- 172.21.15.46:0/2747489748 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4aa40d3730 0x7f4aa40cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:03.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.361+0000 7f4aa995b640 1 --2- 172.21.15.46:0/2747489748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4aa415d330 0x7f4aa415d710 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c00efc0 tx=0x7f4a9c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:03.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.361+0000 7f4a99ffb640 1 -- 172.21.15.46:0/2747489748 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c014480 con 0x7f4aa415d330 2023-12-02T00:29:03.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.361+0000 7f4a99ffb640 1 -- 172.21.15.46:0/2747489748 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a9c00f040 con 0x7f4aa415d330 2023-12-02T00:29:03.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.361+0000 7f4aab3be640 1 -- 172.21.15.46:0/2747489748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aa40c7c70 con 0x7f4aa415d330 2023-12-02T00:29:03.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.361+0000 7f4a99ffb640 1 -- 172.21.15.46:0/2747489748 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c014820 con 0x7f4aa415d330 2023-12-02T00:29:03.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.361+0000 7f4aab3be640 1 -- 172.21.15.46:0/2747489748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aa4149d10 con 0x7f4aa415d330 2023-12-02T00:29:03.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.362+0000 7f4aab3be640 1 -- 172.21.15.46:0/2747489748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4aa4066da0 con 0x7f4aa415d330 2023-12-02T00:29:03.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.363+0000 7f4a99ffb640 1 -- 172.21.15.46:0/2747489748 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4a9c014980 con 0x7f4aa415d330 2023-12-02T00:29:03.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.363+0000 7f4a99ffb640 1 --2- 172.21.15.46:0/2747489748 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f4a8806ae40 0x7f4a8806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:03.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.363+0000 7f4a99ffb640 1 -- 172.21.15.46:0/2747489748 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4a9c08a560 con 0x7f4aa415d330 2023-12-02T00:29:03.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.363+0000 7f4aa915a640 1 --2- 172.21.15.46:0/2747489748 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f4a8806ae40 0x7f4a8806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:03.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.364+0000 7f4aa915a640 1 --2- 172.21.15.46:0/2747489748 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f4a8806ae40 0x7f4a8806d300 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0005880 tx=0x7f4aa0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:03.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.367+0000 7f4a99ffb640 1 -- 172.21.15.46:0/2747489748 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4a9c0554e0 con 0x7f4aa415d330 2023-12-02T00:29:03.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.542+0000 7f4aab3be640 1 -- 172.21.15.46:0/2747489748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4aa4069400 con 0x7f4aa415d330 2023-12-02T00:29:03.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.543+0000 7f4a99ffb640 1 -- 172.21.15.46:0/2747489748 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4a9c018020 con 0x7f4aa415d330 2023-12-02T00:29:03.544 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-12-02T00:29:03.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.545+0000 7f4aab3be640 1 -- 172.21.15.46:0/2747489748 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f4a8806ae40 msgr2=0x7f4a8806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:03.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.545+0000 7f4aab3be640 1 --2- 172.21.15.46:0/2747489748 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f4a8806ae40 0x7f4a8806d300 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0005880 tx=0x7f4aa0009040 comp rx=0 tx=0).stop 2023-12-02T00:29:03.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.545+0000 7f4aab3be640 1 -- 172.21.15.46:0/2747489748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4aa415d330 msgr2=0x7f4aa415d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:03.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.545+0000 7f4aab3be640 1 --2- 172.21.15.46:0/2747489748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4aa415d330 0x7f4aa415d710 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c00efc0 tx=0x7f4a9c00c490 comp rx=0 tx=0).stop 2023-12-02T00:29:03.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.545+0000 7f4aab3be640 1 -- 172.21.15.46:0/2747489748 shutdown_connections 2023-12-02T00:29:03.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.545+0000 7f4aab3be640 1 --2- 172.21.15.46:0/2747489748 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f4a8806ae40 0x7f4a8806d300 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:03.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.545+0000 7f4aab3be640 1 --2- 172.21.15.46:0/2747489748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4aa415d330 0x7f4aa415d710 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:03.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.545+0000 7f4aab3be640 1 --2- 172.21.15.46:0/2747489748 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4aa40d3730 0x7f4aa40cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:03.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.545+0000 7f4aab3be640 1 --2- 172.21.15.46:0/2747489748 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4aa40d2910 0x7f4aa41497d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:03.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.545+0000 7f4aab3be640 1 -- 172.21.15.46:0/2747489748 >> 172.21.15.46:0/2747489748 conn(0x7f4aa4072020 msgr2=0x7f4aa40b4510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:03.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.545+0000 7f4aab3be640 1 -- 172.21.15.46:0/2747489748 shutdown_connections 2023-12-02T00:29:03.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.545+0000 7f4aab3be640 1 -- 172.21.15.46:0/2747489748 wait complete. 2023-12-02T00:29:03.557 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-02T00:29:03.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.662+0000 7fbced9b0640 1 Processor -- start 2023-12-02T00:29:03.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.662+0000 7fbced9b0640 1 -- start start 2023-12-02T00:29:03.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.662+0000 7fbced9b0640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbce8144980 0x7fbce815c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:03.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.662+0000 7fbced9b0640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbce80b4370 0x7fbce80b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:03.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.662+0000 7fbced9b0640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbce8071770 con 0x7fbce80ceb70 2023-12-02T00:29:03.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.662+0000 7fbced9b0640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbce80ccb60 con 0x7fbce8144980 2023-12-02T00:29:03.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.662+0000 7fbced9b0640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbce80b4ce0 con 0x7fbce80b4370 2023-12-02T00:29:03.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.663+0000 7fbce6ffd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbce8144980 0x7fbce815c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:03.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.663+0000 7fbce6ffd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbce8144980 0x7fbce815c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37906/0 (socket says 172.21.15.46:37906) 2023-12-02T00:29:03.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.663+0000 7fbce6ffd640 1 -- 172.21.15.46:0/1333601927 learned_addr learned my addr 172.21.15.46:0/1333601927 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:03.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.663+0000 7fbce77fe640 1 --2- 172.21.15.46:0/1333601927 >> v2:172.21.15.162:3300/0 conn(0x7fbce80b4370 0x7fbce80b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:03.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.663+0000 7fbce6ffd640 1 -- 172.21.15.46:0/1333601927 >> v2:172.21.15.162:3300/0 conn(0x7fbce80b4370 msgr2=0x7fbce80b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:03.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.663+0000 7fbce6ffd640 1 --2- 172.21.15.46:0/1333601927 >> v2:172.21.15.162:3300/0 conn(0x7fbce80b4370 0x7fbce80b47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:03.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.663+0000 7fbce6ffd640 1 -- 172.21.15.46:0/1333601927 >> v1:172.21.15.162:6789/0 conn(0x7fbce80ceb70 legacy=0x7fbce80cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:03.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.663+0000 7fbce6ffd640 1 -- 172.21.15.46:0/1333601927 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce80ccdc0 con 0x7fbce8144980 2023-12-02T00:29:03.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.663+0000 7fbce77fe640 1 --2- 172.21.15.46:0/1333601927 >> v2:172.21.15.162:3300/0 conn(0x7fbce80b4370 0x7fbce80b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:03.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.664+0000 7fbce6ffd640 1 --2- 172.21.15.46:0/1333601927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbce8144980 0x7fbce815c7c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbce80716e0 tx=0x7fbcd802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=badaff3c13ec484 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:03.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.664+0000 7fbce67fc640 1 -- 172.21.15.46:0/1333601927 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd8030860 con 0x7fbce8144980 2023-12-02T00:29:03.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.664+0000 7fbce67fc640 1 -- 172.21.15.46:0/1333601927 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcd80309c0 con 0x7fbce8144980 2023-12-02T00:29:03.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.664+0000 7fbce67fc640 1 -- 172.21.15.46:0/1333601927 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd8030d50 con 0x7fbce8144980 2023-12-02T00:29:03.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.664+0000 7fbced9b0640 1 -- 172.21.15.46:0/1333601927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbce8144980 msgr2=0x7fbce815c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:03.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.664+0000 7fbced9b0640 1 --2- 172.21.15.46:0/1333601927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbce8144980 0x7fbce815c7c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbce80716e0 tx=0x7fbcd802b9a0 comp rx=0 tx=0).stop 2023-12-02T00:29:03.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.664+0000 7fbced9b0640 1 -- 172.21.15.46:0/1333601927 shutdown_connections 2023-12-02T00:29:03.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.664+0000 7fbced9b0640 1 --2- 172.21.15.46:0/1333601927 >> v2:172.21.15.162:3300/0 conn(0x7fbce80b4370 0x7fbce80b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:03.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.664+0000 7fbced9b0640 1 --2- 172.21.15.46:0/1333601927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbce8144980 0x7fbce815c7c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:03.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.664+0000 7fbced9b0640 1 -- 172.21.15.46:0/1333601927 >> 172.21.15.46:0/1333601927 conn(0x7fbce806a4d0 msgr2=0x7fbce806a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:03.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.664+0000 7fbced9b0640 1 -- 172.21.15.46:0/1333601927 shutdown_connections 2023-12-02T00:29:03.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.664+0000 7fbced9b0640 1 -- 172.21.15.46:0/1333601927 wait complete. 2023-12-02T00:29:03.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.665+0000 7fbced9b0640 1 Processor -- start 2023-12-02T00:29:03.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.666+0000 7fbced9b0640 1 -- start start 2023-12-02T00:29:03.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.666+0000 7fbced9b0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbce80b4370 0x7fbce80ae320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:03.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.666+0000 7fbced9b0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbce80ceb70 0x7fbce80b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:03.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.666+0000 7fbced9b0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbce8144980 0x7fbce80b39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:03.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.666+0000 7fbced9b0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce80b3510 con 0x7fbce8144980 2023-12-02T00:29:03.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.666+0000 7fbced9b0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce80b26a0 con 0x7fbce80b4370 2023-12-02T00:29:03.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.666+0000 7fbced9b0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbce80b3ee0 con 0x7fbce80ceb70 2023-12-02T00:29:03.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.667+0000 7fbce77fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbce8144980 0x7fbce80b39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:03.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.667+0000 7fbce77fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbce8144980 0x7fbce80b39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37908/0 (socket says 172.21.15.46:37908) 2023-12-02T00:29:03.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.667+0000 7fbce77fe640 1 -- 172.21.15.46:0/2570796163 learned_addr learned my addr 172.21.15.46:0/2570796163 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:03.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.667+0000 7fbce6ffd640 1 --2- 172.21.15.46:0/2570796163 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbce80b4370 0x7fbce80ae320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:03.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.667+0000 7fbcde7fc640 1 --2- 172.21.15.46:0/2570796163 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbce80ceb70 0x7fbce80b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:03.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.667+0000 7fbce77fe640 1 -- 172.21.15.46:0/2570796163 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbce80ceb70 msgr2=0x7fbce80b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:03.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.667+0000 7fbce77fe640 1 --2- 172.21.15.46:0/2570796163 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbce80ceb70 0x7fbce80b2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:03.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.667+0000 7fbce77fe640 1 -- 172.21.15.46:0/2570796163 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbce80b4370 msgr2=0x7fbce80ae320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:03.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.667+0000 7fbce77fe640 1 --2- 172.21.15.46:0/2570796163 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbce80b4370 0x7fbce80ae320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:03.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.667+0000 7fbce77fe640 1 -- 172.21.15.46:0/2570796163 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd80094d0 con 0x7fbce8144980 2023-12-02T00:29:03.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.667+0000 7fbce77fe640 1 --2- 172.21.15.46:0/2570796163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbce8144980 0x7fbce80b39a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbce0009fa0 tx=0x7fbce000d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:03.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.667+0000 7fbce4ff9640 1 -- 172.21.15.46:0/2570796163 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce0003b90 con 0x7fbce8144980 2023-12-02T00:29:03.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.667+0000 7fbce4ff9640 1 -- 172.21.15.46:0/2570796163 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbce000f040 con 0x7fbce8144980 2023-12-02T00:29:03.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.667+0000 7fbced9b0640 1 -- 172.21.15.46:0/2570796163 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbce80b2900 con 0x7fbce8144980 2023-12-02T00:29:03.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.668+0000 7fbce4ff9640 1 -- 172.21.15.46:0/2570796163 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce0004030 con 0x7fbce8144980 2023-12-02T00:29:03.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.668+0000 7fbced9b0640 1 -- 172.21.15.46:0/2570796163 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbce80b2ec0 con 0x7fbce8144980 2023-12-02T00:29:03.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.668+0000 7fbced9b0640 1 -- 172.21.15.46:0/2570796163 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbce80b3680 con 0x7fbce8144980 2023-12-02T00:29:03.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.669+0000 7fbce4ff9640 1 -- 172.21.15.46:0/2570796163 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbce0004190 con 0x7fbce8144980 2023-12-02T00:29:03.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.669+0000 7fbce4ff9640 1 --2- 172.21.15.46:0/2570796163 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fbcc806ae40 0x7fbcc806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:03.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.669+0000 7fbce4ff9640 1 -- 172.21.15.46:0/2570796163 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbce0089890 con 0x7fbce8144980 2023-12-02T00:29:03.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.670+0000 7fbce6ffd640 1 --2- 172.21.15.46:0/2570796163 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fbcc806ae40 0x7fbcc806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:03.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.670+0000 7fbce6ffd640 1 --2- 172.21.15.46:0/2570796163 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fbcc806ae40 0x7fbcc806d300 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbcd8036000 tx=0x7fbcd8008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:03.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.673+0000 7fbce4ff9640 1 -- 172.21.15.46:0/2570796163 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbce0054a80 con 0x7fbce8144980 2023-12-02T00:29:03.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.887+0000 7fbced9b0640 1 -- 172.21.15.46:0/2570796163 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbce8069400 con 0x7fbce8144980 2023-12-02T00:29:03.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.887+0000 7fbce4ff9640 1 -- 172.21.15.46:0/2570796163 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4907 (secure 0 0 0) 0x7fbce001a090 con 0x7fbce8144980 2023-12-02T00:29:03.890 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:29:03.890 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/2059113401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2059113401},{"type":"v1","addr":"172.21.15.46:6834","nonce":2059113401}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/987532381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":987532381},{"type":"v1","addr":"172.21.15.46:6835","nonce":987532381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/449974862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":449974862},{"type":"v1","addr":"172.21.15.162:6836","nonce":449974862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-02T00:28:49.086548+0000","modified":"2023-12-02T00:28:51.082469+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4253":{"gid":4253,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/1231844478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":1231844478},{"type":"v1","addr":"172.21.15.162:6837","nonce":1231844478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-02T00:29:03.891 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 6 2023-12-02T00:29:03.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.889+0000 7fbced9b0640 1 -- 172.21.15.46:0/2570796163 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fbcc806ae40 msgr2=0x7fbcc806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:03.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.889+0000 7fbced9b0640 1 --2- 172.21.15.46:0/2570796163 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fbcc806ae40 0x7fbcc806d300 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbcd8036000 tx=0x7fbcd8008030 comp rx=0 tx=0).stop 2023-12-02T00:29:03.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.889+0000 7fbced9b0640 1 -- 172.21.15.46:0/2570796163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbce8144980 msgr2=0x7fbce80b39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:03.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.889+0000 7fbced9b0640 1 --2- 172.21.15.46:0/2570796163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbce8144980 0x7fbce80b39a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbce0009fa0 tx=0x7fbce000d560 comp rx=0 tx=0).stop 2023-12-02T00:29:03.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.889+0000 7fbced9b0640 1 -- 172.21.15.46:0/2570796163 shutdown_connections 2023-12-02T00:29:03.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.889+0000 7fbced9b0640 1 --2- 172.21.15.46:0/2570796163 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fbcc806ae40 0x7fbcc806d300 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:03.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.889+0000 7fbced9b0640 1 --2- 172.21.15.46:0/2570796163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbce8144980 0x7fbce80b39a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:03.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.889+0000 7fbced9b0640 1 --2- 172.21.15.46:0/2570796163 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbce80ceb70 0x7fbce80b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:03.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.889+0000 7fbced9b0640 1 --2- 172.21.15.46:0/2570796163 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbce80b4370 0x7fbce80ae320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:03.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.889+0000 7fbced9b0640 1 -- 172.21.15.46:0/2570796163 >> 172.21.15.46:0/2570796163 conn(0x7fbce806a4d0 msgr2=0x7fbce80b0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:03.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.889+0000 7fbced9b0640 1 -- 172.21.15.46:0/2570796163 shutdown_connections 2023-12-02T00:29:03.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:03.889+0000 7fbced9b0640 1 -- 172.21.15.46:0/2570796163 wait complete. 2023-12-02T00:29:03.901 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:29:04.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.008+0000 7f89661d0640 1 Processor -- start 2023-12-02T00:29:04.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.008+0000 7f89661d0640 1 -- start start 2023-12-02T00:29:04.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.008+0000 7f89661d0640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8960140340 0x7f8960166000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:04.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.008+0000 7f89661d0640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f896015ba40 con 0x7f896015c2f0 2023-12-02T00:29:04.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.008+0000 7f89661d0640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89601416e0 con 0x7f896006f210 2023-12-02T00:29:04.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.008+0000 7f89661d0640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f896015c6d0 con 0x7f8960140340 2023-12-02T00:29:04.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.009+0000 7f895ffff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8960140340 0x7f8960166000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:04.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.009+0000 7f895f7fe640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f896015c2f0 0x7f896006eb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58278/0 (socket says 172.21.15.46:58278) 2023-12-02T00:29:04.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.009+0000 7f895f7fe640 1 -- 172.21.15.46:0/3275815339 learned_addr learned my addr 172.21.15.46:0/3275815339 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:04.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.009+0000 7f895ffff640 1 -- 172.21.15.46:0/3275815339 >> v1:172.21.15.162:6789/0 conn(0x7f896006f210 legacy=0x7f8960140800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:04.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.009+0000 7f895ffff640 1 -- 172.21.15.46:0/3275815339 >> v1:172.21.15.46:6789/0 conn(0x7f896015c2f0 legacy=0x7f896006eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:04.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.009+0000 7f895e7fc640 1 -- 172.21.15.46:0/3275815339 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 587412919 0 0) 0x7f896015ba40 con 0x7f896015c2f0 2023-12-02T00:29:04.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.009+0000 7f895ffff640 1 -- 172.21.15.46:0/3275815339 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8960141400 con 0x7f8960140340 2023-12-02T00:29:04.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.010+0000 7f895ffff640 1 --2- 172.21.15.46:0/3275815339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8960140340 0x7f8960166000 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f89601418d0 tx=0x7f895802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db4c1b09d5d96053 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:04.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.010+0000 7f895e7fc640 1 -- 172.21.15.46:0/3275815339 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895802cc50 con 0x7f8960140340 2023-12-02T00:29:04.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.010+0000 7f895e7fc640 1 -- 172.21.15.46:0/3275815339 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f895802cdb0 con 0x7f8960140340 2023-12-02T00:29:04.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.010+0000 7f895e7fc640 1 -- 172.21.15.46:0/3275815339 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8958034660 con 0x7f8960140340 2023-12-02T00:29:04.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.010+0000 7f89661d0640 1 -- 172.21.15.46:0/3275815339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8960140340 msgr2=0x7f8960166000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:04.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.010+0000 7f89661d0640 1 --2- 172.21.15.46:0/3275815339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8960140340 0x7f8960166000 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f89601418d0 tx=0x7f895802bea0 comp rx=0 tx=0).stop 2023-12-02T00:29:04.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.010+0000 7f89661d0640 1 -- 172.21.15.46:0/3275815339 shutdown_connections 2023-12-02T00:29:04.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.010+0000 7f89661d0640 1 --2- 172.21.15.46:0/3275815339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8960140340 0x7f8960166000 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:04.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.010+0000 7f89661d0640 1 -- 172.21.15.46:0/3275815339 >> 172.21.15.46:0/3275815339 conn(0x7f896006a6b0 msgr2=0x7f896006aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:04.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.010+0000 7f89661d0640 1 -- 172.21.15.46:0/3275815339 shutdown_connections 2023-12-02T00:29:04.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.010+0000 7f89661d0640 1 -- 172.21.15.46:0/3275815339 wait complete. 2023-12-02T00:29:04.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.011+0000 7f89661d0640 1 Processor -- start 2023-12-02T00:29:04.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.012+0000 7f89661d0640 1 -- start start 2023-12-02T00:29:04.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.012+0000 7f89661d0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f896006f210 0x7f89600a9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:04.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.012+0000 7f89661d0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8960140340 0x7f89601509c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:04.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.012+0000 7f89661d0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f896015c2f0 0x7f89600a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:04.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.012+0000 7f89661d0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89600a8f20 con 0x7f896006f210 2023-12-02T00:29:04.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.012+0000 7f89661d0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89600a9200 con 0x7f8960140340 2023-12-02T00:29:04.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.012+0000 7f89661d0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89600a94e0 con 0x7f896015c2f0 2023-12-02T00:29:04.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.012+0000 7f895f7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f896006f210 0x7f89600a9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:04.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.013+0000 7f895f7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f896006f210 0x7f89600a9fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37930/0 (socket says 172.21.15.46:37930) 2023-12-02T00:29:04.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.013+0000 7f895f7fe640 1 -- 172.21.15.46:0/3407716916 learned_addr learned my addr 172.21.15.46:0/3407716916 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:04.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.013+0000 7f895effd640 1 --2- 172.21.15.46:0/3407716916 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8960140340 0x7f89601509c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:04.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.013+0000 7f895ffff640 1 --2- 172.21.15.46:0/3407716916 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f896015c2f0 0x7f89600a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:04.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.013+0000 7f895f7fe640 1 -- 172.21.15.46:0/3407716916 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f896015c2f0 msgr2=0x7f89600a84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:04.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.013+0000 7f895f7fe640 1 --2- 172.21.15.46:0/3407716916 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f896015c2f0 0x7f89600a84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:04.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.013+0000 7f895f7fe640 1 -- 172.21.15.46:0/3407716916 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8960140340 msgr2=0x7f89601509c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:04.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.013+0000 7f895f7fe640 1 --2- 172.21.15.46:0/3407716916 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8960140340 0x7f89601509c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:04.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.013+0000 7f895f7fe640 1 -- 172.21.15.46:0/3407716916 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8958009a70 con 0x7f896006f210 2023-12-02T00:29:04.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.013+0000 7f895f7fe640 1 --2- 172.21.15.46:0/3407716916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f896006f210 0x7f89600a9fc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f895000c430 tx=0x7f895000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:04.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.013+0000 7f8947fff640 1 -- 172.21.15.46:0/3407716916 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89500074e0 con 0x7f896006f210 2023-12-02T00:29:04.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.013+0000 7f8947fff640 1 -- 172.21.15.46:0/3407716916 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8950007660 con 0x7f896006f210 2023-12-02T00:29:04.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.013+0000 7f89661d0640 1 -- 172.21.15.46:0/3407716916 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89600a97c0 con 0x7f896006f210 2023-12-02T00:29:04.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.013+0000 7f8947fff640 1 -- 172.21.15.46:0/3407716916 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8950004a20 con 0x7f896006f210 2023-12-02T00:29:04.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.013+0000 7f89661d0640 1 -- 172.21.15.46:0/3407716916 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89601f0da0 con 0x7f896006f210 2023-12-02T00:29:04.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.014+0000 7f89661d0640 1 -- 172.21.15.46:0/3407716916 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8960069400 con 0x7f896006f210 2023-12-02T00:29:04.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.015+0000 7f8947fff640 1 -- 172.21.15.46:0/3407716916 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8950004b80 con 0x7f896006f210 2023-12-02T00:29:04.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.015+0000 7f8947fff640 1 --2- 172.21.15.46:0/3407716916 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f894c06ae40 0x7f894c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:04.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.015+0000 7f8947fff640 1 -- 172.21.15.46:0/3407716916 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8950089360 con 0x7f896006f210 2023-12-02T00:29:04.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.015+0000 7f895effd640 1 --2- 172.21.15.46:0/3407716916 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f894c06ae40 0x7f894c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:04.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.016+0000 7f895effd640 1 --2- 172.21.15.46:0/3407716916 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f894c06ae40 0x7f894c06d300 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f89600aa500 tx=0x7f8954007e30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:04.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.020+0000 7f8947fff640 1 -- 172.21.15.46:0/3407716916 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f89500544c0 con 0x7f896006f210 2023-12-02T00:29:04.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.197+0000 7f89661d0640 1 -- 172.21.15.46:0/3407716916 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f89600d1b90 con 0x7f896006f210 2023-12-02T00:29:04.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.198+0000 7f8947fff640 1 -- 172.21.15.46:0/3407716916 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f8950057ae0 con 0x7f896006f210 2023-12-02T00:29:04.199 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:29:04.199 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":21,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:28:50.700112+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-02T00:28:47.273337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-02T00:28:48.060115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:29:04.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.199+0000 7f89661d0640 1 -- 172.21.15.46:0/3407716916 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f894c06ae40 msgr2=0x7f894c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:04.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.199+0000 7f89661d0640 1 --2- 172.21.15.46:0/3407716916 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f894c06ae40 0x7f894c06d300 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f89600aa500 tx=0x7f8954007e30 comp rx=0 tx=0).stop 2023-12-02T00:29:04.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.199+0000 7f89661d0640 1 -- 172.21.15.46:0/3407716916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f896006f210 msgr2=0x7f89600a9fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:04.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.199+0000 7f89661d0640 1 --2- 172.21.15.46:0/3407716916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f896006f210 0x7f89600a9fc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f895000c430 tx=0x7f895000c8f0 comp rx=0 tx=0).stop 2023-12-02T00:29:04.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.199+0000 7f89661d0640 1 -- 172.21.15.46:0/3407716916 shutdown_connections 2023-12-02T00:29:04.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.199+0000 7f89661d0640 1 --2- 172.21.15.46:0/3407716916 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f894c06ae40 0x7f894c06d300 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:04.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.199+0000 7f89661d0640 1 --2- 172.21.15.46:0/3407716916 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f896015c2f0 0x7f89600a84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:04.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.199+0000 7f89661d0640 1 --2- 172.21.15.46:0/3407716916 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8960140340 0x7f89601509c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:04.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.199+0000 7f89661d0640 1 --2- 172.21.15.46:0/3407716916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f896006f210 0x7f89600a9fc0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:04.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.200+0000 7f89661d0640 1 -- 172.21.15.46:0/3407716916 >> 172.21.15.46:0/3407716916 conn(0x7f896006a6b0 msgr2=0x7f896006aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:04.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.200+0000 7f89661d0640 1 -- 172.21.15.46:0/3407716916 shutdown_connections 2023-12-02T00:29:04.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.200+0000 7f89661d0640 1 -- 172.21.15.46:0/3407716916 wait complete. 2023-12-02T00:29:04.212 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-02T00:28:37.213645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-02T00:28:47.273337+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-02T00:28:48.060115+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-02T00:29:04.212 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-02T00:29:04.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.320+0000 7f0ac6a90640 1 Processor -- start 2023-12-02T00:29:04.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.320+0000 7f0ac6a90640 1 -- start start 2023-12-02T00:29:04.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.321+0000 7f0ac6a90640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ac015ac90 0x7f0ac0161740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:04.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.321+0000 7f0ac6a90640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0ac00b3000 0x7f0ac00b3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:04.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.321+0000 7f0ac6a90640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ac01612b0 con 0x7f0ac00b3f10 2023-12-02T00:29:04.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.321+0000 7f0ac6a90640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ac00b39d0 con 0x7f0ac00b3000 2023-12-02T00:29:04.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.321+0000 7f0ac6a90640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac015df70 con 0x7f0ac015ac90 2023-12-02T00:29:04.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.321+0000 7f0ac502d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0ac00b3000 0x7f0ac00b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:04.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.321+0000 7f0ac502d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0ac00b3000 0x7f0ac00b3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37942/0 (socket says 172.21.15.46:37942) 2023-12-02T00:29:04.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.321+0000 7f0ac502d640 1 -- 172.21.15.46:0/3634814152 learned_addr learned my addr 172.21.15.46:0/3634814152 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:04.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.321+0000 7f0ac482c640 1 --2- 172.21.15.46:0/3634814152 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ac015ac90 0x7f0ac0161740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:04.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.321+0000 7f0ac502d640 1 -- 172.21.15.46:0/3634814152 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ac015ac90 msgr2=0x7f0ac0161740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:04.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.321+0000 7f0ac502d640 1 --2- 172.21.15.46:0/3634814152 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ac015ac90 0x7f0ac0161740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:04.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.321+0000 7f0ac502d640 1 -- 172.21.15.46:0/3634814152 >> v1:172.21.15.162:6789/0 conn(0x7f0ac00b3f10 legacy=0x7f0ac015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:04.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.321+0000 7f0ac502d640 1 -- 172.21.15.46:0/3634814152 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac015e190 con 0x7f0ac00b3000 2023-12-02T00:29:04.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.321+0000 7f0ac482c640 1 --2- 172.21.15.46:0/3634814152 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ac015ac90 0x7f0ac0161740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:29:04.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.322+0000 7f0ac502d640 1 --2- 172.21.15.46:0/3634814152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ac00b3000 0x7f0ac00b3490 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0ab00098a0 tx=0x7f0ab002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8e8c3070f1a921 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:04.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.322+0000 7f0abf7fe640 1 -- 172.21.15.46:0/3634814152 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab0031640 con 0x7f0ac00b3000 2023-12-02T00:29:04.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.322+0000 7f0abf7fe640 1 -- 172.21.15.46:0/3634814152 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab00317a0 con 0x7f0ac00b3000 2023-12-02T00:29:04.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.322+0000 7f0abf7fe640 1 -- 172.21.15.46:0/3634814152 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab0031b30 con 0x7f0ac00b3000 2023-12-02T00:29:04.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.323+0000 7f0ac6a90640 1 -- 172.21.15.46:0/3634814152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ac00b3000 msgr2=0x7f0ac00b3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:04.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.323+0000 7f0ac6a90640 1 --2- 172.21.15.46:0/3634814152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ac00b3000 0x7f0ac00b3490 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0ab00098a0 tx=0x7f0ab002b8f0 comp rx=0 tx=0).stop 2023-12-02T00:29:04.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.323+0000 7f0ac6a90640 1 -- 172.21.15.46:0/3634814152 shutdown_connections 2023-12-02T00:29:04.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.323+0000 7f0ac6a90640 1 --2- 172.21.15.46:0/3634814152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ac00b3000 0x7f0ac00b3490 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:04.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.323+0000 7f0ac6a90640 1 --2- 172.21.15.46:0/3634814152 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ac015ac90 0x7f0ac0161740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:04.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.323+0000 7f0ac6a90640 1 -- 172.21.15.46:0/3634814152 >> 172.21.15.46:0/3634814152 conn(0x7f0ac006a8c0 msgr2=0x7f0ac006acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:04.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.323+0000 7f0ac6a90640 1 -- 172.21.15.46:0/3634814152 shutdown_connections 2023-12-02T00:29:04.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.323+0000 7f0ac6a90640 1 -- 172.21.15.46:0/3634814152 wait complete. 2023-12-02T00:29:04.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.324+0000 7f0ac6a90640 1 Processor -- start 2023-12-02T00:29:04.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.324+0000 7f0ac6a90640 1 -- start start 2023-12-02T00:29:04.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.325+0000 7f0ac6a90640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ac00b3000 0x7f0ac0146d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:04.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.325+0000 7f0ac6a90640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ac00b3f10 0x7f0ac00d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:04.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.325+0000 7f0ac6a90640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ac015ac90 0x7f0ac00ce800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:04.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.325+0000 7f0ac6a90640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac00d0af0 con 0x7f0ac00b3000 2023-12-02T00:29:04.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.325+0000 7f0ac6a90640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac0145c70 con 0x7f0ac015ac90 2023-12-02T00:29:04.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.325+0000 7f0ac6a90640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac0145f50 con 0x7f0ac00b3f10 2023-12-02T00:29:04.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.325+0000 7f0ac482c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ac00b3000 0x7f0ac0146d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:04.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.325+0000 7f0ac482c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ac00b3000 0x7f0ac0146d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37952/0 (socket says 172.21.15.46:37952) 2023-12-02T00:29:04.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.325+0000 7f0ac482c640 1 -- 172.21.15.46:0/4166170183 learned_addr learned my addr 172.21.15.46:0/4166170183 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:04.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.325+0000 7f0ac502d640 1 --2- 172.21.15.46:0/4166170183 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ac015ac90 0x7f0ac00ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:04.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.325+0000 7f0abffff640 1 --2- 172.21.15.46:0/4166170183 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ac00b3f10 0x7f0ac00d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:04.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.325+0000 7f0ac482c640 1 -- 172.21.15.46:0/4166170183 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ac00b3f10 msgr2=0x7f0ac00d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:04.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.325+0000 7f0ac482c640 1 --2- 172.21.15.46:0/4166170183 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ac00b3f10 0x7f0ac00d05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:04.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.325+0000 7f0ac482c640 1 -- 172.21.15.46:0/4166170183 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ac015ac90 msgr2=0x7f0ac00ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:04.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.325+0000 7f0ac482c640 1 --2- 172.21.15.46:0/4166170183 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ac015ac90 0x7f0ac00ce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:04.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.325+0000 7f0ac482c640 1 -- 172.21.15.46:0/4166170183 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab00094d0 con 0x7f0ac00b3000 2023-12-02T00:29:04.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.326+0000 7f0ac482c640 1 --2- 172.21.15.46:0/4166170183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ac00b3000 0x7f0ac0146d10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0ab800ce00 tx=0x7f0ab800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:04.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.326+0000 7f0abd7fa640 1 -- 172.21.15.46:0/4166170183 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab800f890 con 0x7f0ac00b3000 2023-12-02T00:29:04.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.326+0000 7f0abd7fa640 1 -- 172.21.15.46:0/4166170183 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab800f9f0 con 0x7f0ac00b3000 2023-12-02T00:29:04.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.326+0000 7f0ac6a90640 1 -- 172.21.15.46:0/4166170183 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ac0146230 con 0x7f0ac00b3000 2023-12-02T00:29:04.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.326+0000 7f0abd7fa640 1 -- 172.21.15.46:0/4166170183 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab800fdc0 con 0x7f0ac00b3000 2023-12-02T00:29:04.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.326+0000 7f0ac6a90640 1 -- 172.21.15.46:0/4166170183 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ac01467f0 con 0x7f0ac00b3000 2023-12-02T00:29:04.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.327+0000 7f0ac6a90640 1 -- 172.21.15.46:0/4166170183 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ac0067420 con 0x7f0ac00b3000 2023-12-02T00:29:04.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.327+0000 7f0abd7fa640 1 -- 172.21.15.46:0/4166170183 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0ab800b040 con 0x7f0ac00b3000 2023-12-02T00:29:04.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.328+0000 7f0abd7fa640 1 --2- 172.21.15.46:0/4166170183 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f0aac06ae40 0x7f0aac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:04.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.328+0000 7f0abd7fa640 1 -- 172.21.15.46:0/4166170183 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0ab808a0c0 con 0x7f0ac00b3000 2023-12-02T00:29:04.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.328+0000 7f0abffff640 1 --2- 172.21.15.46:0/4166170183 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f0aac06ae40 0x7f0aac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:04.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.329+0000 7f0abffff640 1 --2- 172.21.15.46:0/4166170183 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f0aac06ae40 0x7f0aac06d300 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0ac014b970 tx=0x7f0ab4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:04.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.332+0000 7f0abd7fa640 1 -- 172.21.15.46:0/4166170183 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0ab80550d0 con 0x7f0ac00b3000 2023-12-02T00:29:04.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.506+0000 7f0ac6a90640 1 -- 172.21.15.46:0/4166170183 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0ac006aeb0 con 0x7f0ac00b3000 2023-12-02T00:29:04.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.506+0000 7f0abd7fa640 1 -- 172.21.15.46:0/4166170183 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0ab801d0a0 con 0x7f0ac00b3000 2023-12-02T00:29:04.507 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-12-02T00:29:04.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.508+0000 7f0ac6a90640 1 -- 172.21.15.46:0/4166170183 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f0aac06ae40 msgr2=0x7f0aac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:04.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.508+0000 7f0ac6a90640 1 --2- 172.21.15.46:0/4166170183 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f0aac06ae40 0x7f0aac06d300 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0ac014b970 tx=0x7f0ab4009040 comp rx=0 tx=0).stop 2023-12-02T00:29:04.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.508+0000 7f0ac6a90640 1 -- 172.21.15.46:0/4166170183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ac00b3000 msgr2=0x7f0ac0146d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:04.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.508+0000 7f0ac6a90640 1 --2- 172.21.15.46:0/4166170183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ac00b3000 0x7f0ac0146d10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0ab800ce00 tx=0x7f0ab800a650 comp rx=0 tx=0).stop 2023-12-02T00:29:04.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.508+0000 7f0ac6a90640 1 -- 172.21.15.46:0/4166170183 shutdown_connections 2023-12-02T00:29:04.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.508+0000 7f0ac6a90640 1 --2- 172.21.15.46:0/4166170183 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f0aac06ae40 0x7f0aac06d300 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:04.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.508+0000 7f0ac6a90640 1 --2- 172.21.15.46:0/4166170183 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ac015ac90 0x7f0ac00ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:04.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.508+0000 7f0ac6a90640 1 --2- 172.21.15.46:0/4166170183 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ac00b3f10 0x7f0ac00d05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:04.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.508+0000 7f0ac6a90640 1 --2- 172.21.15.46:0/4166170183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ac00b3000 0x7f0ac0146d10 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:04.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.508+0000 7f0ac6a90640 1 -- 172.21.15.46:0/4166170183 >> 172.21.15.46:0/4166170183 conn(0x7f0ac006a8c0 msgr2=0x7f0ac015d600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:04.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.508+0000 7f0ac6a90640 1 -- 172.21.15.46:0/4166170183 shutdown_connections 2023-12-02T00:29:04.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.508+0000 7f0ac6a90640 1 -- 172.21.15.46:0/4166170183 wait complete. 2023-12-02T00:29:04.520 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-02T00:29:04.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.628+0000 7fc8d8368640 1 Processor -- start 2023-12-02T00:29:04.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.628+0000 7fc8d8368640 1 -- start start 2023-12-02T00:29:04.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.629+0000 7fc8d8368640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc8d006ea50 0x7fc8d006ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:04.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.629+0000 7fc8d8368640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc8d0160fb0 0x7fc8d0161440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:04.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.629+0000 7fc8d8368640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8d0161d20 con 0x7fc8d00b3ff0 2023-12-02T00:29:04.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.629+0000 7fc8d8368640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8d00d1ee0 con 0x7fc8d0160fb0 2023-12-02T00:29:04.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.629+0000 7fc8d8368640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8d006f370 con 0x7fc8d006ea50 2023-12-02T00:29:04.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.629+0000 7fc8d5903640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc8d0160fb0 0x7fc8d0161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:04.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.629+0000 7fc8d6905640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc8d00b3ff0 0x7fc8d00b4460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58292/0 (socket says 172.21.15.46:58292) 2023-12-02T00:29:04.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.629+0000 7fc8d6905640 1 -- 172.21.15.46:0/2135603644 learned_addr learned my addr 172.21.15.46:0/2135603644 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:04.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.630+0000 7fc8d6104640 1 --2- 172.21.15.46:0/2135603644 >> v2:172.21.15.162:3300/0 conn(0x7fc8d006ea50 0x7fc8d006ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:04.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.630+0000 7fc8d5903640 1 -- 172.21.15.46:0/2135603644 >> v2:172.21.15.162:3300/0 conn(0x7fc8d006ea50 msgr2=0x7fc8d006ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:04.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.630+0000 7fc8d5903640 1 --2- 172.21.15.46:0/2135603644 >> v2:172.21.15.162:3300/0 conn(0x7fc8d006ea50 0x7fc8d006ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:04.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.630+0000 7fc8d5903640 1 -- 172.21.15.46:0/2135603644 >> v1:172.21.15.46:6789/0 conn(0x7fc8d00b3ff0 legacy=0x7fc8d00b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:04.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.630+0000 7fc8d5903640 1 -- 172.21.15.46:0/2135603644 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d015e760 con 0x7fc8d0160fb0 2023-12-02T00:29:04.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.630+0000 7fc8d5102640 1 -- 172.21.15.46:0/2135603644 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 710764502 0 0) 0x7fc8d0161d20 con 0x7fc8d00b3ff0 2023-12-02T00:29:04.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.630+0000 7fc8d6104640 1 --2- 172.21.15.46:0/2135603644 >> v2:172.21.15.162:3300/0 conn(0x7fc8d006ea50 0x7fc8d006ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:29:04.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.630+0000 7fc8d5903640 1 --2- 172.21.15.46:0/2135603644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8d0160fb0 0x7fc8d0161440 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc8d006f680 tx=0x7fc8c402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c44af08f16c6102c server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:04.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.631+0000 7fc8d5102640 1 -- 172.21.15.46:0/2135603644 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c402e070 con 0x7fc8d0160fb0 2023-12-02T00:29:04.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.631+0000 7fc8d5102640 1 -- 172.21.15.46:0/2135603644 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8c40315a0 con 0x7fc8d0160fb0 2023-12-02T00:29:04.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.631+0000 7fc8d5102640 1 -- 172.21.15.46:0/2135603644 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c4031930 con 0x7fc8d0160fb0 2023-12-02T00:29:04.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.631+0000 7fc8d8368640 1 -- 172.21.15.46:0/2135603644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8d0160fb0 msgr2=0x7fc8d0161440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:04.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.631+0000 7fc8d8368640 1 --2- 172.21.15.46:0/2135603644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8d0160fb0 0x7fc8d0161440 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc8d006f680 tx=0x7fc8c402b840 comp rx=0 tx=0).stop 2023-12-02T00:29:04.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.631+0000 7fc8d8368640 1 -- 172.21.15.46:0/2135603644 shutdown_connections 2023-12-02T00:29:04.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.631+0000 7fc8d8368640 1 --2- 172.21.15.46:0/2135603644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8d0160fb0 0x7fc8d0161440 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:04.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.631+0000 7fc8d8368640 1 --2- 172.21.15.46:0/2135603644 >> v2:172.21.15.162:3300/0 conn(0x7fc8d006ea50 0x7fc8d006ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:04.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.631+0000 7fc8d8368640 1 -- 172.21.15.46:0/2135603644 >> 172.21.15.46:0/2135603644 conn(0x7fc8d006a6b0 msgr2=0x7fc8d006aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:04.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.631+0000 7fc8d8368640 1 -- 172.21.15.46:0/2135603644 shutdown_connections 2023-12-02T00:29:04.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.631+0000 7fc8d8368640 1 -- 172.21.15.46:0/2135603644 wait complete. 2023-12-02T00:29:04.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.632+0000 7fc8d8368640 1 Processor -- start 2023-12-02T00:29:04.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.632+0000 7fc8d8368640 1 -- start start 2023-12-02T00:29:04.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.633+0000 7fc8d8368640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8d006ea50 0x7fc8d006cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:04.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.633+0000 7fc8d8368640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8d00b3ff0 0x7fc8d006c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:04.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.633+0000 7fc8d8368640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8d0160fb0 0x7fc8d01e8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:04.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.633+0000 7fc8d8368640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d01e7440 con 0x7fc8d006ea50 2023-12-02T00:29:04.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.633+0000 7fc8d8368640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d006d250 con 0x7fc8d00b3ff0 2023-12-02T00:29:04.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.633+0000 7fc8d8368640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d01e7690 con 0x7fc8d0160fb0 2023-12-02T00:29:04.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.633+0000 7fc8d6104640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8d006ea50 0x7fc8d006cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:04.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.633+0000 7fc8d6104640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8d006ea50 0x7fc8d006cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37968/0 (socket says 172.21.15.46:37968) 2023-12-02T00:29:04.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.633+0000 7fc8d6104640 1 -- 172.21.15.46:0/2038936560 learned_addr learned my addr 172.21.15.46:0/2038936560 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:04.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.633+0000 7fc8d6104640 1 -- 172.21.15.46:0/2038936560 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8d0160fb0 msgr2=0x7fc8d01e8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:04.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.633+0000 7fc8d5903640 1 --2- 172.21.15.46:0/2038936560 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8d00b3ff0 0x7fc8d006c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:04.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.633+0000 7fc8d6104640 1 --2- 172.21.15.46:0/2038936560 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8d0160fb0 0x7fc8d01e8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:04.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.633+0000 7fc8d6104640 1 -- 172.21.15.46:0/2038936560 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8d00b3ff0 msgr2=0x7fc8d006c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:04.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.633+0000 7fc8d6104640 1 --2- 172.21.15.46:0/2038936560 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8d00b3ff0 0x7fc8d006c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:04.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.633+0000 7fc8d6104640 1 -- 172.21.15.46:0/2038936560 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8c4009400 con 0x7fc8d006ea50 2023-12-02T00:29:04.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.634+0000 7fc8d6104640 1 --2- 172.21.15.46:0/2038936560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8d006ea50 0x7fc8d006cc00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc8c800c7f0 tx=0x7fc8c800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:04.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.634+0000 7fc8ceffd640 1 -- 172.21.15.46:0/2038936560 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c8007820 con 0x7fc8d006ea50 2023-12-02T00:29:04.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.634+0000 7fc8ceffd640 1 -- 172.21.15.46:0/2038936560 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8c8007980 con 0x7fc8d006ea50 2023-12-02T00:29:04.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.634+0000 7fc8d8368640 1 -- 172.21.15.46:0/2038936560 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8d01e7970 con 0x7fc8d006ea50 2023-12-02T00:29:04.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.634+0000 7fc8ceffd640 1 -- 172.21.15.46:0/2038936560 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c8007cf0 con 0x7fc8d006ea50 2023-12-02T00:29:04.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.634+0000 7fc8d8368640 1 -- 172.21.15.46:0/2038936560 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8d0156520 con 0x7fc8d006ea50 2023-12-02T00:29:04.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.635+0000 7fc8d8368640 1 -- 172.21.15.46:0/2038936560 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8d00b5330 con 0x7fc8d006ea50 2023-12-02T00:29:04.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.635+0000 7fc8ceffd640 1 -- 172.21.15.46:0/2038936560 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc8c80113f0 con 0x7fc8d006ea50 2023-12-02T00:29:04.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.636+0000 7fc8ceffd640 1 --2- 172.21.15.46:0/2038936560 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc8bc06ae70 0x7fc8bc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:04.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.636+0000 7fc8ceffd640 1 -- 172.21.15.46:0/2038936560 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc8c8089930 con 0x7fc8d006ea50 2023-12-02T00:29:04.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.636+0000 7fc8d5903640 1 --2- 172.21.15.46:0/2038936560 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc8bc06ae70 0x7fc8bc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:04.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.636+0000 7fc8d5903640 1 --2- 172.21.15.46:0/2038936560 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc8bc06ae70 0x7fc8bc06d330 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc8c4036000 tx=0x7fc8c403d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:04.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.640+0000 7fc8ceffd640 1 -- 172.21.15.46:0/2038936560 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc8c80549e0 con 0x7fc8d006ea50 2023-12-02T00:29:04.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.819+0000 7fc8d8368640 1 -- 172.21.15.46:0/2038936560 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc8d0069400 con 0x7fc8d006ea50 2023-12-02T00:29:04.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.819+0000 7fc8ceffd640 1 -- 172.21.15.46:0/2038936560 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc8c801f020 con 0x7fc8d006ea50 2023-12-02T00:29:04.821 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-12-02T00:29:04.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.821+0000 7fc8d8368640 1 -- 172.21.15.46:0/2038936560 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc8bc06ae70 msgr2=0x7fc8bc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:04.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.821+0000 7fc8d8368640 1 --2- 172.21.15.46:0/2038936560 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc8bc06ae70 0x7fc8bc06d330 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc8c4036000 tx=0x7fc8c403d040 comp rx=0 tx=0).stop 2023-12-02T00:29:04.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.821+0000 7fc8d8368640 1 -- 172.21.15.46:0/2038936560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8d006ea50 msgr2=0x7fc8d006cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:04.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.821+0000 7fc8d8368640 1 --2- 172.21.15.46:0/2038936560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8d006ea50 0x7fc8d006cc00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc8c800c7f0 tx=0x7fc8c800ccb0 comp rx=0 tx=0).stop 2023-12-02T00:29:04.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.821+0000 7fc8d8368640 1 -- 172.21.15.46:0/2038936560 shutdown_connections 2023-12-02T00:29:04.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.821+0000 7fc8d8368640 1 --2- 172.21.15.46:0/2038936560 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc8bc06ae70 0x7fc8bc06d330 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:04.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.821+0000 7fc8d8368640 1 --2- 172.21.15.46:0/2038936560 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8d0160fb0 0x7fc8d01e8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:04.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.821+0000 7fc8d8368640 1 --2- 172.21.15.46:0/2038936560 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8d00b3ff0 0x7fc8d006c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:04.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.821+0000 7fc8d8368640 1 --2- 172.21.15.46:0/2038936560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8d006ea50 0x7fc8d006cc00 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:04.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.821+0000 7fc8d8368640 1 -- 172.21.15.46:0/2038936560 >> 172.21.15.46:0/2038936560 conn(0x7fc8d006a6b0 msgr2=0x7fc8d013e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:04.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.821+0000 7fc8d8368640 1 -- 172.21.15.46:0/2038936560 shutdown_connections 2023-12-02T00:29:04.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.821+0000 7fc8d8368640 1 -- 172.21.15.46:0/2038936560 wait complete. 2023-12-02T00:29:04.833 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-02T00:29:04.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.945+0000 7f9c02f00640 1 Processor -- start 2023-12-02T00:29:04.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.945+0000 7f9c02f00640 1 -- start start 2023-12-02T00:29:04.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.945+0000 7f9c02f00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9bfc15d330 0x7f9bfc0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:04.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.945+0000 7f9c02f00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9bfc0d2910 0x7f9bfc0d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:04.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.945+0000 7f9c02f00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bfc0b2b10 con 0x7f9bfc0d3730 2023-12-02T00:29:04.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.945+0000 7f9c02f00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bfc0d3bc0 con 0x7f9bfc0d2910 2023-12-02T00:29:04.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.945+0000 7f9c02f00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bfc0d1590 con 0x7f9bfc15d330 2023-12-02T00:29:04.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.945+0000 7f9bfbfff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9bfc0d2910 0x7f9bfc0d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:04.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.946+0000 7f9bfbfff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9bfc0d2910 0x7f9bfc0d31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37972/0 (socket says 172.21.15.46:37972) 2023-12-02T00:29:04.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.946+0000 7f9bfbfff640 1 -- 172.21.15.46:0/2296077093 learned_addr learned my addr 172.21.15.46:0/2296077093 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:04.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.946+0000 7f9c00c9c640 1 --2- 172.21.15.46:0/2296077093 >> v2:172.21.15.162:3300/0 conn(0x7f9bfc15d330 0x7f9bfc0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:04.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.946+0000 7f9bfbfff640 1 -- 172.21.15.46:0/2296077093 >> v2:172.21.15.162:3300/0 conn(0x7f9bfc15d330 msgr2=0x7f9bfc0b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:04.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.946+0000 7f9bfbfff640 1 --2- 172.21.15.46:0/2296077093 >> v2:172.21.15.162:3300/0 conn(0x7f9bfc15d330 0x7f9bfc0b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:04.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.946+0000 7f9bfbfff640 1 -- 172.21.15.46:0/2296077093 >> v1:172.21.15.162:6789/0 conn(0x7f9bfc0d3730 legacy=0x7f9bfc0b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:04.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.946+0000 7f9bfbfff640 1 -- 172.21.15.46:0/2296077093 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bfc0d1870 con 0x7f9bfc0d2910 2023-12-02T00:29:04.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.947+0000 7f9bfbfff640 1 --2- 172.21.15.46:0/2296077093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bfc0d2910 0x7f9bfc0d31f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc0d0d10 tx=0x7f9bec02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb6502b27b978038 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:04.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.947+0000 7f9bfb7fe640 1 -- 172.21.15.46:0/2296077093 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec0317e0 con 0x7f9bfc0d2910 2023-12-02T00:29:04.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.947+0000 7f9bfb7fe640 1 -- 172.21.15.46:0/2296077093 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bec031940 con 0x7f9bfc0d2910 2023-12-02T00:29:04.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.947+0000 7f9bfb7fe640 1 -- 172.21.15.46:0/2296077093 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec031cd0 con 0x7f9bfc0d2910 2023-12-02T00:29:04.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.947+0000 7f9c02f00640 1 -- 172.21.15.46:0/2296077093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bfc0d2910 msgr2=0x7f9bfc0d31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:04.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.947+0000 7f9c02f00640 1 --2- 172.21.15.46:0/2296077093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bfc0d2910 0x7f9bfc0d31f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc0d0d10 tx=0x7f9bec02b880 comp rx=0 tx=0).stop 2023-12-02T00:29:04.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.947+0000 7f9c02f00640 1 -- 172.21.15.46:0/2296077093 shutdown_connections 2023-12-02T00:29:04.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.947+0000 7f9c02f00640 1 --2- 172.21.15.46:0/2296077093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bfc0d2910 0x7f9bfc0d31f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:04.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.947+0000 7f9c02f00640 1 --2- 172.21.15.46:0/2296077093 >> v2:172.21.15.162:3300/0 conn(0x7f9bfc15d330 0x7f9bfc0b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:04.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.947+0000 7f9c02f00640 1 -- 172.21.15.46:0/2296077093 >> 172.21.15.46:0/2296077093 conn(0x7f9bfc072020 msgr2=0x7f9bfc072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:04.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.947+0000 7f9c02f00640 1 -- 172.21.15.46:0/2296077093 shutdown_connections 2023-12-02T00:29:04.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.947+0000 7f9c02f00640 1 -- 172.21.15.46:0/2296077093 wait complete. 2023-12-02T00:29:04.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.948+0000 7f9c02f00640 1 Processor -- start 2023-12-02T00:29:04.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.949+0000 7f9c02f00640 1 -- start start 2023-12-02T00:29:04.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.949+0000 7f9c02f00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bfc0d2910 0x7f9bfc0c6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:04.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.949+0000 7f9c02f00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9bfc0d3730 0x7f9bfc0b3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:04.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.949+0000 7f9c02f00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9bfc15d330 0x7f9bfc0b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:04.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.949+0000 7f9c02f00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc153910 con 0x7f9bfc0d2910 2023-12-02T00:29:04.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.949+0000 7f9c02f00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc0c67b0 con 0x7f9bfc0d3730 2023-12-02T00:29:04.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.949+0000 7f9c02f00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc0cad00 con 0x7f9bfc15d330 2023-12-02T00:29:04.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.950+0000 7f9c00c9c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bfc0d2910 0x7f9bfc0c6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:04.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.950+0000 7f9c00c9c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bfc0d2910 0x7f9bfc0c6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37982/0 (socket says 172.21.15.46:37982) 2023-12-02T00:29:04.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.950+0000 7f9c00c9c640 1 -- 172.21.15.46:0/2854524876 learned_addr learned my addr 172.21.15.46:0/2854524876 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:04.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.950+0000 7f9bfbfff640 1 --2- 172.21.15.46:0/2854524876 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9bfc0d3730 0x7f9bfc0b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:04.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.950+0000 7f9c0149d640 1 --2- 172.21.15.46:0/2854524876 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9bfc15d330 0x7f9bfc0b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:04.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.950+0000 7f9c00c9c640 1 -- 172.21.15.46:0/2854524876 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9bfc15d330 msgr2=0x7f9bfc0b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:04.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.950+0000 7f9c00c9c640 1 --2- 172.21.15.46:0/2854524876 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9bfc15d330 0x7f9bfc0b3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:04.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.950+0000 7f9c00c9c640 1 -- 172.21.15.46:0/2854524876 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9bfc0d3730 msgr2=0x7f9bfc0b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:04.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.950+0000 7f9c00c9c640 1 --2- 172.21.15.46:0/2854524876 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9bfc0d3730 0x7f9bfc0b3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:04.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.950+0000 7f9c00c9c640 1 -- 172.21.15.46:0/2854524876 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bec0094d0 con 0x7f9bfc0d2910 2023-12-02T00:29:04.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.950+0000 7f9c00c9c640 1 --2- 172.21.15.46:0/2854524876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bfc0d2910 0x7f9bfc0c6270 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4007e90 tx=0x7f9bf400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:04.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.950+0000 7f9bf97fa640 1 -- 172.21.15.46:0/2854524876 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4014420 con 0x7f9bfc0d2910 2023-12-02T00:29:04.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.950+0000 7f9bf97fa640 1 -- 172.21.15.46:0/2854524876 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf40145a0 con 0x7f9bfc0d2910 2023-12-02T00:29:04.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.950+0000 7f9c02f00640 1 -- 172.21.15.46:0/2854524876 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bfc0c9b90 con 0x7f9bfc0d2910 2023-12-02T00:29:04.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.950+0000 7f9bf97fa640 1 -- 172.21.15.46:0/2854524876 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4014950 con 0x7f9bfc0d2910 2023-12-02T00:29:04.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.950+0000 7f9c02f00640 1 -- 172.21.15.46:0/2854524876 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bfc0ca6b0 con 0x7f9bfc0d2910 2023-12-02T00:29:04.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.951+0000 7f9c02f00640 1 -- 172.21.15.46:0/2854524876 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bfc069400 con 0x7f9bfc0d2910 2023-12-02T00:29:04.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.952+0000 7f9bf97fa640 1 -- 172.21.15.46:0/2854524876 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9bf4014ab0 con 0x7f9bfc0d2910 2023-12-02T00:29:04.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.952+0000 7f9bf97fa640 1 --2- 172.21.15.46:0/2854524876 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9bdc06ae40 0x7f9bdc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:04.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.952+0000 7f9bf97fa640 1 -- 172.21.15.46:0/2854524876 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9bf4089ef0 con 0x7f9bfc0d2910 2023-12-02T00:29:04.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.952+0000 7f9bfbfff640 1 --2- 172.21.15.46:0/2854524876 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9bdc06ae40 0x7f9bdc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:04.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.953+0000 7f9bfbfff640 1 --2- 172.21.15.46:0/2854524876 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9bdc06ae40 0x7f9bdc06d300 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9bec036000 tx=0x7f9bec03d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:04.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:04.956+0000 7f9bf97fa640 1 -- 172.21.15.46:0/2854524876 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9bf4054fa0 con 0x7f9bfc0d2910 2023-12-02T00:29:05.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.133+0000 7f9c02f00640 1 -- 172.21.15.46:0/2854524876 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9bfc0725a0 con 0x7f9bfc0d2910 2023-12-02T00:29:05.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.134+0000 7f9bf97fa640 1 -- 172.21.15.46:0/2854524876 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9bf40585c0 con 0x7f9bfc0d2910 2023-12-02T00:29:05.135 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-12-02T00:29:05.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.136+0000 7f9c02f00640 1 -- 172.21.15.46:0/2854524876 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9bdc06ae40 msgr2=0x7f9bdc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:05.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.136+0000 7f9c02f00640 1 --2- 172.21.15.46:0/2854524876 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9bdc06ae40 0x7f9bdc06d300 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9bec036000 tx=0x7f9bec03d040 comp rx=0 tx=0).stop 2023-12-02T00:29:05.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.136+0000 7f9c02f00640 1 -- 172.21.15.46:0/2854524876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bfc0d2910 msgr2=0x7f9bfc0c6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:05.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.136+0000 7f9c02f00640 1 --2- 172.21.15.46:0/2854524876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bfc0d2910 0x7f9bfc0c6270 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4007e90 tx=0x7f9bf400e4b0 comp rx=0 tx=0).stop 2023-12-02T00:29:05.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.136+0000 7f9c02f00640 1 -- 172.21.15.46:0/2854524876 shutdown_connections 2023-12-02T00:29:05.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.136+0000 7f9c02f00640 1 --2- 172.21.15.46:0/2854524876 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9bdc06ae40 0x7f9bdc06d300 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:05.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.136+0000 7f9c02f00640 1 --2- 172.21.15.46:0/2854524876 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9bfc15d330 0x7f9bfc0b3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:05.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.136+0000 7f9c02f00640 1 --2- 172.21.15.46:0/2854524876 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9bfc0d3730 0x7f9bfc0b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:05.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.136+0000 7f9c02f00640 1 --2- 172.21.15.46:0/2854524876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bfc0d2910 0x7f9bfc0c6270 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:05.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.136+0000 7f9c02f00640 1 -- 172.21.15.46:0/2854524876 >> 172.21.15.46:0/2854524876 conn(0x7f9bfc072020 msgr2=0x7f9bfc0b5600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:05.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.136+0000 7f9c02f00640 1 -- 172.21.15.46:0/2854524876 shutdown_connections 2023-12-02T00:29:05.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.136+0000 7f9c02f00640 1 -- 172.21.15.46:0/2854524876 wait complete. 2023-12-02T00:29:05.148 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-02T00:29:05.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.253+0000 7fda73100640 1 Processor -- start 2023-12-02T00:29:05.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.253+0000 7fda73100640 1 -- start start 2023-12-02T00:29:05.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.254+0000 7fda73100640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda6c140310 0x7fda6c1618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:05.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.254+0000 7fda73100640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda6c1406f0 con 0x7fda6c1409d0 2023-12-02T00:29:05.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.254+0000 7fda73100640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda6c143960 con 0x7fda6c06abe0 2023-12-02T00:29:05.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.254+0000 7fda73100640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda6c161e20 con 0x7fda6c140310 2023-12-02T00:29:05.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.254+0000 7fda6bfff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fda6c1409d0 0x7fda6c06a4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58296/0 (socket says 172.21.15.46:58296) 2023-12-02T00:29:05.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.254+0000 7fda6bfff640 1 -- 172.21.15.46:0/3569000462 learned_addr learned my addr 172.21.15.46:0/3569000462 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:05.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.254+0000 7fda70e9c640 1 --2- 172.21.15.46:0/3569000462 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda6c140310 0x7fda6c1618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:05.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.254+0000 7fda6b7fe640 1 -- 172.21.15.46:0/3569000462 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2337031572 0 0) 0x7fda6c1406f0 con 0x7fda6c1409d0 2023-12-02T00:29:05.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.254+0000 7fda6b7fe640 1 -- 172.21.15.46:0/3569000462 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda54003520 con 0x7fda6c1409d0 2023-12-02T00:29:05.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.254+0000 7fda6b7fe640 1 -- 172.21.15.46:0/3569000462 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2683196016 0 0) 0x7fda6c143960 con 0x7fda6c06abe0 2023-12-02T00:29:05.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.254+0000 7fda6b7fe640 1 -- 172.21.15.46:0/3569000462 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda6c1406f0 con 0x7fda6c06abe0 2023-12-02T00:29:05.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.254+0000 7fda70e9c640 1 -- 172.21.15.46:0/3569000462 >> v1:172.21.15.162:6789/0 conn(0x7fda6c06abe0 legacy=0x7fda6c06efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:05.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.254+0000 7fda70e9c640 1 -- 172.21.15.46:0/3569000462 >> v1:172.21.15.46:6789/0 conn(0x7fda6c1409d0 legacy=0x7fda6c06a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:05.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.254+0000 7fda6b7fe640 1 -- 172.21.15.46:0/3569000462 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1521547 0 0) 0x7fda54003520 con 0x7fda6c1409d0 2023-12-02T00:29:05.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.254+0000 7fda70e9c640 1 -- 172.21.15.46:0/3569000462 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda6c142de0 con 0x7fda6c140310 2023-12-02T00:29:05.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.255+0000 7fda70e9c640 1 --2- 172.21.15.46:0/3569000462 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda6c140310 0x7fda6c1618e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fda64009dc0 tx=0x7fda6402bd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8801ae342cdaf3d server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:05.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.255+0000 7fda6b7fe640 1 -- 172.21.15.46:0/3569000462 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda60001670 con 0x7fda6c140310 2023-12-02T00:29:05.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.255+0000 7fda6b7fe640 1 -- 172.21.15.46:0/3569000462 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda6402c880 con 0x7fda6c140310 2023-12-02T00:29:05.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.255+0000 7fda6b7fe640 1 -- 172.21.15.46:0/3569000462 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6402cbf0 con 0x7fda6c140310 2023-12-02T00:29:05.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.256+0000 7fda73100640 1 -- 172.21.15.46:0/3569000462 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda6c140310 msgr2=0x7fda6c1618e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:05.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.256+0000 7fda73100640 1 --2- 172.21.15.46:0/3569000462 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda6c140310 0x7fda6c1618e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fda64009dc0 tx=0x7fda6402bd50 comp rx=0 tx=0).stop 2023-12-02T00:29:05.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.256+0000 7fda73100640 1 -- 172.21.15.46:0/3569000462 shutdown_connections 2023-12-02T00:29:05.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.256+0000 7fda73100640 1 --2- 172.21.15.46:0/3569000462 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda6c140310 0x7fda6c1618e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:05.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.256+0000 7fda73100640 1 -- 172.21.15.46:0/3569000462 >> 172.21.15.46:0/3569000462 conn(0x7fda6c0b1e50 msgr2=0x7fda6c0b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:05.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.256+0000 7fda73100640 1 -- 172.21.15.46:0/3569000462 shutdown_connections 2023-12-02T00:29:05.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.256+0000 7fda73100640 1 -- 172.21.15.46:0/3569000462 wait complete. 2023-12-02T00:29:05.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.257+0000 7fda73100640 1 Processor -- start 2023-12-02T00:29:05.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.257+0000 7fda73100640 1 -- start start 2023-12-02T00:29:05.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.258+0000 7fda73100640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda6c06abe0 0x7fda6c1f2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:05.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.258+0000 7fda73100640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda6c140310 0x7fda6c145c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:05.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.258+0000 7fda73100640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda6c1409d0 0x7fda6c0ae950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:05.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.258+0000 7fda73100640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6c1465c0 con 0x7fda6c140310 2023-12-02T00:29:05.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.258+0000 7fda73100640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6c1468a0 con 0x7fda6c06abe0 2023-12-02T00:29:05.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.258+0000 7fda73100640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda6c146b80 con 0x7fda6c1409d0 2023-12-02T00:29:05.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.258+0000 7fda6bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda6c140310 0x7fda6c145c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:05.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.258+0000 7fda6bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda6c140310 0x7fda6c145c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37994/0 (socket says 172.21.15.46:37994) 2023-12-02T00:29:05.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.258+0000 7fda6bfff640 1 -- 172.21.15.46:0/1582516090 learned_addr learned my addr 172.21.15.46:0/1582516090 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:05.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.258+0000 7fda70e9c640 1 --2- 172.21.15.46:0/1582516090 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda6c06abe0 0x7fda6c1f2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:05.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.258+0000 7fda7169d640 1 --2- 172.21.15.46:0/1582516090 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda6c1409d0 0x7fda6c0ae950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:05.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.258+0000 7fda6bfff640 1 -- 172.21.15.46:0/1582516090 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda6c1409d0 msgr2=0x7fda6c0ae950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:05.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.258+0000 7fda6bfff640 1 --2- 172.21.15.46:0/1582516090 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda6c1409d0 0x7fda6c0ae950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:05.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.258+0000 7fda6bfff640 1 -- 172.21.15.46:0/1582516090 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda6c06abe0 msgr2=0x7fda6c1f2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:05.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.258+0000 7fda6bfff640 1 --2- 172.21.15.46:0/1582516090 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda6c06abe0 0x7fda6c1f2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:05.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.258+0000 7fda6bfff640 1 -- 172.21.15.46:0/1582516090 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda64009a70 con 0x7fda6c140310 2023-12-02T00:29:05.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.258+0000 7fda70e9c640 1 --2- 172.21.15.46:0/1582516090 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda6c06abe0 0x7fda6c1f2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:29:05.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.258+0000 7fda6bfff640 1 --2- 172.21.15.46:0/1582516090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda6c140310 0x7fda6c145c70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fda60000fd0 tx=0x7fda6000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:05.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.259+0000 7fda697fa640 1 -- 172.21.15.46:0/1582516090 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda60005ad0 con 0x7fda6c140310 2023-12-02T00:29:05.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.259+0000 7fda697fa640 1 -- 172.21.15.46:0/1582516090 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda60005c30 con 0x7fda6c140310 2023-12-02T00:29:05.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.259+0000 7fda73100640 1 -- 172.21.15.46:0/1582516090 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda6c143e10 con 0x7fda6c140310 2023-12-02T00:29:05.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.259+0000 7fda697fa640 1 -- 172.21.15.46:0/1582516090 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda600072b0 con 0x7fda6c140310 2023-12-02T00:29:05.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.259+0000 7fda73100640 1 -- 172.21.15.46:0/1582516090 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda6c1461b0 con 0x7fda6c140310 2023-12-02T00:29:05.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.260+0000 7fda73100640 1 -- 172.21.15.46:0/1582516090 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda6c161030 con 0x7fda6c140310 2023-12-02T00:29:05.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.260+0000 7fda697fa640 1 -- 172.21.15.46:0/1582516090 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fda60007410 con 0x7fda6c140310 2023-12-02T00:29:05.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.261+0000 7fda697fa640 1 --2- 172.21.15.46:0/1582516090 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fda5806ae40 0x7fda5806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:05.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.261+0000 7fda697fa640 1 -- 172.21.15.46:0/1582516090 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fda60088c40 con 0x7fda6c140310 2023-12-02T00:29:05.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.261+0000 7fda70e9c640 1 --2- 172.21.15.46:0/1582516090 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fda5806ae40 0x7fda5806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:05.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.261+0000 7fda70e9c640 1 --2- 172.21.15.46:0/1582516090 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fda5806ae40 0x7fda5806d300 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fda64036000 tx=0x7fda64030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:05.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.265+0000 7fda697fa640 1 -- 172.21.15.46:0/1582516090 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fda60053da0 con 0x7fda6c140310 2023-12-02T00:29:05.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.478+0000 7fda73100640 1 -- 172.21.15.46:0/1582516090 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda6c069400 con 0x7fda6c140310 2023-12-02T00:29:05.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.478+0000 7fda697fa640 1 -- 172.21.15.46:0/1582516090 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4907 (secure 0 0 0) 0x7fda60017020 con 0x7fda6c140310 2023-12-02T00:29:05.480 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:29:05.480 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/2059113401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2059113401},{"type":"v1","addr":"172.21.15.46:6834","nonce":2059113401}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/987532381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":987532381},{"type":"v1","addr":"172.21.15.46:6835","nonce":987532381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/449974862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":449974862},{"type":"v1","addr":"172.21.15.162:6836","nonce":449974862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-02T00:28:49.086548+0000","modified":"2023-12-02T00:28:51.082469+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4253":{"gid":4253,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/1231844478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":1231844478},{"type":"v1","addr":"172.21.15.162:6837","nonce":1231844478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-02T00:29:05.480 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 6 2023-12-02T00:29:05.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.480+0000 7fda73100640 1 -- 172.21.15.46:0/1582516090 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fda5806ae40 msgr2=0x7fda5806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:05.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.480+0000 7fda73100640 1 --2- 172.21.15.46:0/1582516090 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fda5806ae40 0x7fda5806d300 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fda64036000 tx=0x7fda64030000 comp rx=0 tx=0).stop 2023-12-02T00:29:05.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.481+0000 7fda73100640 1 -- 172.21.15.46:0/1582516090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda6c140310 msgr2=0x7fda6c145c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:05.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.481+0000 7fda73100640 1 --2- 172.21.15.46:0/1582516090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda6c140310 0x7fda6c145c70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fda60000fd0 tx=0x7fda6000cbc0 comp rx=0 tx=0).stop 2023-12-02T00:29:05.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.481+0000 7fda73100640 1 -- 172.21.15.46:0/1582516090 shutdown_connections 2023-12-02T00:29:05.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.481+0000 7fda73100640 1 --2- 172.21.15.46:0/1582516090 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fda5806ae40 0x7fda5806d300 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:05.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.481+0000 7fda73100640 1 --2- 172.21.15.46:0/1582516090 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda6c1409d0 0x7fda6c0ae950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:05.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.481+0000 7fda73100640 1 --2- 172.21.15.46:0/1582516090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda6c140310 0x7fda6c145c70 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:05.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.481+0000 7fda73100640 1 --2- 172.21.15.46:0/1582516090 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda6c06abe0 0x7fda6c1f2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:05.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.481+0000 7fda73100640 1 -- 172.21.15.46:0/1582516090 >> 172.21.15.46:0/1582516090 conn(0x7fda6c0b1e50 msgr2=0x7fda6c15ed10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:05.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.481+0000 7fda73100640 1 -- 172.21.15.46:0/1582516090 shutdown_connections 2023-12-02T00:29:05.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.481+0000 7fda73100640 1 -- 172.21.15.46:0/1582516090 wait complete. 2023-12-02T00:29:05.493 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:29:05.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.598+0000 7f641a361640 1 Processor -- start 2023-12-02T00:29:05.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.598+0000 7f641a361640 1 -- start start 2023-12-02T00:29:05.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.599+0000 7f641a361640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6414160fb0 0x7f6414161440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:05.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.599+0000 7f641a361640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f64140d0ed0 0x7f64140d1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:05.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.599+0000 7f641a361640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6414161d20 con 0x7f641406ea50 2023-12-02T00:29:05.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.599+0000 7f641a361640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f641415e760 con 0x7f6414160fb0 2023-12-02T00:29:05.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.599+0000 7f641a361640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f641415d9a0 con 0x7f64140d0ed0 2023-12-02T00:29:05.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.599+0000 7f64137fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6414160fb0 0x7f6414161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:05.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.599+0000 7f64137fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6414160fb0 0x7f6414161440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37996/0 (socket says 172.21.15.46:37996) 2023-12-02T00:29:05.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.599+0000 7f6413fff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f641406ea50 0x7f641406ee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58312/0 (socket says 172.21.15.46:58312) 2023-12-02T00:29:05.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.599+0000 7f64137fe640 1 -- 172.21.15.46:0/3609102947 learned_addr learned my addr 172.21.15.46:0/3609102947 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:05.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.599+0000 7f64188fe640 1 --2- 172.21.15.46:0/3609102947 >> v2:172.21.15.162:3300/0 conn(0x7f64140d0ed0 0x7f64140d1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:05.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.599+0000 7f64137fe640 1 -- 172.21.15.46:0/3609102947 >> v2:172.21.15.162:3300/0 conn(0x7f64140d0ed0 msgr2=0x7f64140d1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:05.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.599+0000 7f64137fe640 1 --2- 172.21.15.46:0/3609102947 >> v2:172.21.15.162:3300/0 conn(0x7f64140d0ed0 0x7f64140d1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:05.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.599+0000 7f64137fe640 1 -- 172.21.15.46:0/3609102947 >> v1:172.21.15.46:6789/0 conn(0x7f641406ea50 legacy=0x7f641406ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:05.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.599+0000 7f64137fe640 1 -- 172.21.15.46:0/3609102947 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f641415dc80 con 0x7f6414160fb0 2023-12-02T00:29:05.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.599+0000 7f6412ffd640 1 -- 172.21.15.46:0/3609102947 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 626023355 0 0) 0x7f6414161d20 con 0x7f641406ea50 2023-12-02T00:29:05.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.600+0000 7f64137fe640 1 --2- 172.21.15.46:0/3609102947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6414160fb0 0x7f6414161440 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f64140b55a0 tx=0x7f640402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8be860c15ab1577d server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:05.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.600+0000 7f6412ffd640 1 -- 172.21.15.46:0/3609102947 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6404031870 con 0x7f6414160fb0 2023-12-02T00:29:05.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.600+0000 7f6412ffd640 1 -- 172.21.15.46:0/3609102947 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64040319d0 con 0x7f6414160fb0 2023-12-02T00:29:05.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.600+0000 7f6412ffd640 1 -- 172.21.15.46:0/3609102947 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6404031d60 con 0x7f6414160fb0 2023-12-02T00:29:05.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.601+0000 7f641a361640 1 -- 172.21.15.46:0/3609102947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6414160fb0 msgr2=0x7f6414161440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:05.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.601+0000 7f641a361640 1 --2- 172.21.15.46:0/3609102947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6414160fb0 0x7f6414161440 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f64140b55a0 tx=0x7f640402b910 comp rx=0 tx=0).stop 2023-12-02T00:29:05.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.601+0000 7f641a361640 1 -- 172.21.15.46:0/3609102947 shutdown_connections 2023-12-02T00:29:05.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.601+0000 7f641a361640 1 --2- 172.21.15.46:0/3609102947 >> v2:172.21.15.162:3300/0 conn(0x7f64140d0ed0 0x7f64140d1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:05.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.601+0000 7f641a361640 1 --2- 172.21.15.46:0/3609102947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6414160fb0 0x7f6414161440 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:05.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.601+0000 7f641a361640 1 -- 172.21.15.46:0/3609102947 >> 172.21.15.46:0/3609102947 conn(0x7f641406a6b0 msgr2=0x7f641406aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:05.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.601+0000 7f641a361640 1 -- 172.21.15.46:0/3609102947 shutdown_connections 2023-12-02T00:29:05.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.601+0000 7f641a361640 1 -- 172.21.15.46:0/3609102947 wait complete. 2023-12-02T00:29:05.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.602+0000 7f641a361640 1 Processor -- start 2023-12-02T00:29:05.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.602+0000 7f641a361640 1 -- start start 2023-12-02T00:29:05.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.603+0000 7f641a361640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f641406ea50 0x7f6414156970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:05.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.603+0000 7f641a361640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64140d0ed0 0x7f6414159640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:05.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.603+0000 7f641a361640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6414160fb0 0x7f641413f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:05.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.603+0000 7f641a361640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64141f32e0 con 0x7f64140d0ed0 2023-12-02T00:29:05.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.603+0000 7f641a361640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f641415a130 con 0x7f641406ea50 2023-12-02T00:29:05.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.603+0000 7f641a361640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f641415a410 con 0x7f6414160fb0 2023-12-02T00:29:05.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.603+0000 7f64137fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64140d0ed0 0x7f6414159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:05.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.603+0000 7f64137fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64140d0ed0 0x7f6414159640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38004/0 (socket says 172.21.15.46:38004) 2023-12-02T00:29:05.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.603+0000 7f64137fe640 1 -- 172.21.15.46:0/1270812717 learned_addr learned my addr 172.21.15.46:0/1270812717 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:05.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.603+0000 7f64188fe640 1 --2- 172.21.15.46:0/1270812717 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6414160fb0 0x7f641413f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:05.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.603+0000 7f6413fff640 1 --2- 172.21.15.46:0/1270812717 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f641406ea50 0x7f6414156970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:05.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.603+0000 7f64137fe640 1 -- 172.21.15.46:0/1270812717 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6414160fb0 msgr2=0x7f641413f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:05.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.603+0000 7f64137fe640 1 --2- 172.21.15.46:0/1270812717 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6414160fb0 0x7f641413f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:05.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.603+0000 7f64137fe640 1 -- 172.21.15.46:0/1270812717 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f641406ea50 msgr2=0x7f6414156970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:05.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.603+0000 7f64137fe640 1 --2- 172.21.15.46:0/1270812717 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f641406ea50 0x7f6414156970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:05.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.603+0000 7f64137fe640 1 -- 172.21.15.46:0/1270812717 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64040094d0 con 0x7f64140d0ed0 2023-12-02T00:29:05.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.603+0000 7f64188fe640 1 --2- 172.21.15.46:0/1270812717 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6414160fb0 0x7f641413f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:05.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.603+0000 7f6413fff640 1 --2- 172.21.15.46:0/1270812717 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f641406ea50 0x7f6414156970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:05.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.603+0000 7f64137fe640 1 --2- 172.21.15.46:0/1270812717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64140d0ed0 0x7f6414159640 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6404000c00 tx=0x7f64040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:05.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.603+0000 7f6410ff9640 1 -- 172.21.15.46:0/1270812717 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6404004230 con 0x7f64140d0ed0 2023-12-02T00:29:05.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.604+0000 7f6410ff9640 1 -- 172.21.15.46:0/1270812717 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64040043b0 con 0x7f64140d0ed0 2023-12-02T00:29:05.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.604+0000 7f641a361640 1 -- 172.21.15.46:0/1270812717 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f641415a6f0 con 0x7f64140d0ed0 2023-12-02T00:29:05.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.604+0000 7f6410ff9640 1 -- 172.21.15.46:0/1270812717 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6404004520 con 0x7f64140d0ed0 2023-12-02T00:29:05.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.604+0000 7f641a361640 1 -- 172.21.15.46:0/1270812717 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64141e7560 con 0x7f64140d0ed0 2023-12-02T00:29:05.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.604+0000 7f641a361640 1 -- 172.21.15.46:0/1270812717 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64141f3520 con 0x7f64140d0ed0 2023-12-02T00:29:05.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.605+0000 7f6410ff9640 1 -- 172.21.15.46:0/1270812717 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6404043070 con 0x7f64140d0ed0 2023-12-02T00:29:05.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.605+0000 7f6410ff9640 1 --2- 172.21.15.46:0/1270812717 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f640006ae40 0x7f640006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:05.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.605+0000 7f6410ff9640 1 -- 172.21.15.46:0/1270812717 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f640402d040 con 0x7f64140d0ed0 2023-12-02T00:29:05.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.605+0000 7f6413fff640 1 --2- 172.21.15.46:0/1270812717 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f640006ae40 0x7f640006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:05.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.606+0000 7f6413fff640 1 --2- 172.21.15.46:0/1270812717 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f640006ae40 0x7f640006d300 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6408000f60 tx=0x7f6408007ca0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:05.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.609+0000 7f6410ff9640 1 -- 172.21.15.46:0/1270812717 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f64141f3520 con 0x7f64140d0ed0 2023-12-02T00:29:05.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.789+0000 7f641a361640 1 -- 172.21.15.46:0/1270812717 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f64141f3520 con 0x7f64140d0ed0 2023-12-02T00:29:05.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.790+0000 7f6410ff9640 1 -- 172.21.15.46:0/1270812717 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f64141f3520 con 0x7f64140d0ed0 2023-12-02T00:29:05.791 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:29:05.792 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":21,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:28:50.700112+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-02T00:28:47.273337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-02T00:28:48.060115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:29:05.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.792+0000 7f641a361640 1 -- 172.21.15.46:0/1270812717 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f640006ae40 msgr2=0x7f640006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:05.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.792+0000 7f641a361640 1 --2- 172.21.15.46:0/1270812717 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f640006ae40 0x7f640006d300 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6408000f60 tx=0x7f6408007ca0 comp rx=0 tx=0).stop 2023-12-02T00:29:05.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.792+0000 7f641a361640 1 -- 172.21.15.46:0/1270812717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64140d0ed0 msgr2=0x7f6414159640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:05.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.792+0000 7f641a361640 1 --2- 172.21.15.46:0/1270812717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64140d0ed0 0x7f6414159640 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6404000c00 tx=0x7f64040046c0 comp rx=0 tx=0).stop 2023-12-02T00:29:05.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.792+0000 7f641a361640 1 -- 172.21.15.46:0/1270812717 shutdown_connections 2023-12-02T00:29:05.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.792+0000 7f641a361640 1 --2- 172.21.15.46:0/1270812717 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f640006ae40 0x7f640006d300 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:05.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.792+0000 7f641a361640 1 --2- 172.21.15.46:0/1270812717 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6414160fb0 0x7f641413f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:05.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.792+0000 7f641a361640 1 --2- 172.21.15.46:0/1270812717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64140d0ed0 0x7f6414159640 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:05.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.792+0000 7f641a361640 1 --2- 172.21.15.46:0/1270812717 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f641406ea50 0x7f6414156970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:05.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.792+0000 7f641a361640 1 -- 172.21.15.46:0/1270812717 >> 172.21.15.46:0/1270812717 conn(0x7f641406a6b0 msgr2=0x7f641415c840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:05.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.792+0000 7f641a361640 1 -- 172.21.15.46:0/1270812717 shutdown_connections 2023-12-02T00:29:05.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.792+0000 7f641a361640 1 -- 172.21.15.46:0/1270812717 wait complete. 2023-12-02T00:29:05.805 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-02T00:29:05.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.910+0000 7feea7ddc640 1 Processor -- start 2023-12-02T00:29:05.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.911+0000 7feea7ddc640 1 -- start start 2023-12-02T00:29:05.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.911+0000 7feea7ddc640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7feea015d330 0x7feea00b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:05.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.911+0000 7feea7ddc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feea00d3900 0x7feea00b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:05.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.911+0000 7feea7ddc640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feea00b2b10 con 0x7feea00d2910 2023-12-02T00:29:05.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.911+0000 7feea7ddc640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feea00d1590 con 0x7feea015d330 2023-12-02T00:29:05.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.911+0000 7feea7ddc640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feea00d1870 con 0x7feea00d3900 2023-12-02T00:29:05.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.912+0000 7feea5b78640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7feea015d330 0x7feea00b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:05.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.912+0000 7feea6379640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feea00d3900 0x7feea00b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:05.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.912+0000 7feea5377640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7feea00d2910 0x7feea00d31f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.46:46928/0 (socket says 172.21.15.46:46928) 2023-12-02T00:29:05.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.912+0000 7feea5377640 1 -- 172.21.15.46:0/767661014 learned_addr learned my addr 172.21.15.46:0/767661014 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:05.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.912+0000 7feea5b78640 1 -- 172.21.15.46:0/767661014 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feea00d3900 msgr2=0x7feea00b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:05.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.912+0000 7feea5b78640 1 --2- 172.21.15.46:0/767661014 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feea00d3900 0x7feea00b6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:05.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.912+0000 7feea5b78640 1 -- 172.21.15.46:0/767661014 >> v1:172.21.15.162:6789/0 conn(0x7feea00d2910 legacy=0x7feea00d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:05.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.912+0000 7feea5b78640 1 -- 172.21.15.46:0/767661014 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea00d1b50 con 0x7feea015d330 2023-12-02T00:29:05.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.912+0000 7feea6379640 1 --2- 172.21.15.46:0/767661014 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feea00d3900 0x7feea00b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:29:05.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.912+0000 7feea4b76640 1 -- 172.21.15.46:0/767661014 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 912669900 0 0) 0x7feea00b2b10 con 0x7feea00d2910 2023-12-02T00:29:05.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.913+0000 7feea5b78640 1 --2- 172.21.15.46:0/767661014 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feea015d330 0x7feea00b8b80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fee9c0097a0 tx=0x7fee9c02b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59696623c7bd54b8 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:05.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.913+0000 7feea4b76640 1 -- 172.21.15.46:0/767661014 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9c02e070 con 0x7feea015d330 2023-12-02T00:29:05.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.913+0000 7feea4b76640 1 -- 172.21.15.46:0/767661014 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee9c0315a0 con 0x7feea015d330 2023-12-02T00:29:05.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.913+0000 7feea4b76640 1 -- 172.21.15.46:0/767661014 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9c031930 con 0x7feea015d330 2023-12-02T00:29:05.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.914+0000 7feea7ddc640 1 -- 172.21.15.46:0/767661014 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feea015d330 msgr2=0x7feea00b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:05.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.914+0000 7feea7ddc640 1 --2- 172.21.15.46:0/767661014 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feea015d330 0x7feea00b8b80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fee9c0097a0 tx=0x7fee9c02b840 comp rx=0 tx=0).stop 2023-12-02T00:29:05.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.914+0000 7feea7ddc640 1 -- 172.21.15.46:0/767661014 shutdown_connections 2023-12-02T00:29:05.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.914+0000 7feea7ddc640 1 --2- 172.21.15.46:0/767661014 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feea00d3900 0x7feea00b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:05.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.914+0000 7feea7ddc640 1 --2- 172.21.15.46:0/767661014 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feea015d330 0x7feea00b8b80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:05.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.914+0000 7feea7ddc640 1 -- 172.21.15.46:0/767661014 >> 172.21.15.46:0/767661014 conn(0x7feea0072020 msgr2=0x7feea0072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:05.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.914+0000 7feea7ddc640 1 -- 172.21.15.46:0/767661014 shutdown_connections 2023-12-02T00:29:05.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.914+0000 7feea7ddc640 1 -- 172.21.15.46:0/767661014 wait complete. 2023-12-02T00:29:05.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.915+0000 7feea7ddc640 1 Processor -- start 2023-12-02T00:29:05.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.915+0000 7feea7ddc640 1 -- start start 2023-12-02T00:29:05.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.916+0000 7feea7ddc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feea00d2910 0x7feea014e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:05.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.916+0000 7feea7ddc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feea00d3900 0x7feea00b2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:05.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.916+0000 7feea7ddc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feea015d330 0x7feea00b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:05.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.916+0000 7feea7ddc640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea00c7b90 con 0x7feea00d2910 2023-12-02T00:29:05.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.916+0000 7feea7ddc640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea00c7fb0 con 0x7feea015d330 2023-12-02T00:29:05.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.916+0000 7feea7ddc640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feea014ed80 con 0x7feea00d3900 2023-12-02T00:29:05.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.916+0000 7feea5b78640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feea00d2910 0x7feea014e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:05.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.916+0000 7feea5b78640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feea00d2910 0x7feea014e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38008/0 (socket says 172.21.15.46:38008) 2023-12-02T00:29:05.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.916+0000 7feea5b78640 1 -- 172.21.15.46:0/2942336477 learned_addr learned my addr 172.21.15.46:0/2942336477 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:05.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.916+0000 7feea6379640 1 --2- 172.21.15.46:0/2942336477 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feea015d330 0x7feea00b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:05.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.916+0000 7feea5377640 1 --2- 172.21.15.46:0/2942336477 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feea00d3900 0x7feea00b2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:05.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.916+0000 7feea5b78640 1 -- 172.21.15.46:0/2942336477 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feea00d3900 msgr2=0x7feea00b2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:05.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.916+0000 7feea5b78640 1 --2- 172.21.15.46:0/2942336477 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feea00d3900 0x7feea00b2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:05.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.916+0000 7feea5b78640 1 -- 172.21.15.46:0/2942336477 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feea015d330 msgr2=0x7feea00b32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:05.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.916+0000 7feea5b78640 1 --2- 172.21.15.46:0/2942336477 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feea015d330 0x7feea00b32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:05.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.916+0000 7feea5b78640 1 -- 172.21.15.46:0/2942336477 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee98009a70 con 0x7feea00d2910 2023-12-02T00:29:05.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.916+0000 7feea6379640 1 --2- 172.21.15.46:0/2942336477 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feea015d330 0x7feea00b32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:29:05.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.916+0000 7feea5377640 1 --2- 172.21.15.46:0/2942336477 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feea00d3900 0x7feea00b2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:29:05.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.916+0000 7feea5b78640 1 --2- 172.21.15.46:0/2942336477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feea00d2910 0x7feea014e840 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fee9c0098c0 tx=0x7fee9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:05.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.916+0000 7fee967fc640 1 -- 172.21.15.46:0/2942336477 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9c032840 con 0x7feea00d2910 2023-12-02T00:29:05.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.917+0000 7feea7ddc640 1 -- 172.21.15.46:0/2942336477 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee9c009400 con 0x7feea00d2910 2023-12-02T00:29:05.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.917+0000 7fee967fc640 1 -- 172.21.15.46:0/2942336477 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee9c031f00 con 0x7feea00d2910 2023-12-02T00:29:05.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.917+0000 7fee967fc640 1 -- 172.21.15.46:0/2942336477 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9c032be0 con 0x7feea00d2910 2023-12-02T00:29:05.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.917+0000 7feea7ddc640 1 -- 172.21.15.46:0/2942336477 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feea00c71f0 con 0x7feea00d2910 2023-12-02T00:29:05.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.917+0000 7feea7ddc640 1 -- 172.21.15.46:0/2942336477 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feea0069400 con 0x7feea00d2910 2023-12-02T00:29:05.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.918+0000 7fee967fc640 1 -- 172.21.15.46:0/2942336477 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fee9c032d40 con 0x7feea00d2910 2023-12-02T00:29:05.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.918+0000 7fee967fc640 1 --2- 172.21.15.46:0/2942336477 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fee8406ae40 0x7fee8406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:05.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.918+0000 7fee967fc640 1 -- 172.21.15.46:0/2942336477 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fee9c0ab5f0 con 0x7feea00d2910 2023-12-02T00:29:05.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.918+0000 7feea5377640 1 --2- 172.21.15.46:0/2942336477 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fee8406ae40 0x7fee8406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:05.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.919+0000 7feea5377640 1 --2- 172.21.15.46:0/2942336477 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fee8406ae40 0x7fee8406d300 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7feea00b3de0 tx=0x7fee90009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:05.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:05.923+0000 7fee967fc640 1 -- 172.21.15.46:0/2942336477 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fee9c076620 con 0x7feea00d2910 2023-12-02T00:29:06.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.138+0000 7feea7ddc640 1 -- 172.21.15.46:0/2942336477 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feea006c080 con 0x7feea00d2910 2023-12-02T00:29:06.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.138+0000 7fee967fc640 1 -- 172.21.15.46:0/2942336477 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4907 (secure 0 0 0) 0x7fee9c080070 con 0x7feea00d2910 2023-12-02T00:29:06.141 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:29:06.142 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/2059113401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2059113401},{"type":"v1","addr":"172.21.15.46:6834","nonce":2059113401}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/987532381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":987532381},{"type":"v1","addr":"172.21.15.46:6835","nonce":987532381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/449974862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":449974862},{"type":"v1","addr":"172.21.15.162:6836","nonce":449974862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-02T00:28:49.086548+0000","modified":"2023-12-02T00:28:51.082469+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4253":{"gid":4253,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/1231844478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":1231844478},{"type":"v1","addr":"172.21.15.162:6837","nonce":1231844478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-02T00:29:06.142 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 6 2023-12-02T00:29:06.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.140+0000 7feea7ddc640 1 -- 172.21.15.46:0/2942336477 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fee8406ae40 msgr2=0x7fee8406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:06.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.140+0000 7feea7ddc640 1 --2- 172.21.15.46:0/2942336477 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fee8406ae40 0x7fee8406d300 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7feea00b3de0 tx=0x7fee90009040 comp rx=0 tx=0).stop 2023-12-02T00:29:06.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.140+0000 7feea7ddc640 1 -- 172.21.15.46:0/2942336477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feea00d2910 msgr2=0x7feea014e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:06.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.140+0000 7feea7ddc640 1 --2- 172.21.15.46:0/2942336477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feea00d2910 0x7feea014e840 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fee9c0098c0 tx=0x7fee9c0046c0 comp rx=0 tx=0).stop 2023-12-02T00:29:06.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.140+0000 7feea7ddc640 1 -- 172.21.15.46:0/2942336477 shutdown_connections 2023-12-02T00:29:06.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.140+0000 7feea7ddc640 1 --2- 172.21.15.46:0/2942336477 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fee8406ae40 0x7fee8406d300 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:06.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.140+0000 7feea7ddc640 1 --2- 172.21.15.46:0/2942336477 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feea015d330 0x7feea00b32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:06.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.140+0000 7feea7ddc640 1 --2- 172.21.15.46:0/2942336477 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feea00d3900 0x7feea00b2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:06.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.140+0000 7feea7ddc640 1 --2- 172.21.15.46:0/2942336477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feea00d2910 0x7feea014e840 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:06.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.140+0000 7feea7ddc640 1 -- 172.21.15.46:0/2942336477 >> 172.21.15.46:0/2942336477 conn(0x7feea0072020 msgr2=0x7feea00d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:06.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.140+0000 7feea7ddc640 1 -- 172.21.15.46:0/2942336477 shutdown_connections 2023-12-02T00:29:06.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.140+0000 7feea7ddc640 1 -- 172.21.15.46:0/2942336477 wait complete. 2023-12-02T00:29:06.554 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-02T00:28:49.086548+0000', 'modified': '2023-12-02T00:28:51.082469+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4253}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4253': {'gid': 4253, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.162:6837/1231844478', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6835', 'nonce': 1231844478}, {'type': 'v1', 'addr': '172.21.15.162:6837', 'nonce': 1231844478}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-02T00:29:06.555 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-02T00:29:06.555 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-12-02T00:29:06.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.665+0000 7f3e9b4e5640 1 Processor -- start 2023-12-02T00:29:06.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.666+0000 7f3e9b4e5640 1 -- start start 2023-12-02T00:29:06.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.666+0000 7f3e9b4e5640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3e940d2910 0x7f3e940d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:06.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.666+0000 7f3e9b4e5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e940d3730 0x7f3e940b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:06.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.666+0000 7f3e9b4e5640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e940b2b10 con 0x7f3e9415d330 2023-12-02T00:29:06.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.666+0000 7f3e9b4e5640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e940d1590 con 0x7f3e940d2910 2023-12-02T00:29:06.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.666+0000 7f3e9b4e5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e940d1870 con 0x7f3e940d3730 2023-12-02T00:29:06.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.666+0000 7f3e98a80640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3e940d2910 0x7f3e940d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:06.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.666+0000 7f3e99281640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f3e9415d330 0x7f3e940b8b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58318/0 (socket says 172.21.15.46:58318) 2023-12-02T00:29:06.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.666+0000 7f3e99281640 1 -- 172.21.15.46:0/3376239444 learned_addr learned my addr 172.21.15.46:0/3376239444 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:06.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.667+0000 7f3e99a82640 1 --2- 172.21.15.46:0/3376239444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e940d3730 0x7f3e940b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:06.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.667+0000 7f3e98a80640 1 -- 172.21.15.46:0/3376239444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e940d3730 msgr2=0x7f3e940b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:06.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.667+0000 7f3e98a80640 1 --2- 172.21.15.46:0/3376239444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e940d3730 0x7f3e940b6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:06.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.667+0000 7f3e98a80640 1 -- 172.21.15.46:0/3376239444 >> v1:172.21.15.46:6789/0 conn(0x7f3e9415d330 legacy=0x7f3e940b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:06.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.667+0000 7f3e93fff640 1 -- 172.21.15.46:0/3376239444 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2312285527 0 0) 0x7f3e940b2b10 con 0x7f3e9415d330 2023-12-02T00:29:06.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.667+0000 7f3e98a80640 1 -- 172.21.15.46:0/3376239444 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e940d3b90 con 0x7f3e940d2910 2023-12-02T00:29:06.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.668+0000 7f3e98a80640 1 --2- 172.21.15.46:0/3376239444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e940d2910 0x7f3e940d31f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3e880097a0 tx=0x7f3e8802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f65da6aba35fbeeb server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:06.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.668+0000 7f3e93fff640 1 -- 172.21.15.46:0/3376239444 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8802e070 con 0x7f3e940d2910 2023-12-02T00:29:06.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.668+0000 7f3e93fff640 1 -- 172.21.15.46:0/3376239444 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e880315a0 con 0x7f3e940d2910 2023-12-02T00:29:06.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.668+0000 7f3e93fff640 1 -- 172.21.15.46:0/3376239444 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e88031930 con 0x7f3e940d2910 2023-12-02T00:29:06.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.668+0000 7f3e9b4e5640 1 -- 172.21.15.46:0/3376239444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e940d2910 msgr2=0x7f3e940d31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:06.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.668+0000 7f3e9b4e5640 1 --2- 172.21.15.46:0/3376239444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e940d2910 0x7f3e940d31f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3e880097a0 tx=0x7f3e8802b840 comp rx=0 tx=0).stop 2023-12-02T00:29:06.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.668+0000 7f3e9b4e5640 1 -- 172.21.15.46:0/3376239444 shutdown_connections 2023-12-02T00:29:06.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.668+0000 7f3e9b4e5640 1 --2- 172.21.15.46:0/3376239444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e940d3730 0x7f3e940b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:06.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.668+0000 7f3e9b4e5640 1 --2- 172.21.15.46:0/3376239444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e940d2910 0x7f3e940d31f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:06.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.668+0000 7f3e9b4e5640 1 -- 172.21.15.46:0/3376239444 >> 172.21.15.46:0/3376239444 conn(0x7f3e94072020 msgr2=0x7f3e94072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:06.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.668+0000 7f3e9b4e5640 1 -- 172.21.15.46:0/3376239444 shutdown_connections 2023-12-02T00:29:06.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.668+0000 7f3e9b4e5640 1 -- 172.21.15.46:0/3376239444 wait complete. 2023-12-02T00:29:06.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.670+0000 7f3e9b4e5640 1 Processor -- start 2023-12-02T00:29:06.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.670+0000 7f3e9b4e5640 1 -- start start 2023-12-02T00:29:06.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.670+0000 7f3e9b4e5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e940d2910 0x7f3e94153e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:06.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.670+0000 7f3e9b4e5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e940d3730 0x7f3e940ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:06.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.670+0000 7f3e9b4e5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e9415d330 0x7f3e940b8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:06.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.670+0000 7f3e9b4e5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e941543b0 con 0x7f3e940d3730 2023-12-02T00:29:06.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.670+0000 7f3e9b4e5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e940cd180 con 0x7f3e940d2910 2023-12-02T00:29:06.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.670+0000 7f3e9b4e5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e940c6c60 con 0x7f3e9415d330 2023-12-02T00:29:06.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.670+0000 7f3e98a80640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e940d3730 0x7f3e940ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:06.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.670+0000 7f3e98a80640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e940d3730 0x7f3e940ccc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38014/0 (socket says 172.21.15.46:38014) 2023-12-02T00:29:06.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.671+0000 7f3e98a80640 1 -- 172.21.15.46:0/1817538399 learned_addr learned my addr 172.21.15.46:0/1817538399 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:06.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.671+0000 7f3e99281640 1 --2- 172.21.15.46:0/1817538399 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e940d2910 0x7f3e94153e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:06.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.671+0000 7f3e99a82640 1 --2- 172.21.15.46:0/1817538399 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e9415d330 0x7f3e940b8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:06.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.671+0000 7f3e98a80640 1 -- 172.21.15.46:0/1817538399 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e9415d330 msgr2=0x7f3e940b8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:06.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.671+0000 7f3e98a80640 1 --2- 172.21.15.46:0/1817538399 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e9415d330 0x7f3e940b8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:06.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.671+0000 7f3e98a80640 1 -- 172.21.15.46:0/1817538399 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e940d2910 msgr2=0x7f3e94153e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:06.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.671+0000 7f3e98a80640 1 --2- 172.21.15.46:0/1817538399 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e940d2910 0x7f3e94153e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:06.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.671+0000 7f3e98a80640 1 -- 172.21.15.46:0/1817538399 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e88009400 con 0x7f3e940d3730 2023-12-02T00:29:06.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.671+0000 7f3e99281640 1 --2- 172.21.15.46:0/1817538399 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e940d2910 0x7f3e94153e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:06.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.671+0000 7f3e98a80640 1 --2- 172.21.15.46:0/1817538399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e940d3730 0x7f3e940ccc40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3e88034040 tx=0x7f3e880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:06.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.671+0000 7f3e91ffb640 1 -- 172.21.15.46:0/1817538399 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e88032840 con 0x7f3e940d3730 2023-12-02T00:29:06.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.671+0000 7f3e91ffb640 1 -- 172.21.15.46:0/1817538399 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e88031f00 con 0x7f3e940d3730 2023-12-02T00:29:06.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.671+0000 7f3e91ffb640 1 -- 172.21.15.46:0/1817538399 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e88032be0 con 0x7f3e940d3730 2023-12-02T00:29:06.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.671+0000 7f3e9b4e5640 1 -- 172.21.15.46:0/1817538399 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e9415f470 con 0x7f3e940d3730 2023-12-02T00:29:06.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.671+0000 7f3e9b4e5640 1 -- 172.21.15.46:0/1817538399 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e9415fa30 con 0x7f3e940d3730 2023-12-02T00:29:06.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.673+0000 7f3e91ffb640 1 -- 172.21.15.46:0/1817538399 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3e88032d40 con 0x7f3e940d3730 2023-12-02T00:29:06.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.673+0000 7f3e91ffb640 1 --2- 172.21.15.46:0/1817538399 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3e8006ae70 0x7f3e8006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:06.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.673+0000 7f3e91ffb640 1 -- 172.21.15.46:0/1817538399 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e880ab470 con 0x7f3e940d3730 2023-12-02T00:29:06.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.673+0000 7f3e99281640 1 --2- 172.21.15.46:0/1817538399 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3e8006ae70 0x7f3e8006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:06.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.674+0000 7f3e99281640 1 --2- 172.21.15.46:0/1817538399 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3e8006ae70 0x7f3e8006d330 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c000f60 tx=0x7f3e8c007cc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:06.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.674+0000 7f3e9e078740 1 Processor -- start 2023-12-02T00:29:06.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.674+0000 7f3e9e078740 1 -- start start 2023-12-02T00:29:06.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.675+0000 7f3e9e078740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f447a8dc50 0x55f447a60bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:06.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.675+0000 7f3e9e078740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f447a8e030 0x55f447a909e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:06.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.675+0000 7f3e9e078740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f447a61130 0x55f447a90f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:06.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.675+0000 7f3e9e078740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f447a21980 con 0x55f447a8e030 2023-12-02T00:29:06.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.675+0000 7f3e9e078740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4479d3b90 con 0x55f447a8dc50 2023-12-02T00:29:06.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.675+0000 7f3e9e078740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f4479d1280 con 0x55f447a61130 2023-12-02T00:29:06.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.675+0000 7f3e98a80640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f447a8e030 0x55f447a909e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:06.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.675+0000 7f3e98a80640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f447a8e030 0x55f447a909e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38018/0 (socket says 172.21.15.46:38018) 2023-12-02T00:29:06.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.675+0000 7f3e98a80640 1 -- 172.21.15.46:0/4214651725 learned_addr learned my addr 172.21.15.46:0/4214651725 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:06.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.675+0000 7f3e99281640 1 --2- 172.21.15.46:0/4214651725 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f447a8dc50 0x55f447a60bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:06.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.675+0000 7f3e99a82640 1 --2- 172.21.15.46:0/4214651725 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f447a61130 0x55f447a90f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:06.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.675+0000 7f3e98a80640 1 -- 172.21.15.46:0/4214651725 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f447a61130 msgr2=0x55f447a90f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:06.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.675+0000 7f3e98a80640 1 --2- 172.21.15.46:0/4214651725 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f447a61130 0x55f447a90f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:06.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.675+0000 7f3e98a80640 1 -- 172.21.15.46:0/4214651725 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f447a8dc50 msgr2=0x55f447a60bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:06.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.675+0000 7f3e98a80640 1 --2- 172.21.15.46:0/4214651725 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f447a8dc50 0x55f447a60bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:06.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.675+0000 7f3e98a80640 1 -- 172.21.15.46:0/4214651725 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f447992500 con 0x55f447a8e030 2023-12-02T00:29:06.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.675+0000 7f3e99a82640 1 --2- 172.21.15.46:0/4214651725 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f447a61130 0x55f447a90f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:06.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.675+0000 7f3e98a80640 1 --2- 172.21.15.46:0/4214651725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f447a8e030 0x55f447a909e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3e880a9a50 tx=0x7f3e8807aa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=354568b36aa4d854 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:06.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.676+0000 7f3e7affd640 1 -- 172.21.15.46:0/4214651725 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e88082570 con 0x55f447a8e030 2023-12-02T00:29:06.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.676+0000 7f3e7affd640 1 -- 172.21.15.46:0/4214651725 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e8807e120 con 0x55f447a8e030 2023-12-02T00:29:06.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.676+0000 7f3e7affd640 1 -- 172.21.15.46:0/4214651725 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e88076980 con 0x55f447a8e030 2023-12-02T00:29:06.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.676+0000 7f3e9e078740 1 -- 172.21.15.46:0/4214651725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f447a8e030 msgr2=0x55f447a909e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:06.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.676+0000 7f3e9e078740 1 --2- 172.21.15.46:0/4214651725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f447a8e030 0x55f447a909e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3e880a9a50 tx=0x7f3e8807aa30 comp rx=0 tx=0).stop 2023-12-02T00:29:06.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.676+0000 7f3e9e078740 1 -- 172.21.15.46:0/4214651725 shutdown_connections 2023-12-02T00:29:06.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.676+0000 7f3e9e078740 1 --2- 172.21.15.46:0/4214651725 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f447a61130 0x55f447a90f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:06.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.676+0000 7f3e9e078740 1 --2- 172.21.15.46:0/4214651725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f447a8e030 0x55f447a909e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:06.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.676+0000 7f3e9e078740 1 --2- 172.21.15.46:0/4214651725 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f447a8dc50 0x55f447a60bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:06.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.676+0000 7f3e9e078740 1 -- 172.21.15.46:0/4214651725 >> 172.21.15.46:0/4214651725 conn(0x55f447a62e60 msgr2=0x55f447aab9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:06.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.676+0000 7f3e9e078740 1 -- 172.21.15.46:0/4214651725 shutdown_connections 2023-12-02T00:29:06.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.676+0000 7f3e9e078740 1 -- 172.21.15.46:0/4214651725 wait complete. 2023-12-02T00:29:06.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.677+0000 7f3e9e078740 1 -- start start 2023-12-02T00:29:06.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.677+0000 7f3e9e078740 1 Processor -- start 2023-12-02T00:29:06.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.678+0000 7f3e9e078740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f447a8dc50 0x55f447aa8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:06.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.678+0000 7f3e9e078740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f447a8e030 0x55f447aa9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:06.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.678+0000 7f3e9e078740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f447aa9870 0x55f447aab450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:06.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.678+0000 7f3e9e078740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4479ccb70 con 0x55f447aa9870 2023-12-02T00:29:06.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.678+0000 7f3e9e078740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f447986e00 con 0x55f447a8dc50 2023-12-02T00:29:06.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.678+0000 7f3e9e078740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f447986aa0 con 0x55f447a8e030 2023-12-02T00:29:06.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.678+0000 7f3e9e078740 10 client.0 _ll_register_callbacks cb 0x55f4476deac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-02T00:29:06.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.678+0000 7f3e99a82640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f447aa9870 0x55f447aab450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:06.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.678+0000 7f3e99a82640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f447aa9870 0x55f447aab450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38028/0 (socket says 172.21.15.46:38028) 2023-12-02T00:29:06.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.678+0000 7f3e99a82640 1 -- 172.21.15.46:0/1210396951 learned_addr learned my addr 172.21.15.46:0/1210396951 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:06.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.678+0000 7f3e99281640 1 --2- 172.21.15.46:0/1210396951 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f447a8dc50 0x55f447aa8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:06.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.678+0000 7f3e98a80640 1 --2- 172.21.15.46:0/1210396951 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f447a8e030 0x55f447aa9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:06.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.678+0000 7f3e99a82640 1 -- 172.21.15.46:0/1210396951 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f447a8e030 msgr2=0x55f447aa9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:06.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.678+0000 7f3e99a82640 1 --2- 172.21.15.46:0/1210396951 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f447a8e030 0x55f447aa9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:06.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.678+0000 7f3e99a82640 1 -- 172.21.15.46:0/1210396951 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f447a8dc50 msgr2=0x55f447aa8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:06.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.678+0000 7f3e99a82640 1 --2- 172.21.15.46:0/1210396951 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f447a8dc50 0x55f447aa8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:06.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.678+0000 7f3e99a82640 1 -- 172.21.15.46:0/1210396951 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e9415fa30 con 0x55f447aa9870 2023-12-02T00:29:06.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.678+0000 7f3e99a82640 1 --2- 172.21.15.46:0/1210396951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f447aa9870 0x55f447aab450 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3e84007dd0 tx=0x7f3e84012610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44f2611ef5c43d48 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:06.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.678+0000 7f3e6ffff640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2023-12-02T00:29:06.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.679+0000 7f3e6ffff640 1 -- 172.21.15.46:0/1210396951 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8400e040 con 0x55f447aa9870 2023-12-02T00:29:06.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.679+0000 7f3e6ffff640 1 -- 172.21.15.46:0/1210396951 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e84018470 con 0x55f447aa9870 2023-12-02T00:29:06.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.679+0000 7f3e9e078740 1 -- 172.21.15.46:0/1210396951 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f4479c5b50 con 0x55f447aa9870 2023-12-02T00:29:06.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.679+0000 7f3e6ffff640 1 -- 172.21.15.46:0/1210396951 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e84012ea0 con 0x55f447aa9870 2023-12-02T00:29:06.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.679+0000 7f3e9e078740 10 client.4695 fetch_fsmap learned FSMap version 6 2023-12-02T00:29:06.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.679+0000 7f3e9e078740 1 -- 172.21.15.46:0/1210396951 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x55f4479cb1f0 con 0x55f447aa9870 2023-12-02T00:29:06.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.679+0000 7f3e6ffff640 1 -- 172.21.15.46:0/1210396951 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f4479c5b50 con 0x55f447aa9870 2023-12-02T00:29:06.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.679+0000 7f3e6ffff640 1 -- 172.21.15.46:0/1210396951 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 6) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3e84014040 con 0x55f447aa9870 2023-12-02T00:29:06.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.679+0000 7f3e9e078740 10 client.4695 fetch_fsmap finished waiting for FSMap version 6 2023-12-02T00:29:06.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.679+0000 7f3e9e078740 10 client.4695 resolve_mds: resolved name 'd' to daemon mds.d 2023-12-02T00:29:06.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.679+0000 7f3e9e078740 20 client.4695 populate_metadata read hostname 'smithi046' 2023-12-02T00:29:06.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.679+0000 7f3e9e078740 1 --2- 172.21.15.46:0/1210396951 >> [v2:172.21.15.162:6835/1231844478,v1:172.21.15.162:6837/1231844478] conn(0x55f447abfb60 0x55f447ac92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:06.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.679+0000 7f3e9e078740 4 client.4695 mds_command: new command op to 4253 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-02T00:29:06.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.679+0000 7f3e9e078740 1 -- 172.21.15.46:0/1210396951 --> [v2:172.21.15.162:6835/1231844478,v1:172.21.15.162:6837/1231844478] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f4479c5870 con 0x55f447abfb60 2023-12-02T00:29:06.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.679+0000 7f3e99281640 1 --2- 172.21.15.46:0/1210396951 >> [v2:172.21.15.162:6835/1231844478,v1:172.21.15.162:6837/1231844478] conn(0x55f447abfb60 0x55f447ac92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:06.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.680+0000 7f3e99281640 1 --2- 172.21.15.46:0/1210396951 >> [v2:172.21.15.162:6835/1231844478,v1:172.21.15.162:6837/1231844478] conn(0x55f447abfb60 0x55f447ac92a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99ab0a0a7196afb4 server_cookie=1902d1202bd47034 in_seq=0 out_seq=0 2023-12-02T00:29:06.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.680+0000 7f3e6ffff640 10 client.4695 ms_handle_connect on v2:172.21.15.162:6835/1231844478 2023-12-02T00:29:06.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.680+0000 7f3e6ffff640 1 -- 172.21.15.46:0/1210396951 <== mds.0 v2:172.21.15.162:6835/1231844478 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f4479c5870 con 0x55f447abfb60 2023-12-02T00:29:06.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.680+0000 7f3e6ffff640 10 client.4695 handle_command_reply: tid=0 2023-12-02T00:29:06.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.680+0000 7f3e9e078740 1 client.4695 shutdown 2023-12-02T00:29:06.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.681+0000 7f3e9e078740 1 -- 172.21.15.46:0/1210396951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f447aa9870 msgr2=0x55f447aab450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:06.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.681+0000 7f3e9e078740 1 --2- 172.21.15.46:0/1210396951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f447aa9870 0x55f447aab450 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3e84007dd0 tx=0x7f3e84012610 comp rx=0 tx=0).stop 2023-12-02T00:29:06.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.681+0000 7f3e9e078740 1 -- 172.21.15.46:0/1210396951 shutdown_connections 2023-12-02T00:29:06.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.681+0000 7f3e9e078740 1 --2- 172.21.15.46:0/1210396951 >> [v2:172.21.15.162:6835/1231844478,v1:172.21.15.162:6837/1231844478] conn(0x55f447abfb60 0x55f447ac92a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:06.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.681+0000 7f3e9e078740 1 --2- 172.21.15.46:0/1210396951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f447aa9870 0x55f447aab450 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:06.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.681+0000 7f3e9e078740 1 --2- 172.21.15.46:0/1210396951 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f447a8e030 0x55f447aa9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:06.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.681+0000 7f3e6ffff640 0 client.4695 ms_handle_reset on v2:172.21.15.162:6835/1231844478 2023-12-02T00:29:06.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.681+0000 7f3e9e078740 1 --2- 172.21.15.46:0/1210396951 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f447a8dc50 0x55f447aa8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:06.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.681+0000 7f3e9e078740 1 -- 172.21.15.46:0/1210396951 >> 172.21.15.46:0/1210396951 conn(0x55f447a61130 msgr2=0x55f447a658f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:06.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.681+0000 7f3e9e078740 1 -- 172.21.15.46:0/1210396951 shutdown_connections 2023-12-02T00:29:06.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.681+0000 7f3e9e078740 1 -- 172.21.15.46:0/1210396951 wait complete. 2023-12-02T00:29:06.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.681+0000 7f3e9e078740 20 client.4695 trim_cache size 0 max 16384 2023-12-02T00:29:06.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.697+0000 7f3e9e078740 1 Processor -- start 2023-12-02T00:29:06.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.697+0000 7f3e9e078740 1 -- start start 2023-12-02T00:29:06.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.697+0000 7f3e9e078740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f447a8dc50 0x55f447a5c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:06.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.697+0000 7f3e9e078740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f447a8e030 0x55f447abeea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:06.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.697+0000 7f3e9e078740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f447aa9870 0x55f447aa8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:06.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.697+0000 7f3e9e078740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f44794c270 con 0x55f447aa9870 2023-12-02T00:29:06.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.697+0000 7f3e9e078740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f44799f230 con 0x55f447a8dc50 2023-12-02T00:29:06.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.697+0000 7f3e9e078740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f447a92f20 con 0x55f447a8e030 2023-12-02T00:29:06.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.698+0000 7f3e99a82640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f447aa9870 0x55f447aa8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:06.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.698+0000 7f3e99a82640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f447aa9870 0x55f447aa8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38034/0 (socket says 172.21.15.46:38034) 2023-12-02T00:29:06.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.698+0000 7f3e99a82640 1 -- 172.21.15.46:0/3458756316 learned_addr learned my addr 172.21.15.46:0/3458756316 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:06.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.698+0000 7f3e99281640 1 --2- 172.21.15.46:0/3458756316 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f447a8dc50 0x55f447a5c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:06.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.698+0000 7f3e98a80640 1 --2- 172.21.15.46:0/3458756316 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f447a8e030 0x55f447abeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:06.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.698+0000 7f3e99a82640 1 -- 172.21.15.46:0/3458756316 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f447a8e030 msgr2=0x55f447abeea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:06.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.698+0000 7f3e99a82640 1 --2- 172.21.15.46:0/3458756316 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f447a8e030 0x55f447abeea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:06.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.698+0000 7f3e99a82640 1 -- 172.21.15.46:0/3458756316 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f447a8dc50 msgr2=0x55f447a5c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:06.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.698+0000 7f3e99a82640 1 --2- 172.21.15.46:0/3458756316 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f447a8dc50 0x55f447a5c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:06.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.698+0000 7f3e99a82640 1 -- 172.21.15.46:0/3458756316 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f447968d00 con 0x55f447aa9870 2023-12-02T00:29:06.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.698+0000 7f3e98a80640 1 --2- 172.21.15.46:0/3458756316 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f447a8e030 0x55f447abeea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:06.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.698+0000 7f3e99a82640 1 --2- 172.21.15.46:0/3458756316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f447aa9870 0x55f447aa8d10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3e84007a10 tx=0x7f3e84018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35f0b23ac2c870af server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:06.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.698+0000 7f3e7b7fe640 1 -- 172.21.15.46:0/3458756316 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8400e040 con 0x55f447aa9870 2023-12-02T00:29:06.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.698+0000 7f3e7b7fe640 1 -- 172.21.15.46:0/3458756316 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e84009b10 con 0x55f447aa9870 2023-12-02T00:29:06.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.698+0000 7f3e7b7fe640 1 -- 172.21.15.46:0/3458756316 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e84012ca0 con 0x55f447aa9870 2023-12-02T00:29:06.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.699+0000 7f3e9e078740 1 -- 172.21.15.46:0/3458756316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f447aa9870 msgr2=0x55f447aa8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:06.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.699+0000 7f3e9e078740 1 --2- 172.21.15.46:0/3458756316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f447aa9870 0x55f447aa8d10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3e84007a10 tx=0x7f3e84018000 comp rx=0 tx=0).stop 2023-12-02T00:29:06.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.699+0000 7f3e9e078740 1 -- 172.21.15.46:0/3458756316 shutdown_connections 2023-12-02T00:29:06.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.699+0000 7f3e9e078740 1 --2- 172.21.15.46:0/3458756316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f447aa9870 0x55f447aa8d10 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:06.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.699+0000 7f3e9e078740 1 --2- 172.21.15.46:0/3458756316 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f447a8e030 0x55f447abeea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:06.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.699+0000 7f3e9e078740 1 --2- 172.21.15.46:0/3458756316 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f447a8dc50 0x55f447a5c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:06.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.699+0000 7f3e9e078740 1 -- 172.21.15.46:0/3458756316 >> 172.21.15.46:0/3458756316 conn(0x55f447a61130 msgr2=0x55f447a66ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:06.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.699+0000 7f3e9e078740 1 -- 172.21.15.46:0/3458756316 shutdown_connections 2023-12-02T00:29:06.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.699+0000 7f3e9e078740 1 -- 172.21.15.46:0/3458756316 wait complete. 2023-12-02T00:29:06.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.699+0000 7f3e9e078740 1 -- start start 2023-12-02T00:29:06.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.699+0000 7f3e9e078740 1 Processor -- start 2023-12-02T00:29:06.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.700+0000 7f3e9e078740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f447a8dc50 0x55f447b1eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:06.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.700+0000 7f3e9e078740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f447a8e030 0x55f447aa8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:06.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.700+0000 7f3e9e078740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f447aa9870 0x55f447aa9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:06.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.700+0000 7f3e9e078740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f447a92750 con 0x55f447a8e030 2023-12-02T00:29:06.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.700+0000 7f3e9e078740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f447aae2c0 con 0x55f447aa9870 2023-12-02T00:29:06.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.700+0000 7f3e9e078740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f447a8c170 con 0x55f447a8dc50 2023-12-02T00:29:06.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.700+0000 7f3e9e078740 10 client.0 _ll_register_callbacks cb 0x7f3e880765b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-02T00:29:06.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.700+0000 7f3e98a80640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f447a8e030 0x55f447aa8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:06.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.700+0000 7f3e98a80640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f447a8e030 0x55f447aa8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38038/0 (socket says 172.21.15.46:38038) 2023-12-02T00:29:06.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.700+0000 7f3e98a80640 1 -- 172.21.15.46:0/2601045791 learned_addr learned my addr 172.21.15.46:0/2601045791 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:06.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.700+0000 7f3e99a82640 1 --2- 172.21.15.46:0/2601045791 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f447aa9870 0x55f447aa9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:06.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.700+0000 7f3e99281640 1 --2- 172.21.15.46:0/2601045791 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f447a8dc50 0x55f447b1eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:06.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.701+0000 7f3e98a80640 1 -- 172.21.15.46:0/2601045791 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f447a8dc50 msgr2=0x55f447b1eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:06.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.701+0000 7f3e98a80640 1 --2- 172.21.15.46:0/2601045791 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f447a8dc50 0x55f447b1eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:06.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.701+0000 7f3e98a80640 1 -- 172.21.15.46:0/2601045791 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f447aa9870 msgr2=0x55f447aa9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:06.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.701+0000 7f3e98a80640 1 --2- 172.21.15.46:0/2601045791 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f447aa9870 0x55f447aa9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:06.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.701+0000 7f3e98a80640 1 -- 172.21.15.46:0/2601045791 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f4479cb1f0 con 0x55f447a8e030 2023-12-02T00:29:06.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.701+0000 7f3e98a80640 1 --2- 172.21.15.46:0/2601045791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f447a8e030 0x55f447aa8d10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3e8803cfd0 tx=0x7f3e8802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88701a56cde437ff server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:06.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.701+0000 7f3e6ffff640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2023-12-02T00:29:06.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.701+0000 7f3e6ffff640 1 -- 172.21.15.46:0/2601045791 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e88082570 con 0x55f447a8e030 2023-12-02T00:29:06.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.701+0000 7f3e6ffff640 1 -- 172.21.15.46:0/2601045791 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e880a8040 con 0x55f447a8e030 2023-12-02T00:29:06.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.701+0000 7f3e9e078740 1 -- 172.21.15.46:0/2601045791 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3e8807e120 con 0x55f447a8e030 2023-12-02T00:29:06.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.701+0000 7f3e6ffff640 1 -- 172.21.15.46:0/2601045791 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e880ab0b0 con 0x55f447a8e030 2023-12-02T00:29:06.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.701+0000 7f3e9e078740 10 client.4701 fetch_fsmap learned FSMap version 6 2023-12-02T00:29:06.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.701+0000 7f3e9e078740 1 -- 172.21.15.46:0/2601045791 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x55f44798e4f0 con 0x55f447a8e030 2023-12-02T00:29:06.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.701+0000 7f3e6ffff640 1 -- 172.21.15.46:0/2601045791 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e8807e120 con 0x55f447a8e030 2023-12-02T00:29:06.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.701+0000 7f3e6ffff640 1 -- 172.21.15.46:0/2601045791 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 6) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3e88079760 con 0x55f447a8e030 2023-12-02T00:29:06.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.701+0000 7f3e9e078740 10 client.4701 fetch_fsmap finished waiting for FSMap version 6 2023-12-02T00:29:06.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.701+0000 7f3e9e078740 10 client.4701 resolve_mds: resolved name 'd' to daemon mds.d 2023-12-02T00:29:06.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.701+0000 7f3e9e078740 20 client.4701 populate_metadata read hostname 'smithi046' 2023-12-02T00:29:06.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.701+0000 7f3e9e078740 1 --2- 172.21.15.46:0/2601045791 >> [v2:172.21.15.162:6835/1231844478,v1:172.21.15.162:6837/1231844478] conn(0x55f447abf380 0x55f447a917c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:06.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.701+0000 7f3e9e078740 4 client.4701 mds_command: new command op to 4253 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-02T00:29:06.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.701+0000 7f3e9e078740 1 -- 172.21.15.46:0/2601045791 --> [v2:172.21.15.162:6835/1231844478,v1:172.21.15.162:6837/1231844478] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f4476cd6c0 con 0x55f447abf380 2023-12-02T00:29:06.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.702+0000 7f3e99281640 1 --2- 172.21.15.46:0/2601045791 >> [v2:172.21.15.162:6835/1231844478,v1:172.21.15.162:6837/1231844478] conn(0x55f447abf380 0x55f447a917c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:06.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.702+0000 7f3e99281640 1 --2- 172.21.15.46:0/2601045791 >> [v2:172.21.15.162:6835/1231844478,v1:172.21.15.162:6837/1231844478] conn(0x55f447abf380 0x55f447a917c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=464911dc35440c49 server_cookie=4962719c96993e9c in_seq=0 out_seq=0 2023-12-02T00:29:06.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.702+0000 7f3e6ffff640 10 client.4701 ms_handle_connect on v2:172.21.15.162:6835/1231844478 2023-12-02T00:29:06.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.702+0000 7f3e6ffff640 1 -- 172.21.15.46:0/2601045791 <== mds.0 v2:172.21.15.162:6835/1231844478 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55f4476cd6c0 con 0x55f447abf380 2023-12-02T00:29:06.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.702+0000 7f3e6ffff640 10 client.4701 handle_command_reply: tid=0 2023-12-02T00:29:06.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.702+0000 7f3e9e078740 1 client.4701 shutdown 2023-12-02T00:29:06.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.702+0000 7f3e9e078740 1 -- 172.21.15.46:0/2601045791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f447a8e030 msgr2=0x55f447aa8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:06.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.702+0000 7f3e9e078740 1 --2- 172.21.15.46:0/2601045791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f447a8e030 0x55f447aa8d10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3e8803cfd0 tx=0x7f3e8802b840 comp rx=0 tx=0).stop 2023-12-02T00:29:06.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.702+0000 7f3e9e078740 1 -- 172.21.15.46:0/2601045791 shutdown_connections 2023-12-02T00:29:06.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.702+0000 7f3e9e078740 1 --2- 172.21.15.46:0/2601045791 >> [v2:172.21.15.162:6835/1231844478,v1:172.21.15.162:6837/1231844478] conn(0x55f447abf380 0x55f447a917c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:06.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.702+0000 7f3e9e078740 1 --2- 172.21.15.46:0/2601045791 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f447aa9870 0x55f447aa9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:06.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.702+0000 7f3e9e078740 1 --2- 172.21.15.46:0/2601045791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f447a8e030 0x55f447aa8d10 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:06.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.702+0000 7f3e6ffff640 0 client.4701 ms_handle_reset on v2:172.21.15.162:6835/1231844478 2023-12-02T00:29:06.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.702+0000 7f3e9e078740 1 --2- 172.21.15.46:0/2601045791 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f447a8dc50 0x55f447b1eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:06.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.702+0000 7f3e9e078740 1 -- 172.21.15.46:0/2601045791 >> 172.21.15.46:0/2601045791 conn(0x55f447a61130 msgr2=0x55f447aabf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:06.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.702+0000 7f3e9e078740 1 -- 172.21.15.46:0/2601045791 shutdown_connections 2023-12-02T00:29:06.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.702+0000 7f3e9e078740 1 -- 172.21.15.46:0/2601045791 wait complete. 2023-12-02T00:29:06.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.703+0000 7f3e9e078740 20 client.4701 trim_cache size 0 max 16384 2023-12-02T00:29:06.714 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-12-02T00:29:06.714 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "6d588070-5443-40f5-81f6-108a1e3f56c3", 2023-12-02T00:29:06.715 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2023-12-02T00:29:06.715 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 4253, 2023-12-02T00:29:06.715 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2023-12-02T00:29:06.715 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2023-12-02T00:29:06.715 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2023-12-02T00:29:06.715 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 16.974933643, 2023-12-02T00:29:06.715 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 6, 2023-12-02T00:29:06.715 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 21, 2023-12-02T00:29:06.715 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 21, 2023-12-02T00:29:06.715 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 23.937199290999999 2023-12-02T00:29:06.715 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-12-02T00:29:06.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.703+0000 7f3e9b4e5640 1 -- 172.21.15.46:0/1817538399 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3e8006ae70 msgr2=0x7f3e8006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:06.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.703+0000 7f3e9b4e5640 1 --2- 172.21.15.46:0/1817538399 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3e8006ae70 0x7f3e8006d330 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c000f60 tx=0x7f3e8c007cc0 comp rx=0 tx=0).stop 2023-12-02T00:29:06.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.703+0000 7f3e9b4e5640 1 -- 172.21.15.46:0/1817538399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e940d3730 msgr2=0x7f3e940ccc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:06.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.703+0000 7f3e9b4e5640 1 --2- 172.21.15.46:0/1817538399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e940d3730 0x7f3e940ccc40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3e88034040 tx=0x7f3e880046c0 comp rx=0 tx=0).stop 2023-12-02T00:29:06.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.703+0000 7f3e9b4e5640 1 -- 172.21.15.46:0/1817538399 shutdown_connections 2023-12-02T00:29:06.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.703+0000 7f3e9b4e5640 1 --2- 172.21.15.46:0/1817538399 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3e8006ae70 0x7f3e8006d330 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:06.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.703+0000 7f3e9b4e5640 1 --2- 172.21.15.46:0/1817538399 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e9415d330 0x7f3e940b8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:06.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.703+0000 7f3e9b4e5640 1 --2- 172.21.15.46:0/1817538399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e940d3730 0x7f3e940ccc40 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:06.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.703+0000 7f3e9b4e5640 1 --2- 172.21.15.46:0/1817538399 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e940d2910 0x7f3e94153e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:06.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.703+0000 7f3e9b4e5640 1 -- 172.21.15.46:0/1817538399 >> 172.21.15.46:0/1817538399 conn(0x7f3e94072020 msgr2=0x7f3e94072400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:06.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.703+0000 7f3e9b4e5640 1 -- 172.21.15.46:0/1817538399 shutdown_connections 2023-12-02T00:29:06.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:06.703+0000 7f3e9b4e5640 1 -- 172.21.15.46:0/1817538399 wait complete. 2023-12-02T00:29:06.722 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-12-02T00:29:06.749 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-12-02T00:29:06.750 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-12-02T00:29:06.750 INFO:tasks.ceph_fuse:client.0 config is {} 2023-12-02T00:29:06.750 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-12-02T00:29:06.750 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-02T00:29:06.750 INFO:tasks.ceph_fuse:client.2 config is {} 2023-12-02T00:29:06.750 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-12-02T00:29:06.751 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-12-02T00:29:06.751 INFO:tasks.ceph_fuse:client.1 config is {} 2023-12-02T00:29:06.751 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-12-02T00:29:06.751 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-12-02T00:29:06.751 INFO:tasks.ceph_fuse:client.3 config is {} 2023-12-02T00:29:06.751 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-12-02T00:29:06.751 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-12-02T00:29:06.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:06.752 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2023-12-02T00:29:06.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:06.768 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link delete ceph-brx 2023-12-02T00:29:06.840 INFO:teuthology.orchestra.run.smithi046.stderr:Cannot find device "ceph-brx" 2023-12-02T00:29:06.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-02T00:29:06.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:06.842 DEBUG:teuthology.orchestra.run.smithi162:> ip netns list 2023-12-02T00:29:06.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:06.860 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link delete ceph-brx 2023-12-02T00:29:06.934 INFO:teuthology.orchestra.run.smithi162.stderr:Cannot find device "ceph-brx" 2023-12-02T00:29:06.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-02T00:29:06.936 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-12-02T00:29:06.936 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-02T00:29:07.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.042+0000 7f1915b53640 1 Processor -- start 2023-12-02T00:29:07.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.042+0000 7f1915b53640 1 -- start start 2023-12-02T00:29:07.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.043+0000 7f1915b53640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f191015d330 0x7f19100b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:07.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.043+0000 7f1915b53640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f19100d2910 0x7f19100d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:07.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.043+0000 7f1915b53640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19100b2b10 con 0x7f19100d3730 2023-12-02T00:29:07.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.043+0000 7f1915b53640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19100d3b50 con 0x7f191015d330 2023-12-02T00:29:07.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.043+0000 7f1915b53640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19100d1590 con 0x7f19100d2910 2023-12-02T00:29:07.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.043+0000 7f190ffff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f19100d3730 0x7f19100b6390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58324/0 (socket says 172.21.15.46:58324) 2023-12-02T00:29:07.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.043+0000 7f190ffff640 1 -- 172.21.15.46:0/2694561766 learned_addr learned my addr 172.21.15.46:0/2694561766 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:07.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.044+0000 7f190f7fe640 1 --2- 172.21.15.46:0/2694561766 >> v2:172.21.15.162:3300/0 conn(0x7f191015d330 0x7f19100b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:07.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.044+0000 7f190effd640 1 --2- 172.21.15.46:0/2694561766 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f19100d2910 0x7f19100d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:07.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.044+0000 7f190e7fc640 1 -- 172.21.15.46:0/2694561766 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3707913584 0 0) 0x7f19100b2b10 con 0x7f19100d3730 2023-12-02T00:29:07.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.044+0000 7f190e7fc640 1 -- 172.21.15.46:0/2694561766 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18f8003520 con 0x7f19100d3730 2023-12-02T00:29:07.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.044+0000 7f190e7fc640 1 -- 172.21.15.46:0/2694561766 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1132857226 0 0) 0x7f18f8003520 con 0x7f19100d3730 2023-12-02T00:29:07.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.044+0000 7f190e7fc640 1 -- 172.21.15.46:0/2694561766 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f19100d2910 msgr2=0x7f19100d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:07.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.044+0000 7f190e7fc640 1 --2- 172.21.15.46:0/2694561766 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f19100d2910 0x7f19100d31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:07.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.044+0000 7f190e7fc640 1 -- 172.21.15.46:0/2694561766 >> v2:172.21.15.162:3300/0 conn(0x7f191015d330 msgr2=0x7f19100b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:07.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.044+0000 7f190e7fc640 1 --2- 172.21.15.46:0/2694561766 >> v2:172.21.15.162:3300/0 conn(0x7f191015d330 0x7f19100b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:07.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.044+0000 7f190e7fc640 1 -- 172.21.15.46:0/2694561766 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19100d1870 con 0x7f19100d3730 2023-12-02T00:29:07.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.044+0000 7f190e7fc640 1 -- 172.21.15.46:0/2694561766 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7f19080019f0 con 0x7f19100d3730 2023-12-02T00:29:07.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.044+0000 7f190e7fc640 1 -- 172.21.15.46:0/2694561766 >> v1:172.21.15.46:6789/0 conn(0x7f19100d3730 legacy=0x7f19100b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:07.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.044+0000 7f190e7fc640 1 --2- 172.21.15.46:0/2694561766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18f8003970 0x7f18f8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:07.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.044+0000 7f190e7fc640 1 --2- 172.21.15.46:0/2694561766 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18f80047c0 0x7f18f8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:07.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.044+0000 7f190e7fc640 1 --2- 172.21.15.46:0/2694561766 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f19100d2910 0x7f18f8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:07.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.044+0000 7f190e7fc640 1 -- 172.21.15.46:0/2694561766 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19100d3b50 con 0x7f18f8003970 2023-12-02T00:29:07.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.044+0000 7f190e7fc640 1 -- 172.21.15.46:0/2694561766 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19100d1590 con 0x7f18f80047c0 2023-12-02T00:29:07.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.044+0000 7f190e7fc640 1 -- 172.21.15.46:0/2694561766 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18f800b9b0 con 0x7f19100d2910 2023-12-02T00:29:07.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.044+0000 7f190f7fe640 1 --2- 172.21.15.46:0/2694561766 >> v2:172.21.15.162:3300/0 conn(0x7f191015d330 0x7f19100b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:29:07.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.044+0000 7f190f7fe640 1 --2- 172.21.15.46:0/2694561766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18f8003970 0x7f18f8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:07.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.044+0000 7f190ffff640 1 --2- 172.21.15.46:0/2694561766 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f19100d2910 0x7f18f8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:07.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.044+0000 7f190effd640 1 --2- 172.21.15.46:0/2694561766 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18f80047c0 0x7f18f8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:07.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.044+0000 7f190f7fe640 1 -- 172.21.15.46:0/2694561766 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f19100d2910 msgr2=0x7f18f8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:07.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.044+0000 7f190f7fe640 1 --2- 172.21.15.46:0/2694561766 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f19100d2910 0x7f18f8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:07.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.044+0000 7f190f7fe640 1 -- 172.21.15.46:0/2694561766 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18f80047c0 msgr2=0x7f18f8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:07.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.044+0000 7f190f7fe640 1 --2- 172.21.15.46:0/2694561766 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18f80047c0 0x7f18f8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:07.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.044+0000 7f190f7fe640 1 -- 172.21.15.46:0/2694561766 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f8003520 con 0x7f18f8003970 2023-12-02T00:29:07.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.045+0000 7f190f7fe640 1 --2- 172.21.15.46:0/2694561766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18f8003970 0x7f18f8003d90 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1900009c70 tx=0x7f18f80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1db24346ce1c63e server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:07.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.045+0000 7f190e7fc640 1 -- 172.21.15.46:0/2694561766 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1900004020 con 0x7f18f8003970 2023-12-02T00:29:07.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.045+0000 7f190e7fc640 1 -- 172.21.15.46:0/2694561766 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1900004180 con 0x7f18f8003970 2023-12-02T00:29:07.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.045+0000 7f190e7fc640 1 -- 172.21.15.46:0/2694561766 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19000044f0 con 0x7f18f8003970 2023-12-02T00:29:07.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.046+0000 7f1915b53640 1 -- 172.21.15.46:0/2694561766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18f8003970 msgr2=0x7f18f8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:07.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.046+0000 7f1915b53640 1 --2- 172.21.15.46:0/2694561766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18f8003970 0x7f18f8003d90 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1900009c70 tx=0x7f18f80057f0 comp rx=0 tx=0).stop 2023-12-02T00:29:07.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.046+0000 7f190ffff640 1 -- 172.21.15.46:0/2694561766 reap_dead start 2023-12-02T00:29:07.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.046+0000 7f1915b53640 1 -- 172.21.15.46:0/2694561766 shutdown_connections 2023-12-02T00:29:07.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.046+0000 7f1915b53640 1 -- 172.21.15.46:0/2694561766 >> 172.21.15.46:0/2694561766 conn(0x7f1910072020 msgr2=0x7f1910072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:07.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.046+0000 7f1915b53640 1 -- 172.21.15.46:0/2694561766 shutdown_connections 2023-12-02T00:29:07.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.046+0000 7f1915b53640 1 -- 172.21.15.46:0/2694561766 wait complete. 2023-12-02T00:29:07.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.047+0000 7f1915b53640 1 Processor -- start 2023-12-02T00:29:07.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.047+0000 7f1915b53640 1 -- start start 2023-12-02T00:29:07.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.048+0000 7f1915b53640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18f8003970 0x7f1910157670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:07.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.048+0000 7f1915b53640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f19100ab670 0x7f19100abaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:07.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.048+0000 7f1915b53640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19101f1c20 0x7f19101f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:07.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.048+0000 7f1915b53640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1910157bb0 con 0x7f19101f1c20 2023-12-02T00:29:07.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.048+0000 7f1915b53640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19100ac030 con 0x7f18f8003970 2023-12-02T00:29:07.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.048+0000 7f1915b53640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19100cbda0 con 0x7f19100ab670 2023-12-02T00:29:07.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.048+0000 7f190ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19101f1c20 0x7f19101f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:07.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.048+0000 7f190ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19101f1c20 0x7f19101f20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38060/0 (socket says 172.21.15.46:38060) 2023-12-02T00:29:07.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.048+0000 7f190ffff640 1 -- 172.21.15.46:0/227701486 learned_addr learned my addr 172.21.15.46:0/227701486 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:07.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.048+0000 7f190ffff640 1 -- 172.21.15.46:0/227701486 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f19100ab670 msgr2=0x7f19100abaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:07.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.048+0000 7f190ffff640 1 --2- 172.21.15.46:0/227701486 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f19100ab670 0x7f19100abaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:07.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.048+0000 7f190ffff640 1 -- 172.21.15.46:0/227701486 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18f8003970 msgr2=0x7f1910157670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:07.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.048+0000 7f190f7fe640 1 --2- 172.21.15.46:0/227701486 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18f8003970 0x7f1910157670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:07.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.048+0000 7f190ffff640 1 --2- 172.21.15.46:0/227701486 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18f8003970 0x7f1910157670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:07.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.048+0000 7f190ffff640 1 -- 172.21.15.46:0/227701486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19100ccc40 con 0x7f19101f1c20 2023-12-02T00:29:07.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.048+0000 7f190ffff640 1 --2- 172.21.15.46:0/227701486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19101f1c20 0x7f19101f20a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f191006c530 tx=0x7f1908007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:07.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.049+0000 7f18f7fff640 1 -- 172.21.15.46:0/227701486 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190800abc0 con 0x7f19101f1c20 2023-12-02T00:29:07.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.049+0000 7f18f7fff640 1 -- 172.21.15.46:0/227701486 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f190800ad20 con 0x7f19101f1c20 2023-12-02T00:29:07.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.049+0000 7f1915b53640 1 -- 172.21.15.46:0/227701486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19100cb190 con 0x7f19101f1c20 2023-12-02T00:29:07.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.049+0000 7f18f7fff640 1 -- 172.21.15.46:0/227701486 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1908013630 con 0x7f19101f1c20 2023-12-02T00:29:07.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.049+0000 7f1915b53640 1 -- 172.21.15.46:0/227701486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19100cb750 con 0x7f19101f1c20 2023-12-02T00:29:07.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.050+0000 7f1915b53640 1 -- 172.21.15.46:0/227701486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1910069400 con 0x7f19101f1c20 2023-12-02T00:29:07.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.050+0000 7f18f7fff640 1 -- 172.21.15.46:0/227701486 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1908013790 con 0x7f19101f1c20 2023-12-02T00:29:07.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.050+0000 7f18f7fff640 1 --2- 172.21.15.46:0/227701486 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f18fc06ae40 0x7f18fc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:07.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.050+0000 7f18f7fff640 1 -- 172.21.15.46:0/227701486 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1908089d70 con 0x7f19101f1c20 2023-12-02T00:29:07.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.051+0000 7f190f7fe640 1 --2- 172.21.15.46:0/227701486 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f18fc06ae40 0x7f18fc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:07.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.051+0000 7f190f7fe640 1 --2- 172.21.15.46:0/227701486 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f18fc06ae40 0x7f18fc06d300 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1900035000 tx=0x7f1900030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:07.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.055+0000 7f18f7fff640 1 -- 172.21.15.46:0/227701486 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1908054e10 con 0x7f19101f1c20 2023-12-02T00:29:07.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.268+0000 7f1915b53640 1 -- 172.21.15.46:0/227701486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f19100b2890 con 0x7f19101f1c20 2023-12-02T00:29:07.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.269+0000 7f18f7fff640 1 -- 172.21.15.46:0/227701486 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1908058430 con 0x7f19101f1c20 2023-12-02T00:29:07.270 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-02T00:29:07.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.271+0000 7f1915b53640 1 -- 172.21.15.46:0/227701486 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f18fc06ae40 msgr2=0x7f18fc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:07.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.271+0000 7f1915b53640 1 --2- 172.21.15.46:0/227701486 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f18fc06ae40 0x7f18fc06d300 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1900035000 tx=0x7f1900030000 comp rx=0 tx=0).stop 2023-12-02T00:29:07.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.271+0000 7f1915b53640 1 -- 172.21.15.46:0/227701486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19101f1c20 msgr2=0x7f19101f20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:07.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.271+0000 7f1915b53640 1 --2- 172.21.15.46:0/227701486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19101f1c20 0x7f19101f20a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f191006c530 tx=0x7f1908007bd0 comp rx=0 tx=0).stop 2023-12-02T00:29:07.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.271+0000 7f1915b53640 1 -- 172.21.15.46:0/227701486 shutdown_connections 2023-12-02T00:29:07.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.271+0000 7f1915b53640 1 --2- 172.21.15.46:0/227701486 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f18fc06ae40 0x7f18fc06d300 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:07.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.271+0000 7f1915b53640 1 --2- 172.21.15.46:0/227701486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19101f1c20 0x7f19101f20a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:07.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.271+0000 7f1915b53640 1 --2- 172.21.15.46:0/227701486 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f19100ab670 0x7f19100abaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:07.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.271+0000 7f1915b53640 1 --2- 172.21.15.46:0/227701486 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18f8003970 0x7f1910157670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:07.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.271+0000 7f1915b53640 1 -- 172.21.15.46:0/227701486 >> 172.21.15.46:0/227701486 conn(0x7f1910072020 msgr2=0x7f19100cf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:07.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.271+0000 7f1915b53640 1 -- 172.21.15.46:0/227701486 shutdown_connections 2023-12-02T00:29:07.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:07.271+0000 7f1915b53640 1 -- 172.21.15.46:0/227701486 wait complete. 2023-12-02T00:29:07.283 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-02T00:29:07.283 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-02T00:29:07.283 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2023-12-02T00:29:07.283 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-02T00:29:07.283 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-02T00:29:07.283 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-02T00:29:07.283 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-02T00:29:07.283 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-02T00:29:07.283 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-02T00:29:07.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:07.283 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2023-12-02T00:29:07.299 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-02T00:29:07.299 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-02T00:29:07.299 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2023-12-02T00:29:07.299 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:29:07.299 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2023-12-02T00:29:07.299 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:29:07.299 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-02T00:29:07.300 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2023-12-02T00:29:07.300 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2023-12-02T00:29:07.300 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-02T00:29:07.300 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2023-12-02T00:29:07.300 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2023-12-02T00:29:07.300 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-02T00:29:07.300 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2023-12-02T00:29:07.300 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2023-12-02T00:29:07.300 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-12-02T00:29:07.300 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42632sec preferred_lft 42632sec 2023-12-02T00:29:07.300 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link noprefixroute 2023-12-02T00:29:07.300 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:29:07.300 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-02T00:29:07.300 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2023-12-02T00:29:07.301 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2023-12-02T00:29:07.301 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-12-02T00:29:07.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:07.301 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2023-12-02T00:29:07.301 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2023-12-02T00:29:07.301 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link add name ceph-brx type bridge 2023-12-02T00:29:07.301 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip addr flush dev ceph-brx 2023-12-02T00:29:07.301 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set ceph-brx up 2023-12-02T00:29:07.301 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-12-02T00:29:07.302 DEBUG:teuthology.orchestra.run.smithi046:> ') 2023-12-02T00:29:07.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:07.463 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-02T00:29:07.495 INFO:teuthology.orchestra.run.smithi046.stdout:1 2023-12-02T00:29:07.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:07.497 DEBUG:teuthology.orchestra.run.smithi046:> route 2023-12-02T00:29:07.571 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2023-12-02T00:29:07.571 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-02T00:29:07.571 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-12-02T00:29:07.571 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-12-02T00:29:07.571 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-02T00:29:07.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:07.572 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2023-12-02T00:29:07.572 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2023-12-02T00:29:07.572 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-02T00:29:07.572 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-02T00:29:07.573 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-02T00:29:07.573 DEBUG:teuthology.orchestra.run.smithi046:> ') 2023-12-02T00:29:07.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:07.837 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2023-12-02T00:29:07.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:07.857 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list-id 2023-12-02T00:29:07.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:07.912 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2023-12-02T00:29:07.912 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2023-12-02T00:29:07.912 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-02T00:29:07.912 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-12-02T00:29:07.912 DEBUG:teuthology.orchestra.run.smithi046:> ') 2023-12-02T00:29:08.005 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-12-02T00:29:08.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:08.005 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2023-12-02T00:29:08.005 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2023-12-02T00:29:08.005 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-12-02T00:29:08.005 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-12-02T00:29:08.005 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-12-02T00:29:08.005 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-12-02T00:29:08.006 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-12-02T00:29:08.006 DEBUG:teuthology.orchestra.run.smithi046:> ') 2023-12-02T00:29:08.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:08.120 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2023-12-02T00:29:08.120 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2023-12-02T00:29:08.120 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2023-12-02T00:29:08.120 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set dev brx.0 master ceph-brx 2023-12-02T00:29:08.120 DEBUG:teuthology.orchestra.run.smithi046:> ') 2023-12-02T00:29:08.169 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-12-02T00:29:08.169 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-02T00:29:08.169 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-02T00:29:08.186 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-02T00:29:08.186 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-02T00:29:08.186 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-02T00:29:08.241 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2023-12-02T00:29:08.313 DEBUG:teuthology.orchestra.run.smithi046:> cat /proc/self/mounts | awk '{print $2}' 2023-12-02T00:29:08.370 INFO:teuthology.orchestra.run.smithi046.stdout:/proc 2023-12-02T00:29:08.370 INFO:teuthology.orchestra.run.smithi046.stdout:/sys 2023-12-02T00:29:08.370 INFO:teuthology.orchestra.run.smithi046.stdout:/dev 2023-12-02T00:29:08.370 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/security 2023-12-02T00:29:08.370 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/shm 2023-12-02T00:29:08.370 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/pts 2023-12-02T00:29:08.371 INFO:teuthology.orchestra.run.smithi046.stdout:/run 2023-12-02T00:29:08.371 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/cgroup 2023-12-02T00:29:08.371 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/pstore 2023-12-02T00:29:08.371 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/bpf 2023-12-02T00:29:08.371 INFO:teuthology.orchestra.run.smithi046.stdout:/ 2023-12-02T00:29:08.371 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/selinux 2023-12-02T00:29:08.371 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2023-12-02T00:29:08.371 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/mqueue 2023-12-02T00:29:08.371 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/hugepages 2023-12-02T00:29:08.371 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/debug 2023-12-02T00:29:08.371 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/tracing 2023-12-02T00:29:08.371 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/fuse/connections 2023-12-02T00:29:08.371 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/config 2023-12-02T00:29:08.371 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-sysctl.service 2023-12-02T00:29:08.371 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-02T00:29:08.372 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-02T00:29:08.372 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/nfs/rpc_pipefs 2023-12-02T00:29:08.372 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/1000 2023-12-02T00:29:08.372 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph 2023-12-02T00:29:08.372 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-0 2023-12-02T00:29:08.372 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-1 2023-12-02T00:29:08.372 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-2 2023-12-02T00:29:08.372 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-3 2023-12-02T00:29:08.372 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns 2023-12-02T00:29:08.372 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-02T00:29:08.372 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-02T00:29:08.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:08.373 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2023-12-02T00:29:08.426 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-12-02T00:29:08.426 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-12-02T00:29:08.469 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2023-12-02T00:29:08.499 DEBUG:teuthology.orchestra.run.smithi046:> cat /proc/self/mounts | awk '{print $2}' 2023-12-02T00:29:08.557 INFO:teuthology.orchestra.run.smithi046.stdout:/proc 2023-12-02T00:29:08.557 INFO:teuthology.orchestra.run.smithi046.stdout:/sys 2023-12-02T00:29:08.557 INFO:teuthology.orchestra.run.smithi046.stdout:/dev 2023-12-02T00:29:08.558 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/security 2023-12-02T00:29:08.558 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/shm 2023-12-02T00:29:08.558 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/pts 2023-12-02T00:29:08.558 INFO:teuthology.orchestra.run.smithi046.stdout:/run 2023-12-02T00:29:08.558 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/cgroup 2023-12-02T00:29:08.559 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/pstore 2023-12-02T00:29:08.559 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/bpf 2023-12-02T00:29:08.559 INFO:teuthology.orchestra.run.smithi046.stdout:/ 2023-12-02T00:29:08.559 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/selinux 2023-12-02T00:29:08.559 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2023-12-02T00:29:08.559 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/mqueue 2023-12-02T00:29:08.559 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/hugepages 2023-12-02T00:29:08.559 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/debug 2023-12-02T00:29:08.559 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/tracing 2023-12-02T00:29:08.559 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/fuse/connections 2023-12-02T00:29:08.559 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/config 2023-12-02T00:29:08.560 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-sysctl.service 2023-12-02T00:29:08.560 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-02T00:29:08.560 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-02T00:29:08.560 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/nfs/rpc_pipefs 2023-12-02T00:29:08.560 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/1000 2023-12-02T00:29:08.560 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph 2023-12-02T00:29:08.560 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-0 2023-12-02T00:29:08.560 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-1 2023-12-02T00:29:08.560 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-2 2023-12-02T00:29:08.561 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-3 2023-12-02T00:29:08.561 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns 2023-12-02T00:29:08.561 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-02T00:29:08.561 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-02T00:29:08.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:08.562 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2023-12-02T00:29:08.574 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:2023-12-02T00:29:08.571+0000 7fb593f59480 -1 init, newargv = 0x55f738ecf1a0 newargc=15 2023-12-02T00:29:08.575 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[98651]: starting ceph client 2023-12-02T00:29:08.578 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[98651]: starting fuse 2023-12-02T00:29:08.590 INFO:teuthology.orchestra.run.smithi046.stdout:41 2023-12-02T00:29:08.590 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-12-02T00:29:08.590 DEBUG:teuthology.orchestra.run.smithi046:> sudo stdin-killer -- python3 -c ' 2023-12-02T00:29:08.590 DEBUG:teuthology.orchestra.run.smithi046:> import glob 2023-12-02T00:29:08.591 DEBUG:teuthology.orchestra.run.smithi046:> import re 2023-12-02T00:29:08.591 DEBUG:teuthology.orchestra.run.smithi046:> import os 2023-12-02T00:29:08.591 DEBUG:teuthology.orchestra.run.smithi046:> import subprocess 2023-12-02T00:29:08.591 DEBUG:teuthology.orchestra.run.smithi046:> 2023-12-02T00:29:08.591 DEBUG:teuthology.orchestra.run.smithi046:> def _find_admin_socket(client_name): 2023-12-02T00:29:08.591 DEBUG:teuthology.orchestra.run.smithi046:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-12-02T00:29:08.591 DEBUG:teuthology.orchestra.run.smithi046:> files = glob.glob(asok_path) 2023-12-02T00:29:08.591 DEBUG:teuthology.orchestra.run.smithi046:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-12-02T00:29:08.591 DEBUG:teuthology.orchestra.run.smithi046:> 2023-12-02T00:29:08.591 DEBUG:teuthology.orchestra.run.smithi046:> # Given a non-glob path, it better be there 2023-12-02T00:29:08.591 DEBUG:teuthology.orchestra.run.smithi046:> if "*" not in asok_path: 2023-12-02T00:29:08.591 DEBUG:teuthology.orchestra.run.smithi046:> assert(len(files) == 1) 2023-12-02T00:29:08.591 DEBUG:teuthology.orchestra.run.smithi046:> return files[0] 2023-12-02T00:29:08.592 DEBUG:teuthology.orchestra.run.smithi046:> 2023-12-02T00:29:08.592 DEBUG:teuthology.orchestra.run.smithi046:> for f in files: 2023-12-02T00:29:08.592 DEBUG:teuthology.orchestra.run.smithi046:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-02T00:29:08.592 DEBUG:teuthology.orchestra.run.smithi046:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-02T00:29:08.592 DEBUG:teuthology.orchestra.run.smithi046:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-02T00:29:08.592 DEBUG:teuthology.orchestra.run.smithi046:> contents = proc_f.read() 2023-12-02T00:29:08.592 DEBUG:teuthology.orchestra.run.smithi046:> if mountpoint in contents: 2023-12-02T00:29:08.592 DEBUG:teuthology.orchestra.run.smithi046:> return f 2023-12-02T00:29:08.592 DEBUG:teuthology.orchestra.run.smithi046:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-02T00:29:08.592 DEBUG:teuthology.orchestra.run.smithi046:> 2023-12-02T00:29:08.592 DEBUG:teuthology.orchestra.run.smithi046:> print(_find_admin_socket("client.0")) 2023-12-02T00:29:08.592 DEBUG:teuthology.orchestra.run.smithi046:> ' 2023-12-02T00:29:08.715 INFO:teuthology.orchestra.run.smithi046.stdout:/var/run/ceph/ceph-client.0.98651.asok 2023-12-02T00:29:08.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-12-02T00:29:08.725 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.98651.asok 2023-12-02T00:29:08.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:08.725 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.98651.asok status 2023-12-02T00:29:08.867 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-12-02T00:29:08.867 INFO:teuthology.orchestra.run.smithi046.stdout: "metadata": { 2023-12-02T00:29:08.867 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_sha1": "bed7681480ccb0192b91a35e691658867e1e96b7", 2023-12-02T00:29:08.867 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_version": "ceph version 18.0.0-7631-gbed76814 (bed7681480ccb0192b91a35e691658867e1e96b7) reef (dev)", 2023-12-02T00:29:08.867 INFO:teuthology.orchestra.run.smithi046.stdout: "entity_id": "0", 2023-12-02T00:29:08.867 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": "smithi046", 2023-12-02T00:29:08.867 INFO:teuthology.orchestra.run.smithi046.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-12-02T00:29:08.867 INFO:teuthology.orchestra.run.smithi046.stdout: "pid": "98651", 2023-12-02T00:29:08.867 INFO:teuthology.orchestra.run.smithi046.stdout: "root": "/" 2023-12-02T00:29:08.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:29:08.868 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_count": 0, 2023-12-02T00:29:08.868 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_pinned_count": 0, 2023-12-02T00:29:08.868 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 4716, 2023-12-02T00:29:08.868 INFO:teuthology.orchestra.run.smithi046.stdout: "inst": { 2023-12-02T00:29:08.868 INFO:teuthology.orchestra.run.smithi046.stdout: "name": { 2023-12-02T00:29:08.868 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "client", 2023-12-02T00:29:08.868 INFO:teuthology.orchestra.run.smithi046.stdout: "num": 4716 2023-12-02T00:29:08.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:29:08.868 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2023-12-02T00:29:08.868 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2023-12-02T00:29:08.868 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2023-12-02T00:29:08.868 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2062147779 2023-12-02T00:29:08.868 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:29:08.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:29:08.869 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2023-12-02T00:29:08.869 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2023-12-02T00:29:08.869 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2023-12-02T00:29:08.869 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2062147779 2023-12-02T00:29:08.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:29:08.869 INFO:teuthology.orchestra.run.smithi046.stdout: "inst_str": "client.4716 192.168.0.1:0/2062147779", 2023-12-02T00:29:08.869 INFO:teuthology.orchestra.run.smithi046.stdout: "addr_str": "192.168.0.1:0/2062147779", 2023-12-02T00:29:08.869 INFO:teuthology.orchestra.run.smithi046.stdout: "inode_count": 1, 2023-12-02T00:29:08.869 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_epoch": 6, 2023-12-02T00:29:08.869 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch": 21, 2023-12-02T00:29:08.869 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch_barrier": 0, 2023-12-02T00:29:08.869 INFO:teuthology.orchestra.run.smithi046.stdout: "blocklisted": false, 2023-12-02T00:29:08.869 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs" 2023-12-02T00:29:08.869 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-12-02T00:29:08.876 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-02T00:29:08.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.982+0000 7f4e97b00640 1 Processor -- start 2023-12-02T00:29:08.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.982+0000 7f4e97b00640 1 -- start start 2023-12-02T00:29:08.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.983+0000 7f4e97b00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4e9006ea50 0x7f4e9006ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:08.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.983+0000 7f4e97b00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4e90160fb0 0x7f4e90161440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:08.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.983+0000 7f4e97b00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4e900b3ff0 0x7f4e900b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:08.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.983+0000 7f4e97b00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e90161d20 con 0x7f4e9006ea50 2023-12-02T00:29:08.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.983+0000 7f4e97b00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e9006f370 con 0x7f4e90160fb0 2023-12-02T00:29:08.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.983+0000 7f4e97b00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9015e760 con 0x7f4e900b3ff0 2023-12-02T00:29:08.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.983+0000 7f4e9589c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4e9006ea50 0x7f4e9006ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:08.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.983+0000 7f4e9589c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4e9006ea50 0x7f4e9006ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38068/0 (socket says 172.21.15.46:38068) 2023-12-02T00:29:08.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.983+0000 7f4e9589c640 1 -- 172.21.15.46:0/697361894 learned_addr learned my addr 172.21.15.46:0/697361894 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:08.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.983+0000 7f4e9509b640 1 --2- 172.21.15.46:0/697361894 >> v2:172.21.15.162:3300/0 conn(0x7f4e90160fb0 0x7f4e90161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:08.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.983+0000 7f4e9609d640 1 --2- 172.21.15.46:0/697361894 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4e900b3ff0 0x7f4e900b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:08.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.983+0000 7f4e9589c640 1 -- 172.21.15.46:0/697361894 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4e900b3ff0 msgr2=0x7f4e900b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:08.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.983+0000 7f4e9589c640 1 --2- 172.21.15.46:0/697361894 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4e900b3ff0 0x7f4e900b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:08.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.983+0000 7f4e9589c640 1 -- 172.21.15.46:0/697361894 >> v2:172.21.15.162:3300/0 conn(0x7f4e90160fb0 msgr2=0x7f4e90161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:08.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.983+0000 7f4e9589c640 1 --2- 172.21.15.46:0/697361894 >> v2:172.21.15.162:3300/0 conn(0x7f4e90160fb0 0x7f4e90161440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:08.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.983+0000 7f4e9589c640 1 -- 172.21.15.46:0/697361894 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e9015d9a0 con 0x7f4e9006ea50 2023-12-02T00:29:08.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.984+0000 7f4e9589c640 1 --2- 172.21.15.46:0/697361894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e9006ea50 0x7f4e9006ee30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4e900b5b30 tx=0x7f4e8c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f03eb73689e5a1d2 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:08.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.984+0000 7f4e9489a640 1 -- 172.21.15.46:0/697361894 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c0317e0 con 0x7f4e9006ea50 2023-12-02T00:29:08.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.984+0000 7f4e9489a640 1 -- 172.21.15.46:0/697361894 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e8c031940 con 0x7f4e9006ea50 2023-12-02T00:29:08.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.984+0000 7f4e9489a640 1 -- 172.21.15.46:0/697361894 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c031cd0 con 0x7f4e9006ea50 2023-12-02T00:29:08.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.984+0000 7f4e97b00640 1 -- 172.21.15.46:0/697361894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e9006ea50 msgr2=0x7f4e9006ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:08.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.984+0000 7f4e97b00640 1 --2- 172.21.15.46:0/697361894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e9006ea50 0x7f4e9006ee30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4e900b5b30 tx=0x7f4e8c02b880 comp rx=0 tx=0).stop 2023-12-02T00:29:08.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.985+0000 7f4e97b00640 1 -- 172.21.15.46:0/697361894 shutdown_connections 2023-12-02T00:29:08.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.985+0000 7f4e97b00640 1 --2- 172.21.15.46:0/697361894 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4e900b3ff0 0x7f4e900b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:08.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.985+0000 7f4e97b00640 1 --2- 172.21.15.46:0/697361894 >> v2:172.21.15.162:3300/0 conn(0x7f4e90160fb0 0x7f4e90161440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:08.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.985+0000 7f4e97b00640 1 --2- 172.21.15.46:0/697361894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e9006ea50 0x7f4e9006ee30 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:08.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.985+0000 7f4e97b00640 1 -- 172.21.15.46:0/697361894 >> 172.21.15.46:0/697361894 conn(0x7f4e9006a6b0 msgr2=0x7f4e9006aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:08.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.985+0000 7f4e97b00640 1 -- 172.21.15.46:0/697361894 shutdown_connections 2023-12-02T00:29:08.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.985+0000 7f4e97b00640 1 -- 172.21.15.46:0/697361894 wait complete. 2023-12-02T00:29:08.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.986+0000 7f4e97b00640 1 Processor -- start 2023-12-02T00:29:08.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.986+0000 7f4e97b00640 1 -- start start 2023-12-02T00:29:08.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.986+0000 7f4e97b00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4e9006ea50 0x7f4e90156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:08.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.986+0000 7f4e97b00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e900b3ff0 0x7f4e901f40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:08.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.986+0000 7f4e97b00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e90160fb0 0x7f4e90158db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:08.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.986+0000 7f4e97b00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e901596c0 con 0x7f4e90160fb0 2023-12-02T00:29:08.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.986+0000 7f4e97b00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9006b760 con 0x7f4e900b3ff0 2023-12-02T00:29:08.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.986+0000 7f4e97b00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9006ba40 con 0x7f4e9006ea50 2023-12-02T00:29:08.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.987+0000 7f4e9609d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e90160fb0 0x7f4e90158db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:08.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.987+0000 7f4e9609d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e90160fb0 0x7f4e90158db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38084/0 (socket says 172.21.15.46:38084) 2023-12-02T00:29:08.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.987+0000 7f4e9609d640 1 -- 172.21.15.46:0/51398098 learned_addr learned my addr 172.21.15.46:0/51398098 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:08.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.987+0000 7f4e9509b640 1 --2- 172.21.15.46:0/51398098 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e900b3ff0 0x7f4e901f40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:08.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.987+0000 7f4e9589c640 1 --2- 172.21.15.46:0/51398098 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4e9006ea50 0x7f4e90156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:08.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.987+0000 7f4e9609d640 1 -- 172.21.15.46:0/51398098 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4e9006ea50 msgr2=0x7f4e90156950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:08.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.987+0000 7f4e9609d640 1 --2- 172.21.15.46:0/51398098 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4e9006ea50 0x7f4e90156950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:08.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.987+0000 7f4e9609d640 1 -- 172.21.15.46:0/51398098 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e900b3ff0 msgr2=0x7f4e901f40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:08.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.987+0000 7f4e9609d640 1 --2- 172.21.15.46:0/51398098 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e900b3ff0 0x7f4e901f40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:08.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.987+0000 7f4e9609d640 1 -- 172.21.15.46:0/51398098 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e8c0094d0 con 0x7f4e90160fb0 2023-12-02T00:29:08.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.987+0000 7f4e9589c640 1 --2- 172.21.15.46:0/51398098 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4e9006ea50 0x7f4e90156950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:29:08.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.987+0000 7f4e9609d640 1 --2- 172.21.15.46:0/51398098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e90160fb0 0x7f4e90158db0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4e88007d90 tx=0x7f4e8800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:08.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.987+0000 7f4e867fc640 1 -- 172.21.15.46:0/51398098 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e88014420 con 0x7f4e90160fb0 2023-12-02T00:29:08.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.987+0000 7f4e867fc640 1 -- 172.21.15.46:0/51398098 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e880145a0 con 0x7f4e90160fb0 2023-12-02T00:29:08.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.987+0000 7f4e97b00640 1 -- 172.21.15.46:0/51398098 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e9006bd20 con 0x7f4e90160fb0 2023-12-02T00:29:08.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.987+0000 7f4e867fc640 1 -- 172.21.15.46:0/51398098 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e88014950 con 0x7f4e90160fb0 2023-12-02T00:29:08.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.987+0000 7f4e97b00640 1 -- 172.21.15.46:0/51398098 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e901f4630 con 0x7f4e90160fb0 2023-12-02T00:29:08.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.988+0000 7f4e97b00640 1 -- 172.21.15.46:0/51398098 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e9006c240 con 0x7f4e90160fb0 2023-12-02T00:29:08.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.989+0000 7f4e867fc640 1 -- 172.21.15.46:0/51398098 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4e88014ab0 con 0x7f4e90160fb0 2023-12-02T00:29:08.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.989+0000 7f4e867fc640 1 --2- 172.21.15.46:0/51398098 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f4e7406ae40 0x7f4e7406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:08.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.989+0000 7f4e867fc640 1 -- 172.21.15.46:0/51398098 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4e88089f60 con 0x7f4e90160fb0 2023-12-02T00:29:08.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.989+0000 7f4e9589c640 1 --2- 172.21.15.46:0/51398098 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f4e7406ae40 0x7f4e7406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:08.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.990+0000 7f4e9589c640 1 --2- 172.21.15.46:0/51398098 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f4e7406ae40 0x7f4e7406d300 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c036000 tx=0x7f4e8c03d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:08.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:08.993+0000 7f4e867fc640 1 -- 172.21.15.46:0/51398098 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4e9006c240 con 0x7f4e90160fb0 2023-12-02T00:29:09.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:09.207+0000 7f4e97b00640 1 -- 172.21.15.46:0/51398098 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4e9006c240 con 0x7f4e90160fb0 2023-12-02T00:29:09.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:09.208+0000 7f4e867fc640 1 -- 172.21.15.46:0/51398098 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4e9006c240 con 0x7f4e90160fb0 2023-12-02T00:29:09.209 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-02T00:29:09.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:09.210+0000 7f4e97b00640 1 -- 172.21.15.46:0/51398098 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f4e7406ae40 msgr2=0x7f4e7406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:09.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:09.210+0000 7f4e97b00640 1 --2- 172.21.15.46:0/51398098 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f4e7406ae40 0x7f4e7406d300 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c036000 tx=0x7f4e8c03d040 comp rx=0 tx=0).stop 2023-12-02T00:29:09.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:09.210+0000 7f4e97b00640 1 -- 172.21.15.46:0/51398098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e90160fb0 msgr2=0x7f4e90158db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:09.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:09.210+0000 7f4e97b00640 1 --2- 172.21.15.46:0/51398098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e90160fb0 0x7f4e90158db0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4e88007d90 tx=0x7f4e8800e4b0 comp rx=0 tx=0).stop 2023-12-02T00:29:09.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:09.210+0000 7f4e97b00640 1 -- 172.21.15.46:0/51398098 shutdown_connections 2023-12-02T00:29:09.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:09.210+0000 7f4e97b00640 1 --2- 172.21.15.46:0/51398098 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f4e7406ae40 0x7f4e7406d300 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:09.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:09.210+0000 7f4e97b00640 1 --2- 172.21.15.46:0/51398098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e90160fb0 0x7f4e90158db0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:09.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:09.210+0000 7f4e97b00640 1 --2- 172.21.15.46:0/51398098 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e900b3ff0 0x7f4e901f40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:09.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:09.210+0000 7f4e97b00640 1 --2- 172.21.15.46:0/51398098 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4e9006ea50 0x7f4e90156950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:09.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:09.210+0000 7f4e97b00640 1 -- 172.21.15.46:0/51398098 >> 172.21.15.46:0/51398098 conn(0x7f4e9006a6b0 msgr2=0x7f4e9015c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:09.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:09.210+0000 7f4e97b00640 1 -- 172.21.15.46:0/51398098 shutdown_connections 2023-12-02T00:29:09.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:09.210+0000 7f4e97b00640 1 -- 172.21.15.46:0/51398098 wait complete. 2023-12-02T00:29:09.222 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-02T00:29:09.222 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-02T00:29:09.222 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2023-12-02T00:29:09.222 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-12-02T00:29:09.222 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-12-02T00:29:09.222 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-02T00:29:09.222 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-02T00:29:09.222 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-02T00:29:09.223 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-12-02T00:29:09.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:09.223 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2023-12-02T00:29:09.239 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-02T00:29:09.239 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-02T00:29:09.239 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2023-12-02T00:29:09.239 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:29:09.239 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2023-12-02T00:29:09.239 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:29:09.239 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-02T00:29:09.239 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2023-12-02T00:29:09.239 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2023-12-02T00:29:09.239 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-02T00:29:09.239 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2023-12-02T00:29:09.240 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2023-12-02T00:29:09.240 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-02T00:29:09.240 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2023-12-02T00:29:09.240 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2023-12-02T00:29:09.240 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-12-02T00:29:09.240 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42630sec preferred_lft 42630sec 2023-12-02T00:29:09.240 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link noprefixroute 2023-12-02T00:29:09.240 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:29:09.240 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-02T00:29:09.240 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2023-12-02T00:29:09.240 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2023-12-02T00:29:09.240 INFO:teuthology.orchestra.run.smithi046.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-02T00:29:09.241 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether b6:1a:8b:80:15:f1 brd ff:ff:ff:ff:ff:ff 2023-12-02T00:29:09.241 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-02T00:29:09.241 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:29:09.241 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::eca8:9ff:feb3:1726/64 scope link tentative 2023-12-02T00:29:09.241 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:29:09.241 INFO:teuthology.orchestra.run.smithi046.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-02T00:29:09.241 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether b6:1a:8b:80:15:f1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-02T00:29:09.241 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::b41a:8bff:fe80:15f1/64 scope link tentative 2023-12-02T00:29:09.241 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:29:09.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:09.242 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-02T00:29:09.310 INFO:teuthology.orchestra.run.smithi046.stdout:1 2023-12-02T00:29:09.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:09.311 DEBUG:teuthology.orchestra.run.smithi046:> route 2023-12-02T00:29:09.369 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2023-12-02T00:29:09.369 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-02T00:29:09.369 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-12-02T00:29:09.369 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-12-02T00:29:09.369 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-02T00:29:09.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:09.370 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2023-12-02T00:29:09.370 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2023-12-02T00:29:09.370 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-02T00:29:09.370 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-02T00:29:09.370 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-02T00:29:09.370 DEBUG:teuthology.orchestra.run.smithi046:> ') 2023-12-02T00:29:09.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:09.479 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2023-12-02T00:29:09.494 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-02T00:29:09.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:09.495 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list-id 2023-12-02T00:29:09.551 INFO:teuthology.orchestra.run.smithi046.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-12-02T00:29:09.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:09.551 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2023-12-02T00:29:09.551 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2023-12-02T00:29:09.551 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-02T00:29:09.551 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-12-02T00:29:09.551 DEBUG:teuthology.orchestra.run.smithi046:> ') 2023-12-02T00:29:09.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:09.643 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-12-02T00:29:09.674 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-02T00:29:09.674 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-02T00:29:09.674 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2023-12-02T00:29:09.674 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:29:09.674 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2023-12-02T00:29:09.674 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:29:09.674 INFO:teuthology.orchestra.run.smithi046.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-02T00:29:09.674 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 9e:00:a5:30:00:a4 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-12-02T00:29:09.674 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-12-02T00:29:09.675 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:29:09.675 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::9c00:a5ff:fe30:a4/64 scope link tentative 2023-12-02T00:29:09.675 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:29:09.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:09.677 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-12-02T00:29:09.746 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-02T00:29:09.747 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-02T00:29:09.747 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2023-12-02T00:29:09.747 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:29:09.747 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2023-12-02T00:29:09.747 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:29:09.747 INFO:teuthology.orchestra.run.smithi046.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-02T00:29:09.747 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 9e:00:a5:30:00:a4 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-12-02T00:29:09.747 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-12-02T00:29:09.747 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:29:09.747 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::9c00:a5ff:fe30:a4/64 scope link tentative 2023-12-02T00:29:09.747 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:29:09.748 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-12-02T00:29:09.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:09.749 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2023-12-02T00:29:09.749 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2023-12-02T00:29:09.749 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-12-02T00:29:09.749 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-12-02T00:29:09.749 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-12-02T00:29:09.749 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-12-02T00:29:09.749 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-12-02T00:29:09.749 DEBUG:teuthology.orchestra.run.smithi046:> ') 2023-12-02T00:29:09.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:09.893 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2023-12-02T00:29:09.893 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2023-12-02T00:29:09.893 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.1 up 2023-12-02T00:29:09.893 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set dev brx.1 master ceph-brx 2023-12-02T00:29:09.893 DEBUG:teuthology.orchestra.run.smithi046:> ') 2023-12-02T00:29:09.942 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2023-12-02T00:29:09.942 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-02T00:29:09.942 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-12-02T00:29:09.957 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-12-02T00:29:09.957 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-02T00:29:09.957 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-12-02T00:29:10.012 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2023-12-02T00:29:10.082 DEBUG:teuthology.orchestra.run.smithi046:> cat /proc/self/mounts | awk '{print $2}' 2023-12-02T00:29:10.139 INFO:teuthology.orchestra.run.smithi046.stdout:/proc 2023-12-02T00:29:10.139 INFO:teuthology.orchestra.run.smithi046.stdout:/sys 2023-12-02T00:29:10.139 INFO:teuthology.orchestra.run.smithi046.stdout:/dev 2023-12-02T00:29:10.139 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/security 2023-12-02T00:29:10.139 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/shm 2023-12-02T00:29:10.140 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/pts 2023-12-02T00:29:10.140 INFO:teuthology.orchestra.run.smithi046.stdout:/run 2023-12-02T00:29:10.140 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/cgroup 2023-12-02T00:29:10.140 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/pstore 2023-12-02T00:29:10.140 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/bpf 2023-12-02T00:29:10.140 INFO:teuthology.orchestra.run.smithi046.stdout:/ 2023-12-02T00:29:10.140 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/selinux 2023-12-02T00:29:10.140 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2023-12-02T00:29:10.140 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/mqueue 2023-12-02T00:29:10.140 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/hugepages 2023-12-02T00:29:10.140 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/debug 2023-12-02T00:29:10.140 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/tracing 2023-12-02T00:29:10.140 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/fuse/connections 2023-12-02T00:29:10.140 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/config 2023-12-02T00:29:10.140 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-sysctl.service 2023-12-02T00:29:10.141 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-02T00:29:10.141 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-02T00:29:10.141 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/nfs/rpc_pipefs 2023-12-02T00:29:10.141 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/1000 2023-12-02T00:29:10.141 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph 2023-12-02T00:29:10.141 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-0 2023-12-02T00:29:10.141 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-1 2023-12-02T00:29:10.141 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-2 2023-12-02T00:29:10.141 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-3 2023-12-02T00:29:10.141 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns 2023-12-02T00:29:10.141 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-02T00:29:10.141 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-02T00:29:10.141 INFO:teuthology.orchestra.run.smithi046.stdout:/home/ubuntu/cephtest/mnt.0 2023-12-02T00:29:10.141 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-02T00:29:10.142 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-02T00:29:10.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:10.142 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2023-12-02T00:29:10.197 INFO:teuthology.orchestra.run.smithi046.stdout:41 2023-12-02T00:29:10.197 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [41] 2023-12-02T00:29:10.197 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2023-12-02T00:29:10.239 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2023-12-02T00:29:10.269 DEBUG:teuthology.orchestra.run.smithi046:> cat /proc/self/mounts | awk '{print $2}' 2023-12-02T00:29:10.326 INFO:teuthology.orchestra.run.smithi046.stdout:/proc 2023-12-02T00:29:10.326 INFO:teuthology.orchestra.run.smithi046.stdout:/sys 2023-12-02T00:29:10.326 INFO:teuthology.orchestra.run.smithi046.stdout:/dev 2023-12-02T00:29:10.326 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/security 2023-12-02T00:29:10.326 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/shm 2023-12-02T00:29:10.326 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/pts 2023-12-02T00:29:10.327 INFO:teuthology.orchestra.run.smithi046.stdout:/run 2023-12-02T00:29:10.327 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/cgroup 2023-12-02T00:29:10.327 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/pstore 2023-12-02T00:29:10.327 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/bpf 2023-12-02T00:29:10.327 INFO:teuthology.orchestra.run.smithi046.stdout:/ 2023-12-02T00:29:10.327 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/selinux 2023-12-02T00:29:10.327 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2023-12-02T00:29:10.327 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/mqueue 2023-12-02T00:29:10.327 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/hugepages 2023-12-02T00:29:10.327 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/debug 2023-12-02T00:29:10.327 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/tracing 2023-12-02T00:29:10.327 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/fuse/connections 2023-12-02T00:29:10.327 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/config 2023-12-02T00:29:10.327 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-sysctl.service 2023-12-02T00:29:10.327 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-02T00:29:10.328 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-02T00:29:10.328 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/nfs/rpc_pipefs 2023-12-02T00:29:10.328 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/1000 2023-12-02T00:29:10.328 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph 2023-12-02T00:29:10.328 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-0 2023-12-02T00:29:10.328 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-1 2023-12-02T00:29:10.328 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-2 2023-12-02T00:29:10.328 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-3 2023-12-02T00:29:10.328 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns 2023-12-02T00:29:10.328 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-02T00:29:10.328 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-02T00:29:10.328 INFO:teuthology.orchestra.run.smithi046.stdout:/home/ubuntu/cephtest/mnt.0 2023-12-02T00:29:10.328 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-02T00:29:10.328 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-02T00:29:10.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:10.329 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2023-12-02T00:29:10.335 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi046.stderr:2023-12-02T00:29:10.334+0000 7fa0f5439480 -1 init, newargv = 0x560fff24b900 newargc=15 2023-12-02T00:29:10.335 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi046.stderr:ceph-fuse[99143]: starting ceph client 2023-12-02T00:29:10.340 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi046.stderr:ceph-fuse[99143]: starting fuse 2023-12-02T00:29:10.350 INFO:teuthology.orchestra.run.smithi046.stdout:41 2023-12-02T00:29:10.350 INFO:teuthology.orchestra.run.smithi046.stdout:44 2023-12-02T00:29:10.350 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41, 44] 2023-12-02T00:29:10.350 DEBUG:teuthology.orchestra.run.smithi046:> sudo stdin-killer -- python3 -c ' 2023-12-02T00:29:10.351 DEBUG:teuthology.orchestra.run.smithi046:> import glob 2023-12-02T00:29:10.351 DEBUG:teuthology.orchestra.run.smithi046:> import re 2023-12-02T00:29:10.351 DEBUG:teuthology.orchestra.run.smithi046:> import os 2023-12-02T00:29:10.351 DEBUG:teuthology.orchestra.run.smithi046:> import subprocess 2023-12-02T00:29:10.351 DEBUG:teuthology.orchestra.run.smithi046:> 2023-12-02T00:29:10.351 DEBUG:teuthology.orchestra.run.smithi046:> def _find_admin_socket(client_name): 2023-12-02T00:29:10.352 DEBUG:teuthology.orchestra.run.smithi046:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2023-12-02T00:29:10.352 DEBUG:teuthology.orchestra.run.smithi046:> files = glob.glob(asok_path) 2023-12-02T00:29:10.352 DEBUG:teuthology.orchestra.run.smithi046:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2023-12-02T00:29:10.352 DEBUG:teuthology.orchestra.run.smithi046:> 2023-12-02T00:29:10.352 DEBUG:teuthology.orchestra.run.smithi046:> # Given a non-glob path, it better be there 2023-12-02T00:29:10.352 DEBUG:teuthology.orchestra.run.smithi046:> if "*" not in asok_path: 2023-12-02T00:29:10.352 DEBUG:teuthology.orchestra.run.smithi046:> assert(len(files) == 1) 2023-12-02T00:29:10.352 DEBUG:teuthology.orchestra.run.smithi046:> return files[0] 2023-12-02T00:29:10.352 DEBUG:teuthology.orchestra.run.smithi046:> 2023-12-02T00:29:10.352 DEBUG:teuthology.orchestra.run.smithi046:> for f in files: 2023-12-02T00:29:10.352 DEBUG:teuthology.orchestra.run.smithi046:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-02T00:29:10.353 DEBUG:teuthology.orchestra.run.smithi046:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-02T00:29:10.353 DEBUG:teuthology.orchestra.run.smithi046:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-02T00:29:10.353 DEBUG:teuthology.orchestra.run.smithi046:> contents = proc_f.read() 2023-12-02T00:29:10.353 DEBUG:teuthology.orchestra.run.smithi046:> if mountpoint in contents: 2023-12-02T00:29:10.353 DEBUG:teuthology.orchestra.run.smithi046:> return f 2023-12-02T00:29:10.353 DEBUG:teuthology.orchestra.run.smithi046:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-02T00:29:10.353 DEBUG:teuthology.orchestra.run.smithi046:> 2023-12-02T00:29:10.353 DEBUG:teuthology.orchestra.run.smithi046:> print(_find_admin_socket("client.2")) 2023-12-02T00:29:10.353 DEBUG:teuthology.orchestra.run.smithi046:> ' 2023-12-02T00:29:10.475 INFO:teuthology.orchestra.run.smithi046.stdout:/var/run/ceph/ceph-client.2.99143.asok 2023-12-02T00:29:10.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-12-02T00:29:10.486 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.99143.asok 2023-12-02T00:29:10.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:10.486 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.99143.asok status 2023-12-02T00:29:10.627 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-12-02T00:29:10.628 INFO:teuthology.orchestra.run.smithi046.stdout: "metadata": { 2023-12-02T00:29:10.628 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_sha1": "bed7681480ccb0192b91a35e691658867e1e96b7", 2023-12-02T00:29:10.628 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_version": "ceph version 18.0.0-7631-gbed76814 (bed7681480ccb0192b91a35e691658867e1e96b7) reef (dev)", 2023-12-02T00:29:10.628 INFO:teuthology.orchestra.run.smithi046.stdout: "entity_id": "2", 2023-12-02T00:29:10.628 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": "smithi046", 2023-12-02T00:29:10.628 INFO:teuthology.orchestra.run.smithi046.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2023-12-02T00:29:10.628 INFO:teuthology.orchestra.run.smithi046.stdout: "pid": "99143", 2023-12-02T00:29:10.628 INFO:teuthology.orchestra.run.smithi046.stdout: "root": "/" 2023-12-02T00:29:10.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:29:10.628 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_count": 0, 2023-12-02T00:29:10.628 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_pinned_count": 0, 2023-12-02T00:29:10.628 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 4728, 2023-12-02T00:29:10.628 INFO:teuthology.orchestra.run.smithi046.stdout: "inst": { 2023-12-02T00:29:10.628 INFO:teuthology.orchestra.run.smithi046.stdout: "name": { 2023-12-02T00:29:10.628 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "client", 2023-12-02T00:29:10.628 INFO:teuthology.orchestra.run.smithi046.stdout: "num": 4728 2023-12-02T00:29:10.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:29:10.629 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2023-12-02T00:29:10.629 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2023-12-02T00:29:10.629 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.2:0", 2023-12-02T00:29:10.629 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 764149235 2023-12-02T00:29:10.629 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:29:10.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:29:10.629 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2023-12-02T00:29:10.629 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2023-12-02T00:29:10.629 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.2:0", 2023-12-02T00:29:10.629 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 764149235 2023-12-02T00:29:10.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:29:10.629 INFO:teuthology.orchestra.run.smithi046.stdout: "inst_str": "client.4728 192.168.0.2:0/764149235", 2023-12-02T00:29:10.629 INFO:teuthology.orchestra.run.smithi046.stdout: "addr_str": "192.168.0.2:0/764149235", 2023-12-02T00:29:10.629 INFO:teuthology.orchestra.run.smithi046.stdout: "inode_count": 1, 2023-12-02T00:29:10.629 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_epoch": 6, 2023-12-02T00:29:10.630 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch": 21, 2023-12-02T00:29:10.630 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch_barrier": 0, 2023-12-02T00:29:10.630 INFO:teuthology.orchestra.run.smithi046.stdout: "blocklisted": false, 2023-12-02T00:29:10.630 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs" 2023-12-02T00:29:10.630 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-12-02T00:29:10.636 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-02T00:29:10.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.742+0000 7f50e4fbe640 1 Processor -- start 2023-12-02T00:29:10.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.742+0000 7f50e4fbe640 1 -- start start 2023-12-02T00:29:10.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.743+0000 7f50e4fbe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f50e006abe0 0x7f50e01618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:10.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.743+0000 7f50e4fbe640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f50e01409d0 0x7f50e006efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:10.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.743+0000 7f50e4fbe640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50e01406f0 con 0x7f50e0140310 2023-12-02T00:29:10.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.743+0000 7f50e4fbe640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50e0161e20 con 0x7f50e01409d0 2023-12-02T00:29:10.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.743+0000 7f50e4fbe640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50e0143960 con 0x7f50e006abe0 2023-12-02T00:29:10.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.743+0000 7f50ddd9b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f50e006abe0 0x7f50e01618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:10.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.743+0000 7f50ded9d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f50e01409d0 0x7f50e006efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:10.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.743+0000 7f50ddd9b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f50e006abe0 0x7f50e01618e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.46:48194/0 (socket says 172.21.15.46:48194) 2023-12-02T00:29:10.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.743+0000 7f50de59c640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f50e0140310 0x7f50e006a4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.46:46934/0 (socket says 172.21.15.46:46934) 2023-12-02T00:29:10.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.743+0000 7f50ddd9b640 1 -- 172.21.15.46:0/1500791333 learned_addr learned my addr 172.21.15.46:0/1500791333 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:10.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.743+0000 7f50ded9d640 1 --2- 172.21.15.46:0/1500791333 >> v2:172.21.15.162:3300/0 conn(0x7f50e01409d0 0x7f50e006efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.46:35230/0 (socket says 172.21.15.46:35230) 2023-12-02T00:29:10.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.744+0000 7f50ddd9b640 1 -- 172.21.15.46:0/1500791333 >> v2:172.21.15.162:3300/0 conn(0x7f50e01409d0 msgr2=0x7f50e006efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:10.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.744+0000 7f50ddd9b640 1 --2- 172.21.15.46:0/1500791333 >> v2:172.21.15.162:3300/0 conn(0x7f50e01409d0 0x7f50e006efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:10.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.744+0000 7f50dd59a640 1 -- 172.21.15.46:0/1500791333 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2864959386 0 0) 0x7f50e01406f0 con 0x7f50e0140310 2023-12-02T00:29:10.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.744+0000 7f50ddd9b640 1 -- 172.21.15.46:0/1500791333 >> v1:172.21.15.162:6789/0 conn(0x7f50e0140310 legacy=0x7f50e006a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:10.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.744+0000 7f50ddd9b640 1 -- 172.21.15.46:0/1500791333 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e0142de0 con 0x7f50e006abe0 2023-12-02T00:29:10.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.744+0000 7f50ddd9b640 1 --2- 172.21.15.46:0/1500791333 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f50e006abe0 0x7f50e01618e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f50d4009870 tx=0x7f50d402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6a5b62e78232d12 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:10.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.745+0000 7f50dd59a640 1 -- 172.21.15.46:0/1500791333 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d4031640 con 0x7f50e006abe0 2023-12-02T00:29:10.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.745+0000 7f50dd59a640 1 -- 172.21.15.46:0/1500791333 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50d40317a0 con 0x7f50e006abe0 2023-12-02T00:29:10.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.745+0000 7f50dd59a640 1 -- 172.21.15.46:0/1500791333 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d4031b30 con 0x7f50e006abe0 2023-12-02T00:29:10.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.745+0000 7f50e4fbe640 1 -- 172.21.15.46:0/1500791333 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f50e006abe0 msgr2=0x7f50e01618e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:10.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.745+0000 7f50e4fbe640 1 --2- 172.21.15.46:0/1500791333 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f50e006abe0 0x7f50e01618e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f50d4009870 tx=0x7f50d402b8f0 comp rx=0 tx=0).stop 2023-12-02T00:29:10.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.745+0000 7f50e4fbe640 1 -- 172.21.15.46:0/1500791333 shutdown_connections 2023-12-02T00:29:10.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.745+0000 7f50e4fbe640 1 --2- 172.21.15.46:0/1500791333 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f50e006abe0 0x7f50e01618e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:10.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.745+0000 7f50e4fbe640 1 --2- 172.21.15.46:0/1500791333 >> v2:172.21.15.162:3300/0 conn(0x7f50e01409d0 0x7f50e006efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:10.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.745+0000 7f50e4fbe640 1 -- 172.21.15.46:0/1500791333 >> 172.21.15.46:0/1500791333 conn(0x7f50e00b1e50 msgr2=0x7f50e00b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:10.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.745+0000 7f50e4fbe640 1 -- 172.21.15.46:0/1500791333 shutdown_connections 2023-12-02T00:29:10.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.745+0000 7f50e4fbe640 1 -- 172.21.15.46:0/1500791333 wait complete. 2023-12-02T00:29:10.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.746+0000 7f50e4fbe640 1 Processor -- start 2023-12-02T00:29:10.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.746+0000 7f50e4fbe640 1 -- start start 2023-12-02T00:29:10.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.747+0000 7f50e4fbe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f50e006abe0 0x7f50e01328a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:10.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.747+0000 7f50e4fbe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50e0140310 0x7f50e0157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:10.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.747+0000 7f50e4fbe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50e01409d0 0x7f50e01f6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:10.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.747+0000 7f50e4fbe640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e01f6a20 con 0x7f50e01409d0 2023-12-02T00:29:10.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.747+0000 7f50e4fbe640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e012f2d0 con 0x7f50e0140310 2023-12-02T00:29:10.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.747+0000 7f50e4fbe640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50e01579b0 con 0x7f50e006abe0 2023-12-02T00:29:10.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.747+0000 7f50ded9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50e01409d0 0x7f50e01f6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:10.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.747+0000 7f50ded9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50e01409d0 0x7f50e01f6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38098/0 (socket says 172.21.15.46:38098) 2023-12-02T00:29:10.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.747+0000 7f50ded9d640 1 -- 172.21.15.46:0/2615492269 learned_addr learned my addr 172.21.15.46:0/2615492269 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:10.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.747+0000 7f50de59c640 1 --2- 172.21.15.46:0/2615492269 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f50e006abe0 0x7f50e01328a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:10.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.747+0000 7f50ddd9b640 1 --2- 172.21.15.46:0/2615492269 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50e0140310 0x7f50e0157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:10.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.748+0000 7f50ded9d640 1 -- 172.21.15.46:0/2615492269 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f50e006abe0 msgr2=0x7f50e01328a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:10.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.748+0000 7f50ded9d640 1 --2- 172.21.15.46:0/2615492269 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f50e006abe0 0x7f50e01328a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:10.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.748+0000 7f50ded9d640 1 -- 172.21.15.46:0/2615492269 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50e0140310 msgr2=0x7f50e0157470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:10.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.748+0000 7f50ded9d640 1 --2- 172.21.15.46:0/2615492269 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50e0140310 0x7f50e0157470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:10.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.748+0000 7f50ded9d640 1 -- 172.21.15.46:0/2615492269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d40094d0 con 0x7f50e01409d0 2023-12-02T00:29:10.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.748+0000 7f50ded9d640 1 --2- 172.21.15.46:0/2615492269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50e01409d0 0x7f50e01f6e40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f50d800efc0 tx=0x7f50d800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:10.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.748+0000 7f50ceffd640 1 -- 172.21.15.46:0/2615492269 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d8009ca0 con 0x7f50e01409d0 2023-12-02T00:29:10.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.748+0000 7f50ceffd640 1 -- 172.21.15.46:0/2615492269 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50d800f040 con 0x7f50e01409d0 2023-12-02T00:29:10.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.748+0000 7f50e4fbe640 1 -- 172.21.15.46:0/2615492269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50e012f570 con 0x7f50e01409d0 2023-12-02T00:29:10.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.748+0000 7f50ceffd640 1 -- 172.21.15.46:0/2615492269 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d80146c0 con 0x7f50e01409d0 2023-12-02T00:29:10.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.748+0000 7f50e4fbe640 1 -- 172.21.15.46:0/2615492269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50e01f7380 con 0x7f50e01409d0 2023-12-02T00:29:10.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.749+0000 7f50e4fbe640 1 -- 172.21.15.46:0/2615492269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50e01f64a0 con 0x7f50e01409d0 2023-12-02T00:29:10.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.750+0000 7f50ceffd640 1 -- 172.21.15.46:0/2615492269 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f50d8014820 con 0x7f50e01409d0 2023-12-02T00:29:10.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.750+0000 7f50ceffd640 1 --2- 172.21.15.46:0/2615492269 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f50c006ae40 0x7f50c006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:10.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.750+0000 7f50ceffd640 1 -- 172.21.15.46:0/2615492269 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f50d808a380 con 0x7f50e01409d0 2023-12-02T00:29:10.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.750+0000 7f50de59c640 1 --2- 172.21.15.46:0/2615492269 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f50c006ae40 0x7f50c006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:10.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.751+0000 7f50de59c640 1 --2- 172.21.15.46:0/2615492269 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f50c006ae40 0x7f50c006d300 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f50d0000f60 tx=0x7f50d0007d50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:10.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.755+0000 7f50ceffd640 1 -- 172.21.15.46:0/2615492269 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f50e01f64a0 con 0x7f50e01409d0 2023-12-02T00:29:10.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.969+0000 7f50e4fbe640 1 -- 172.21.15.46:0/2615492269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f50e01f64a0 con 0x7f50e01409d0 2023-12-02T00:29:10.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.970+0000 7f50ceffd640 1 -- 172.21.15.46:0/2615492269 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f50e01f64a0 con 0x7f50e01409d0 2023-12-02T00:29:10.971 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-02T00:29:10.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.971+0000 7f50e4fbe640 1 -- 172.21.15.46:0/2615492269 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f50c006ae40 msgr2=0x7f50c006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:10.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.971+0000 7f50e4fbe640 1 --2- 172.21.15.46:0/2615492269 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f50c006ae40 0x7f50c006d300 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f50d0000f60 tx=0x7f50d0007d50 comp rx=0 tx=0).stop 2023-12-02T00:29:10.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.971+0000 7f50e4fbe640 1 -- 172.21.15.46:0/2615492269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50e01409d0 msgr2=0x7f50e01f6e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:10.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.971+0000 7f50e4fbe640 1 --2- 172.21.15.46:0/2615492269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50e01409d0 0x7f50e01f6e40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f50d800efc0 tx=0x7f50d800c490 comp rx=0 tx=0).stop 2023-12-02T00:29:10.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.971+0000 7f50e4fbe640 1 -- 172.21.15.46:0/2615492269 shutdown_connections 2023-12-02T00:29:10.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.971+0000 7f50e4fbe640 1 --2- 172.21.15.46:0/2615492269 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f50c006ae40 0x7f50c006d300 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:10.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.971+0000 7f50e4fbe640 1 --2- 172.21.15.46:0/2615492269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50e01409d0 0x7f50e01f6e40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:10.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.971+0000 7f50e4fbe640 1 --2- 172.21.15.46:0/2615492269 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50e0140310 0x7f50e0157470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:10.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.971+0000 7f50e4fbe640 1 --2- 172.21.15.46:0/2615492269 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f50e006abe0 0x7f50e01328a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:10.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.971+0000 7f50e4fbe640 1 -- 172.21.15.46:0/2615492269 >> 172.21.15.46:0/2615492269 conn(0x7f50e00b1e50 msgr2=0x7f50e006a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:10.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.972+0000 7f50e4fbe640 1 -- 172.21.15.46:0/2615492269 shutdown_connections 2023-12-02T00:29:10.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:10.972+0000 7f50e4fbe640 1 -- 172.21.15.46:0/2615492269 wait complete. 2023-12-02T00:29:10.983 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-02T00:29:10.983 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-02T00:29:10.983 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi162.front.sepia.ceph.com 2023-12-02T00:29:10.983 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-12-02T00:29:10.984 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-12-02T00:29:10.984 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-02T00:29:10.984 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-02T00:29:10.984 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-02T00:29:10.984 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-12-02T00:29:10.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:10.984 DEBUG:teuthology.orchestra.run.smithi162:> ip addr 2023-12-02T00:29:11.003 INFO:teuthology.orchestra.run.smithi162.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-02T00:29:11.003 INFO:teuthology.orchestra.run.smithi162.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-02T00:29:11.003 INFO:teuthology.orchestra.run.smithi162.stdout: inet 127.0.0.1/8 scope host lo 2023-12-02T00:29:11.003 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:29:11.003 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 ::1/128 scope host 2023-12-02T00:29:11.003 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:29:11.003 INFO:teuthology.orchestra.run.smithi162.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-02T00:29:11.003 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:d9:37:f8 brd ff:ff:ff:ff:ff:ff 2023-12-02T00:29:11.004 INFO:teuthology.orchestra.run.smithi162.stdout: altname enp7s0f0 2023-12-02T00:29:11.004 INFO:teuthology.orchestra.run.smithi162.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-02T00:29:11.004 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:d9:37:f9 brd ff:ff:ff:ff:ff:ff 2023-12-02T00:29:11.004 INFO:teuthology.orchestra.run.smithi162.stdout: altname enp7s0f1 2023-12-02T00:29:11.004 INFO:teuthology.orchestra.run.smithi162.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-02T00:29:11.004 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:88:7f:40 brd ff:ff:ff:ff:ff:ff 2023-12-02T00:29:11.004 INFO:teuthology.orchestra.run.smithi162.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-02T00:29:11.004 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:88:7f:41 brd ff:ff:ff:ff:ff:ff 2023-12-02T00:29:11.004 INFO:teuthology.orchestra.run.smithi162.stdout: inet 172.21.15.162/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-02T00:29:11.004 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft 42609sec preferred_lft 42609sec 2023-12-02T00:29:11.004 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::ec4:7aff:fe88:7f41/64 scope link 2023-12-02T00:29:11.004 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:29:11.004 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-12-02T00:29:11.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:11.005 DEBUG:teuthology.orchestra.run.smithi162:> (cd / && exec bash -c ' 2023-12-02T00:29:11.005 DEBUG:teuthology.orchestra.run.smithi162:> set -e 2023-12-02T00:29:11.005 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link add name ceph-brx type bridge 2023-12-02T00:29:11.005 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip addr flush dev ceph-brx 2023-12-02T00:29:11.005 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link set ceph-brx up 2023-12-02T00:29:11.006 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-12-02T00:29:11.006 DEBUG:teuthology.orchestra.run.smithi162:> ') 2023-12-02T00:29:11.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:11.187 DEBUG:teuthology.orchestra.run.smithi162:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-02T00:29:11.215 INFO:teuthology.orchestra.run.smithi162.stdout:1 2023-12-02T00:29:11.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:11.216 DEBUG:teuthology.orchestra.run.smithi162:> route 2023-12-02T00:29:11.281 INFO:teuthology.orchestra.run.smithi162.stdout:Kernel IP routing table 2023-12-02T00:29:11.281 INFO:teuthology.orchestra.run.smithi162.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-02T00:29:11.281 INFO:teuthology.orchestra.run.smithi162.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-02T00:29:11.281 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-02T00:29:11.281 INFO:teuthology.orchestra.run.smithi162.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-02T00:29:11.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:11.282 DEBUG:teuthology.orchestra.run.smithi162:> (cd / && exec bash -c ' 2023-12-02T00:29:11.282 DEBUG:teuthology.orchestra.run.smithi162:> set -e 2023-12-02T00:29:11.283 DEBUG:teuthology.orchestra.run.smithi162:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-02T00:29:11.283 DEBUG:teuthology.orchestra.run.smithi162:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-02T00:29:11.283 DEBUG:teuthology.orchestra.run.smithi162:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-02T00:29:11.283 DEBUG:teuthology.orchestra.run.smithi162:> ') 2023-12-02T00:29:11.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:11.568 DEBUG:teuthology.orchestra.run.smithi162:> ip netns list 2023-12-02T00:29:11.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:11.584 DEBUG:teuthology.orchestra.run.smithi162:> ip netns list-id 2023-12-02T00:29:11.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:11.639 DEBUG:teuthology.orchestra.run.smithi162:> (cd / && exec bash -c ' 2023-12-02T00:29:11.639 DEBUG:teuthology.orchestra.run.smithi162:> set -e 2023-12-02T00:29:11.639 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-02T00:29:11.639 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-12-02T00:29:11.639 DEBUG:teuthology.orchestra.run.smithi162:> ') 2023-12-02T00:29:11.721 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-12-02T00:29:11.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:11.722 DEBUG:teuthology.orchestra.run.smithi162:> (cd / && exec bash -c ' 2023-12-02T00:29:11.722 DEBUG:teuthology.orchestra.run.smithi162:> set -e 2023-12-02T00:29:11.722 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-12-02T00:29:11.722 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-12-02T00:29:11.722 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-12-02T00:29:11.723 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-12-02T00:29:11.723 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-12-02T00:29:11.723 DEBUG:teuthology.orchestra.run.smithi162:> ') 2023-12-02T00:29:11.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:11.865 DEBUG:teuthology.orchestra.run.smithi162:> (cd / && exec bash -c ' 2023-12-02T00:29:11.865 DEBUG:teuthology.orchestra.run.smithi162:> set -e 2023-12-02T00:29:11.865 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link set brx.0 up 2023-12-02T00:29:11.865 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link set dev brx.0 master ceph-brx 2023-12-02T00:29:11.865 DEBUG:teuthology.orchestra.run.smithi162:> ') 2023-12-02T00:29:11.906 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-12-02T00:29:11.906 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-02T00:29:11.906 DEBUG:teuthology.orchestra.run.smithi162:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-12-02T00:29:11.959 INFO:teuthology.orchestra.run.smithi162.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-12-02T00:29:11.960 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-02T00:29:11.960 DEBUG:teuthology.orchestra.run.smithi162:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-12-02T00:29:12.014 DEBUG:teuthology.orchestra.run.smithi162:> sudo modprobe fuse 2023-12-02T00:29:12.082 DEBUG:teuthology.orchestra.run.smithi162:> cat /proc/self/mounts | awk '{print $2}' 2023-12-02T00:29:12.138 INFO:teuthology.orchestra.run.smithi162.stdout:/proc 2023-12-02T00:29:12.138 INFO:teuthology.orchestra.run.smithi162.stdout:/sys 2023-12-02T00:29:12.138 INFO:teuthology.orchestra.run.smithi162.stdout:/dev 2023-12-02T00:29:12.138 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/security 2023-12-02T00:29:12.138 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/shm 2023-12-02T00:29:12.138 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/pts 2023-12-02T00:29:12.138 INFO:teuthology.orchestra.run.smithi162.stdout:/run 2023-12-02T00:29:12.138 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/cgroup 2023-12-02T00:29:12.138 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/pstore 2023-12-02T00:29:12.138 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/bpf 2023-12-02T00:29:12.138 INFO:teuthology.orchestra.run.smithi162.stdout:/ 2023-12-02T00:29:12.138 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/selinux 2023-12-02T00:29:12.138 INFO:teuthology.orchestra.run.smithi162.stdout:/proc/sys/fs/binfmt_misc 2023-12-02T00:29:12.139 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/mqueue 2023-12-02T00:29:12.139 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/hugepages 2023-12-02T00:29:12.139 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/debug 2023-12-02T00:29:12.139 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/tracing 2023-12-02T00:29:12.139 INFO:teuthology.orchestra.run.smithi162.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-02T00:29:12.139 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/fuse/connections 2023-12-02T00:29:12.139 INFO:teuthology.orchestra.run.smithi162.stdout:/run/credentials/systemd-sysctl.service 2023-12-02T00:29:12.139 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/config 2023-12-02T00:29:12.139 INFO:teuthology.orchestra.run.smithi162.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-02T00:29:12.139 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/nfs/rpc_pipefs 2023-12-02T00:29:12.139 INFO:teuthology.orchestra.run.smithi162.stdout:/run/user/1000 2023-12-02T00:29:12.139 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph 2023-12-02T00:29:12.139 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-4 2023-12-02T00:29:12.139 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-5 2023-12-02T00:29:12.139 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-6 2023-12-02T00:29:12.139 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-7 2023-12-02T00:29:12.140 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns 2023-12-02T00:29:12.140 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-02T00:29:12.140 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-02T00:29:12.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:12.140 DEBUG:teuthology.orchestra.run.smithi162:> ls /sys/fs/fuse/connections 2023-12-02T00:29:12.192 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-12-02T00:29:12.192 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-12-02T00:29:12.235 DEBUG:teuthology.orchestra.run.smithi162:> sudo modprobe fuse 2023-12-02T00:29:12.261 DEBUG:teuthology.orchestra.run.smithi162:> cat /proc/self/mounts | awk '{print $2}' 2023-12-02T00:29:12.317 INFO:teuthology.orchestra.run.smithi162.stdout:/proc 2023-12-02T00:29:12.317 INFO:teuthology.orchestra.run.smithi162.stdout:/sys 2023-12-02T00:29:12.317 INFO:teuthology.orchestra.run.smithi162.stdout:/dev 2023-12-02T00:29:12.318 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/security 2023-12-02T00:29:12.318 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/shm 2023-12-02T00:29:12.318 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/pts 2023-12-02T00:29:12.318 INFO:teuthology.orchestra.run.smithi162.stdout:/run 2023-12-02T00:29:12.318 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/cgroup 2023-12-02T00:29:12.318 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/pstore 2023-12-02T00:29:12.318 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/bpf 2023-12-02T00:29:12.318 INFO:teuthology.orchestra.run.smithi162.stdout:/ 2023-12-02T00:29:12.318 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/selinux 2023-12-02T00:29:12.318 INFO:teuthology.orchestra.run.smithi162.stdout:/proc/sys/fs/binfmt_misc 2023-12-02T00:29:12.319 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/mqueue 2023-12-02T00:29:12.319 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/hugepages 2023-12-02T00:29:12.319 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/debug 2023-12-02T00:29:12.319 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/tracing 2023-12-02T00:29:12.319 INFO:teuthology.orchestra.run.smithi162.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-02T00:29:12.319 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/fuse/connections 2023-12-02T00:29:12.319 INFO:teuthology.orchestra.run.smithi162.stdout:/run/credentials/systemd-sysctl.service 2023-12-02T00:29:12.319 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/config 2023-12-02T00:29:12.319 INFO:teuthology.orchestra.run.smithi162.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-02T00:29:12.319 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/nfs/rpc_pipefs 2023-12-02T00:29:12.319 INFO:teuthology.orchestra.run.smithi162.stdout:/run/user/1000 2023-12-02T00:29:12.319 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph 2023-12-02T00:29:12.319 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-4 2023-12-02T00:29:12.319 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-5 2023-12-02T00:29:12.319 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-6 2023-12-02T00:29:12.320 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-7 2023-12-02T00:29:12.320 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns 2023-12-02T00:29:12.320 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-02T00:29:12.320 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-02T00:29:12.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:12.320 DEBUG:teuthology.orchestra.run.smithi162:> ls /sys/fs/fuse/connections 2023-12-02T00:29:12.340 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi162.stderr:2023-12-02T00:29:12.339+0000 7f5630e37480 -1 init, newargv = 0x557db1b81540 newargc=15 2023-12-02T00:29:12.341 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi162.stderr:ceph-fuse[95190]: starting ceph client 2023-12-02T00:29:12.345 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi162.stderr:ceph-fuse[95190]: starting fuse 2023-12-02T00:29:12.357 INFO:teuthology.orchestra.run.smithi162.stdout:43 2023-12-02T00:29:12.357 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-12-02T00:29:12.357 DEBUG:teuthology.orchestra.run.smithi162:> sudo stdin-killer -- python3 -c ' 2023-12-02T00:29:12.357 DEBUG:teuthology.orchestra.run.smithi162:> import glob 2023-12-02T00:29:12.357 DEBUG:teuthology.orchestra.run.smithi162:> import re 2023-12-02T00:29:12.357 DEBUG:teuthology.orchestra.run.smithi162:> import os 2023-12-02T00:29:12.357 DEBUG:teuthology.orchestra.run.smithi162:> import subprocess 2023-12-02T00:29:12.358 DEBUG:teuthology.orchestra.run.smithi162:> 2023-12-02T00:29:12.358 DEBUG:teuthology.orchestra.run.smithi162:> def _find_admin_socket(client_name): 2023-12-02T00:29:12.358 DEBUG:teuthology.orchestra.run.smithi162:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-12-02T00:29:12.358 DEBUG:teuthology.orchestra.run.smithi162:> files = glob.glob(asok_path) 2023-12-02T00:29:12.358 DEBUG:teuthology.orchestra.run.smithi162:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-12-02T00:29:12.358 DEBUG:teuthology.orchestra.run.smithi162:> 2023-12-02T00:29:12.358 DEBUG:teuthology.orchestra.run.smithi162:> # Given a non-glob path, it better be there 2023-12-02T00:29:12.358 DEBUG:teuthology.orchestra.run.smithi162:> if "*" not in asok_path: 2023-12-02T00:29:12.358 DEBUG:teuthology.orchestra.run.smithi162:> assert(len(files) == 1) 2023-12-02T00:29:12.358 DEBUG:teuthology.orchestra.run.smithi162:> return files[0] 2023-12-02T00:29:12.358 DEBUG:teuthology.orchestra.run.smithi162:> 2023-12-02T00:29:12.358 DEBUG:teuthology.orchestra.run.smithi162:> for f in files: 2023-12-02T00:29:12.359 DEBUG:teuthology.orchestra.run.smithi162:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-02T00:29:12.359 DEBUG:teuthology.orchestra.run.smithi162:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-02T00:29:12.359 DEBUG:teuthology.orchestra.run.smithi162:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-02T00:29:12.359 DEBUG:teuthology.orchestra.run.smithi162:> contents = proc_f.read() 2023-12-02T00:29:12.359 DEBUG:teuthology.orchestra.run.smithi162:> if mountpoint in contents: 2023-12-02T00:29:12.359 DEBUG:teuthology.orchestra.run.smithi162:> return f 2023-12-02T00:29:12.359 DEBUG:teuthology.orchestra.run.smithi162:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-02T00:29:12.359 DEBUG:teuthology.orchestra.run.smithi162:> 2023-12-02T00:29:12.359 DEBUG:teuthology.orchestra.run.smithi162:> print(_find_admin_socket("client.1")) 2023-12-02T00:29:12.359 DEBUG:teuthology.orchestra.run.smithi162:> ' 2023-12-02T00:29:12.476 INFO:teuthology.orchestra.run.smithi162.stdout:/var/run/ceph/ceph-client.1.95190.asok 2023-12-02T00:29:12.480 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:29:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-12-02T00:29:12.487 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.95190.asok 2023-12-02T00:29:12.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:12.487 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.95190.asok status 2023-12-02T00:29:12.623 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2023-12-02T00:29:12.623 INFO:teuthology.orchestra.run.smithi162.stdout: "metadata": { 2023-12-02T00:29:12.624 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_sha1": "bed7681480ccb0192b91a35e691658867e1e96b7", 2023-12-02T00:29:12.624 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_version": "ceph version 18.0.0-7631-gbed76814 (bed7681480ccb0192b91a35e691658867e1e96b7) reef (dev)", 2023-12-02T00:29:12.624 INFO:teuthology.orchestra.run.smithi162.stdout: "entity_id": "1", 2023-12-02T00:29:12.624 INFO:teuthology.orchestra.run.smithi162.stdout: "hostname": "smithi162", 2023-12-02T00:29:12.624 INFO:teuthology.orchestra.run.smithi162.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-12-02T00:29:12.624 INFO:teuthology.orchestra.run.smithi162.stdout: "pid": "95190", 2023-12-02T00:29:12.624 INFO:teuthology.orchestra.run.smithi162.stdout: "root": "/" 2023-12-02T00:29:12.624 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-12-02T00:29:12.624 INFO:teuthology.orchestra.run.smithi162.stdout: "dentry_count": 0, 2023-12-02T00:29:12.624 INFO:teuthology.orchestra.run.smithi162.stdout: "dentry_pinned_count": 0, 2023-12-02T00:29:12.624 INFO:teuthology.orchestra.run.smithi162.stdout: "id": 4523, 2023-12-02T00:29:12.624 INFO:teuthology.orchestra.run.smithi162.stdout: "inst": { 2023-12-02T00:29:12.625 INFO:teuthology.orchestra.run.smithi162.stdout: "name": { 2023-12-02T00:29:12.625 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "client", 2023-12-02T00:29:12.625 INFO:teuthology.orchestra.run.smithi162.stdout: "num": 4523 2023-12-02T00:29:12.625 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-12-02T00:29:12.625 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": { 2023-12-02T00:29:12.625 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2023-12-02T00:29:12.625 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "192.168.0.1:0", 2023-12-02T00:29:12.625 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 341444654 2023-12-02T00:29:12.625 INFO:teuthology.orchestra.run.smithi162.stdout: } 2023-12-02T00:29:12.625 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-12-02T00:29:12.625 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": { 2023-12-02T00:29:12.625 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2023-12-02T00:29:12.625 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "192.168.0.1:0", 2023-12-02T00:29:12.625 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 341444654 2023-12-02T00:29:12.625 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-12-02T00:29:12.625 INFO:teuthology.orchestra.run.smithi162.stdout: "inst_str": "client.4523 192.168.0.1:0/341444654", 2023-12-02T00:29:12.626 INFO:teuthology.orchestra.run.smithi162.stdout: "addr_str": "192.168.0.1:0/341444654", 2023-12-02T00:29:12.626 INFO:teuthology.orchestra.run.smithi162.stdout: "inode_count": 1, 2023-12-02T00:29:12.626 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_epoch": 6, 2023-12-02T00:29:12.626 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_epoch": 21, 2023-12-02T00:29:12.626 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_epoch_barrier": 0, 2023-12-02T00:29:12.626 INFO:teuthology.orchestra.run.smithi162.stdout: "blocklisted": false, 2023-12-02T00:29:12.626 INFO:teuthology.orchestra.run.smithi162.stdout: "fs_name": "cephfs" 2023-12-02T00:29:12.626 INFO:teuthology.orchestra.run.smithi162.stdout:} 2023-12-02T00:29:12.632 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-02T00:29:12.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.750+0000 7f4b279d0640 1 Processor -- start 2023-12-02T00:29:12.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.750+0000 7f4b279d0640 1 -- start start 2023-12-02T00:29:12.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.751+0000 7f4b279d0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b2015d330 0x7f4b200b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:12.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.751+0000 7f4b279d0640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4b200d2910 0x7f4b200d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:12.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.751+0000 7f4b279d0640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b200b2b10 con 0x7f4b200d2d90 2023-12-02T00:29:12.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.751+0000 7f4b279d0640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b200d1590 con 0x7f4b200d2910 2023-12-02T00:29:12.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.751+0000 7f4b279d0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b200d1870 con 0x7f4b2015d330 2023-12-02T00:29:12.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.751+0000 7f4b25f6d640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4b200d2d90 0x7f4b200b6390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38526/0 (socket says 172.21.15.46:38526) 2023-12-02T00:29:12.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.751+0000 7f4b25f6d640 1 -- 172.21.15.46:0/217048521 learned_addr learned my addr 172.21.15.46:0/217048521 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:12.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.751+0000 7f4b2576c640 1 --2- 172.21.15.46:0/217048521 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b2015d330 0x7f4b200b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:12.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.751+0000 7f4b24f6b640 1 --2- 172.21.15.46:0/217048521 >> v2:172.21.15.162:3300/0 conn(0x7f4b200d2910 0x7f4b200d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:12.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.751+0000 7f4b17fff640 1 -- 172.21.15.46:0/217048521 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 934528879 0 0) 0x7f4b200b2b10 con 0x7f4b200d2d90 2023-12-02T00:29:12.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.751+0000 7f4b17fff640 1 -- 172.21.15.46:0/217048521 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b0c003520 con 0x7f4b200d2d90 2023-12-02T00:29:12.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.751+0000 7f4b17fff640 1 -- 172.21.15.46:0/217048521 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3839982761 0 0) 0x7f4b0c003520 con 0x7f4b200d2d90 2023-12-02T00:29:12.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.751+0000 7f4b2576c640 1 -- 172.21.15.46:0/217048521 >> v2:172.21.15.162:3300/0 conn(0x7f4b200d2910 msgr2=0x7f4b200d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:12.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.751+0000 7f4b2576c640 1 --2- 172.21.15.46:0/217048521 >> v2:172.21.15.162:3300/0 conn(0x7f4b200d2910 0x7f4b200d31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:12.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.751+0000 7f4b2576c640 1 -- 172.21.15.46:0/217048521 >> v1:172.21.15.46:6789/0 conn(0x7f4b200d2d90 legacy=0x7f4b200b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:12.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.751+0000 7f4b2576c640 1 -- 172.21.15.46:0/217048521 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b200d1b50 con 0x7f4b2015d330 2023-12-02T00:29:12.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.751+0000 7f4b24f6b640 1 --2- 172.21.15.46:0/217048521 >> v2:172.21.15.162:3300/0 conn(0x7f4b200d2910 0x7f4b200d31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:29:12.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.752+0000 7f4b2576c640 1 --2- 172.21.15.46:0/217048521 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b2015d330 0x7f4b200b8b80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c0097a0 tx=0x7f4b1c02b710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=329cbfa4d762929 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:12.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.752+0000 7f4b17fff640 1 -- 172.21.15.46:0/217048521 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b180019f0 con 0x7f4b2015d330 2023-12-02T00:29:12.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.752+0000 7f4b17fff640 1 -- 172.21.15.46:0/217048521 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b1c02e050 con 0x7f4b2015d330 2023-12-02T00:29:12.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.752+0000 7f4b17fff640 1 -- 172.21.15.46:0/217048521 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c030710 con 0x7f4b2015d330 2023-12-02T00:29:12.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.753+0000 7f4b279d0640 1 -- 172.21.15.46:0/217048521 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b2015d330 msgr2=0x7f4b200b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:12.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.753+0000 7f4b279d0640 1 --2- 172.21.15.46:0/217048521 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b2015d330 0x7f4b200b8b80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c0097a0 tx=0x7f4b1c02b710 comp rx=0 tx=0).stop 2023-12-02T00:29:12.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.753+0000 7f4b279d0640 1 -- 172.21.15.46:0/217048521 shutdown_connections 2023-12-02T00:29:12.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.753+0000 7f4b279d0640 1 --2- 172.21.15.46:0/217048521 >> v2:172.21.15.162:3300/0 conn(0x7f4b200d2910 0x7f4b200d31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:12.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.753+0000 7f4b279d0640 1 --2- 172.21.15.46:0/217048521 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b2015d330 0x7f4b200b8b80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:12.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.753+0000 7f4b279d0640 1 -- 172.21.15.46:0/217048521 >> 172.21.15.46:0/217048521 conn(0x7f4b20072020 msgr2=0x7f4b20072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:12.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.753+0000 7f4b279d0640 1 -- 172.21.15.46:0/217048521 shutdown_connections 2023-12-02T00:29:12.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.753+0000 7f4b279d0640 1 -- 172.21.15.46:0/217048521 wait complete. 2023-12-02T00:29:12.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.754+0000 7f4b279d0640 1 Processor -- start 2023-12-02T00:29:12.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.754+0000 7f4b279d0640 1 -- start start 2023-12-02T00:29:12.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.755+0000 7f4b279d0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b200d2910 0x7f4b2015ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:12.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.755+0000 7f4b279d0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b200d2d90 0x7f4b2015e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:12.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.755+0000 7f4b279d0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b2015d330 0x7f4b2014d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:12.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.755+0000 7f4b279d0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2015e8e0 con 0x7f4b2015d330 2023-12-02T00:29:12.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.755+0000 7f4b279d0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b200cab00 con 0x7f4b200d2910 2023-12-02T00:29:12.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.755+0000 7f4b279d0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2015f2f0 con 0x7f4b200d2d90 2023-12-02T00:29:12.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.755+0000 7f4b25f6d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b2015d330 0x7f4b2014d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:12.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.755+0000 7f4b25f6d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b2015d330 0x7f4b2014d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44606/0 (socket says 172.21.15.46:44606) 2023-12-02T00:29:12.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.755+0000 7f4b25f6d640 1 -- 172.21.15.46:0/184913842 learned_addr learned my addr 172.21.15.46:0/184913842 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:12.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.755+0000 7f4b2576c640 1 --2- 172.21.15.46:0/184913842 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b200d2910 0x7f4b2015ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:12.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.755+0000 7f4b24f6b640 1 --2- 172.21.15.46:0/184913842 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b200d2d90 0x7f4b2015e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:12.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.756+0000 7f4b25f6d640 1 -- 172.21.15.46:0/184913842 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b200d2d90 msgr2=0x7f4b2015e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:12.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.756+0000 7f4b25f6d640 1 --2- 172.21.15.46:0/184913842 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b200d2d90 0x7f4b2015e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:12.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.756+0000 7f4b25f6d640 1 -- 172.21.15.46:0/184913842 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b200d2910 msgr2=0x7f4b2015ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:12.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.756+0000 7f4b25f6d640 1 --2- 172.21.15.46:0/184913842 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b200d2910 0x7f4b2015ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:12.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.756+0000 7f4b25f6d640 1 -- 172.21.15.46:0/184913842 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b10009400 con 0x7f4b2015d330 2023-12-02T00:29:12.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.756+0000 7f4b25f6d640 1 --2- 172.21.15.46:0/184913842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b2015d330 0x7f4b2014d830 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4b18001560 tx=0x7f4b1800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:12.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.756+0000 7f4b15ffb640 1 -- 172.21.15.46:0/184913842 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1800ce90 con 0x7f4b2015d330 2023-12-02T00:29:12.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.756+0000 7f4b15ffb640 1 -- 172.21.15.46:0/184913842 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b18012400 con 0x7f4b2015d330 2023-12-02T00:29:12.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.756+0000 7f4b279d0640 1 -- 172.21.15.46:0/184913842 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b1c009400 con 0x7f4b2015d330 2023-12-02T00:29:12.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.756+0000 7f4b15ffb640 1 -- 172.21.15.46:0/184913842 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b180127f0 con 0x7f4b2015d330 2023-12-02T00:29:12.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.756+0000 7f4b279d0640 1 -- 172.21.15.46:0/184913842 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b2015f8b0 con 0x7f4b2015d330 2023-12-02T00:29:12.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.757+0000 7f4b279d0640 1 -- 172.21.15.46:0/184913842 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b20069400 con 0x7f4b2015d330 2023-12-02T00:29:12.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.758+0000 7f4b15ffb640 1 -- 172.21.15.46:0/184913842 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4b18012950 con 0x7f4b2015d330 2023-12-02T00:29:12.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.758+0000 7f4b15ffb640 1 --2- 172.21.15.46:0/184913842 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f4b0406ae40 0x7f4b0406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:12.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.758+0000 7f4b15ffb640 1 -- 172.21.15.46:0/184913842 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4b18089c20 con 0x7f4b2015d330 2023-12-02T00:29:12.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.759+0000 7f4b2576c640 1 --2- 172.21.15.46:0/184913842 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f4b0406ae40 0x7f4b0406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:12.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.759+0000 7f4b2576c640 1 --2- 172.21.15.46:0/184913842 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f4b0406ae40 0x7f4b0406d300 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c034000 tx=0x7f4b1c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:12.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.762+0000 7f4b15ffb640 1 -- 172.21.15.46:0/184913842 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4b18054d90 con 0x7f4b2015d330 2023-12-02T00:29:12.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.975+0000 7f4b279d0640 1 -- 172.21.15.46:0/184913842 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4b200b3e20 con 0x7f4b2015d330 2023-12-02T00:29:12.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.976+0000 7f4b15ffb640 1 -- 172.21.15.46:0/184913842 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4b180583b0 con 0x7f4b2015d330 2023-12-02T00:29:12.977 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-02T00:29:12.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.978+0000 7f4b279d0640 1 -- 172.21.15.46:0/184913842 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f4b0406ae40 msgr2=0x7f4b0406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:12.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.978+0000 7f4b279d0640 1 --2- 172.21.15.46:0/184913842 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f4b0406ae40 0x7f4b0406d300 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c034000 tx=0x7f4b1c008030 comp rx=0 tx=0).stop 2023-12-02T00:29:12.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.978+0000 7f4b279d0640 1 -- 172.21.15.46:0/184913842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b2015d330 msgr2=0x7f4b2014d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:12.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.978+0000 7f4b279d0640 1 --2- 172.21.15.46:0/184913842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b2015d330 0x7f4b2014d830 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4b18001560 tx=0x7f4b1800a430 comp rx=0 tx=0).stop 2023-12-02T00:29:12.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.978+0000 7f4b279d0640 1 -- 172.21.15.46:0/184913842 shutdown_connections 2023-12-02T00:29:12.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.978+0000 7f4b279d0640 1 --2- 172.21.15.46:0/184913842 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f4b0406ae40 0x7f4b0406d300 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:12.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.978+0000 7f4b279d0640 1 --2- 172.21.15.46:0/184913842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b2015d330 0x7f4b2014d830 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:12.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.978+0000 7f4b279d0640 1 --2- 172.21.15.46:0/184913842 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b200d2d90 0x7f4b2015e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:12.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.978+0000 7f4b279d0640 1 --2- 172.21.15.46:0/184913842 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b200d2910 0x7f4b2015ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:12.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.978+0000 7f4b279d0640 1 -- 172.21.15.46:0/184913842 >> 172.21.15.46:0/184913842 conn(0x7f4b20072020 msgr2=0x7f4b20072400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:12.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.978+0000 7f4b279d0640 1 -- 172.21.15.46:0/184913842 shutdown_connections 2023-12-02T00:29:12.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:12.978+0000 7f4b279d0640 1 -- 172.21.15.46:0/184913842 wait complete. 2023-12-02T00:29:12.989 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-02T00:29:12.989 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-02T00:29:12.990 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi162.front.sepia.ceph.com 2023-12-02T00:29:12.990 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-12-02T00:29:12.990 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-12-02T00:29:12.990 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-02T00:29:12.990 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-02T00:29:12.990 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-02T00:29:12.990 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-12-02T00:29:12.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:12.990 DEBUG:teuthology.orchestra.run.smithi162:> ip addr 2023-12-02T00:29:13.006 INFO:teuthology.orchestra.run.smithi162.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-02T00:29:13.006 INFO:teuthology.orchestra.run.smithi162.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-02T00:29:13.006 INFO:teuthology.orchestra.run.smithi162.stdout: inet 127.0.0.1/8 scope host lo 2023-12-02T00:29:13.006 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:29:13.006 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 ::1/128 scope host 2023-12-02T00:29:13.006 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:29:13.007 INFO:teuthology.orchestra.run.smithi162.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-02T00:29:13.007 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:d9:37:f8 brd ff:ff:ff:ff:ff:ff 2023-12-02T00:29:13.007 INFO:teuthology.orchestra.run.smithi162.stdout: altname enp7s0f0 2023-12-02T00:29:13.007 INFO:teuthology.orchestra.run.smithi162.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-02T00:29:13.007 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:d9:37:f9 brd ff:ff:ff:ff:ff:ff 2023-12-02T00:29:13.007 INFO:teuthology.orchestra.run.smithi162.stdout: altname enp7s0f1 2023-12-02T00:29:13.007 INFO:teuthology.orchestra.run.smithi162.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-02T00:29:13.007 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:88:7f:40 brd ff:ff:ff:ff:ff:ff 2023-12-02T00:29:13.007 INFO:teuthology.orchestra.run.smithi162.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-02T00:29:13.007 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:88:7f:41 brd ff:ff:ff:ff:ff:ff 2023-12-02T00:29:13.007 INFO:teuthology.orchestra.run.smithi162.stdout: inet 172.21.15.162/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-02T00:29:13.007 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft 42607sec preferred_lft 42607sec 2023-12-02T00:29:13.007 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::ec4:7aff:fe88:7f41/64 scope link 2023-12-02T00:29:13.008 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:29:13.008 INFO:teuthology.orchestra.run.smithi162.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-02T00:29:13.008 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 4e:8e:16:25:6e:5e brd ff:ff:ff:ff:ff:ff 2023-12-02T00:29:13.008 INFO:teuthology.orchestra.run.smithi162.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-02T00:29:13.008 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:29:13.008 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::88ef:d3ff:fefc:f2b8/64 scope link 2023-12-02T00:29:13.008 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:29:13.008 INFO:teuthology.orchestra.run.smithi162.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-02T00:29:13.008 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 4e:8e:16:25:6e:5e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-02T00:29:13.008 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::4c8e:16ff:fe25:6e5e/64 scope link tentative 2023-12-02T00:29:13.008 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:29:13.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:13.009 DEBUG:teuthology.orchestra.run.smithi162:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-02T00:29:13.075 INFO:teuthology.orchestra.run.smithi162.stdout:1 2023-12-02T00:29:13.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:13.075 DEBUG:teuthology.orchestra.run.smithi162:> route 2023-12-02T00:29:13.131 INFO:teuthology.orchestra.run.smithi162.stdout:Kernel IP routing table 2023-12-02T00:29:13.132 INFO:teuthology.orchestra.run.smithi162.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-02T00:29:13.132 INFO:teuthology.orchestra.run.smithi162.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-02T00:29:13.132 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-02T00:29:13.132 INFO:teuthology.orchestra.run.smithi162.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-02T00:29:13.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:13.132 DEBUG:teuthology.orchestra.run.smithi162:> (cd / && exec bash -c ' 2023-12-02T00:29:13.132 DEBUG:teuthology.orchestra.run.smithi162:> set -e 2023-12-02T00:29:13.132 DEBUG:teuthology.orchestra.run.smithi162:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-02T00:29:13.132 DEBUG:teuthology.orchestra.run.smithi162:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-02T00:29:13.132 DEBUG:teuthology.orchestra.run.smithi162:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-02T00:29:13.133 DEBUG:teuthology.orchestra.run.smithi162:> ') 2023-12-02T00:29:13.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:13.227 DEBUG:teuthology.orchestra.run.smithi162:> ip netns list 2023-12-02T00:29:13.241 INFO:teuthology.orchestra.run.smithi162.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-12-02T00:29:13.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:13.242 DEBUG:teuthology.orchestra.run.smithi162:> ip netns list-id 2023-12-02T00:29:13.296 INFO:teuthology.orchestra.run.smithi162.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-12-02T00:29:13.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:13.297 DEBUG:teuthology.orchestra.run.smithi162:> (cd / && exec bash -c ' 2023-12-02T00:29:13.297 DEBUG:teuthology.orchestra.run.smithi162:> set -e 2023-12-02T00:29:13.297 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-12-02T00:29:13.297 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-12-02T00:29:13.297 DEBUG:teuthology.orchestra.run.smithi162:> ') 2023-12-02T00:29:13.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:13.378 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-12-02T00:29:13.446 INFO:teuthology.orchestra.run.smithi162.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-02T00:29:13.446 INFO:teuthology.orchestra.run.smithi162.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-02T00:29:13.446 INFO:teuthology.orchestra.run.smithi162.stdout: inet 127.0.0.1/8 scope host lo 2023-12-02T00:29:13.446 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:29:13.446 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 ::1/128 scope host 2023-12-02T00:29:13.446 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:29:13.446 INFO:teuthology.orchestra.run.smithi162.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-02T00:29:13.446 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 52:04:9e:bb:4b:d7 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-12-02T00:29:13.446 INFO:teuthology.orchestra.run.smithi162.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-12-02T00:29:13.446 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:29:13.446 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::5004:9eff:febb:4bd7/64 scope link tentative 2023-12-02T00:29:13.446 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:29:13.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:13.448 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-12-02T00:29:13.514 INFO:teuthology.orchestra.run.smithi162.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-02T00:29:13.515 INFO:teuthology.orchestra.run.smithi162.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-02T00:29:13.515 INFO:teuthology.orchestra.run.smithi162.stdout: inet 127.0.0.1/8 scope host lo 2023-12-02T00:29:13.515 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:29:13.515 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 ::1/128 scope host 2023-12-02T00:29:13.515 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:29:13.515 INFO:teuthology.orchestra.run.smithi162.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-02T00:29:13.515 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 52:04:9e:bb:4b:d7 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-12-02T00:29:13.515 INFO:teuthology.orchestra.run.smithi162.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-12-02T00:29:13.515 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:29:13.515 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::5004:9eff:febb:4bd7/64 scope link tentative 2023-12-02T00:29:13.515 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:29:13.517 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-12-02T00:29:13.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:13.517 DEBUG:teuthology.orchestra.run.smithi162:> (cd / && exec bash -c ' 2023-12-02T00:29:13.517 DEBUG:teuthology.orchestra.run.smithi162:> set -e 2023-12-02T00:29:13.517 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-12-02T00:29:13.517 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-12-02T00:29:13.517 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-12-02T00:29:13.518 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-12-02T00:29:13.518 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-12-02T00:29:13.518 DEBUG:teuthology.orchestra.run.smithi162:> ') 2023-12-02T00:29:13.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:13.642 DEBUG:teuthology.orchestra.run.smithi162:> (cd / && exec bash -c ' 2023-12-02T00:29:13.642 DEBUG:teuthology.orchestra.run.smithi162:> set -e 2023-12-02T00:29:13.642 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link set brx.1 up 2023-12-02T00:29:13.642 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link set dev brx.1 master ceph-brx 2023-12-02T00:29:13.642 DEBUG:teuthology.orchestra.run.smithi162:> ') 2023-12-02T00:29:13.683 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2023-12-02T00:29:13.684 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-02T00:29:13.684 DEBUG:teuthology.orchestra.run.smithi162:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-12-02T00:29:13.739 INFO:teuthology.orchestra.run.smithi162.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-12-02T00:29:13.739 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-02T00:29:13.739 DEBUG:teuthology.orchestra.run.smithi162:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-12-02T00:29:13.794 DEBUG:teuthology.orchestra.run.smithi162:> sudo modprobe fuse 2023-12-02T00:29:13.862 DEBUG:teuthology.orchestra.run.smithi162:> cat /proc/self/mounts | awk '{print $2}' 2023-12-02T00:29:13.918 INFO:teuthology.orchestra.run.smithi162.stdout:/proc 2023-12-02T00:29:13.918 INFO:teuthology.orchestra.run.smithi162.stdout:/sys 2023-12-02T00:29:13.918 INFO:teuthology.orchestra.run.smithi162.stdout:/dev 2023-12-02T00:29:13.918 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/security 2023-12-02T00:29:13.918 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/shm 2023-12-02T00:29:13.919 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/pts 2023-12-02T00:29:13.919 INFO:teuthology.orchestra.run.smithi162.stdout:/run 2023-12-02T00:29:13.919 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/cgroup 2023-12-02T00:29:13.919 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/pstore 2023-12-02T00:29:13.919 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/bpf 2023-12-02T00:29:13.919 INFO:teuthology.orchestra.run.smithi162.stdout:/ 2023-12-02T00:29:13.919 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/selinux 2023-12-02T00:29:13.919 INFO:teuthology.orchestra.run.smithi162.stdout:/proc/sys/fs/binfmt_misc 2023-12-02T00:29:13.919 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/mqueue 2023-12-02T00:29:13.919 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/hugepages 2023-12-02T00:29:13.919 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/debug 2023-12-02T00:29:13.919 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/tracing 2023-12-02T00:29:13.919 INFO:teuthology.orchestra.run.smithi162.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-02T00:29:13.919 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/fuse/connections 2023-12-02T00:29:13.919 INFO:teuthology.orchestra.run.smithi162.stdout:/run/credentials/systemd-sysctl.service 2023-12-02T00:29:13.920 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/config 2023-12-02T00:29:13.920 INFO:teuthology.orchestra.run.smithi162.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-02T00:29:13.920 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/nfs/rpc_pipefs 2023-12-02T00:29:13.920 INFO:teuthology.orchestra.run.smithi162.stdout:/run/user/1000 2023-12-02T00:29:13.920 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph 2023-12-02T00:29:13.920 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-4 2023-12-02T00:29:13.920 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-5 2023-12-02T00:29:13.920 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-6 2023-12-02T00:29:13.920 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-7 2023-12-02T00:29:13.920 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns 2023-12-02T00:29:13.920 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-02T00:29:13.920 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-02T00:29:13.920 INFO:teuthology.orchestra.run.smithi162.stdout:/home/ubuntu/cephtest/mnt.1 2023-12-02T00:29:13.920 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-12-02T00:29:13.920 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-12-02T00:29:13.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:13.921 DEBUG:teuthology.orchestra.run.smithi162:> ls /sys/fs/fuse/connections 2023-12-02T00:29:13.973 INFO:teuthology.orchestra.run.smithi162.stdout:43 2023-12-02T00:29:13.974 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2023-12-02T00:29:13.974 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2023-12-02T00:29:14.016 DEBUG:teuthology.orchestra.run.smithi162:> sudo modprobe fuse 2023-12-02T00:29:14.042 DEBUG:teuthology.orchestra.run.smithi162:> cat /proc/self/mounts | awk '{print $2}' 2023-12-02T00:29:14.101 INFO:teuthology.orchestra.run.smithi162.stdout:/proc 2023-12-02T00:29:14.102 INFO:teuthology.orchestra.run.smithi162.stdout:/sys 2023-12-02T00:29:14.102 INFO:teuthology.orchestra.run.smithi162.stdout:/dev 2023-12-02T00:29:14.102 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/security 2023-12-02T00:29:14.102 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/shm 2023-12-02T00:29:14.102 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/pts 2023-12-02T00:29:14.102 INFO:teuthology.orchestra.run.smithi162.stdout:/run 2023-12-02T00:29:14.102 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/cgroup 2023-12-02T00:29:14.102 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/pstore 2023-12-02T00:29:14.102 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/bpf 2023-12-02T00:29:14.102 INFO:teuthology.orchestra.run.smithi162.stdout:/ 2023-12-02T00:29:14.103 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/selinux 2023-12-02T00:29:14.103 INFO:teuthology.orchestra.run.smithi162.stdout:/proc/sys/fs/binfmt_misc 2023-12-02T00:29:14.103 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/mqueue 2023-12-02T00:29:14.103 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/hugepages 2023-12-02T00:29:14.103 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/debug 2023-12-02T00:29:14.103 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/tracing 2023-12-02T00:29:14.103 INFO:teuthology.orchestra.run.smithi162.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-02T00:29:14.103 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/fuse/connections 2023-12-02T00:29:14.103 INFO:teuthology.orchestra.run.smithi162.stdout:/run/credentials/systemd-sysctl.service 2023-12-02T00:29:14.103 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/config 2023-12-02T00:29:14.103 INFO:teuthology.orchestra.run.smithi162.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-02T00:29:14.103 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/nfs/rpc_pipefs 2023-12-02T00:29:14.103 INFO:teuthology.orchestra.run.smithi162.stdout:/run/user/1000 2023-12-02T00:29:14.103 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph 2023-12-02T00:29:14.103 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-4 2023-12-02T00:29:14.104 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-5 2023-12-02T00:29:14.104 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-6 2023-12-02T00:29:14.104 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-7 2023-12-02T00:29:14.104 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns 2023-12-02T00:29:14.104 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-02T00:29:14.104 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-02T00:29:14.104 INFO:teuthology.orchestra.run.smithi162.stdout:/home/ubuntu/cephtest/mnt.1 2023-12-02T00:29:14.104 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-12-02T00:29:14.104 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-12-02T00:29:14.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:14.105 DEBUG:teuthology.orchestra.run.smithi162:> ls /sys/fs/fuse/connections 2023-12-02T00:29:14.105 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi162.stderr:2023-12-02T00:29:14.100+0000 7fb0dac0c480 -1 init, newargv = 0x562dd6f2f120 newargc=15 2023-12-02T00:29:14.105 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi162.stderr:ceph-fuse[95624]: starting ceph client 2023-12-02T00:29:14.106 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi162.stderr:ceph-fuse[95624]: starting fuse 2023-12-02T00:29:14.119 INFO:teuthology.orchestra.run.smithi162.stdout:43 2023-12-02T00:29:14.120 INFO:teuthology.orchestra.run.smithi162.stdout:44 2023-12-02T00:29:14.120 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 44] 2023-12-02T00:29:14.120 DEBUG:teuthology.orchestra.run.smithi162:> sudo stdin-killer -- python3 -c ' 2023-12-02T00:29:14.120 DEBUG:teuthology.orchestra.run.smithi162:> import glob 2023-12-02T00:29:14.121 DEBUG:teuthology.orchestra.run.smithi162:> import re 2023-12-02T00:29:14.121 DEBUG:teuthology.orchestra.run.smithi162:> import os 2023-12-02T00:29:14.121 DEBUG:teuthology.orchestra.run.smithi162:> import subprocess 2023-12-02T00:29:14.121 DEBUG:teuthology.orchestra.run.smithi162:> 2023-12-02T00:29:14.121 DEBUG:teuthology.orchestra.run.smithi162:> def _find_admin_socket(client_name): 2023-12-02T00:29:14.121 DEBUG:teuthology.orchestra.run.smithi162:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2023-12-02T00:29:14.121 DEBUG:teuthology.orchestra.run.smithi162:> files = glob.glob(asok_path) 2023-12-02T00:29:14.121 DEBUG:teuthology.orchestra.run.smithi162:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2023-12-02T00:29:14.121 DEBUG:teuthology.orchestra.run.smithi162:> 2023-12-02T00:29:14.121 DEBUG:teuthology.orchestra.run.smithi162:> # Given a non-glob path, it better be there 2023-12-02T00:29:14.122 DEBUG:teuthology.orchestra.run.smithi162:> if "*" not in asok_path: 2023-12-02T00:29:14.122 DEBUG:teuthology.orchestra.run.smithi162:> assert(len(files) == 1) 2023-12-02T00:29:14.123 DEBUG:teuthology.orchestra.run.smithi162:> return files[0] 2023-12-02T00:29:14.123 DEBUG:teuthology.orchestra.run.smithi162:> 2023-12-02T00:29:14.123 DEBUG:teuthology.orchestra.run.smithi162:> for f in files: 2023-12-02T00:29:14.123 DEBUG:teuthology.orchestra.run.smithi162:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-02T00:29:14.123 DEBUG:teuthology.orchestra.run.smithi162:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-02T00:29:14.123 DEBUG:teuthology.orchestra.run.smithi162:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-02T00:29:14.123 DEBUG:teuthology.orchestra.run.smithi162:> contents = proc_f.read() 2023-12-02T00:29:14.123 DEBUG:teuthology.orchestra.run.smithi162:> if mountpoint in contents: 2023-12-02T00:29:14.123 DEBUG:teuthology.orchestra.run.smithi162:> return f 2023-12-02T00:29:14.123 DEBUG:teuthology.orchestra.run.smithi162:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-02T00:29:14.123 DEBUG:teuthology.orchestra.run.smithi162:> 2023-12-02T00:29:14.123 DEBUG:teuthology.orchestra.run.smithi162:> print(_find_admin_socket("client.3")) 2023-12-02T00:29:14.123 DEBUG:teuthology.orchestra.run.smithi162:> ' 2023-12-02T00:29:14.239 INFO:teuthology.orchestra.run.smithi162.stdout:/var/run/ceph/ceph-client.3.95624.asok 2023-12-02T00:29:14.242 INFO:teuthology.orchestra.run.smithi162.stderr:2023-12-02T00:29:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-12-02T00:29:14.250 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.95624.asok 2023-12-02T00:29:14.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:14.250 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.95624.asok status 2023-12-02T00:29:14.385 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2023-12-02T00:29:14.385 INFO:teuthology.orchestra.run.smithi162.stdout: "metadata": { 2023-12-02T00:29:14.385 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_sha1": "bed7681480ccb0192b91a35e691658867e1e96b7", 2023-12-02T00:29:14.386 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_version": "ceph version 18.0.0-7631-gbed76814 (bed7681480ccb0192b91a35e691658867e1e96b7) reef (dev)", 2023-12-02T00:29:14.386 INFO:teuthology.orchestra.run.smithi162.stdout: "entity_id": "3", 2023-12-02T00:29:14.386 INFO:teuthology.orchestra.run.smithi162.stdout: "hostname": "smithi162", 2023-12-02T00:29:14.386 INFO:teuthology.orchestra.run.smithi162.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2023-12-02T00:29:14.386 INFO:teuthology.orchestra.run.smithi162.stdout: "pid": "95624", 2023-12-02T00:29:14.386 INFO:teuthology.orchestra.run.smithi162.stdout: "root": "/" 2023-12-02T00:29:14.387 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-12-02T00:29:14.387 INFO:teuthology.orchestra.run.smithi162.stdout: "dentry_count": 0, 2023-12-02T00:29:14.387 INFO:teuthology.orchestra.run.smithi162.stdout: "dentry_pinned_count": 0, 2023-12-02T00:29:14.388 INFO:teuthology.orchestra.run.smithi162.stdout: "id": 4648, 2023-12-02T00:29:14.388 INFO:teuthology.orchestra.run.smithi162.stdout: "inst": { 2023-12-02T00:29:14.388 INFO:teuthology.orchestra.run.smithi162.stdout: "name": { 2023-12-02T00:29:14.388 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "client", 2023-12-02T00:29:14.388 INFO:teuthology.orchestra.run.smithi162.stdout: "num": 4648 2023-12-02T00:29:14.389 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-12-02T00:29:14.389 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": { 2023-12-02T00:29:14.389 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2023-12-02T00:29:14.389 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "192.168.0.2:0", 2023-12-02T00:29:14.389 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 4132943145 2023-12-02T00:29:14.389 INFO:teuthology.orchestra.run.smithi162.stdout: } 2023-12-02T00:29:14.389 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-12-02T00:29:14.389 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": { 2023-12-02T00:29:14.389 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2023-12-02T00:29:14.389 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "192.168.0.2:0", 2023-12-02T00:29:14.389 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 4132943145 2023-12-02T00:29:14.389 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2023-12-02T00:29:14.389 INFO:teuthology.orchestra.run.smithi162.stdout: "inst_str": "client.4648 192.168.0.2:0/4132943145", 2023-12-02T00:29:14.389 INFO:teuthology.orchestra.run.smithi162.stdout: "addr_str": "192.168.0.2:0/4132943145", 2023-12-02T00:29:14.389 INFO:teuthology.orchestra.run.smithi162.stdout: "inode_count": 1, 2023-12-02T00:29:14.390 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_epoch": 6, 2023-12-02T00:29:14.390 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_epoch": 21, 2023-12-02T00:29:14.390 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_epoch_barrier": 0, 2023-12-02T00:29:14.390 INFO:teuthology.orchestra.run.smithi162.stdout: "blocklisted": false, 2023-12-02T00:29:14.390 INFO:teuthology.orchestra.run.smithi162.stdout: "fs_name": "cephfs" 2023-12-02T00:29:14.390 INFO:teuthology.orchestra.run.smithi162.stdout:} 2023-12-02T00:29:14.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:14.394 DEBUG:teuthology.orchestra.run.smithi046:> stat --file-system '--printf=%T 2023-12-02T00:29:14.394 DEBUG:teuthology.orchestra.run.smithi046:> ' -- /home/ubuntu/cephtest/mnt.0 2023-12-02T00:29:14.414 INFO:teuthology.orchestra.run.smithi046.stdout:fuseblk 2023-12-02T00:29:14.414 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-12-02T00:29:14.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:14.414 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-02T00:29:14.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:14.492 DEBUG:teuthology.orchestra.run.smithi046:> stat --file-system '--printf=%T 2023-12-02T00:29:14.493 DEBUG:teuthology.orchestra.run.smithi046:> ' -- /home/ubuntu/cephtest/mnt.2 2023-12-02T00:29:14.552 INFO:teuthology.orchestra.run.smithi046.stdout:fuseblk 2023-12-02T00:29:14.552 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2023-12-02T00:29:14.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:14.552 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-12-02T00:29:14.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:14.629 DEBUG:teuthology.orchestra.run.smithi162:> stat --file-system '--printf=%T 2023-12-02T00:29:14.629 DEBUG:teuthology.orchestra.run.smithi162:> ' -- /home/ubuntu/cephtest/mnt.1 2023-12-02T00:29:14.644 INFO:teuthology.orchestra.run.smithi162.stdout:fuseblk 2023-12-02T00:29:14.645 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-12-02T00:29:14.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:14.645 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-12-02T00:29:14.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:14.715 DEBUG:teuthology.orchestra.run.smithi162:> stat --file-system '--printf=%T 2023-12-02T00:29:14.715 DEBUG:teuthology.orchestra.run.smithi162:> ' -- /home/ubuntu/cephtest/mnt.3 2023-12-02T00:29:14.770 INFO:teuthology.orchestra.run.smithi162.stdout:fuseblk 2023-12-02T00:29:14.770 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2023-12-02T00:29:14.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:14.770 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-12-02T00:29:14.838 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-12-02T00:29:14.856 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:29:14.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.973+0000 7f378041e640 1 Processor -- start 2023-12-02T00:29:14.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.974+0000 7f378041e640 1 -- start start 2023-12-02T00:29:14.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.974+0000 7f378041e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f377815d330 0x7f37780b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:14.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.975+0000 7f378041e640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37780b2b10 con 0x7f37780d3900 2023-12-02T00:29:14.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.975+0000 7f378041e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37780d1590 con 0x7f37780d2910 2023-12-02T00:29:14.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.975+0000 7f378041e640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37780d1870 con 0x7f377815d330 2023-12-02T00:29:14.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.975+0000 7f377e1ba640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f377815d330 0x7f37780b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:14.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.975+0000 7f377e1ba640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f377815d330 0x7f37780b8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44614/0 (socket says 172.21.15.46:44614) 2023-12-02T00:29:14.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.975+0000 7f377e1ba640 1 -- 172.21.15.46:0/3777373393 learned_addr learned my addr 172.21.15.46:0/3777373393 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:14.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.975+0000 7f377e9bb640 1 --1- 172.21.15.46:0/3777373393 >> v1:172.21.15.46:6789/0 conn(0x7f37780d3900 0x7f37780b6390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38532/0 (socket says 172.21.15.46:38532) 2023-12-02T00:29:14.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.975+0000 7f377e1ba640 1 -- 172.21.15.46:0/3777373393 >> v1:172.21.15.162:6789/0 conn(0x7f37780d2910 legacy=0x7f37780d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:14.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.975+0000 7f377e1ba640 1 -- 172.21.15.46:0/3777373393 >> v1:172.21.15.46:6789/0 conn(0x7f37780d3900 legacy=0x7f37780b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:14.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.975+0000 7f377e1ba640 1 -- 172.21.15.46:0/3777373393 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37780d1b50 con 0x7f377815d330 2023-12-02T00:29:14.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.976+0000 7f377e1ba640 1 --2- 172.21.15.46:0/3777373393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f377815d330 0x7f37780b8b80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3774009870 tx=0x7f377402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5664ace67d950bc server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:14.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.976+0000 7f377d1b8640 1 -- 172.21.15.46:0/3777373393 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3774031870 con 0x7f377815d330 2023-12-02T00:29:14.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.976+0000 7f377d1b8640 1 -- 172.21.15.46:0/3777373393 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37740319d0 con 0x7f377815d330 2023-12-02T00:29:14.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.976+0000 7f377d1b8640 1 -- 172.21.15.46:0/3777373393 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3774031d60 con 0x7f377815d330 2023-12-02T00:29:14.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.976+0000 7f378041e640 1 -- 172.21.15.46:0/3777373393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f377815d330 msgr2=0x7f37780b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:14.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.976+0000 7f378041e640 1 --2- 172.21.15.46:0/3777373393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f377815d330 0x7f37780b8b80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3774009870 tx=0x7f377402b910 comp rx=0 tx=0).stop 2023-12-02T00:29:14.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.977+0000 7f378041e640 1 -- 172.21.15.46:0/3777373393 shutdown_connections 2023-12-02T00:29:14.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.977+0000 7f378041e640 1 --2- 172.21.15.46:0/3777373393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f377815d330 0x7f37780b8b80 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:14.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.977+0000 7f378041e640 1 -- 172.21.15.46:0/3777373393 >> 172.21.15.46:0/3777373393 conn(0x7f3778072020 msgr2=0x7f3778072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:14.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.977+0000 7f378041e640 1 -- 172.21.15.46:0/3777373393 shutdown_connections 2023-12-02T00:29:14.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.977+0000 7f378041e640 1 -- 172.21.15.46:0/3777373393 wait complete. 2023-12-02T00:29:14.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.978+0000 7f378041e640 1 Processor -- start 2023-12-02T00:29:14.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.978+0000 7f378041e640 1 -- start start 2023-12-02T00:29:14.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.979+0000 7f378041e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37780d2910 0x7f377814c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:14.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.979+0000 7f378041e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37780d3900 0x7f37780cd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:14.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.979+0000 7f378041e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f377815d330 0x7f37780ca690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:14.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.979+0000 7f378041e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377814cb20 con 0x7f37780d2910 2023-12-02T00:29:14.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.979+0000 7f378041e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377814ce00 con 0x7f37780d3900 2023-12-02T00:29:14.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.979+0000 7f378041e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f377814d0e0 con 0x7f377815d330 2023-12-02T00:29:14.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.979+0000 7f377e1ba640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37780d2910 0x7f377814c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:14.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.979+0000 7f377e1ba640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37780d2910 0x7f377814c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44624/0 (socket says 172.21.15.46:44624) 2023-12-02T00:29:14.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.979+0000 7f377e1ba640 1 -- 172.21.15.46:0/814913382 learned_addr learned my addr 172.21.15.46:0/814913382 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:14.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.979+0000 7f377e9bb640 1 --2- 172.21.15.46:0/814913382 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f377815d330 0x7f37780ca690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:14.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.979+0000 7f377d9b9640 1 --2- 172.21.15.46:0/814913382 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37780d3900 0x7f37780cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:14.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.979+0000 7f377e1ba640 1 -- 172.21.15.46:0/814913382 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f377815d330 msgr2=0x7f37780ca690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:14.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.979+0000 7f377e1ba640 1 --2- 172.21.15.46:0/814913382 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f377815d330 0x7f37780ca690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:14.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.979+0000 7f377e1ba640 1 -- 172.21.15.46:0/814913382 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37780d3900 msgr2=0x7f37780cd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:14.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.979+0000 7f377e1ba640 1 --2- 172.21.15.46:0/814913382 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37780d3900 0x7f37780cd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:14.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.979+0000 7f377e1ba640 1 -- 172.21.15.46:0/814913382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37740094d0 con 0x7f37780d2910 2023-12-02T00:29:14.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.979+0000 7f377e1ba640 1 --2- 172.21.15.46:0/814913382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37780d2910 0x7f377814c240 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3774009870 tx=0x7f37740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:14.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.980+0000 7f376effd640 1 -- 172.21.15.46:0/814913382 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3774031820 con 0x7f37780d2910 2023-12-02T00:29:14.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.980+0000 7f376effd640 1 -- 172.21.15.46:0/814913382 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37740443f0 con 0x7f37780d2910 2023-12-02T00:29:14.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.980+0000 7f378041e640 1 -- 172.21.15.46:0/814913382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f377814d3c0 con 0x7f37780d2910 2023-12-02T00:29:14.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.980+0000 7f376effd640 1 -- 172.21.15.46:0/814913382 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37740447a0 con 0x7f37780d2910 2023-12-02T00:29:14.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.980+0000 7f378041e640 1 -- 172.21.15.46:0/814913382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f377814d980 con 0x7f37780d2910 2023-12-02T00:29:14.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.981+0000 7f378041e640 1 -- 172.21.15.46:0/814913382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37780b78f0 con 0x7f37780d2910 2023-12-02T00:29:14.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.981+0000 7f376effd640 1 -- 172.21.15.46:0/814913382 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f377404d450 con 0x7f37780d2910 2023-12-02T00:29:14.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.982+0000 7f376effd640 1 --2- 172.21.15.46:0/814913382 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f375c06ae40 0x7f375c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:14.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.982+0000 7f376effd640 1 -- 172.21.15.46:0/814913382 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f377402d040 con 0x7f37780d2910 2023-12-02T00:29:14.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.982+0000 7f377d9b9640 1 --2- 172.21.15.46:0/814913382 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f375c06ae40 0x7f375c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:14.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.982+0000 7f377d9b9640 1 --2- 172.21.15.46:0/814913382 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f375c06ae40 0x7f375c06d300 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f37780c9b90 tx=0x7f3768007e30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:14.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:14.986+0000 7f376effd640 1 -- 172.21.15.46:0/814913382 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3774076d40 con 0x7f37780d2910 2023-12-02T00:29:15.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.162+0000 7f378041e640 1 -- 172.21.15.46:0/814913382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f37780b9170 con 0x7f37780d2910 2023-12-02T00:29:15.164 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.163+0000 7f376effd640 1 -- 172.21.15.46:0/814913382 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f377407a360 con 0x7f37780d2910 2023-12-02T00:29:15.164 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:29:15.164 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":21,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:28:50.700112+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-02T00:28:47.273337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-02T00:28:48.060115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:29:15.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.165+0000 7f378041e640 1 -- 172.21.15.46:0/814913382 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f375c06ae40 msgr2=0x7f375c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:15.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.165+0000 7f378041e640 1 --2- 172.21.15.46:0/814913382 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f375c06ae40 0x7f375c06d300 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f37780c9b90 tx=0x7f3768007e30 comp rx=0 tx=0).stop 2023-12-02T00:29:15.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.165+0000 7f378041e640 1 -- 172.21.15.46:0/814913382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37780d2910 msgr2=0x7f377814c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:15.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.165+0000 7f378041e640 1 --2- 172.21.15.46:0/814913382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37780d2910 0x7f377814c240 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3774009870 tx=0x7f37740046c0 comp rx=0 tx=0).stop 2023-12-02T00:29:15.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.165+0000 7f378041e640 1 -- 172.21.15.46:0/814913382 shutdown_connections 2023-12-02T00:29:15.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.165+0000 7f378041e640 1 --2- 172.21.15.46:0/814913382 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f375c06ae40 0x7f375c06d300 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:15.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.165+0000 7f378041e640 1 --2- 172.21.15.46:0/814913382 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f377815d330 0x7f37780ca690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:15.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.165+0000 7f378041e640 1 --2- 172.21.15.46:0/814913382 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37780d3900 0x7f37780cd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:15.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.165+0000 7f378041e640 1 --2- 172.21.15.46:0/814913382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37780d2910 0x7f377814c240 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:15.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.165+0000 7f378041e640 1 -- 172.21.15.46:0/814913382 >> 172.21.15.46:0/814913382 conn(0x7f3778072020 msgr2=0x7f3778072400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:15.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.165+0000 7f378041e640 1 -- 172.21.15.46:0/814913382 shutdown_connections 2023-12-02T00:29:15.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.165+0000 7f378041e640 1 -- 172.21.15.46:0/814913382 wait complete. 2023-12-02T00:29:15.177 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-02T00:28:37.213645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-02T00:28:47.273337+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-02T00:28:48.060115+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-02T00:29:15.178 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-02T00:29:15.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.286+0000 7f9ba5e6b640 1 Processor -- start 2023-12-02T00:29:15.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.286+0000 7f9ba5e6b640 1 -- start start 2023-12-02T00:29:15.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.287+0000 7f9ba5e6b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9ba0143490 0x7f9ba00b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:15.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.287+0000 7f9ba5e6b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9ba015e730 0x7f9ba00b1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:15.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.287+0000 7f9ba5e6b640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ba015e2a0 con 0x7f9ba015a4e0 2023-12-02T00:29:15.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.287+0000 7f9ba5e6b640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ba015b520 con 0x7f9ba015e730 2023-12-02T00:29:15.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.287+0000 7f9ba5e6b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba0143870 con 0x7f9ba0143490 2023-12-02T00:29:15.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.287+0000 7f9b9effd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9ba015e730 0x7f9ba00b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:15.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.287+0000 7f9b9effd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9ba015e730 0x7f9ba00b1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44632/0 (socket says 172.21.15.46:44632) 2023-12-02T00:29:15.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.287+0000 7f9b9effd640 1 -- 172.21.15.46:0/4199151411 learned_addr learned my addr 172.21.15.46:0/4199151411 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:15.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.287+0000 7f9b9f7fe640 1 --2- 172.21.15.46:0/4199151411 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9ba0143490 0x7f9ba00b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:15.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.288+0000 7f9b9effd640 1 -- 172.21.15.46:0/4199151411 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9ba0143490 msgr2=0x7f9ba00b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:15.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.288+0000 7f9b9effd640 1 --2- 172.21.15.46:0/4199151411 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9ba0143490 0x7f9ba00b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:15.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.288+0000 7f9b9effd640 1 -- 172.21.15.46:0/4199151411 >> v1:172.21.15.162:6789/0 conn(0x7f9ba015a4e0 legacy=0x7f9ba015a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:15.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.288+0000 7f9b9effd640 1 -- 172.21.15.46:0/4199151411 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ba015b240 con 0x7f9ba015e730 2023-12-02T00:29:15.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.288+0000 7f9b9effd640 1 --2- 172.21.15.46:0/4199151411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ba015e730 0x7f9ba00b1230 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9b94009870 tx=0x7f9b9402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=971a63cf2eb66f05 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:15.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.288+0000 7f9b9e7fc640 1 -- 172.21.15.46:0/4199151411 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b94030840 con 0x7f9ba015e730 2023-12-02T00:29:15.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.289+0000 7f9b9e7fc640 1 -- 172.21.15.46:0/4199151411 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b940309a0 con 0x7f9ba015e730 2023-12-02T00:29:15.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.289+0000 7f9b9e7fc640 1 -- 172.21.15.46:0/4199151411 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b94030d30 con 0x7f9ba015e730 2023-12-02T00:29:15.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.289+0000 7f9ba5e6b640 1 -- 172.21.15.46:0/4199151411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ba015e730 msgr2=0x7f9ba00b1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:15.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.289+0000 7f9ba5e6b640 1 --2- 172.21.15.46:0/4199151411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ba015e730 0x7f9ba00b1230 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9b94009870 tx=0x7f9b9402b980 comp rx=0 tx=0).stop 2023-12-02T00:29:15.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.289+0000 7f9ba5e6b640 1 -- 172.21.15.46:0/4199151411 shutdown_connections 2023-12-02T00:29:15.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.289+0000 7f9ba5e6b640 1 --2- 172.21.15.46:0/4199151411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ba015e730 0x7f9ba00b1230 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:15.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.289+0000 7f9ba5e6b640 1 --2- 172.21.15.46:0/4199151411 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9ba0143490 0x7f9ba00b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:15.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.289+0000 7f9ba5e6b640 1 -- 172.21.15.46:0/4199151411 >> 172.21.15.46:0/4199151411 conn(0x7f9ba006a4d0 msgr2=0x7f9ba006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:15.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.289+0000 7f9ba5e6b640 1 -- 172.21.15.46:0/4199151411 shutdown_connections 2023-12-02T00:29:15.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.289+0000 7f9ba5e6b640 1 -- 172.21.15.46:0/4199151411 wait complete. 2023-12-02T00:29:15.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.290+0000 7f9ba5e6b640 1 Processor -- start 2023-12-02T00:29:15.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.290+0000 7f9ba5e6b640 1 -- start start 2023-12-02T00:29:15.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.291+0000 7f9ba5e6b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ba0143490 0x7f9ba0139510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:15.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.291+0000 7f9ba5e6b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ba015a4e0 0x7f9ba01e38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:15.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.291+0000 7f9ba5e6b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9ba015e730 0x7f9ba013aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:15.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.291+0000 7f9ba5e6b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba0139a50 con 0x7f9ba0143490 2023-12-02T00:29:15.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.291+0000 7f9ba5e6b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba01e3e30 con 0x7f9ba015a4e0 2023-12-02T00:29:15.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.291+0000 7f9ba5e6b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba0158620 con 0x7f9ba015e730 2023-12-02T00:29:15.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.291+0000 7f9b9f7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ba0143490 0x7f9ba0139510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:15.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.291+0000 7f9b9f7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ba0143490 0x7f9ba0139510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44644/0 (socket says 172.21.15.46:44644) 2023-12-02T00:29:15.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.291+0000 7f9b9f7fe640 1 -- 172.21.15.46:0/2114222802 learned_addr learned my addr 172.21.15.46:0/2114222802 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:15.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.291+0000 7f9b9effd640 1 --2- 172.21.15.46:0/2114222802 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ba015a4e0 0x7f9ba01e38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:15.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.291+0000 7f9b9ffff640 1 --2- 172.21.15.46:0/2114222802 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9ba015e730 0x7f9ba013aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:15.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.292+0000 7f9b9f7fe640 1 -- 172.21.15.46:0/2114222802 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9ba015e730 msgr2=0x7f9ba013aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:15.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.292+0000 7f9b9f7fe640 1 --2- 172.21.15.46:0/2114222802 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9ba015e730 0x7f9ba013aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:15.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.292+0000 7f9b9f7fe640 1 -- 172.21.15.46:0/2114222802 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ba015a4e0 msgr2=0x7f9ba01e38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:15.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.292+0000 7f9b9f7fe640 1 --2- 172.21.15.46:0/2114222802 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ba015a4e0 0x7f9ba01e38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:15.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.292+0000 7f9b9f7fe640 1 -- 172.21.15.46:0/2114222802 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b940094d0 con 0x7f9ba0143490 2023-12-02T00:29:15.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.292+0000 7f9b9f7fe640 1 --2- 172.21.15.46:0/2114222802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ba0143490 0x7f9ba0139510 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9b9000e9b0 tx=0x7f9b9000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:15.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.292+0000 7f9b87fff640 1 -- 172.21.15.46:0/2114222802 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b90009670 con 0x7f9ba0143490 2023-12-02T00:29:15.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.292+0000 7f9b87fff640 1 -- 172.21.15.46:0/2114222802 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b9000c9a0 con 0x7f9ba0143490 2023-12-02T00:29:15.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.292+0000 7f9ba5e6b640 1 -- 172.21.15.46:0/2114222802 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ba013d940 con 0x7f9ba0143490 2023-12-02T00:29:15.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.292+0000 7f9b87fff640 1 -- 172.21.15.46:0/2114222802 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9000cd50 con 0x7f9ba0143490 2023-12-02T00:29:15.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.292+0000 7f9ba5e6b640 1 -- 172.21.15.46:0/2114222802 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ba0157fd0 con 0x7f9ba0143490 2023-12-02T00:29:15.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.293+0000 7f9ba5e6b640 1 -- 172.21.15.46:0/2114222802 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ba0069400 con 0x7f9ba0143490 2023-12-02T00:29:15.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.293+0000 7f9b87fff640 1 -- 172.21.15.46:0/2114222802 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9b90013470 con 0x7f9ba0143490 2023-12-02T00:29:15.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.294+0000 7f9b87fff640 1 --2- 172.21.15.46:0/2114222802 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9b8c06ae40 0x7f9b8c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:15.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.294+0000 7f9b87fff640 1 -- 172.21.15.46:0/2114222802 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9b900896e0 con 0x7f9ba0143490 2023-12-02T00:29:15.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.294+0000 7f9b9effd640 1 --2- 172.21.15.46:0/2114222802 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9b8c06ae40 0x7f9b8c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:15.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.294+0000 7f9b9effd640 1 --2- 172.21.15.46:0/2114222802 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9b8c06ae40 0x7f9b8c06d300 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9b94036000 tx=0x7f9b94008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:15.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.298+0000 7f9b87fff640 1 -- 172.21.15.46:0/2114222802 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9b900548d0 con 0x7f9ba0143490 2023-12-02T00:29:15.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.476+0000 7f9ba5e6b640 1 -- 172.21.15.46:0/2114222802 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9ba00b2820 con 0x7f9ba0143490 2023-12-02T00:29:15.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.476+0000 7f9b87fff640 1 -- 172.21.15.46:0/2114222802 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9b90057ef0 con 0x7f9ba0143490 2023-12-02T00:29:15.478 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-12-02T00:29:15.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.478+0000 7f9ba5e6b640 1 -- 172.21.15.46:0/2114222802 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9b8c06ae40 msgr2=0x7f9b8c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:15.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.478+0000 7f9ba5e6b640 1 --2- 172.21.15.46:0/2114222802 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9b8c06ae40 0x7f9b8c06d300 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9b94036000 tx=0x7f9b94008030 comp rx=0 tx=0).stop 2023-12-02T00:29:15.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.478+0000 7f9ba5e6b640 1 -- 172.21.15.46:0/2114222802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ba0143490 msgr2=0x7f9ba0139510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:15.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.478+0000 7f9ba5e6b640 1 --2- 172.21.15.46:0/2114222802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ba0143490 0x7f9ba0139510 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9b9000e9b0 tx=0x7f9b9000ee70 comp rx=0 tx=0).stop 2023-12-02T00:29:15.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.478+0000 7f9ba5e6b640 1 -- 172.21.15.46:0/2114222802 shutdown_connections 2023-12-02T00:29:15.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.478+0000 7f9ba5e6b640 1 --2- 172.21.15.46:0/2114222802 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9b8c06ae40 0x7f9b8c06d300 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:15.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.478+0000 7f9ba5e6b640 1 --2- 172.21.15.46:0/2114222802 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9ba015e730 0x7f9ba013aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:15.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.478+0000 7f9ba5e6b640 1 --2- 172.21.15.46:0/2114222802 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ba015a4e0 0x7f9ba01e38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:15.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.478+0000 7f9ba5e6b640 1 --2- 172.21.15.46:0/2114222802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ba0143490 0x7f9ba0139510 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:15.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.478+0000 7f9ba5e6b640 1 -- 172.21.15.46:0/2114222802 >> 172.21.15.46:0/2114222802 conn(0x7f9ba006a4d0 msgr2=0x7f9ba00ce710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:15.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.478+0000 7f9ba5e6b640 1 -- 172.21.15.46:0/2114222802 shutdown_connections 2023-12-02T00:29:15.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.478+0000 7f9ba5e6b640 1 -- 172.21.15.46:0/2114222802 wait complete. 2023-12-02T00:29:15.490 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-02T00:29:15.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.598+0000 7f92a4525640 1 Processor -- start 2023-12-02T00:29:15.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.598+0000 7f92a4525640 1 -- start start 2023-12-02T00:29:15.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.599+0000 7f92a4525640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f929c143490 0x7f929c0b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:15.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.599+0000 7f92a4525640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f929c15abf0 0x7f929c0b1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:15.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.599+0000 7f92a4525640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f929c15e2a0 con 0x7f929c15e730 2023-12-02T00:29:15.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.599+0000 7f92a4525640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f929c15b520 con 0x7f929c15abf0 2023-12-02T00:29:15.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.599+0000 7f92a4525640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f929c143870 con 0x7f929c143490 2023-12-02T00:29:15.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.600+0000 7f92a2ac2640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f929c15abf0 0x7f929c0b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:15.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.600+0000 7f92a1ac0640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f929c15e730 0x7f929c15a4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38540/0 (socket says 172.21.15.46:38540) 2023-12-02T00:29:15.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.600+0000 7f92a1ac0640 1 -- 172.21.15.46:0/3163218455 learned_addr learned my addr 172.21.15.46:0/3163218455 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:15.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.600+0000 7f92a22c1640 1 --2- 172.21.15.46:0/3163218455 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f929c143490 0x7f929c0b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:15.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.600+0000 7f92a2ac2640 1 -- 172.21.15.46:0/3163218455 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f929c143490 msgr2=0x7f929c0b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:15.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.600+0000 7f92a2ac2640 1 --2- 172.21.15.46:0/3163218455 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f929c143490 0x7f929c0b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:15.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.600+0000 7f92a2ac2640 1 -- 172.21.15.46:0/3163218455 >> v1:172.21.15.46:6789/0 conn(0x7f929c15e730 legacy=0x7f929c15a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:15.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.600+0000 7f92a12bf640 1 -- 172.21.15.46:0/3163218455 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2566310173 0 0) 0x7f929c15e2a0 con 0x7f929c15e730 2023-12-02T00:29:15.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.600+0000 7f92a2ac2640 1 -- 172.21.15.46:0/3163218455 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f929c15b240 con 0x7f929c15abf0 2023-12-02T00:29:15.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.601+0000 7f92a2ac2640 1 --2- 172.21.15.46:0/3163218455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f929c15abf0 0x7f929c0b1230 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9294009d90 tx=0x7f929402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e108c7268a8fa27 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:15.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.601+0000 7f92a12bf640 1 -- 172.21.15.46:0/3163218455 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929402cc50 con 0x7f929c15abf0 2023-12-02T00:29:15.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.601+0000 7f92a12bf640 1 -- 172.21.15.46:0/3163218455 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f929402cdb0 con 0x7f929c15abf0 2023-12-02T00:29:15.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.601+0000 7f92a12bf640 1 -- 172.21.15.46:0/3163218455 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9294034660 con 0x7f929c15abf0 2023-12-02T00:29:15.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.601+0000 7f92a4525640 1 -- 172.21.15.46:0/3163218455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f929c15abf0 msgr2=0x7f929c0b1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:15.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.601+0000 7f92a4525640 1 --2- 172.21.15.46:0/3163218455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f929c15abf0 0x7f929c0b1230 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9294009d90 tx=0x7f929402bea0 comp rx=0 tx=0).stop 2023-12-02T00:29:15.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.601+0000 7f92a4525640 1 -- 172.21.15.46:0/3163218455 shutdown_connections 2023-12-02T00:29:15.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.601+0000 7f92a4525640 1 --2- 172.21.15.46:0/3163218455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f929c15abf0 0x7f929c0b1230 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:15.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.601+0000 7f92a4525640 1 --2- 172.21.15.46:0/3163218455 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f929c143490 0x7f929c0b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:15.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.601+0000 7f92a4525640 1 -- 172.21.15.46:0/3163218455 >> 172.21.15.46:0/3163218455 conn(0x7f929c06a4d0 msgr2=0x7f929c06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:15.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.601+0000 7f92a4525640 1 -- 172.21.15.46:0/3163218455 shutdown_connections 2023-12-02T00:29:15.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.601+0000 7f92a4525640 1 -- 172.21.15.46:0/3163218455 wait complete. 2023-12-02T00:29:15.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.603+0000 7f92a4525640 1 Processor -- start 2023-12-02T00:29:15.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.603+0000 7f92a4525640 1 -- start start 2023-12-02T00:29:15.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.603+0000 7f92a4525640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f929c143490 0x7f929c1420a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:15.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.603+0000 7f92a4525640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f929c15abf0 0x7f929c159250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:15.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.603+0000 7f92a4525640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f929c15e730 0x7f929c13b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:15.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.603+0000 7f92a4525640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929c0ac8e0 con 0x7f929c15e730 2023-12-02T00:29:15.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.603+0000 7f92a4525640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929c15c3c0 con 0x7f929c143490 2023-12-02T00:29:15.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.603+0000 7f92a4525640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f929c15c6a0 con 0x7f929c15abf0 2023-12-02T00:29:15.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.603+0000 7f92a2ac2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f929c15e730 0x7f929c13b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:15.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.603+0000 7f92a2ac2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f929c15e730 0x7f929c13b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44674/0 (socket says 172.21.15.46:44674) 2023-12-02T00:29:15.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.603+0000 7f92a2ac2640 1 -- 172.21.15.46:0/1233327692 learned_addr learned my addr 172.21.15.46:0/1233327692 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:15.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.604+0000 7f92a22c1640 1 --2- 172.21.15.46:0/1233327692 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f929c143490 0x7f929c1420a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:15.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.604+0000 7f92a1ac0640 1 --2- 172.21.15.46:0/1233327692 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f929c15abf0 0x7f929c159250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:15.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.604+0000 7f92a2ac2640 1 -- 172.21.15.46:0/1233327692 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f929c15abf0 msgr2=0x7f929c159250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:15.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.604+0000 7f92a2ac2640 1 --2- 172.21.15.46:0/1233327692 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f929c15abf0 0x7f929c159250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:15.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.604+0000 7f92a2ac2640 1 -- 172.21.15.46:0/1233327692 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f929c143490 msgr2=0x7f929c1420a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:15.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.604+0000 7f92a2ac2640 1 --2- 172.21.15.46:0/1233327692 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f929c143490 0x7f929c1420a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:15.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.604+0000 7f92a2ac2640 1 -- 172.21.15.46:0/1233327692 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9294009a70 con 0x7f929c15e730 2023-12-02T00:29:15.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.604+0000 7f92a22c1640 1 --2- 172.21.15.46:0/1233327692 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f929c143490 0x7f929c1420a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:29:15.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.604+0000 7f92a1ac0640 1 --2- 172.21.15.46:0/1233327692 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f929c15abf0 0x7f929c159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:29:15.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.604+0000 7f92a2ac2640 1 --2- 172.21.15.46:0/1233327692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f929c15e730 0x7f929c13b450 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9294009eb0 tx=0x7f9294004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:15.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.604+0000 7f9292ffd640 1 -- 172.21.15.46:0/1233327692 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9294009450 con 0x7f929c15e730 2023-12-02T00:29:15.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.604+0000 7f9292ffd640 1 -- 172.21.15.46:0/1233327692 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f929403f420 con 0x7f929c15e730 2023-12-02T00:29:15.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.604+0000 7f9292ffd640 1 -- 172.21.15.46:0/1233327692 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929403f790 con 0x7f929c15e730 2023-12-02T00:29:15.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.604+0000 7f92a4525640 1 -- 172.21.15.46:0/1233327692 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f929c15c980 con 0x7f929c15e730 2023-12-02T00:29:15.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.604+0000 7f92a4525640 1 -- 172.21.15.46:0/1233327692 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f929c15c070 con 0x7f929c15e730 2023-12-02T00:29:15.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.605+0000 7f92a4525640 1 -- 172.21.15.46:0/1233327692 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f929c0acb00 con 0x7f929c15e730 2023-12-02T00:29:15.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.606+0000 7f9292ffd640 1 -- 172.21.15.46:0/1233327692 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f929403f8f0 con 0x7f929c15e730 2023-12-02T00:29:15.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.606+0000 7f9292ffd640 1 --2- 172.21.15.46:0/1233327692 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f928006ae40 0x7f928006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:15.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.606+0000 7f9292ffd640 1 -- 172.21.15.46:0/1233327692 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f92940ac220 con 0x7f929c15e730 2023-12-02T00:29:15.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.606+0000 7f92a22c1640 1 --2- 172.21.15.46:0/1233327692 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f928006ae40 0x7f928006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:15.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.607+0000 7f92a22c1640 1 --2- 172.21.15.46:0/1233327692 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f928006ae40 0x7f928006d300 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9298009ac0 tx=0x7f9298008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:15.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.610+0000 7f9292ffd640 1 -- 172.21.15.46:0/1233327692 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f929c0acb00 con 0x7f929c15e730 2023-12-02T00:29:15.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.786+0000 7f92a4525640 1 -- 172.21.15.46:0/1233327692 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f929c0acb00 con 0x7f929c15e730 2023-12-02T00:29:15.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.786+0000 7f9292ffd640 1 -- 172.21.15.46:0/1233327692 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f929c0acb00 con 0x7f929c15e730 2023-12-02T00:29:15.787 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-12-02T00:29:15.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.788+0000 7f92a4525640 1 -- 172.21.15.46:0/1233327692 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f928006ae40 msgr2=0x7f928006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:15.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.788+0000 7f92a4525640 1 --2- 172.21.15.46:0/1233327692 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f928006ae40 0x7f928006d300 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9298009ac0 tx=0x7f9298008040 comp rx=0 tx=0).stop 2023-12-02T00:29:15.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.788+0000 7f92a4525640 1 -- 172.21.15.46:0/1233327692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f929c15e730 msgr2=0x7f929c13b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:15.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.788+0000 7f92a4525640 1 --2- 172.21.15.46:0/1233327692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f929c15e730 0x7f929c13b450 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9294009eb0 tx=0x7f9294004a40 comp rx=0 tx=0).stop 2023-12-02T00:29:15.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.788+0000 7f92a4525640 1 -- 172.21.15.46:0/1233327692 shutdown_connections 2023-12-02T00:29:15.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.788+0000 7f92a4525640 1 --2- 172.21.15.46:0/1233327692 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f928006ae40 0x7f928006d300 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:15.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.788+0000 7f92a4525640 1 --2- 172.21.15.46:0/1233327692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f929c15e730 0x7f929c13b450 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:15.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.788+0000 7f92a4525640 1 --2- 172.21.15.46:0/1233327692 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f929c15abf0 0x7f929c159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:15.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.788+0000 7f92a4525640 1 --2- 172.21.15.46:0/1233327692 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f929c143490 0x7f929c1420a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:15.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.788+0000 7f92a4525640 1 -- 172.21.15.46:0/1233327692 >> 172.21.15.46:0/1233327692 conn(0x7f929c06a4d0 msgr2=0x7f929c15a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:15.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.788+0000 7f92a4525640 1 -- 172.21.15.46:0/1233327692 shutdown_connections 2023-12-02T00:29:15.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.788+0000 7f92a4525640 1 -- 172.21.15.46:0/1233327692 wait complete. 2023-12-02T00:29:15.800 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-02T00:29:15.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.909+0000 7fc5880ec640 1 Processor -- start 2023-12-02T00:29:15.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.909+0000 7fc5880ec640 1 -- start start 2023-12-02T00:29:15.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.910+0000 7fc5880ec640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc580146aa0 0x7fc5800b2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:15.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.910+0000 7fc5880ec640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc5800b4ed0 0x7fc5800b1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:15.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.910+0000 7fc5880ec640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5800b1690 con 0x7fc5800b0cf0 2023-12-02T00:29:15.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.910+0000 7fc5880ec640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc580161bb0 con 0x7fc5800b4ed0 2023-12-02T00:29:15.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.910+0000 7fc5880ec640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc580160fe0 con 0x7fc580146aa0 2023-12-02T00:29:15.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.910+0000 7fc586689640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc5800b4ed0 0x7fc5800b1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:15.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.910+0000 7fc585e88640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc580146aa0 0x7fc5800b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:15.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.910+0000 7fc586689640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc5800b4ed0 0x7fc5800b1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.46:60104/0 (socket says 172.21.15.46:60104) 2023-12-02T00:29:15.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.910+0000 7fc585e88640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc580146aa0 0x7fc5800b2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.46:56900/0 (socket says 172.21.15.46:56900) 2023-12-02T00:29:15.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.910+0000 7fc586689640 1 -- 172.21.15.46:0/47593757 learned_addr learned my addr 172.21.15.46:0/47593757 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:15.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.910+0000 7fc585687640 1 --1- 172.21.15.46:0/47593757 >> v1:172.21.15.162:6789/0 conn(0x7fc5800b0cf0 0x7fc5800ac8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.46:38070/0 (socket says 172.21.15.46:38070) 2023-12-02T00:29:15.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.911+0000 7fc585e88640 1 -- 172.21.15.46:0/47593757 >> v2:172.21.15.162:3300/0 conn(0x7fc5800b4ed0 msgr2=0x7fc5800b1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:15.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.911+0000 7fc585e88640 1 --2- 172.21.15.46:0/47593757 >> v2:172.21.15.162:3300/0 conn(0x7fc5800b4ed0 0x7fc5800b1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:15.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.911+0000 7fc585e88640 1 -- 172.21.15.46:0/47593757 >> v1:172.21.15.162:6789/0 conn(0x7fc5800b0cf0 legacy=0x7fc5800ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:15.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.911+0000 7fc585e88640 1 -- 172.21.15.46:0/47593757 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc580146e80 con 0x7fc580146aa0 2023-12-02T00:29:15.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.911+0000 7fc586689640 1 --2- 172.21.15.46:0/47593757 >> v2:172.21.15.162:3300/0 conn(0x7fc5800b4ed0 0x7fc5800b1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:29:15.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.911+0000 7fc585e88640 1 --2- 172.21.15.46:0/47593757 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc580146aa0 0x7fc5800b2140 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc57c0097a0 tx=0x7fc57c02b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4128af2ea78f715 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:15.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.912+0000 7fc584e86640 1 -- 172.21.15.46:0/47593757 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57c02e070 con 0x7fc580146aa0 2023-12-02T00:29:15.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.912+0000 7fc584e86640 1 -- 172.21.15.46:0/47593757 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc57c0315a0 con 0x7fc580146aa0 2023-12-02T00:29:15.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.912+0000 7fc584e86640 1 -- 172.21.15.46:0/47593757 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57c031930 con 0x7fc580146aa0 2023-12-02T00:29:15.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.912+0000 7fc5880ec640 1 -- 172.21.15.46:0/47593757 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc580146aa0 msgr2=0x7fc5800b2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:15.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.912+0000 7fc5880ec640 1 --2- 172.21.15.46:0/47593757 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc580146aa0 0x7fc5800b2140 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc57c0097a0 tx=0x7fc57c02b840 comp rx=0 tx=0).stop 2023-12-02T00:29:15.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.912+0000 7fc5880ec640 1 -- 172.21.15.46:0/47593757 shutdown_connections 2023-12-02T00:29:15.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.912+0000 7fc5880ec640 1 --2- 172.21.15.46:0/47593757 >> v2:172.21.15.162:3300/0 conn(0x7fc5800b4ed0 0x7fc5800b1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:15.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.912+0000 7fc5880ec640 1 --2- 172.21.15.46:0/47593757 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc580146aa0 0x7fc5800b2140 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:15.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.912+0000 7fc5880ec640 1 -- 172.21.15.46:0/47593757 >> 172.21.15.46:0/47593757 conn(0x7fc58006a4d0 msgr2=0x7fc58006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:15.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.912+0000 7fc5880ec640 1 -- 172.21.15.46:0/47593757 shutdown_connections 2023-12-02T00:29:15.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.912+0000 7fc5880ec640 1 -- 172.21.15.46:0/47593757 wait complete. 2023-12-02T00:29:15.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.913+0000 7fc5880ec640 1 Processor -- start 2023-12-02T00:29:15.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.913+0000 7fc5880ec640 1 -- start start 2023-12-02T00:29:15.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.914+0000 7fc5880ec640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5800b0cf0 0x7fc580159d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:15.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.914+0000 7fc5880ec640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5800b4ed0 0x7fc5800706a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:15.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.914+0000 7fc5880ec640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc580146aa0 0x7fc580070be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:15.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.914+0000 7fc5880ec640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc580158950 con 0x7fc5800b4ed0 2023-12-02T00:29:15.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.914+0000 7fc5880ec640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58015a250 con 0x7fc5800b0cf0 2023-12-02T00:29:15.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.914+0000 7fc5880ec640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5801e8f50 con 0x7fc580146aa0 2023-12-02T00:29:15.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.914+0000 7fc585687640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5800b4ed0 0x7fc5800706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:15.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.914+0000 7fc585687640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5800b4ed0 0x7fc5800706a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44686/0 (socket says 172.21.15.46:44686) 2023-12-02T00:29:15.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.914+0000 7fc585687640 1 -- 172.21.15.46:0/3299402245 learned_addr learned my addr 172.21.15.46:0/3299402245 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:15.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.914+0000 7fc586689640 1 --2- 172.21.15.46:0/3299402245 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc580146aa0 0x7fc580070be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:15.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.914+0000 7fc585e88640 1 --2- 172.21.15.46:0/3299402245 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5800b0cf0 0x7fc580159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:15.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.915+0000 7fc585687640 1 -- 172.21.15.46:0/3299402245 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc580146aa0 msgr2=0x7fc580070be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:15.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.915+0000 7fc585687640 1 --2- 172.21.15.46:0/3299402245 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc580146aa0 0x7fc580070be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:15.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.915+0000 7fc585687640 1 -- 172.21.15.46:0/3299402245 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5800b0cf0 msgr2=0x7fc580159d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:15.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.915+0000 7fc585687640 1 --2- 172.21.15.46:0/3299402245 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5800b0cf0 0x7fc580159d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:15.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.915+0000 7fc585687640 1 -- 172.21.15.46:0/3299402245 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc578009b30 con 0x7fc5800b4ed0 2023-12-02T00:29:15.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.915+0000 7fc585687640 1 --2- 172.21.15.46:0/3299402245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5800b4ed0 0x7fc5800706a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc57000c4d0 tx=0x7fc57000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:15.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.915+0000 7fc5767fc640 1 -- 172.21.15.46:0/3299402245 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc570007480 con 0x7fc5800b4ed0 2023-12-02T00:29:15.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.915+0000 7fc5767fc640 1 -- 172.21.15.46:0/3299402245 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5700075e0 con 0x7fc5800b4ed0 2023-12-02T00:29:15.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.915+0000 7fc5880ec640 1 -- 172.21.15.46:0/3299402245 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc57c009400 con 0x7fc5800b4ed0 2023-12-02T00:29:15.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.915+0000 7fc5767fc640 1 -- 172.21.15.46:0/3299402245 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5700049e0 con 0x7fc5800b4ed0 2023-12-02T00:29:15.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.915+0000 7fc5880ec640 1 -- 172.21.15.46:0/3299402245 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc580158290 con 0x7fc5800b4ed0 2023-12-02T00:29:15.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.916+0000 7fc5880ec640 1 -- 172.21.15.46:0/3299402245 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc580158a90 con 0x7fc5800b4ed0 2023-12-02T00:29:15.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.916+0000 7fc5767fc640 1 -- 172.21.15.46:0/3299402245 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc570004b40 con 0x7fc5800b4ed0 2023-12-02T00:29:15.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.917+0000 7fc5767fc640 1 --2- 172.21.15.46:0/3299402245 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc56406ae40 0x7fc56406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:15.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.917+0000 7fc5767fc640 1 -- 172.21.15.46:0/3299402245 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc570088a30 con 0x7fc5800b4ed0 2023-12-02T00:29:15.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.917+0000 7fc585e88640 1 --2- 172.21.15.46:0/3299402245 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc56406ae40 0x7fc56406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:15.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.918+0000 7fc585e88640 1 --2- 172.21.15.46:0/3299402245 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc56406ae40 0x7fc56406d300 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc57c036000 tx=0x7fc57c03d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:15.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:15.921+0000 7fc5767fc640 1 -- 172.21.15.46:0/3299402245 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc580158a90 con 0x7fc5800b4ed0 2023-12-02T00:29:16.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.098+0000 7fc5880ec640 1 -- 172.21.15.46:0/3299402245 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc580158a90 con 0x7fc5800b4ed0 2023-12-02T00:29:16.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.098+0000 7fc5767fc640 1 -- 172.21.15.46:0/3299402245 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc580158a90 con 0x7fc5800b4ed0 2023-12-02T00:29:16.099 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-12-02T00:29:16.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.100+0000 7fc5880ec640 1 -- 172.21.15.46:0/3299402245 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc56406ae40 msgr2=0x7fc56406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:16.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.100+0000 7fc5880ec640 1 --2- 172.21.15.46:0/3299402245 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc56406ae40 0x7fc56406d300 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc57c036000 tx=0x7fc57c03d040 comp rx=0 tx=0).stop 2023-12-02T00:29:16.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.100+0000 7fc5880ec640 1 -- 172.21.15.46:0/3299402245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5800b4ed0 msgr2=0x7fc5800706a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:16.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.100+0000 7fc5880ec640 1 --2- 172.21.15.46:0/3299402245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5800b4ed0 0x7fc5800706a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc57000c4d0 tx=0x7fc57000c990 comp rx=0 tx=0).stop 2023-12-02T00:29:16.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.100+0000 7fc5880ec640 1 -- 172.21.15.46:0/3299402245 shutdown_connections 2023-12-02T00:29:16.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.100+0000 7fc5880ec640 1 --2- 172.21.15.46:0/3299402245 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc56406ae40 0x7fc56406d300 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:16.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.100+0000 7fc5880ec640 1 --2- 172.21.15.46:0/3299402245 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc580146aa0 0x7fc580070be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:16.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.100+0000 7fc5880ec640 1 --2- 172.21.15.46:0/3299402245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5800b4ed0 0x7fc5800706a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:16.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.100+0000 7fc5880ec640 1 --2- 172.21.15.46:0/3299402245 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5800b0cf0 0x7fc580159d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:16.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.100+0000 7fc5880ec640 1 -- 172.21.15.46:0/3299402245 >> 172.21.15.46:0/3299402245 conn(0x7fc58006a4d0 msgr2=0x7fc58015f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:16.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.100+0000 7fc5880ec640 1 -- 172.21.15.46:0/3299402245 shutdown_connections 2023-12-02T00:29:16.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.100+0000 7fc5880ec640 1 -- 172.21.15.46:0/3299402245 wait complete. 2023-12-02T00:29:16.112 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:29:16.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.218+0000 7f2c133b0640 1 Processor -- start 2023-12-02T00:29:16.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.219+0000 7f2c133b0640 1 -- start start 2023-12-02T00:29:16.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.219+0000 7f2c133b0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2c0c15d330 0x7f2c0c0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:16.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.219+0000 7f2c133b0640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2c0c0d2910 0x7f2c0c0d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:16.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.219+0000 7f2c133b0640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c0c0b2b10 con 0x7f2c0c0d2d90 2023-12-02T00:29:16.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.219+0000 7f2c133b0640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c0c0d1590 con 0x7f2c0c0d2910 2023-12-02T00:29:16.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.219+0000 7f2c133b0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0c0d1870 con 0x7f2c0c15d330 2023-12-02T00:29:16.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.220+0000 7f2c1094b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2c0c0d2910 0x7f2c0c0d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:16.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.220+0000 7f2c1094b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2c0c0d2910 0x7f2c0c0d31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44698/0 (socket says 172.21.15.46:44698) 2023-12-02T00:29:16.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.220+0000 7f2c1094b640 1 -- 172.21.15.46:0/4155917403 learned_addr learned my addr 172.21.15.46:0/4155917403 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:16.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.220+0000 7f2c1114c640 1 --2- 172.21.15.46:0/4155917403 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2c0c15d330 0x7f2c0c0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:16.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.220+0000 7f2c1094b640 1 -- 172.21.15.46:0/4155917403 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2c0c15d330 msgr2=0x7f2c0c0b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:16.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.220+0000 7f2c1094b640 1 --2- 172.21.15.46:0/4155917403 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2c0c15d330 0x7f2c0c0b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:16.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.220+0000 7f2c1094b640 1 -- 172.21.15.46:0/4155917403 >> v1:172.21.15.162:6789/0 conn(0x7f2c0c0d2d90 legacy=0x7f2c0c0b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:16.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.220+0000 7f2c1094b640 1 -- 172.21.15.46:0/4155917403 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c0c0d1b50 con 0x7f2c0c0d2910 2023-12-02T00:29:16.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.221+0000 7f2c1094b640 1 --2- 172.21.15.46:0/4155917403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c0c0d2910 0x7f2c0c0d31f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc009870 tx=0x7f2bfc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7681f66242a761c7 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:16.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.221+0000 7f2c03fff640 1 -- 172.21.15.46:0/4155917403 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc031640 con 0x7f2c0c0d2910 2023-12-02T00:29:16.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.221+0000 7f2c03fff640 1 -- 172.21.15.46:0/4155917403 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bfc0317a0 con 0x7f2c0c0d2910 2023-12-02T00:29:16.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.221+0000 7f2c03fff640 1 -- 172.21.15.46:0/4155917403 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc031b30 con 0x7f2c0c0d2910 2023-12-02T00:29:16.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.221+0000 7f2c133b0640 1 -- 172.21.15.46:0/4155917403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c0c0d2910 msgr2=0x7f2c0c0d31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:16.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.221+0000 7f2c133b0640 1 --2- 172.21.15.46:0/4155917403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c0c0d2910 0x7f2c0c0d31f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc009870 tx=0x7f2bfc02b8f0 comp rx=0 tx=0).stop 2023-12-02T00:29:16.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.221+0000 7f2c133b0640 1 -- 172.21.15.46:0/4155917403 shutdown_connections 2023-12-02T00:29:16.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.221+0000 7f2c133b0640 1 --2- 172.21.15.46:0/4155917403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c0c0d2910 0x7f2c0c0d31f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:16.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.221+0000 7f2c133b0640 1 --2- 172.21.15.46:0/4155917403 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2c0c15d330 0x7f2c0c0b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:16.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.221+0000 7f2c133b0640 1 -- 172.21.15.46:0/4155917403 >> 172.21.15.46:0/4155917403 conn(0x7f2c0c072020 msgr2=0x7f2c0c072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:16.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.221+0000 7f2c133b0640 1 -- 172.21.15.46:0/4155917403 shutdown_connections 2023-12-02T00:29:16.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.221+0000 7f2c133b0640 1 -- 172.21.15.46:0/4155917403 wait complete. 2023-12-02T00:29:16.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.223+0000 7f2c133b0640 1 Processor -- start 2023-12-02T00:29:16.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.223+0000 7f2c133b0640 1 -- start start 2023-12-02T00:29:16.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.223+0000 7f2c133b0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2c0c0d2910 0x7f2c0c14d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:16.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.223+0000 7f2c133b0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c0c0d2d90 0x7f2c0c0cb490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:16.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.223+0000 7f2c133b0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2c0c15d330 0x7f2c0c1f2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:16.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.223+0000 7f2c133b0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0c0cb9d0 con 0x7f2c0c0d2d90 2023-12-02T00:29:16.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.223+0000 7f2c133b0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0c15f810 con 0x7f2c0c0d2910 2023-12-02T00:29:16.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.223+0000 7f2c133b0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0c1f33a0 con 0x7f2c0c15d330 2023-12-02T00:29:16.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.223+0000 7f2c1094b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c0c0d2d90 0x7f2c0c0cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:16.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.223+0000 7f2c1094b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c0c0d2d90 0x7f2c0c0cb490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44710/0 (socket says 172.21.15.46:44710) 2023-12-02T00:29:16.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.223+0000 7f2c1094b640 1 -- 172.21.15.46:0/92223223 learned_addr learned my addr 172.21.15.46:0/92223223 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:16.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.224+0000 7f2c1194d640 1 --2- 172.21.15.46:0/92223223 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2c0c15d330 0x7f2c0c1f2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:16.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.224+0000 7f2c1114c640 1 --2- 172.21.15.46:0/92223223 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2c0c0d2910 0x7f2c0c14d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:16.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.224+0000 7f2c1094b640 1 -- 172.21.15.46:0/92223223 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2c0c15d330 msgr2=0x7f2c0c1f2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:16.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.224+0000 7f2c1094b640 1 --2- 172.21.15.46:0/92223223 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2c0c15d330 0x7f2c0c1f2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:16.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.224+0000 7f2c1094b640 1 -- 172.21.15.46:0/92223223 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2c0c0d2910 msgr2=0x7f2c0c14d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:16.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.224+0000 7f2c1094b640 1 --2- 172.21.15.46:0/92223223 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2c0c0d2910 0x7f2c0c14d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:16.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.224+0000 7f2c1094b640 1 -- 172.21.15.46:0/92223223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bfc0094d0 con 0x7f2c0c0d2d90 2023-12-02T00:29:16.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.224+0000 7f2c1094b640 1 --2- 172.21.15.46:0/92223223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c0c0d2d90 0x7f2c0c0cb490 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc000c00 tx=0x7f2bfc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:16.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.224+0000 7f2c01ffb640 1 -- 172.21.15.46:0/92223223 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc03ac60 con 0x7f2c0c0d2d90 2023-12-02T00:29:16.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.224+0000 7f2c01ffb640 1 -- 172.21.15.46:0/92223223 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bfc03adc0 con 0x7f2c0c0d2d90 2023-12-02T00:29:16.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.224+0000 7f2c133b0640 1 -- 172.21.15.46:0/92223223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c0c15ec00 con 0x7f2c0c0d2d90 2023-12-02T00:29:16.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.224+0000 7f2c01ffb640 1 -- 172.21.15.46:0/92223223 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc042630 con 0x7f2c0c0d2d90 2023-12-02T00:29:16.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.224+0000 7f2c133b0640 1 -- 172.21.15.46:0/92223223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c0c15f1c0 con 0x7f2c0c0d2d90 2023-12-02T00:29:16.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.225+0000 7f2c133b0640 1 -- 172.21.15.46:0/92223223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c0c15f9c0 con 0x7f2c0c0d2d90 2023-12-02T00:29:16.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.226+0000 7f2c01ffb640 1 -- 172.21.15.46:0/92223223 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2bfc02f050 con 0x7f2c0c0d2d90 2023-12-02T00:29:16.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.226+0000 7f2c01ffb640 1 --2- 172.21.15.46:0/92223223 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2bf006ae40 0x7f2bf006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:16.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.226+0000 7f2c01ffb640 1 -- 172.21.15.46:0/92223223 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2bfc0ab080 con 0x7f2c0c0d2d90 2023-12-02T00:29:16.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.226+0000 7f2c1114c640 1 --2- 172.21.15.46:0/92223223 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2bf006ae40 0x7f2bf006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:16.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.227+0000 7f2c1114c640 1 --2- 172.21.15.46:0/92223223 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2bf006ae40 0x7f2bf006d300 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2c08006fd0 tx=0x7f2c08008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:16.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.230+0000 7f2c01ffb640 1 -- 172.21.15.46:0/92223223 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2c0c15f9c0 con 0x7f2c0c0d2d90 2023-12-02T00:29:16.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.410+0000 7f2c133b0640 1 -- 172.21.15.46:0/92223223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2c0c15f9c0 con 0x7f2c0c0d2d90 2023-12-02T00:29:16.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.411+0000 7f2c01ffb640 1 -- 172.21.15.46:0/92223223 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f2c0c15f9c0 con 0x7f2c0c0d2d90 2023-12-02T00:29:16.412 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:29:16.412 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":21,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:28:50.700112+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-02T00:28:47.273337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-02T00:28:48.060115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:29:16.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.413+0000 7f2c133b0640 1 -- 172.21.15.46:0/92223223 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2bf006ae40 msgr2=0x7f2bf006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:16.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.413+0000 7f2c133b0640 1 --2- 172.21.15.46:0/92223223 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2bf006ae40 0x7f2bf006d300 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2c08006fd0 tx=0x7f2c08008040 comp rx=0 tx=0).stop 2023-12-02T00:29:16.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.413+0000 7f2c133b0640 1 -- 172.21.15.46:0/92223223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c0c0d2d90 msgr2=0x7f2c0c0cb490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:16.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.413+0000 7f2c133b0640 1 --2- 172.21.15.46:0/92223223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c0c0d2d90 0x7f2c0c0cb490 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc000c00 tx=0x7f2bfc0046c0 comp rx=0 tx=0).stop 2023-12-02T00:29:16.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.413+0000 7f2c133b0640 1 -- 172.21.15.46:0/92223223 shutdown_connections 2023-12-02T00:29:16.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.413+0000 7f2c133b0640 1 --2- 172.21.15.46:0/92223223 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2bf006ae40 0x7f2bf006d300 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:16.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.413+0000 7f2c133b0640 1 --2- 172.21.15.46:0/92223223 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2c0c15d330 0x7f2c0c1f2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:16.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.413+0000 7f2c133b0640 1 --2- 172.21.15.46:0/92223223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c0c0d2d90 0x7f2c0c0cb490 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:16.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.413+0000 7f2c133b0640 1 --2- 172.21.15.46:0/92223223 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2c0c0d2910 0x7f2c0c14d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:16.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.414+0000 7f2c133b0640 1 -- 172.21.15.46:0/92223223 >> 172.21.15.46:0/92223223 conn(0x7f2c0c072020 msgr2=0x7f2c0c0cf7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:16.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.414+0000 7f2c133b0640 1 -- 172.21.15.46:0/92223223 shutdown_connections 2023-12-02T00:29:16.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.414+0000 7f2c133b0640 1 -- 172.21.15.46:0/92223223 wait complete. 2023-12-02T00:29:16.426 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-02T00:28:37.213645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-02T00:28:47.273337+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-02T00:28:48.060115+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-02T00:29:16.426 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-02T00:29:16.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.534+0000 7f6014a36640 1 Processor -- start 2023-12-02T00:29:16.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.534+0000 7f6014a36640 1 -- start start 2023-12-02T00:29:16.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.535+0000 7f6014a36640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f60100cdee0 0x7f60100cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:16.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.535+0000 7f6014a36640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f601006d580 con 0x7f60100709c0 2023-12-02T00:29:16.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.535+0000 7f6014a36640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60100d0c40 con 0x7f601015d270 2023-12-02T00:29:16.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.535+0000 7f6014a36640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60100ced90 con 0x7f60100cdee0 2023-12-02T00:29:16.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.535+0000 7f600dd9b640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f60100709c0 0x7f60100cd7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38550/0 (socket says 172.21.15.46:38550) 2023-12-02T00:29:16.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.535+0000 7f600dd9b640 1 -- 172.21.15.46:0/162410180 learned_addr learned my addr 172.21.15.46:0/162410180 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:16.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.536+0000 7f600ed9d640 1 --2- 172.21.15.46:0/162410180 >> v2:172.21.15.162:3300/0 conn(0x7f60100cdee0 0x7f60100cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:16.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.536+0000 7f600d59a640 1 -- 172.21.15.46:0/162410180 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1714521162 0 0) 0x7f601006d580 con 0x7f60100709c0 2023-12-02T00:29:16.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.536+0000 7f600d59a640 1 -- 172.21.15.46:0/162410180 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ffc003520 con 0x7f60100709c0 2023-12-02T00:29:16.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.536+0000 7f600d59a640 1 -- 172.21.15.46:0/162410180 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4252951344 0 0) 0x7f5ffc003520 con 0x7f60100709c0 2023-12-02T00:29:16.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.536+0000 7f600ed9d640 1 -- 172.21.15.46:0/162410180 >> v1:172.21.15.162:6789/0 conn(0x7f601015d270 legacy=0x7f601006ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:16.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.536+0000 7f600ed9d640 1 -- 172.21.15.46:0/162410180 >> v1:172.21.15.46:6789/0 conn(0x7f60100709c0 legacy=0x7f60100cd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:16.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.536+0000 7f600ed9d640 1 -- 172.21.15.46:0/162410180 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60100cf070 con 0x7f60100cdee0 2023-12-02T00:29:16.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.537+0000 7f600ed9d640 1 --2- 172.21.15.46:0/162410180 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60100cdee0 0x7f60100cc6a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6000009d90 tx=0x7f600002bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16c0ee4fcdb82c1e server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:16.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.537+0000 7f600d59a640 1 -- 172.21.15.46:0/162410180 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff8001670 con 0x7f60100cdee0 2023-12-02T00:29:16.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.537+0000 7f600d59a640 1 -- 172.21.15.46:0/162410180 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f600002c8b0 con 0x7f60100cdee0 2023-12-02T00:29:16.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.537+0000 7f600d59a640 1 -- 172.21.15.46:0/162410180 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600002cc20 con 0x7f60100cdee0 2023-12-02T00:29:16.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.537+0000 7f6014a36640 1 -- 172.21.15.46:0/162410180 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60100cdee0 msgr2=0x7f60100cc6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:16.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.537+0000 7f6014a36640 1 --2- 172.21.15.46:0/162410180 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60100cdee0 0x7f60100cc6a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6000009d90 tx=0x7f600002bd80 comp rx=0 tx=0).stop 2023-12-02T00:29:16.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.538+0000 7f6014a36640 1 -- 172.21.15.46:0/162410180 shutdown_connections 2023-12-02T00:29:16.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.538+0000 7f6014a36640 1 --2- 172.21.15.46:0/162410180 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60100cdee0 0x7f60100cc6a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:16.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.538+0000 7f6014a36640 1 -- 172.21.15.46:0/162410180 >> 172.21.15.46:0/162410180 conn(0x7f60100b2c10 msgr2=0x7f60100b2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:16.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.538+0000 7f6014a36640 1 -- 172.21.15.46:0/162410180 shutdown_connections 2023-12-02T00:29:16.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.538+0000 7f6014a36640 1 -- 172.21.15.46:0/162410180 wait complete. 2023-12-02T00:29:16.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.539+0000 7f6014a36640 1 Processor -- start 2023-12-02T00:29:16.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.539+0000 7f6014a36640 1 -- start start 2023-12-02T00:29:16.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.540+0000 7f6014a36640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60100709c0 0x7f6010151850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:16.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.540+0000 7f6014a36640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60100cdee0 0x7f601015a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:16.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.540+0000 7f6014a36640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f601015d270 0x7f601015a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:16.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.540+0000 7f6014a36640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60100c8110 con 0x7f601015d270 2023-12-02T00:29:16.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.540+0000 7f6014a36640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6010151d90 con 0x7f60100cdee0 2023-12-02T00:29:16.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.540+0000 7f6014a36640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60100b4910 con 0x7f60100709c0 2023-12-02T00:29:16.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.540+0000 7f600ed9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f601015d270 0x7f601015a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:16.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.540+0000 7f600ed9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f601015d270 0x7f601015a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44720/0 (socket says 172.21.15.46:44720) 2023-12-02T00:29:16.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.540+0000 7f600ed9d640 1 -- 172.21.15.46:0/1566824933 learned_addr learned my addr 172.21.15.46:0/1566824933 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:16.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.540+0000 7f600e59c640 1 --2- 172.21.15.46:0/1566824933 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60100709c0 0x7f6010151850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:16.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.540+0000 7f600dd9b640 1 --2- 172.21.15.46:0/1566824933 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60100cdee0 0x7f601015a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:16.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.540+0000 7f600ed9d640 1 -- 172.21.15.46:0/1566824933 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60100709c0 msgr2=0x7f6010151850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:16.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.540+0000 7f600ed9d640 1 --2- 172.21.15.46:0/1566824933 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60100709c0 0x7f6010151850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:16.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.540+0000 7f600ed9d640 1 -- 172.21.15.46:0/1566824933 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60100cdee0 msgr2=0x7f601015a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:16.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.540+0000 7f600ed9d640 1 --2- 172.21.15.46:0/1566824933 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60100cdee0 0x7f601015a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:16.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.540+0000 7f600ed9d640 1 -- 172.21.15.46:0/1566824933 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6000009a70 con 0x7f601015d270 2023-12-02T00:29:16.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.540+0000 7f600ed9d640 1 --2- 172.21.15.46:0/1566824933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f601015d270 0x7f601015a5f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6000009eb0 tx=0x7f6000004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:16.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.540+0000 7f6006ffd640 1 -- 172.21.15.46:0/1566824933 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600002cea0 con 0x7f601015d270 2023-12-02T00:29:16.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.540+0000 7f6006ffd640 1 -- 172.21.15.46:0/1566824933 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6000009450 con 0x7f601015d270 2023-12-02T00:29:16.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.540+0000 7f6014a36640 1 -- 172.21.15.46:0/1566824933 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60100b4bf0 con 0x7f601015d270 2023-12-02T00:29:16.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.540+0000 7f6006ffd640 1 -- 172.21.15.46:0/1566824933 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600003f420 con 0x7f601015d270 2023-12-02T00:29:16.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.541+0000 7f6014a36640 1 -- 172.21.15.46:0/1566824933 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60100c8640 con 0x7f601015d270 2023-12-02T00:29:16.545 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.542+0000 7f6014a36640 1 -- 172.21.15.46:0/1566824933 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6010067fc0 con 0x7f601015d270 2023-12-02T00:29:16.545 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.542+0000 7f6006ffd640 1 -- 172.21.15.46:0/1566824933 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f600003f580 con 0x7f601015d270 2023-12-02T00:29:16.545 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.542+0000 7f6006ffd640 1 --2- 172.21.15.46:0/1566824933 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5ff006ae40 0x7f5ff006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:16.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.543+0000 7f6006ffd640 1 -- 172.21.15.46:0/1566824933 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f60000abc20 con 0x7f601015d270 2023-12-02T00:29:16.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.543+0000 7f600e59c640 1 --2- 172.21.15.46:0/1566824933 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5ff006ae40 0x7f5ff006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:16.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.543+0000 7f600e59c640 1 --2- 172.21.15.46:0/1566824933 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5ff006ae40 0x7f5ff006d300 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6008000f60 tx=0x7f6008009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:16.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.547+0000 7f6006ffd640 1 -- 172.21.15.46:0/1566824933 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6000076de0 con 0x7f601015d270 2023-12-02T00:29:16.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.720+0000 7f6014a36640 1 -- 172.21.15.46:0/1566824933 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6010067420 con 0x7f601015d270 2023-12-02T00:29:16.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.721+0000 7f6006ffd640 1 -- 172.21.15.46:0/1566824933 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f60000390a0 con 0x7f601015d270 2023-12-02T00:29:16.722 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-12-02T00:29:16.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.723+0000 7f6014a36640 1 -- 172.21.15.46:0/1566824933 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5ff006ae40 msgr2=0x7f5ff006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:16.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.723+0000 7f6014a36640 1 --2- 172.21.15.46:0/1566824933 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5ff006ae40 0x7f5ff006d300 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6008000f60 tx=0x7f6008009040 comp rx=0 tx=0).stop 2023-12-02T00:29:16.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.723+0000 7f6014a36640 1 -- 172.21.15.46:0/1566824933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f601015d270 msgr2=0x7f601015a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:16.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.723+0000 7f6014a36640 1 --2- 172.21.15.46:0/1566824933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f601015d270 0x7f601015a5f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6000009eb0 tx=0x7f6000004a40 comp rx=0 tx=0).stop 2023-12-02T00:29:16.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.723+0000 7f6014a36640 1 -- 172.21.15.46:0/1566824933 shutdown_connections 2023-12-02T00:29:16.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.723+0000 7f6014a36640 1 --2- 172.21.15.46:0/1566824933 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5ff006ae40 0x7f5ff006d300 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:16.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.723+0000 7f6014a36640 1 --2- 172.21.15.46:0/1566824933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f601015d270 0x7f601015a5f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:16.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.723+0000 7f6014a36640 1 --2- 172.21.15.46:0/1566824933 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60100cdee0 0x7f601015a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:16.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.723+0000 7f6014a36640 1 --2- 172.21.15.46:0/1566824933 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60100709c0 0x7f6010151850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:16.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.723+0000 7f6014a36640 1 -- 172.21.15.46:0/1566824933 >> 172.21.15.46:0/1566824933 conn(0x7f60100b2c10 msgr2=0x7f601006b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:16.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.723+0000 7f6014a36640 1 -- 172.21.15.46:0/1566824933 shutdown_connections 2023-12-02T00:29:16.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.723+0000 7f6014a36640 1 -- 172.21.15.46:0/1566824933 wait complete. 2023-12-02T00:29:16.736 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-02T00:29:16.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.845+0000 7fc4757a0640 1 Processor -- start 2023-12-02T00:29:16.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.845+0000 7fc4757a0640 1 -- start start 2023-12-02T00:29:16.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.846+0000 7fc4757a0640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc47015d1b0 0x7fc4700c92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:16.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.846+0000 7fc4757a0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc4700d14f0 0x7fc4700ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:16.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.846+0000 7fc4757a0640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4700d1a20 con 0x7fc4700ceaf0 2023-12-02T00:29:16.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.846+0000 7fc4757a0640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4700d1d00 con 0x7fc47015d1b0 2023-12-02T00:29:16.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.846+0000 7fc4757a0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4700cddb0 con 0x7fc4700d14f0 2023-12-02T00:29:16.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.846+0000 7fc46ffff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc47015d1b0 0x7fc4700c92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:16.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.846+0000 7fc46ffff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc47015d1b0 0x7fc4700c92f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44734/0 (socket says 172.21.15.46:44734) 2023-12-02T00:29:16.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.846+0000 7fc474f9f640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc4700ceaf0 0x7fc4700cefa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38558/0 (socket says 172.21.15.46:38558) 2023-12-02T00:29:16.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.846+0000 7fc46ffff640 1 -- 172.21.15.46:0/845175771 learned_addr learned my addr 172.21.15.46:0/845175771 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:16.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.846+0000 7fc46f7fe640 1 --2- 172.21.15.46:0/845175771 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc4700d14f0 0x7fc4700ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:16.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.846+0000 7fc46ffff640 1 -- 172.21.15.46:0/845175771 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc4700d14f0 msgr2=0x7fc4700ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:16.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.846+0000 7fc46ffff640 1 --2- 172.21.15.46:0/845175771 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc4700d14f0 0x7fc4700ce5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:16.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.846+0000 7fc46ffff640 1 -- 172.21.15.46:0/845175771 >> v1:172.21.15.46:6789/0 conn(0x7fc4700ceaf0 legacy=0x7fc4700cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:16.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.846+0000 7fc46ffff640 1 -- 172.21.15.46:0/845175771 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4700ce090 con 0x7fc47015d1b0 2023-12-02T00:29:16.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.846+0000 7fc46effd640 1 -- 172.21.15.46:0/845175771 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3503130472 0 0) 0x7fc4700d1a20 con 0x7fc4700ceaf0 2023-12-02T00:29:16.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.846+0000 7fc46f7fe640 1 --2- 172.21.15.46:0/845175771 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc4700d14f0 0x7fc4700ce5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:29:16.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.847+0000 7fc46ffff640 1 --2- 172.21.15.46:0/845175771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc47015d1b0 0x7fc4700c92f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc464009870 tx=0x7fc46402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a59550e3459041a7 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:16.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.847+0000 7fc46effd640 1 -- 172.21.15.46:0/845175771 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc464030840 con 0x7fc47015d1b0 2023-12-02T00:29:16.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.847+0000 7fc46effd640 1 -- 172.21.15.46:0/845175771 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4640309a0 con 0x7fc47015d1b0 2023-12-02T00:29:16.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.847+0000 7fc46effd640 1 -- 172.21.15.46:0/845175771 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc464030d30 con 0x7fc47015d1b0 2023-12-02T00:29:16.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.848+0000 7fc4757a0640 1 -- 172.21.15.46:0/845175771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc47015d1b0 msgr2=0x7fc4700c92f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:16.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.848+0000 7fc4757a0640 1 --2- 172.21.15.46:0/845175771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc47015d1b0 0x7fc4700c92f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc464009870 tx=0x7fc46402b980 comp rx=0 tx=0).stop 2023-12-02T00:29:16.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.848+0000 7fc4757a0640 1 -- 172.21.15.46:0/845175771 shutdown_connections 2023-12-02T00:29:16.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.848+0000 7fc4757a0640 1 --2- 172.21.15.46:0/845175771 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc4700d14f0 0x7fc4700ce5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:16.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.848+0000 7fc4757a0640 1 --2- 172.21.15.46:0/845175771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc47015d1b0 0x7fc4700c92f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:16.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.848+0000 7fc4757a0640 1 -- 172.21.15.46:0/845175771 >> 172.21.15.46:0/845175771 conn(0x7fc47009c300 msgr2=0x7fc4700d2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:16.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.848+0000 7fc4757a0640 1 -- 172.21.15.46:0/845175771 shutdown_connections 2023-12-02T00:29:16.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.848+0000 7fc4757a0640 1 -- 172.21.15.46:0/845175771 wait complete. 2023-12-02T00:29:16.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.849+0000 7fc4757a0640 1 Processor -- start 2023-12-02T00:29:16.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.849+0000 7fc4757a0640 1 -- start start 2023-12-02T00:29:16.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.850+0000 7fc4757a0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc4700ceaf0 0x7fc4701532a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:16.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.850+0000 7fc4757a0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4700d14f0 0x7fc4700cb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:16.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.850+0000 7fc4757a0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc47015d1b0 0x7fc4701431f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:16.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.850+0000 7fc4757a0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4700cba30 con 0x7fc47015d1b0 2023-12-02T00:29:16.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.850+0000 7fc4757a0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4700cbd10 con 0x7fc4700d14f0 2023-12-02T00:29:16.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.850+0000 7fc4757a0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4700cbff0 con 0x7fc4700ceaf0 2023-12-02T00:29:16.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.850+0000 7fc474f9f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc47015d1b0 0x7fc4701431f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:16.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.850+0000 7fc474f9f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc47015d1b0 0x7fc4701431f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44738/0 (socket says 172.21.15.46:44738) 2023-12-02T00:29:16.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.850+0000 7fc474f9f640 1 -- 172.21.15.46:0/2543971080 learned_addr learned my addr 172.21.15.46:0/2543971080 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:16.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.850+0000 7fc46f7fe640 1 --2- 172.21.15.46:0/2543971080 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4700d14f0 0x7fc4700cb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:16.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.850+0000 7fc46ffff640 1 --2- 172.21.15.46:0/2543971080 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc4700ceaf0 0x7fc4701532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:16.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.850+0000 7fc474f9f640 1 -- 172.21.15.46:0/2543971080 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc4700ceaf0 msgr2=0x7fc4701532a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:16.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.850+0000 7fc474f9f640 1 --2- 172.21.15.46:0/2543971080 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc4700ceaf0 0x7fc4701532a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:16.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.850+0000 7fc474f9f640 1 -- 172.21.15.46:0/2543971080 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4700d14f0 msgr2=0x7fc4700cb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:16.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.850+0000 7fc474f9f640 1 --2- 172.21.15.46:0/2543971080 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4700d14f0 0x7fc4700cb0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:16.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.850+0000 7fc474f9f640 1 -- 172.21.15.46:0/2543971080 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4640094d0 con 0x7fc47015d1b0 2023-12-02T00:29:16.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.850+0000 7fc46f7fe640 1 --2- 172.21.15.46:0/2543971080 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4700d14f0 0x7fc4700cb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:29:16.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.851+0000 7fc474f9f640 1 --2- 172.21.15.46:0/2543971080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc47015d1b0 0x7fc4701431f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc46800c780 tx=0x7fc46800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:16.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.851+0000 7fc46cff9640 1 -- 172.21.15.46:0/2543971080 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46800ce30 con 0x7fc47015d1b0 2023-12-02T00:29:16.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.851+0000 7fc46cff9640 1 -- 172.21.15.46:0/2543971080 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc46800a640 con 0x7fc47015d1b0 2023-12-02T00:29:16.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.851+0000 7fc4757a0640 1 -- 172.21.15.46:0/2543971080 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4700cc2d0 con 0x7fc47015d1b0 2023-12-02T00:29:16.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.851+0000 7fc46cff9640 1 -- 172.21.15.46:0/2543971080 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46800aa70 con 0x7fc47015d1b0 2023-12-02T00:29:16.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.851+0000 7fc4757a0640 1 -- 172.21.15.46:0/2543971080 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4701537e0 con 0x7fc47015d1b0 2023-12-02T00:29:16.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.852+0000 7fc4757a0640 1 -- 172.21.15.46:0/2543971080 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4700ccad0 con 0x7fc47015d1b0 2023-12-02T00:29:16.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.852+0000 7fc46cff9640 1 -- 172.21.15.46:0/2543971080 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc46800abd0 con 0x7fc47015d1b0 2023-12-02T00:29:16.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.853+0000 7fc46cff9640 1 --2- 172.21.15.46:0/2543971080 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc45c06ae40 0x7fc45c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:16.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.853+0000 7fc46cff9640 1 -- 172.21.15.46:0/2543971080 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc468088b30 con 0x7fc47015d1b0 2023-12-02T00:29:16.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.853+0000 7fc46ffff640 1 --2- 172.21.15.46:0/2543971080 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc45c06ae40 0x7fc45c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:16.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.853+0000 7fc46ffff640 1 --2- 172.21.15.46:0/2543971080 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc45c06ae40 0x7fc45c06d300 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc464036000 tx=0x7fc46403b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:16.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:16.857+0000 7fc46cff9640 1 -- 172.21.15.46:0/2543971080 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc468053b50 con 0x7fc47015d1b0 2023-12-02T00:29:17.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.033+0000 7fc4757a0640 1 -- 172.21.15.46:0/2543971080 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc4700cc7f0 con 0x7fc47015d1b0 2023-12-02T00:29:17.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.034+0000 7fc46cff9640 1 -- 172.21.15.46:0/2543971080 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc4700cc7f0 con 0x7fc47015d1b0 2023-12-02T00:29:17.035 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-12-02T00:29:17.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.035+0000 7fc4757a0640 1 -- 172.21.15.46:0/2543971080 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc45c06ae40 msgr2=0x7fc45c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:17.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.035+0000 7fc4757a0640 1 --2- 172.21.15.46:0/2543971080 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc45c06ae40 0x7fc45c06d300 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc464036000 tx=0x7fc46403b040 comp rx=0 tx=0).stop 2023-12-02T00:29:17.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.035+0000 7fc4757a0640 1 -- 172.21.15.46:0/2543971080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc47015d1b0 msgr2=0x7fc4701431f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:17.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.035+0000 7fc4757a0640 1 --2- 172.21.15.46:0/2543971080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc47015d1b0 0x7fc4701431f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc46800c780 tx=0x7fc46800cc40 comp rx=0 tx=0).stop 2023-12-02T00:29:17.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.036+0000 7fc4757a0640 1 -- 172.21.15.46:0/2543971080 shutdown_connections 2023-12-02T00:29:17.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.036+0000 7fc4757a0640 1 --2- 172.21.15.46:0/2543971080 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc45c06ae40 0x7fc45c06d300 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:17.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.036+0000 7fc4757a0640 1 --2- 172.21.15.46:0/2543971080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc47015d1b0 0x7fc4701431f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:17.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.036+0000 7fc4757a0640 1 --2- 172.21.15.46:0/2543971080 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4700d14f0 0x7fc4700cb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:17.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.036+0000 7fc4757a0640 1 --2- 172.21.15.46:0/2543971080 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc4700ceaf0 0x7fc4701532a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:17.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.036+0000 7fc4757a0640 1 -- 172.21.15.46:0/2543971080 >> 172.21.15.46:0/2543971080 conn(0x7fc47009c300 msgr2=0x7fc47015f820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:17.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.036+0000 7fc4757a0640 1 -- 172.21.15.46:0/2543971080 shutdown_connections 2023-12-02T00:29:17.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.036+0000 7fc4757a0640 1 -- 172.21.15.46:0/2543971080 wait complete. 2023-12-02T00:29:17.049 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-02T00:29:17.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.157+0000 7f7e96d41640 1 Processor -- start 2023-12-02T00:29:17.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.157+0000 7f7e96d41640 1 -- start start 2023-12-02T00:29:17.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.158+0000 7f7e96d41640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e9015d2d0 0x7f7e900b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:17.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.158+0000 7f7e96d41640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7e900ce650 0x7f7e900d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:17.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.158+0000 7f7e96d41640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e901614e0 con 0x7f7e90160f70 2023-12-02T00:29:17.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.158+0000 7f7e96d41640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e901617c0 con 0x7f7e900ce650 2023-12-02T00:29:17.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.158+0000 7f7e96d41640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e90161aa0 con 0x7f7e9015d2d0 2023-12-02T00:29:17.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.158+0000 7f7e8ffff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7e90160f70 0x7f7e900cdf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38568/0 (socket says 172.21.15.46:38568) 2023-12-02T00:29:17.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.158+0000 7f7e8ffff640 1 -- 172.21.15.46:0/1154563136 learned_addr learned my addr 172.21.15.46:0/1154563136 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:17.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.158+0000 7f7e94add640 1 --2- 172.21.15.46:0/1154563136 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e9015d2d0 0x7f7e900b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:17.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.158+0000 7f7e952de640 1 --2- 172.21.15.46:0/1154563136 >> v2:172.21.15.162:3300/0 conn(0x7f7e900ce650 0x7f7e900d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:17.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.158+0000 7f7e8f7fe640 1 -- 172.21.15.46:0/1154563136 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3504627928 0 0) 0x7f7e901614e0 con 0x7f7e90160f70 2023-12-02T00:29:17.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.158+0000 7f7e8f7fe640 1 -- 172.21.15.46:0/1154563136 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e7c003520 con 0x7f7e90160f70 2023-12-02T00:29:17.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.159+0000 7f7e94add640 1 -- 172.21.15.46:0/1154563136 >> v2:172.21.15.162:3300/0 conn(0x7f7e900ce650 msgr2=0x7f7e900d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:17.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.159+0000 7f7e94add640 1 --2- 172.21.15.46:0/1154563136 >> v2:172.21.15.162:3300/0 conn(0x7f7e900ce650 0x7f7e900d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:17.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.159+0000 7f7e94add640 1 -- 172.21.15.46:0/1154563136 >> v1:172.21.15.46:6789/0 conn(0x7f7e90160f70 legacy=0x7f7e900cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:17.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.159+0000 7f7e94add640 1 -- 172.21.15.46:0/1154563136 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e90161d80 con 0x7f7e9015d2d0 2023-12-02T00:29:17.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.159+0000 7f7e952de640 1 --2- 172.21.15.46:0/1154563136 >> v2:172.21.15.162:3300/0 conn(0x7f7e900ce650 0x7f7e900d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:29:17.162 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.159+0000 7f7e94add640 1 --2- 172.21.15.46:0/1154563136 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e9015d2d0 0x7f7e900b8b90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7e880097a0 tx=0x7f7e8802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=675d099efcd2d73b server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:17.162 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.160+0000 7f7e8f7fe640 1 -- 172.21.15.46:0/1154563136 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e88031830 con 0x7f7e9015d2d0 2023-12-02T00:29:17.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.160+0000 7f7e8f7fe640 1 -- 172.21.15.46:0/1154563136 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e88031990 con 0x7f7e9015d2d0 2023-12-02T00:29:17.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.160+0000 7f7e8f7fe640 1 -- 172.21.15.46:0/1154563136 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e88031d20 con 0x7f7e9015d2d0 2023-12-02T00:29:17.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.160+0000 7f7e96d41640 1 -- 172.21.15.46:0/1154563136 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e9015d2d0 msgr2=0x7f7e900b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:17.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.160+0000 7f7e96d41640 1 --2- 172.21.15.46:0/1154563136 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e9015d2d0 0x7f7e900b8b90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7e880097a0 tx=0x7f7e8802b8d0 comp rx=0 tx=0).stop 2023-12-02T00:29:17.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.160+0000 7f7e96d41640 1 -- 172.21.15.46:0/1154563136 shutdown_connections 2023-12-02T00:29:17.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.160+0000 7f7e96d41640 1 --2- 172.21.15.46:0/1154563136 >> v2:172.21.15.162:3300/0 conn(0x7f7e900ce650 0x7f7e900d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:17.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.160+0000 7f7e96d41640 1 --2- 172.21.15.46:0/1154563136 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e9015d2d0 0x7f7e900b8b90 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:17.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.160+0000 7f7e96d41640 1 -- 172.21.15.46:0/1154563136 >> 172.21.15.46:0/1154563136 conn(0x7f7e90072230 msgr2=0x7f7e900cfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:17.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.160+0000 7f7e96d41640 1 -- 172.21.15.46:0/1154563136 shutdown_connections 2023-12-02T00:29:17.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.160+0000 7f7e96d41640 1 -- 172.21.15.46:0/1154563136 wait complete. 2023-12-02T00:29:17.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.161+0000 7f7e96d41640 1 Processor -- start 2023-12-02T00:29:17.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.161+0000 7f7e96d41640 1 -- start start 2023-12-02T00:29:17.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.162+0000 7f7e96d41640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e900ce650 0x7f7e90147de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:17.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.162+0000 7f7e96d41640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e9015d2d0 0x7f7e900b63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:17.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.162+0000 7f7e96d41640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7e90160f70 0x7f7e900b6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:17.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.162+0000 7f7e96d41640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9014a000 con 0x7f7e9015d2d0 2023-12-02T00:29:17.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.162+0000 7f7e96d41640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e900cd260 con 0x7f7e90160f70 2023-12-02T00:29:17.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.162+0000 7f7e96d41640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e90149380 con 0x7f7e900ce650 2023-12-02T00:29:17.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.162+0000 7f7e8ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e9015d2d0 0x7f7e900b63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:17.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.162+0000 7f7e8ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e9015d2d0 0x7f7e900b63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44754/0 (socket says 172.21.15.46:44754) 2023-12-02T00:29:17.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.162+0000 7f7e8ffff640 1 -- 172.21.15.46:0/1688671430 learned_addr learned my addr 172.21.15.46:0/1688671430 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:17.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.163+0000 7f7e952de640 1 --2- 172.21.15.46:0/1688671430 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7e90160f70 0x7f7e900b6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:17.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.163+0000 7f7e94add640 1 --2- 172.21.15.46:0/1688671430 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e900ce650 0x7f7e90147de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:17.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.163+0000 7f7e8ffff640 1 -- 172.21.15.46:0/1688671430 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e900ce650 msgr2=0x7f7e90147de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:17.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.163+0000 7f7e8ffff640 1 --2- 172.21.15.46:0/1688671430 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e900ce650 0x7f7e90147de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:17.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.163+0000 7f7e8ffff640 1 -- 172.21.15.46:0/1688671430 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7e90160f70 msgr2=0x7f7e900b6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:17.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.163+0000 7f7e8ffff640 1 --2- 172.21.15.46:0/1688671430 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7e90160f70 0x7f7e900b6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:17.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.163+0000 7f7e8ffff640 1 -- 172.21.15.46:0/1688671430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e78009400 con 0x7f7e9015d2d0 2023-12-02T00:29:17.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.163+0000 7f7e8ffff640 1 --2- 172.21.15.46:0/1688671430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e9015d2d0 0x7f7e900b63c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7e8000cbc0 tx=0x7f7e800021c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:17.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.163+0000 7f7e8d7fa640 1 -- 172.21.15.46:0/1688671430 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8000ac00 con 0x7f7e9015d2d0 2023-12-02T00:29:17.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.163+0000 7f7e8d7fa640 1 -- 172.21.15.46:0/1688671430 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e8000ad60 con 0x7f7e9015d2d0 2023-12-02T00:29:17.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.163+0000 7f7e96d41640 1 -- 172.21.15.46:0/1688671430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e88009400 con 0x7f7e9015d2d0 2023-12-02T00:29:17.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.163+0000 7f7e8d7fa640 1 -- 172.21.15.46:0/1688671430 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e80012620 con 0x7f7e9015d2d0 2023-12-02T00:29:17.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.163+0000 7f7e96d41640 1 -- 172.21.15.46:0/1688671430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e90149940 con 0x7f7e9015d2d0 2023-12-02T00:29:17.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.164+0000 7f7e96d41640 1 -- 172.21.15.46:0/1688671430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e9014a140 con 0x7f7e9015d2d0 2023-12-02T00:29:17.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.165+0000 7f7e8d7fa640 1 -- 172.21.15.46:0/1688671430 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7e80007070 con 0x7f7e9015d2d0 2023-12-02T00:29:17.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.165+0000 7f7e8d7fa640 1 --2- 172.21.15.46:0/1688671430 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7e7006ae40 0x7f7e7006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:17.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.165+0000 7f7e8d7fa640 1 -- 172.21.15.46:0/1688671430 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7e80089460 con 0x7f7e9015d2d0 2023-12-02T00:29:17.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.165+0000 7f7e94add640 1 --2- 172.21.15.46:0/1688671430 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7e7006ae40 0x7f7e7006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:17.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.166+0000 7f7e94add640 1 --2- 172.21.15.46:0/1688671430 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7e7006ae40 0x7f7e7006d300 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7e88036000 tx=0x7f7e8803c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:17.170 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.169+0000 7f7e8d7fa640 1 -- 172.21.15.46:0/1688671430 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7e9014a140 con 0x7f7e9015d2d0 2023-12-02T00:29:17.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.346+0000 7f7e96d41640 1 -- 172.21.15.46:0/1688671430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7e9014a140 con 0x7f7e9015d2d0 2023-12-02T00:29:17.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.347+0000 7f7e8d7fa640 1 -- 172.21.15.46:0/1688671430 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7e80017020 con 0x7f7e9015d2d0 2023-12-02T00:29:17.348 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-12-02T00:29:17.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.349+0000 7f7e96d41640 1 -- 172.21.15.46:0/1688671430 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7e7006ae40 msgr2=0x7f7e7006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:17.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.349+0000 7f7e96d41640 1 --2- 172.21.15.46:0/1688671430 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7e7006ae40 0x7f7e7006d300 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7e88036000 tx=0x7f7e8803c040 comp rx=0 tx=0).stop 2023-12-02T00:29:17.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.349+0000 7f7e96d41640 1 -- 172.21.15.46:0/1688671430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e9015d2d0 msgr2=0x7f7e900b63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:17.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.349+0000 7f7e96d41640 1 --2- 172.21.15.46:0/1688671430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e9015d2d0 0x7f7e900b63c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7e8000cbc0 tx=0x7f7e800021c0 comp rx=0 tx=0).stop 2023-12-02T00:29:17.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.349+0000 7f7e96d41640 1 -- 172.21.15.46:0/1688671430 shutdown_connections 2023-12-02T00:29:17.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.349+0000 7f7e96d41640 1 --2- 172.21.15.46:0/1688671430 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7e7006ae40 0x7f7e7006d300 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:17.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.349+0000 7f7e96d41640 1 --2- 172.21.15.46:0/1688671430 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7e90160f70 0x7f7e900b6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:17.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.349+0000 7f7e96d41640 1 --2- 172.21.15.46:0/1688671430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e9015d2d0 0x7f7e900b63c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:17.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.349+0000 7f7e96d41640 1 --2- 172.21.15.46:0/1688671430 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e900ce650 0x7f7e90147de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:17.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.349+0000 7f7e96d41640 1 -- 172.21.15.46:0/1688671430 >> 172.21.15.46:0/1688671430 conn(0x7f7e90072230 msgr2=0x7f7e900b7100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:17.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.349+0000 7f7e96d41640 1 -- 172.21.15.46:0/1688671430 shutdown_connections 2023-12-02T00:29:17.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.349+0000 7f7e96d41640 1 -- 172.21.15.46:0/1688671430 wait complete. 2023-12-02T00:29:17.361 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:29:17.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.468+0000 7fa7ddc6b640 1 Processor -- start 2023-12-02T00:29:17.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.468+0000 7fa7ddc6b640 1 -- start start 2023-12-02T00:29:17.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.469+0000 7fa7ddc6b640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa7d8150560 0x7fa7d80b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:17.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.469+0000 7fa7ddc6b640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7d80b49f0 con 0x7fa7d8150940 2023-12-02T00:29:17.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.469+0000 7fa7ddc6b640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7d80b3de0 con 0x7fa7d80b3390 2023-12-02T00:29:17.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.469+0000 7fa7ddc6b640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7d80b40c0 con 0x7fa7d8150560 2023-12-02T00:29:17.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.469+0000 7fa7d7fff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa7d8150940 0x7fa7d80ce0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38576/0 (socket says 172.21.15.46:38576) 2023-12-02T00:29:17.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.469+0000 7fa7d7fff640 1 -- 172.21.15.46:0/3960399057 learned_addr learned my addr 172.21.15.46:0/3960399057 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:17.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.469+0000 7fa7d67fc640 1 -- 172.21.15.46:0/3960399057 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1118565197 0 0) 0x7fa7d80b49f0 con 0x7fa7d8150940 2023-12-02T00:29:17.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.469+0000 7fa7d77fe640 1 --2- 172.21.15.46:0/3960399057 >> v2:172.21.15.162:3300/0 conn(0x7fa7d8150560 0x7fa7d80b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:17.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.469+0000 7fa7d67fc640 1 -- 172.21.15.46:0/3960399057 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7c4003520 con 0x7fa7d8150940 2023-12-02T00:29:17.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.469+0000 7fa7d67fc640 1 -- 172.21.15.46:0/3960399057 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3011967848 0 0) 0x7fa7c4003520 con 0x7fa7d8150940 2023-12-02T00:29:17.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.469+0000 7fa7d67fc640 1 -- 172.21.15.46:0/3960399057 >> v2:172.21.15.162:3300/0 conn(0x7fa7d8150560 msgr2=0x7fa7d80b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:17.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.469+0000 7fa7d67fc640 1 --2- 172.21.15.46:0/3960399057 >> v2:172.21.15.162:3300/0 conn(0x7fa7d8150560 0x7fa7d80b2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:17.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.469+0000 7fa7d67fc640 1 -- 172.21.15.46:0/3960399057 >> v1:172.21.15.162:6789/0 conn(0x7fa7d80b3390 legacy=0x7fa7d815d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:17.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.469+0000 7fa7d67fc640 1 -- 172.21.15.46:0/3960399057 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d80b43a0 con 0x7fa7d8150940 2023-12-02T00:29:17.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.469+0000 7fa7d67fc640 1 -- 172.21.15.46:0/3960399057 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7fa7d00019f0 con 0x7fa7d8150940 2023-12-02T00:29:17.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.469+0000 7fa7d67fc640 1 -- 172.21.15.46:0/3960399057 >> v1:172.21.15.46:6789/0 conn(0x7fa7d8150940 legacy=0x7fa7d80ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:17.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.469+0000 7fa7d67fc640 1 --2- 172.21.15.46:0/3960399057 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7c4003970 0x7fa7c4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:17.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.469+0000 7fa7d67fc640 1 --2- 172.21.15.46:0/3960399057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7c40047c0 0x7fa7c4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:17.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.469+0000 7fa7d67fc640 1 --2- 172.21.15.46:0/3960399057 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7c4005180 0x7fa7c400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:17.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.469+0000 7fa7d67fc640 1 -- 172.21.15.46:0/3960399057 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d80b40c0 con 0x7fa7c40047c0 2023-12-02T00:29:17.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.469+0000 7fa7d67fc640 1 -- 172.21.15.46:0/3960399057 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c400c020 con 0x7fa7c4003970 2023-12-02T00:29:17.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.469+0000 7fa7d67fc640 1 -- 172.21.15.46:0/3960399057 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c400c1e0 con 0x7fa7c4005180 2023-12-02T00:29:17.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.469+0000 7fa7d6ffd640 1 --2- 172.21.15.46:0/3960399057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7c40047c0 0x7fa7c4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:17.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.470+0000 7fa7d77fe640 1 --2- 172.21.15.46:0/3960399057 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7c4003970 0x7fa7c4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:17.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.470+0000 7fa7d7fff640 1 --2- 172.21.15.46:0/3960399057 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7c4005180 0x7fa7c400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:17.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.470+0000 7fa7d6ffd640 1 -- 172.21.15.46:0/3960399057 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7c4005180 msgr2=0x7fa7c400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:17.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.470+0000 7fa7d6ffd640 1 --2- 172.21.15.46:0/3960399057 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7c4005180 0x7fa7c400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:17.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.470+0000 7fa7d6ffd640 1 -- 172.21.15.46:0/3960399057 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7c4003970 msgr2=0x7fa7c4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:17.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.470+0000 7fa7d6ffd640 1 --2- 172.21.15.46:0/3960399057 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7c4003970 0x7fa7c4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:17.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.470+0000 7fa7d6ffd640 1 -- 172.21.15.46:0/3960399057 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c4003520 con 0x7fa7c40047c0 2023-12-02T00:29:17.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.470+0000 7fa7d7fff640 1 -- 172.21.15.46:0/3960399057 reap_dead start 2023-12-02T00:29:17.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.471+0000 7fa7d6ffd640 1 --2- 172.21.15.46:0/3960399057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7c40047c0 0x7fa7c4004c40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa7c0001750 tx=0x7fa7c40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbf79ab709c12be5 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:17.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.471+0000 7fa7d67fc640 1 -- 172.21.15.46:0/3960399057 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c00052d0 con 0x7fa7c40047c0 2023-12-02T00:29:17.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.471+0000 7fa7d67fc640 1 -- 172.21.15.46:0/3960399057 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c002e050 con 0x7fa7c40047c0 2023-12-02T00:29:17.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.471+0000 7fa7d67fc640 1 -- 172.21.15.46:0/3960399057 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0003830 con 0x7fa7c40047c0 2023-12-02T00:29:17.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.471+0000 7fa7ddc6b640 1 -- 172.21.15.46:0/3960399057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7c40047c0 msgr2=0x7fa7c4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:17.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.471+0000 7fa7ddc6b640 1 --2- 172.21.15.46:0/3960399057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7c40047c0 0x7fa7c4004c40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa7c0001750 tx=0x7fa7c40057f0 comp rx=0 tx=0).stop 2023-12-02T00:29:17.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.471+0000 7fa7ddc6b640 1 -- 172.21.15.46:0/3960399057 shutdown_connections 2023-12-02T00:29:17.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.471+0000 7fa7ddc6b640 1 --2- 172.21.15.46:0/3960399057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7c40047c0 0x7fa7c4004c40 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:17.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.471+0000 7fa7ddc6b640 1 -- 172.21.15.46:0/3960399057 >> 172.21.15.46:0/3960399057 conn(0x7fa7d806a4d0 msgr2=0x7fa7d806a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:17.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.471+0000 7fa7ddc6b640 1 -- 172.21.15.46:0/3960399057 shutdown_connections 2023-12-02T00:29:17.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.471+0000 7fa7ddc6b640 1 -- 172.21.15.46:0/3960399057 wait complete. 2023-12-02T00:29:17.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.472+0000 7fa7ddc6b640 1 Processor -- start 2023-12-02T00:29:17.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.473+0000 7fa7ddc6b640 1 -- start start 2023-12-02T00:29:17.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.473+0000 7fa7ddc6b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7c40047c0 0x7fa7d815c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:17.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.473+0000 7fa7ddc6b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7d80ac7d0 0x7fa7d80aadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:17.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.473+0000 7fa7ddc6b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7d80ab330 0x7fa7d8151010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:17.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.473+0000 7fa7ddc6b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d80aa9d0 con 0x7fa7d80ac7d0 2023-12-02T00:29:17.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.473+0000 7fa7ddc6b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d80aa070 con 0x7fa7d80ab330 2023-12-02T00:29:17.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.473+0000 7fa7ddc6b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d80aa310 con 0x7fa7c40047c0 2023-12-02T00:29:17.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.473+0000 7fa7d6ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7d80ac7d0 0x7fa7d80aadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:17.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.473+0000 7fa7d6ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7d80ac7d0 0x7fa7d80aadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44772/0 (socket says 172.21.15.46:44772) 2023-12-02T00:29:17.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.473+0000 7fa7d6ffd640 1 -- 172.21.15.46:0/1947587831 learned_addr learned my addr 172.21.15.46:0/1947587831 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:17.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.474+0000 7fa7d77fe640 1 --2- 172.21.15.46:0/1947587831 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7c40047c0 0x7fa7d815c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:17.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.474+0000 7fa7d7fff640 1 --2- 172.21.15.46:0/1947587831 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7d80ab330 0x7fa7d8151010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:17.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.474+0000 7fa7d6ffd640 1 -- 172.21.15.46:0/1947587831 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7c40047c0 msgr2=0x7fa7d815c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:17.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.474+0000 7fa7d6ffd640 1 --2- 172.21.15.46:0/1947587831 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7c40047c0 0x7fa7d815c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:17.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.474+0000 7fa7d6ffd640 1 -- 172.21.15.46:0/1947587831 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7d80ab330 msgr2=0x7fa7d8151010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:17.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.474+0000 7fa7d6ffd640 1 --2- 172.21.15.46:0/1947587831 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7d80ab330 0x7fa7d8151010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:17.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.474+0000 7fa7d6ffd640 1 -- 172.21.15.46:0/1947587831 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d80aa5f0 con 0x7fa7d80ac7d0 2023-12-02T00:29:17.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.474+0000 7fa7d6ffd640 1 --2- 172.21.15.46:0/1947587831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7d80ac7d0 0x7fa7d80aadf0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa7c0005040 tx=0x7fa7c0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:17.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.474+0000 7fa7cffff640 1 -- 172.21.15.46:0/1947587831 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c00076c0 con 0x7fa7d80ac7d0 2023-12-02T00:29:17.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.474+0000 7fa7cffff640 1 -- 172.21.15.46:0/1947587831 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c0007a30 con 0x7fa7d80ac7d0 2023-12-02T00:29:17.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.474+0000 7fa7ddc6b640 1 -- 172.21.15.46:0/1947587831 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7d8140f20 con 0x7fa7d80ac7d0 2023-12-02T00:29:17.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.474+0000 7fa7cffff640 1 -- 172.21.15.46:0/1947587831 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0006be0 con 0x7fa7d80ac7d0 2023-12-02T00:29:17.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.474+0000 7fa7ddc6b640 1 -- 172.21.15.46:0/1947587831 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7d815c5b0 con 0x7fa7d80ac7d0 2023-12-02T00:29:17.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.475+0000 7fa7ddc6b640 1 -- 172.21.15.46:0/1947587831 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7d815c860 con 0x7fa7d80ac7d0 2023-12-02T00:29:17.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.476+0000 7fa7cffff640 1 -- 172.21.15.46:0/1947587831 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa7c0006d40 con 0x7fa7d80ac7d0 2023-12-02T00:29:17.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.476+0000 7fa7cffff640 1 --2- 172.21.15.46:0/1947587831 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa7b806ae40 0x7fa7b806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:17.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.476+0000 7fa7cffff640 1 -- 172.21.15.46:0/1947587831 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa7c00ab590 con 0x7fa7d80ac7d0 2023-12-02T00:29:17.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.476+0000 7fa7d77fe640 1 --2- 172.21.15.46:0/1947587831 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa7b806ae40 0x7fa7b806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:17.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.477+0000 7fa7d77fe640 1 --2- 172.21.15.46:0/1947587831 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa7b806ae40 0x7fa7b806d300 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8050020 tx=0x7fa7c8009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:17.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.480+0000 7fa7cffff640 1 -- 172.21.15.46:0/1947587831 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa7c00766f0 con 0x7fa7d80ac7d0 2023-12-02T00:29:17.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.657+0000 7fa7ddc6b640 1 -- 172.21.15.46:0/1947587831 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa7d8069400 con 0x7fa7d80ac7d0 2023-12-02T00:29:17.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.658+0000 7fa7cffff640 1 -- 172.21.15.46:0/1947587831 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16481 (secure 0 0 0) 0x7fa7c0079d10 con 0x7fa7d80ac7d0 2023-12-02T00:29:17.660 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:29:17.660 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":21,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:28:50.700112+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-02T00:28:47.273337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-02T00:28:48.060115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:29:17.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.660+0000 7fa7ddc6b640 1 -- 172.21.15.46:0/1947587831 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa7b806ae40 msgr2=0x7fa7b806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:17.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.660+0000 7fa7ddc6b640 1 --2- 172.21.15.46:0/1947587831 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa7b806ae40 0x7fa7b806d300 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8050020 tx=0x7fa7c8009190 comp rx=0 tx=0).stop 2023-12-02T00:29:17.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.660+0000 7fa7ddc6b640 1 -- 172.21.15.46:0/1947587831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7d80ac7d0 msgr2=0x7fa7d80aadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:17.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.660+0000 7fa7ddc6b640 1 --2- 172.21.15.46:0/1947587831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7d80ac7d0 0x7fa7d80aadf0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa7c0005040 tx=0x7fa7c0001120 comp rx=0 tx=0).stop 2023-12-02T00:29:17.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.661+0000 7fa7ddc6b640 1 -- 172.21.15.46:0/1947587831 shutdown_connections 2023-12-02T00:29:17.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.661+0000 7fa7ddc6b640 1 --2- 172.21.15.46:0/1947587831 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa7b806ae40 0x7fa7b806d300 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:17.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.661+0000 7fa7ddc6b640 1 --2- 172.21.15.46:0/1947587831 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7d80ab330 0x7fa7d8151010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:17.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.661+0000 7fa7ddc6b640 1 --2- 172.21.15.46:0/1947587831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7d80ac7d0 0x7fa7d80aadf0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:17.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.661+0000 7fa7ddc6b640 1 --2- 172.21.15.46:0/1947587831 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7c40047c0 0x7fa7d815c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:17.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.661+0000 7fa7ddc6b640 1 -- 172.21.15.46:0/1947587831 >> 172.21.15.46:0/1947587831 conn(0x7fa7d806a4d0 msgr2=0x7fa7d80cd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:17.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.661+0000 7fa7ddc6b640 1 -- 172.21.15.46:0/1947587831 shutdown_connections 2023-12-02T00:29:17.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.661+0000 7fa7ddc6b640 1 -- 172.21.15.46:0/1947587831 wait complete. 2023-12-02T00:29:17.673 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-02T00:28:37.213645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-02T00:28:47.273337+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-02T00:28:48.060115+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-02T00:29:17.674 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-02T00:29:17.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.783+0000 7f7129ff1640 1 Processor -- start 2023-12-02T00:29:17.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.783+0000 7f7129ff1640 1 -- start start 2023-12-02T00:29:17.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.784+0000 7f7129ff1640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7124144980 0x7f712415c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:17.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.784+0000 7f7129ff1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f71240b4370 0x7f71240b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:17.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.784+0000 7f7129ff1640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7124071770 con 0x7f71240ceb70 2023-12-02T00:29:17.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.784+0000 7f7129ff1640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71240ccb60 con 0x7f7124144980 2023-12-02T00:29:17.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.784+0000 7f7129ff1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71240b4ce0 con 0x7f71240b4370 2023-12-02T00:29:17.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.784+0000 7f7122ffd640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f71240ceb70 0x7f71240cc450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38582/0 (socket says 172.21.15.46:38582) 2023-12-02T00:29:17.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.784+0000 7f7122ffd640 1 -- 172.21.15.46:0/3846368830 learned_addr learned my addr 172.21.15.46:0/3846368830 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:17.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.784+0000 7f71237fe640 1 --2- 172.21.15.46:0/3846368830 >> v2:172.21.15.162:3300/0 conn(0x7f7124144980 0x7f712415c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:17.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.784+0000 7f7123fff640 1 --2- 172.21.15.46:0/3846368830 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f71240b4370 0x7f71240b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:17.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.784+0000 7f71227fc640 1 -- 172.21.15.46:0/3846368830 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1479347036 0 0) 0x7f7124071770 con 0x7f71240ceb70 2023-12-02T00:29:17.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.784+0000 7f71227fc640 1 -- 172.21.15.46:0/3846368830 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7110003520 con 0x7f71240ceb70 2023-12-02T00:29:17.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.784+0000 7f71237fe640 1 -- 172.21.15.46:0/3846368830 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f71240b4370 msgr2=0x7f71240b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:17.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.784+0000 7f71237fe640 1 --2- 172.21.15.46:0/3846368830 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f71240b4370 0x7f71240b47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:17.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.784+0000 7f71237fe640 1 -- 172.21.15.46:0/3846368830 >> v1:172.21.15.46:6789/0 conn(0x7f71240ceb70 legacy=0x7f71240cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:17.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.784+0000 7f71237fe640 1 -- 172.21.15.46:0/3846368830 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71240ccdc0 con 0x7f7124144980 2023-12-02T00:29:17.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.784+0000 7f7123fff640 1 --2- 172.21.15.46:0/3846368830 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f71240b4370 0x7f71240b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:29:17.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.785+0000 7f71237fe640 1 --2- 172.21.15.46:0/3846368830 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7124144980 0x7f712415c7c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f710c0097a0 tx=0x7f710c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b8186eabb7dabfc server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:17.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.785+0000 7f71227fc640 1 -- 172.21.15.46:0/3846368830 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710c031830 con 0x7f7124144980 2023-12-02T00:29:17.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.785+0000 7f71227fc640 1 -- 172.21.15.46:0/3846368830 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f710c031990 con 0x7f7124144980 2023-12-02T00:29:17.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.785+0000 7f71227fc640 1 -- 172.21.15.46:0/3846368830 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710c031d00 con 0x7f7124144980 2023-12-02T00:29:17.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.786+0000 7f7129ff1640 1 -- 172.21.15.46:0/3846368830 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7124144980 msgr2=0x7f712415c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:17.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.786+0000 7f7129ff1640 1 --2- 172.21.15.46:0/3846368830 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7124144980 0x7f712415c7c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f710c0097a0 tx=0x7f710c02b8d0 comp rx=0 tx=0).stop 2023-12-02T00:29:17.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.786+0000 7f7129ff1640 1 -- 172.21.15.46:0/3846368830 shutdown_connections 2023-12-02T00:29:17.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.786+0000 7f7129ff1640 1 --2- 172.21.15.46:0/3846368830 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f71240b4370 0x7f71240b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:17.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.786+0000 7f7129ff1640 1 --2- 172.21.15.46:0/3846368830 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7124144980 0x7f712415c7c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:17.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.786+0000 7f7129ff1640 1 -- 172.21.15.46:0/3846368830 >> 172.21.15.46:0/3846368830 conn(0x7f712406a4d0 msgr2=0x7f712406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:17.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.786+0000 7f7129ff1640 1 -- 172.21.15.46:0/3846368830 shutdown_connections 2023-12-02T00:29:17.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.786+0000 7f7129ff1640 1 -- 172.21.15.46:0/3846368830 wait complete. 2023-12-02T00:29:17.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.787+0000 7f7129ff1640 1 Processor -- start 2023-12-02T00:29:17.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.787+0000 7f7129ff1640 1 -- start start 2023-12-02T00:29:17.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.788+0000 7f7129ff1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71240b4370 0x7f7124158320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:17.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.788+0000 7f7129ff1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f71240ceb70 0x7f712415ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:17.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.788+0000 7f7129ff1640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7124144980 0x7f71241419c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:17.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.788+0000 7f7129ff1640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712415cb50 con 0x7f71240b4370 2023-12-02T00:29:17.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.788+0000 7f7129ff1640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712415ce30 con 0x7f7124144980 2023-12-02T00:29:17.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.788+0000 7f7129ff1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f712415d110 con 0x7f71240ceb70 2023-12-02T00:29:17.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.788+0000 7f71237fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71240b4370 0x7f7124158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:17.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.788+0000 7f71237fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71240b4370 0x7f7124158320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44776/0 (socket says 172.21.15.46:44776) 2023-12-02T00:29:17.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.788+0000 7f71237fe640 1 -- 172.21.15.46:0/3734480836 learned_addr learned my addr 172.21.15.46:0/3734480836 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:17.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.788+0000 7f7123fff640 1 --2- 172.21.15.46:0/3734480836 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7124144980 0x7f71241419c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:17.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.788+0000 7f7122ffd640 1 --2- 172.21.15.46:0/3734480836 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f71240ceb70 0x7f712415ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:17.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.789+0000 7f71237fe640 1 -- 172.21.15.46:0/3734480836 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f71240ceb70 msgr2=0x7f712415ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:17.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.789+0000 7f71237fe640 1 --2- 172.21.15.46:0/3734480836 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f71240ceb70 0x7f712415ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:17.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.789+0000 7f71237fe640 1 -- 172.21.15.46:0/3734480836 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7124144980 msgr2=0x7f71241419c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:17.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.789+0000 7f71237fe640 1 --2- 172.21.15.46:0/3734480836 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7124144980 0x7f71241419c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:17.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.789+0000 7f71237fe640 1 -- 172.21.15.46:0/3734480836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f711c009a70 con 0x7f71240b4370 2023-12-02T00:29:17.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.789+0000 7f7122ffd640 1 --2- 172.21.15.46:0/3734480836 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f71240ceb70 0x7f712415ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:17.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.789+0000 7f71237fe640 1 --2- 172.21.15.46:0/3734480836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71240b4370 0x7f7124158320 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f710c0098c0 tx=0x7f710c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:17.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.789+0000 7f711bfff640 1 -- 172.21.15.46:0/3734480836 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710c03a630 con 0x7f71240b4370 2023-12-02T00:29:17.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.789+0000 7f711bfff640 1 -- 172.21.15.46:0/3734480836 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f710c03a790 con 0x7f71240b4370 2023-12-02T00:29:17.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.789+0000 7f711bfff640 1 -- 172.21.15.46:0/3734480836 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710c03ab40 con 0x7f71240b4370 2023-12-02T00:29:17.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.789+0000 7f7129ff1640 1 -- 172.21.15.46:0/3734480836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f710c009400 con 0x7f71240b4370 2023-12-02T00:29:17.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.789+0000 7f7129ff1640 1 -- 172.21.15.46:0/3734480836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f712415d6d0 con 0x7f71240b4370 2023-12-02T00:29:17.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.790+0000 7f7129ff1640 1 -- 172.21.15.46:0/3734480836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71240a84d0 con 0x7f71240b4370 2023-12-02T00:29:17.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.790+0000 7f711bfff640 1 -- 172.21.15.46:0/3734480836 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f710c03aca0 con 0x7f71240b4370 2023-12-02T00:29:17.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.791+0000 7f711bfff640 1 --2- 172.21.15.46:0/3734480836 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f710406ae40 0x7f710406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:17.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.791+0000 7f711bfff640 1 -- 172.21.15.46:0/3734480836 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f710c0aada0 con 0x7f71240b4370 2023-12-02T00:29:17.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.791+0000 7f7122ffd640 1 --2- 172.21.15.46:0/3734480836 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f710406ae40 0x7f710406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:17.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.792+0000 7f7122ffd640 1 --2- 172.21.15.46:0/3734480836 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f710406ae40 0x7f710406d300 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7114007fd0 tx=0x7f7114009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:17.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.795+0000 7f711bfff640 1 -- 172.21.15.46:0/3734480836 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f710c075ee0 con 0x7f71240b4370 2023-12-02T00:29:17.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.970+0000 7f7129ff1640 1 -- 172.21.15.46:0/3734480836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7124069400 con 0x7f71240b4370 2023-12-02T00:29:17.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.970+0000 7f711bfff640 1 -- 172.21.15.46:0/3734480836 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f710c0350a0 con 0x7f71240b4370 2023-12-02T00:29:17.972 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-12-02T00:29:17.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.972+0000 7f7129ff1640 1 -- 172.21.15.46:0/3734480836 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f710406ae40 msgr2=0x7f710406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:17.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.972+0000 7f7129ff1640 1 --2- 172.21.15.46:0/3734480836 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f710406ae40 0x7f710406d300 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7114007fd0 tx=0x7f7114009040 comp rx=0 tx=0).stop 2023-12-02T00:29:17.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.972+0000 7f7129ff1640 1 -- 172.21.15.46:0/3734480836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71240b4370 msgr2=0x7f7124158320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:17.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.972+0000 7f7129ff1640 1 --2- 172.21.15.46:0/3734480836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71240b4370 0x7f7124158320 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f710c0098c0 tx=0x7f710c0046c0 comp rx=0 tx=0).stop 2023-12-02T00:29:17.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.972+0000 7f7129ff1640 1 -- 172.21.15.46:0/3734480836 shutdown_connections 2023-12-02T00:29:17.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.972+0000 7f7129ff1640 1 --2- 172.21.15.46:0/3734480836 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f710406ae40 0x7f710406d300 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:17.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.972+0000 7f7129ff1640 1 --2- 172.21.15.46:0/3734480836 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7124144980 0x7f71241419c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:17.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.972+0000 7f7129ff1640 1 --2- 172.21.15.46:0/3734480836 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f71240ceb70 0x7f712415ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:17.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.972+0000 7f7129ff1640 1 --2- 172.21.15.46:0/3734480836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71240b4370 0x7f7124158320 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:17.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.972+0000 7f7129ff1640 1 -- 172.21.15.46:0/3734480836 >> 172.21.15.46:0/3734480836 conn(0x7f712406a4d0 msgr2=0x7f712415a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:17.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.972+0000 7f7129ff1640 1 -- 172.21.15.46:0/3734480836 shutdown_connections 2023-12-02T00:29:17.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:17.972+0000 7f7129ff1640 1 -- 172.21.15.46:0/3734480836 wait complete. 2023-12-02T00:29:17.984 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-02T00:29:18.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.093+0000 7fd46b6d3640 1 Processor -- start 2023-12-02T00:29:18.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.094+0000 7fd46b6d3640 1 -- start start 2023-12-02T00:29:18.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.094+0000 7fd46b6d3640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd4640d3900 0x7fd4640b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:18.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.094+0000 7fd46b6d3640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4640b2b10 con 0x7fd46415d330 2023-12-02T00:29:18.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.094+0000 7fd46b6d3640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4640d1590 con 0x7fd4640d2910 2023-12-02T00:29:18.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.094+0000 7fd46b6d3640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4640d1870 con 0x7fd4640d3900 2023-12-02T00:29:18.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.094+0000 7fd46946f640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd46415d330 0x7fd4640b8b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38598/0 (socket says 172.21.15.46:38598) 2023-12-02T00:29:18.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.094+0000 7fd46946f640 1 -- 172.21.15.46:0/3140953382 learned_addr learned my addr 172.21.15.46:0/3140953382 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:18.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.095+0000 7fd469c70640 1 --2- 172.21.15.46:0/3140953382 >> v2:172.21.15.162:3300/0 conn(0x7fd4640d3900 0x7fd4640b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:18.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.095+0000 7fd45bfff640 1 -- 172.21.15.46:0/3140953382 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 370316806 0 0) 0x7fd4640b2b10 con 0x7fd46415d330 2023-12-02T00:29:18.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.095+0000 7fd45bfff640 1 -- 172.21.15.46:0/3140953382 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd450003520 con 0x7fd46415d330 2023-12-02T00:29:18.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.095+0000 7fd45bfff640 1 -- 172.21.15.46:0/3140953382 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2152124108 0 0) 0x7fd450003520 con 0x7fd46415d330 2023-12-02T00:29:18.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.095+0000 7fd469c70640 1 -- 172.21.15.46:0/3140953382 >> v1:172.21.15.162:6789/0 conn(0x7fd4640d2910 legacy=0x7fd4640d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:18.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.095+0000 7fd469c70640 1 -- 172.21.15.46:0/3140953382 >> v1:172.21.15.46:6789/0 conn(0x7fd46415d330 legacy=0x7fd4640b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:18.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.095+0000 7fd469c70640 1 -- 172.21.15.46:0/3140953382 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4640d1b50 con 0x7fd4640d3900 2023-12-02T00:29:18.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.096+0000 7fd469c70640 1 --2- 172.21.15.46:0/3140953382 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4640d3900 0x7fd4640b6390 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd45c009d90 tx=0x7fd45c02bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1636a2e661276291 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:18.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.096+0000 7fd45bfff640 1 -- 172.21.15.46:0/3140953382 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd460001670 con 0x7fd4640d3900 2023-12-02T00:29:18.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.096+0000 7fd45bfff640 1 -- 172.21.15.46:0/3140953382 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd45c02c8b0 con 0x7fd4640d3900 2023-12-02T00:29:18.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.096+0000 7fd45bfff640 1 -- 172.21.15.46:0/3140953382 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45c02cc20 con 0x7fd4640d3900 2023-12-02T00:29:18.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.096+0000 7fd46b6d3640 1 -- 172.21.15.46:0/3140953382 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4640d3900 msgr2=0x7fd4640b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:18.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.096+0000 7fd46b6d3640 1 --2- 172.21.15.46:0/3140953382 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4640d3900 0x7fd4640b6390 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd45c009d90 tx=0x7fd45c02bd80 comp rx=0 tx=0).stop 2023-12-02T00:29:18.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.097+0000 7fd46b6d3640 1 -- 172.21.15.46:0/3140953382 shutdown_connections 2023-12-02T00:29:18.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.097+0000 7fd46b6d3640 1 --2- 172.21.15.46:0/3140953382 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4640d3900 0x7fd4640b6390 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:18.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.097+0000 7fd46b6d3640 1 -- 172.21.15.46:0/3140953382 >> 172.21.15.46:0/3140953382 conn(0x7fd464072020 msgr2=0x7fd464072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:18.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.097+0000 7fd46b6d3640 1 -- 172.21.15.46:0/3140953382 shutdown_connections 2023-12-02T00:29:18.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.097+0000 7fd46b6d3640 1 -- 172.21.15.46:0/3140953382 wait complete. 2023-12-02T00:29:18.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.098+0000 7fd46b6d3640 1 Processor -- start 2023-12-02T00:29:18.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.098+0000 7fd46b6d3640 1 -- start start 2023-12-02T00:29:18.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.098+0000 7fd46b6d3640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4640d2910 0x7fd464157a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:18.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.098+0000 7fd46b6d3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4640d3900 0x7fd4640c94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:18.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.098+0000 7fd46b6d3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd46415d330 0x7fd46415f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:18.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.098+0000 7fd46b6d3640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd464157fa0 con 0x7fd46415d330 2023-12-02T00:29:18.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.099+0000 7fd46b6d3640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4640c8650 con 0x7fd4640d2910 2023-12-02T00:29:18.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.099+0000 7fd46b6d3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd46415f190 con 0x7fd4640d3900 2023-12-02T00:29:18.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.099+0000 7fd469c70640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd46415d330 0x7fd46415f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:18.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.099+0000 7fd469c70640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd46415d330 0x7fd46415f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44790/0 (socket says 172.21.15.46:44790) 2023-12-02T00:29:18.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.099+0000 7fd469c70640 1 -- 172.21.15.46:0/3638865172 learned_addr learned my addr 172.21.15.46:0/3638865172 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:18.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.099+0000 7fd46946f640 1 --2- 172.21.15.46:0/3638865172 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4640d2910 0x7fd464157a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:18.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.099+0000 7fd468c6e640 1 --2- 172.21.15.46:0/3638865172 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4640d3900 0x7fd4640c94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:18.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.099+0000 7fd469c70640 1 -- 172.21.15.46:0/3638865172 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4640d3900 msgr2=0x7fd4640c94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:18.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.099+0000 7fd469c70640 1 --2- 172.21.15.46:0/3638865172 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4640d3900 0x7fd4640c94b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:18.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.099+0000 7fd469c70640 1 -- 172.21.15.46:0/3638865172 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4640d2910 msgr2=0x7fd464157a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:18.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.099+0000 7fd469c70640 1 --2- 172.21.15.46:0/3638865172 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4640d2910 0x7fd464157a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:18.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.099+0000 7fd469c70640 1 -- 172.21.15.46:0/3638865172 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd45c009a70 con 0x7fd46415d330 2023-12-02T00:29:18.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.099+0000 7fd46946f640 1 --2- 172.21.15.46:0/3638865172 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4640d2910 0x7fd464157a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:29:18.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.099+0000 7fd469c70640 1 --2- 172.21.15.46:0/3638865172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd46415d330 0x7fd46415f9d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd45c02c390 tx=0x7fd45c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:18.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.099+0000 7fd459ffb640 1 -- 172.21.15.46:0/3638865172 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45c02cea0 con 0x7fd46415d330 2023-12-02T00:29:18.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.099+0000 7fd459ffb640 1 -- 172.21.15.46:0/3638865172 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd45c009450 con 0x7fd46415d330 2023-12-02T00:29:18.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.099+0000 7fd46b6d3640 1 -- 172.21.15.46:0/3638865172 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4640c90a0 con 0x7fd46415d330 2023-12-02T00:29:18.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.099+0000 7fd459ffb640 1 -- 172.21.15.46:0/3638865172 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45c03f420 con 0x7fd46415d330 2023-12-02T00:29:18.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.100+0000 7fd46b6d3640 1 -- 172.21.15.46:0/3638865172 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4640cceb0 con 0x7fd46415d330 2023-12-02T00:29:18.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.101+0000 7fd46b6d3640 1 -- 172.21.15.46:0/3638865172 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd464069400 con 0x7fd46415d330 2023-12-02T00:29:18.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.101+0000 7fd459ffb640 1 -- 172.21.15.46:0/3638865172 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd45c03f580 con 0x7fd46415d330 2023-12-02T00:29:18.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.101+0000 7fd459ffb640 1 --2- 172.21.15.46:0/3638865172 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd44806ae40 0x7fd44806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:18.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.101+0000 7fd459ffb640 1 -- 172.21.15.46:0/3638865172 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd45c0abbc0 con 0x7fd46415d330 2023-12-02T00:29:18.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.102+0000 7fd46946f640 1 --2- 172.21.15.46:0/3638865172 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd44806ae40 0x7fd44806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:18.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.102+0000 7fd46946f640 1 --2- 172.21.15.46:0/3638865172 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd44806ae40 0x7fd44806d300 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd4600011e0 tx=0x7fd460009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:18.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.106+0000 7fd459ffb640 1 -- 172.21.15.46:0/3638865172 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd45c076c70 con 0x7fd46415d330 2023-12-02T00:29:18.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.284+0000 7fd46b6d3640 1 -- 172.21.15.46:0/3638865172 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd4640b7480 con 0x7fd46415d330 2023-12-02T00:29:18.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.285+0000 7fd459ffb640 1 -- 172.21.15.46:0/3638865172 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd45c043020 con 0x7fd46415d330 2023-12-02T00:29:18.286 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-12-02T00:29:18.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.286+0000 7fd46b6d3640 1 -- 172.21.15.46:0/3638865172 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd44806ae40 msgr2=0x7fd44806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:18.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.286+0000 7fd46b6d3640 1 --2- 172.21.15.46:0/3638865172 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd44806ae40 0x7fd44806d300 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd4600011e0 tx=0x7fd460009040 comp rx=0 tx=0).stop 2023-12-02T00:29:18.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.286+0000 7fd46b6d3640 1 -- 172.21.15.46:0/3638865172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd46415d330 msgr2=0x7fd46415f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:18.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.286+0000 7fd46b6d3640 1 --2- 172.21.15.46:0/3638865172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd46415d330 0x7fd46415f9d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd45c02c390 tx=0x7fd45c004a40 comp rx=0 tx=0).stop 2023-12-02T00:29:18.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.287+0000 7fd46b6d3640 1 -- 172.21.15.46:0/3638865172 shutdown_connections 2023-12-02T00:29:18.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.287+0000 7fd46b6d3640 1 --2- 172.21.15.46:0/3638865172 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd44806ae40 0x7fd44806d300 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:18.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.287+0000 7fd46b6d3640 1 --2- 172.21.15.46:0/3638865172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd46415d330 0x7fd46415f9d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:18.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.287+0000 7fd46b6d3640 1 --2- 172.21.15.46:0/3638865172 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4640d3900 0x7fd4640c94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:18.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.287+0000 7fd46b6d3640 1 --2- 172.21.15.46:0/3638865172 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4640d2910 0x7fd464157a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:18.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.287+0000 7fd46b6d3640 1 -- 172.21.15.46:0/3638865172 >> 172.21.15.46:0/3638865172 conn(0x7fd464072020 msgr2=0x7fd4640d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:18.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.287+0000 7fd46b6d3640 1 -- 172.21.15.46:0/3638865172 shutdown_connections 2023-12-02T00:29:18.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.287+0000 7fd46b6d3640 1 -- 172.21.15.46:0/3638865172 wait complete. 2023-12-02T00:29:18.299 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-02T00:29:18.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.408+0000 7f0917826640 1 Processor -- start 2023-12-02T00:29:18.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.408+0000 7f0917826640 1 -- start start 2023-12-02T00:29:18.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.409+0000 7f0917826640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f09100ad640 0x7f091006d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:18.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.409+0000 7f0917826640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09100adae0 con 0x7f091006bbf0 2023-12-02T00:29:18.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.409+0000 7f0917826640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09100addc0 con 0x7f091015d1d0 2023-12-02T00:29:18.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.409+0000 7f0917826640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09100ae0a0 con 0x7f09100ad640 2023-12-02T00:29:18.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.409+0000 7f0915dc3640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f091006bbf0 0x7f091006d5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38614/0 (socket says 172.21.15.46:38614) 2023-12-02T00:29:18.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.409+0000 7f0915dc3640 1 -- 172.21.15.46:0/376158460 learned_addr learned my addr 172.21.15.46:0/376158460 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:18.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.409+0000 7f0914dc1640 1 --2- 172.21.15.46:0/376158460 >> v2:172.21.15.162:3300/0 conn(0x7f09100ad640 0x7f091006d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.409+0000 7f090ffff640 1 -- 172.21.15.46:0/376158460 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 690652708 0 0) 0x7f09100adae0 con 0x7f091006bbf0 2023-12-02T00:29:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.409+0000 7f090ffff640 1 -- 172.21.15.46:0/376158460 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08f8003520 con 0x7f091006bbf0 2023-12-02T00:29:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.410+0000 7f090ffff640 1 -- 172.21.15.46:0/376158460 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3828200095 0 0) 0x7f08f8003520 con 0x7f091006bbf0 2023-12-02T00:29:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.410+0000 7f090ffff640 1 -- 172.21.15.46:0/376158460 >> v2:172.21.15.162:3300/0 conn(0x7f09100ad640 msgr2=0x7f091006d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.410+0000 7f090ffff640 1 --2- 172.21.15.46:0/376158460 >> v2:172.21.15.162:3300/0 conn(0x7f09100ad640 0x7f091006d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.410+0000 7f090ffff640 1 -- 172.21.15.46:0/376158460 >> v1:172.21.15.162:6789/0 conn(0x7f091015d1d0 legacy=0x7f091006b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.410+0000 7f090ffff640 1 -- 172.21.15.46:0/376158460 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09100ae380 con 0x7f091006bbf0 2023-12-02T00:29:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.410+0000 7f090ffff640 1 -- 172.21.15.46:0/376158460 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7f09000019f0 con 0x7f091006bbf0 2023-12-02T00:29:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.410+0000 7f090ffff640 1 -- 172.21.15.46:0/376158460 >> v1:172.21.15.46:6789/0 conn(0x7f091006bbf0 legacy=0x7f091006d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.410+0000 7f090ffff640 1 --2- 172.21.15.46:0/376158460 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08f8003970 0x7f08f8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.410+0000 7f090ffff640 1 --2- 172.21.15.46:0/376158460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08f80047c0 0x7f08f8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.410+0000 7f090ffff640 1 --2- 172.21.15.46:0/376158460 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08f8005180 0x7f08f800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.410+0000 7f090ffff640 1 -- 172.21.15.46:0/376158460 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09100ae0a0 con 0x7f08f80047c0 2023-12-02T00:29:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.410+0000 7f090ffff640 1 -- 172.21.15.46:0/376158460 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f800c020 con 0x7f08f8005180 2023-12-02T00:29:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.410+0000 7f090ffff640 1 -- 172.21.15.46:0/376158460 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08f800c1e0 con 0x7f08f8003970 2023-12-02T00:29:18.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.410+0000 7f0914dc1640 1 --2- 172.21.15.46:0/376158460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08f80047c0 0x7f08f8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:18.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.410+0000 7f09155c2640 1 --2- 172.21.15.46:0/376158460 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08f8003970 0x7f08f8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:18.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.410+0000 7f0915dc3640 1 --2- 172.21.15.46:0/376158460 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08f8005180 0x7f08f800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:18.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.410+0000 7f0914dc1640 1 -- 172.21.15.46:0/376158460 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08f8003970 msgr2=0x7f08f8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:18.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.410+0000 7f0914dc1640 1 --2- 172.21.15.46:0/376158460 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08f8003970 0x7f08f8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:18.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.410+0000 7f0914dc1640 1 -- 172.21.15.46:0/376158460 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08f8005180 msgr2=0x7f08f800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:18.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.410+0000 7f0914dc1640 1 --2- 172.21.15.46:0/376158460 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08f8005180 0x7f08f800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:18.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.410+0000 7f0915dc3640 1 -- 172.21.15.46:0/376158460 reap_dead start 2023-12-02T00:29:18.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.410+0000 7f0914dc1640 1 -- 172.21.15.46:0/376158460 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f8003520 con 0x7f08f80047c0 2023-12-02T00:29:18.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.411+0000 7f0914dc1640 1 --2- 172.21.15.46:0/376158460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08f80047c0 0x7f08f8004c40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0904009ac0 tx=0x7f08f80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad0770d352e59650 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:18.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.411+0000 7f090ffff640 1 -- 172.21.15.46:0/376158460 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090402c6b0 con 0x7f08f80047c0 2023-12-02T00:29:18.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.411+0000 7f090ffff640 1 -- 172.21.15.46:0/376158460 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f090402c810 con 0x7f08f80047c0 2023-12-02T00:29:18.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.411+0000 7f090ffff640 1 -- 172.21.15.46:0/376158460 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090402cba0 con 0x7f08f80047c0 2023-12-02T00:29:18.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.412+0000 7f0917826640 1 -- 172.21.15.46:0/376158460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08f80047c0 msgr2=0x7f08f8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:18.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.412+0000 7f0917826640 1 --2- 172.21.15.46:0/376158460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08f80047c0 0x7f08f8004c40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0904009ac0 tx=0x7f08f80057f0 comp rx=0 tx=0).stop 2023-12-02T00:29:18.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.412+0000 7f0917826640 1 -- 172.21.15.46:0/376158460 shutdown_connections 2023-12-02T00:29:18.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.412+0000 7f0917826640 1 --2- 172.21.15.46:0/376158460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08f80047c0 0x7f08f8004c40 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:18.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.412+0000 7f0917826640 1 -- 172.21.15.46:0/376158460 >> 172.21.15.46:0/376158460 conn(0x7f0910070e70 msgr2=0x7f0910071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:18.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.412+0000 7f0917826640 1 -- 172.21.15.46:0/376158460 shutdown_connections 2023-12-02T00:29:18.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.412+0000 7f0917826640 1 -- 172.21.15.46:0/376158460 wait complete. 2023-12-02T00:29:18.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.413+0000 7f0917826640 1 Processor -- start 2023-12-02T00:29:18.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.413+0000 7f0917826640 1 -- start start 2023-12-02T00:29:18.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.414+0000 7f0917826640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08f80047c0 0x7f09101f3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:18.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.414+0000 7f0917826640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09101f2400 0x7f091006e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:18.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.414+0000 7f0917826640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f091006ebe0 0x7f09100ce340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:18.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.414+0000 7f0917826640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091006f060 con 0x7f09101f2400 2023-12-02T00:29:18.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.414+0000 7f0917826640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09101f70e0 con 0x7f08f80047c0 2023-12-02T00:29:18.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.414+0000 7f0917826640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09101f3b40 con 0x7f091006ebe0 2023-12-02T00:29:18.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.414+0000 7f0914dc1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09101f2400 0x7f091006e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:18.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.414+0000 7f0914dc1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09101f2400 0x7f091006e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44814/0 (socket says 172.21.15.46:44814) 2023-12-02T00:29:18.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.414+0000 7f0914dc1640 1 -- 172.21.15.46:0/3407328367 learned_addr learned my addr 172.21.15.46:0/3407328367 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:18.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.414+0000 7f09155c2640 1 --2- 172.21.15.46:0/3407328367 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08f80047c0 0x7f09101f3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:18.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.414+0000 7f0915dc3640 1 --2- 172.21.15.46:0/3407328367 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f091006ebe0 0x7f09100ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:18.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.414+0000 7f0914dc1640 1 -- 172.21.15.46:0/3407328367 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f091006ebe0 msgr2=0x7f09100ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:18.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.414+0000 7f0914dc1640 1 --2- 172.21.15.46:0/3407328367 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f091006ebe0 0x7f09100ce340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:18.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.414+0000 7f0914dc1640 1 -- 172.21.15.46:0/3407328367 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08f80047c0 msgr2=0x7f09101f3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:18.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.414+0000 7f0914dc1640 1 --2- 172.21.15.46:0/3407328367 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08f80047c0 0x7f09101f3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:18.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.414+0000 7f0914dc1640 1 -- 172.21.15.46:0/3407328367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09101f6560 con 0x7f09101f2400 2023-12-02T00:29:18.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.415+0000 7f0914dc1640 1 --2- 172.21.15.46:0/3407328367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09101f2400 0x7f091006e6a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0904037040 tx=0x7f0904004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:18.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.415+0000 7f090dffb640 1 -- 172.21.15.46:0/3407328367 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0904003810 con 0x7f09101f2400 2023-12-02T00:29:18.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.415+0000 7f090dffb640 1 -- 172.21.15.46:0/3407328367 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0904003970 con 0x7f09101f2400 2023-12-02T00:29:18.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.415+0000 7f0917826640 1 -- 172.21.15.46:0/3407328367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09101f6840 con 0x7f09101f2400 2023-12-02T00:29:18.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.415+0000 7f090dffb640 1 -- 172.21.15.46:0/3407328367 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0904003d20 con 0x7f09101f2400 2023-12-02T00:29:18.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.415+0000 7f0917826640 1 -- 172.21.15.46:0/3407328367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09101f6e00 con 0x7f09101f2400 2023-12-02T00:29:18.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.416+0000 7f0917826640 1 -- 172.21.15.46:0/3407328367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f091006efc0 con 0x7f09101f2400 2023-12-02T00:29:18.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.416+0000 7f090dffb640 1 -- 172.21.15.46:0/3407328367 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0904034440 con 0x7f09101f2400 2023-12-02T00:29:18.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.417+0000 7f090dffb640 1 --2- 172.21.15.46:0/3407328367 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f08fc06ae70 0x7f08fc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:18.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.417+0000 7f090dffb640 1 -- 172.21.15.46:0/3407328367 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f09040ab540 con 0x7f09101f2400 2023-12-02T00:29:18.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.417+0000 7f09155c2640 1 --2- 172.21.15.46:0/3407328367 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f08fc06ae70 0x7f08fc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:18.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.418+0000 7f09155c2640 1 --2- 172.21.15.46:0/3407328367 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f08fc06ae70 0x7f08fc06d330 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f09100b00b0 tx=0x7f0908009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:18.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.421+0000 7f090dffb640 1 -- 172.21.15.46:0/3407328367 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0904076850 con 0x7f09101f2400 2023-12-02T00:29:18.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.597+0000 7f0917826640 1 -- 172.21.15.46:0/3407328367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0910069400 con 0x7f09101f2400 2023-12-02T00:29:18.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.598+0000 7f090dffb640 1 -- 172.21.15.46:0/3407328367 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0904079e70 con 0x7f09101f2400 2023-12-02T00:29:18.599 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-12-02T00:29:18.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.600+0000 7f0917826640 1 -- 172.21.15.46:0/3407328367 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f08fc06ae70 msgr2=0x7f08fc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:18.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.600+0000 7f0917826640 1 --2- 172.21.15.46:0/3407328367 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f08fc06ae70 0x7f08fc06d330 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f09100b00b0 tx=0x7f0908009040 comp rx=0 tx=0).stop 2023-12-02T00:29:18.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.600+0000 7f0917826640 1 -- 172.21.15.46:0/3407328367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09101f2400 msgr2=0x7f091006e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:18.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.600+0000 7f0917826640 1 --2- 172.21.15.46:0/3407328367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09101f2400 0x7f091006e6a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0904037040 tx=0x7f0904004730 comp rx=0 tx=0).stop 2023-12-02T00:29:18.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.600+0000 7f0917826640 1 -- 172.21.15.46:0/3407328367 shutdown_connections 2023-12-02T00:29:18.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.600+0000 7f0917826640 1 --2- 172.21.15.46:0/3407328367 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f08fc06ae70 0x7f08fc06d330 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:18.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.600+0000 7f0917826640 1 --2- 172.21.15.46:0/3407328367 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f091006ebe0 0x7f09100ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:18.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.600+0000 7f0917826640 1 --2- 172.21.15.46:0/3407328367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09101f2400 0x7f091006e6a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:18.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.600+0000 7f0917826640 1 --2- 172.21.15.46:0/3407328367 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08f80047c0 0x7f09101f3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:18.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.600+0000 7f0917826640 1 -- 172.21.15.46:0/3407328367 >> 172.21.15.46:0/3407328367 conn(0x7f0910070e70 msgr2=0x7f09100af560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:18.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.600+0000 7f0917826640 1 -- 172.21.15.46:0/3407328367 shutdown_connections 2023-12-02T00:29:18.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.600+0000 7f0917826640 1 -- 172.21.15.46:0/3407328367 wait complete. 2023-12-02T00:29:18.615 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:29:18.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.720+0000 7fb2ae461640 1 Processor -- start 2023-12-02T00:29:18.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.721+0000 7fb2ae461640 1 -- start start 2023-12-02T00:29:18.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.721+0000 7fb2ae461640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb2a806bbf0 0x7fb2a815eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:18.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.721+0000 7fb2ae461640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb2a80cf000 0x7fb2a80b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:18.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.721+0000 7fb2ae461640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2a80b28c0 con 0x7fb2a815af70 2023-12-02T00:29:18.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.721+0000 7fb2ae461640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2a80cdb10 con 0x7fb2a806bbf0 2023-12-02T00:29:18.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.721+0000 7fb2ae461640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a815fc20 con 0x7fb2a80cf000 2023-12-02T00:29:18.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.722+0000 7fb2a77fe640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb2a806bbf0 0x7fb2a815eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:18.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.722+0000 7fb2ac9fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb2a80cf000 0x7fb2a80b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:18.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.722+0000 7fb2a7fff640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fb2a815af70 0x7fb2a806b4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.46:38110/0 (socket says 172.21.15.46:38110) 2023-12-02T00:29:18.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.722+0000 7fb2a7fff640 1 -- 172.21.15.46:0/3925616835 learned_addr learned my addr 172.21.15.46:0/3925616835 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:18.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.722+0000 7fb2ac9fe640 1 -- 172.21.15.46:0/3925616835 >> v2:172.21.15.162:3300/0 conn(0x7fb2a806bbf0 msgr2=0x7fb2a815eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:18.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.722+0000 7fb2ac9fe640 1 --2- 172.21.15.46:0/3925616835 >> v2:172.21.15.162:3300/0 conn(0x7fb2a806bbf0 0x7fb2a815eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:18.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.722+0000 7fb2ac9fe640 1 -- 172.21.15.46:0/3925616835 >> v1:172.21.15.162:6789/0 conn(0x7fb2a815af70 legacy=0x7fb2a806b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:18.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.722+0000 7fb2ac9fe640 1 -- 172.21.15.46:0/3925616835 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a80cf450 con 0x7fb2a80cf000 2023-12-02T00:29:18.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.723+0000 7fb2ac9fe640 1 --2- 172.21.15.46:0/3925616835 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb2a80cf000 0x7fb2a80b2d50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0009d90 tx=0x7fb2a002bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7366bbe7eddbc7ae server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:18.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.723+0000 7fb2a6ffd640 1 -- 172.21.15.46:0/3925616835 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a002cc50 con 0x7fb2a80cf000 2023-12-02T00:29:18.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.723+0000 7fb2a6ffd640 1 -- 172.21.15.46:0/3925616835 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2a002cdb0 con 0x7fb2a80cf000 2023-12-02T00:29:18.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.723+0000 7fb2a6ffd640 1 -- 172.21.15.46:0/3925616835 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a0034660 con 0x7fb2a80cf000 2023-12-02T00:29:18.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.724+0000 7fb2ae461640 1 -- 172.21.15.46:0/3925616835 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb2a80cf000 msgr2=0x7fb2a80b2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:18.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.724+0000 7fb2ae461640 1 --2- 172.21.15.46:0/3925616835 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb2a80cf000 0x7fb2a80b2d50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0009d90 tx=0x7fb2a002bea0 comp rx=0 tx=0).stop 2023-12-02T00:29:18.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.724+0000 7fb2ae461640 1 -- 172.21.15.46:0/3925616835 shutdown_connections 2023-12-02T00:29:18.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.724+0000 7fb2ae461640 1 --2- 172.21.15.46:0/3925616835 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb2a80cf000 0x7fb2a80b2d50 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:18.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.724+0000 7fb2ae461640 1 --2- 172.21.15.46:0/3925616835 >> v2:172.21.15.162:3300/0 conn(0x7fb2a806bbf0 0x7fb2a815eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:18.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.724+0000 7fb2ae461640 1 -- 172.21.15.46:0/3925616835 >> 172.21.15.46:0/3925616835 conn(0x7fb2a8070ec0 msgr2=0x7fb2a80712a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:18.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.724+0000 7fb2ae461640 1 -- 172.21.15.46:0/3925616835 shutdown_connections 2023-12-02T00:29:18.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.724+0000 7fb2ae461640 1 -- 172.21.15.46:0/3925616835 wait complete. 2023-12-02T00:29:18.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.725+0000 7fb2ae461640 1 Processor -- start 2023-12-02T00:29:18.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.725+0000 7fb2ae461640 1 -- start start 2023-12-02T00:29:18.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.726+0000 7fb2ae461640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2a806bbf0 0x7fb2a80c4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:18.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.726+0000 7fb2ae461640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2a80cf000 0x7fb2a81f1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:18.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.726+0000 7fb2ae461640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb2a815af70 0x7fb2a81573f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:18.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.726+0000 7fb2ae461640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a80ac710 con 0x7fb2a806bbf0 2023-12-02T00:29:18.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.726+0000 7fb2ae461640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a8157930 con 0x7fb2a80cf000 2023-12-02T00:29:18.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.726+0000 7fb2ae461640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a80c4d00 con 0x7fb2a815af70 2023-12-02T00:29:18.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.726+0000 7fb2a7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2a806bbf0 0x7fb2a80c4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:18.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.726+0000 7fb2a7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2a806bbf0 0x7fb2a80c4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44830/0 (socket says 172.21.15.46:44830) 2023-12-02T00:29:18.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.726+0000 7fb2a7fff640 1 -- 172.21.15.46:0/242156806 learned_addr learned my addr 172.21.15.46:0/242156806 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:18.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.726+0000 7fb2a77fe640 1 --2- 172.21.15.46:0/242156806 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2a80cf000 0x7fb2a81f1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:18.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.726+0000 7fb2ac9fe640 1 --2- 172.21.15.46:0/242156806 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb2a815af70 0x7fb2a81573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:18.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.726+0000 7fb2a7fff640 1 -- 172.21.15.46:0/242156806 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb2a815af70 msgr2=0x7fb2a81573f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:18.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.726+0000 7fb2a7fff640 1 --2- 172.21.15.46:0/242156806 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb2a815af70 0x7fb2a81573f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:18.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.726+0000 7fb2a7fff640 1 -- 172.21.15.46:0/242156806 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2a80cf000 msgr2=0x7fb2a81f1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:18.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.726+0000 7fb2a7fff640 1 --2- 172.21.15.46:0/242156806 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2a80cf000 0x7fb2a81f1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:18.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.726+0000 7fb2a7fff640 1 -- 172.21.15.46:0/242156806 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a0009a70 con 0x7fb2a806bbf0 2023-12-02T00:29:18.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.726+0000 7fb2ac9fe640 1 --2- 172.21.15.46:0/242156806 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb2a815af70 0x7fb2a81573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:18.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.726+0000 7fb2a77fe640 1 --2- 172.21.15.46:0/242156806 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2a80cf000 0x7fb2a81f1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:18.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.727+0000 7fb2a7fff640 1 --2- 172.21.15.46:0/242156806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2a806bbf0 0x7fb2a80c4690 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb29800c480 tx=0x7fb29800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:18.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.727+0000 7fb2a4ff9640 1 -- 172.21.15.46:0/242156806 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb298005ce0 con 0x7fb2a806bbf0 2023-12-02T00:29:18.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.727+0000 7fb2a4ff9640 1 -- 172.21.15.46:0/242156806 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb298005e40 con 0x7fb2a806bbf0 2023-12-02T00:29:18.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.727+0000 7fb2ae461640 1 -- 172.21.15.46:0/242156806 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2a80c4fe0 con 0x7fb2a806bbf0 2023-12-02T00:29:18.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.727+0000 7fb2a4ff9640 1 -- 172.21.15.46:0/242156806 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2980072f0 con 0x7fb2a806bbf0 2023-12-02T00:29:18.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.727+0000 7fb2ae461640 1 -- 172.21.15.46:0/242156806 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2a80acc00 con 0x7fb2a806bbf0 2023-12-02T00:29:18.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.728+0000 7fb2ae461640 1 -- 172.21.15.46:0/242156806 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2a80c5170 con 0x7fb2a806bbf0 2023-12-02T00:29:18.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.728+0000 7fb2a4ff9640 1 -- 172.21.15.46:0/242156806 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb298007510 con 0x7fb2a806bbf0 2023-12-02T00:29:18.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.729+0000 7fb2a4ff9640 1 --2- 172.21.15.46:0/242156806 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb28806ae40 0x7fb28806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:18.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.729+0000 7fb2a4ff9640 1 -- 172.21.15.46:0/242156806 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb298088ab0 con 0x7fb2a806bbf0 2023-12-02T00:29:18.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.729+0000 7fb2a77fe640 1 --2- 172.21.15.46:0/242156806 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb28806ae40 0x7fb28806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:18.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.729+0000 7fb2a77fe640 1 --2- 172.21.15.46:0/242156806 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb28806ae40 0x7fb28806d300 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb2a80b2620 tx=0x7fb290008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:18.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.733+0000 7fb2a4ff9640 1 -- 172.21.15.46:0/242156806 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb298053ca0 con 0x7fb2a806bbf0 2023-12-02T00:29:18.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.911+0000 7fb2ae461640 1 -- 172.21.15.46:0/242156806 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2a8067420 con 0x7fb2a806bbf0 2023-12-02T00:29:18.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.912+0000 7fb2a4ff9640 1 -- 172.21.15.46:0/242156806 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16481 (secure 0 0 0) 0x7fb298017090 con 0x7fb2a806bbf0 2023-12-02T00:29:18.914 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:29:18.914 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":21,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:28:50.700112+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-02T00:28:47.273337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-02T00:28:48.060115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:29:18.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.913+0000 7fb2ae461640 1 -- 172.21.15.46:0/242156806 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb28806ae40 msgr2=0x7fb28806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:18.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.913+0000 7fb2ae461640 1 --2- 172.21.15.46:0/242156806 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb28806ae40 0x7fb28806d300 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb2a80b2620 tx=0x7fb290008040 comp rx=0 tx=0).stop 2023-12-02T00:29:18.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.913+0000 7fb2ae461640 1 -- 172.21.15.46:0/242156806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2a806bbf0 msgr2=0x7fb2a80c4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:18.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.913+0000 7fb2ae461640 1 --2- 172.21.15.46:0/242156806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2a806bbf0 0x7fb2a80c4690 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb29800c480 tx=0x7fb29800c940 comp rx=0 tx=0).stop 2023-12-02T00:29:18.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.913+0000 7fb2ae461640 1 -- 172.21.15.46:0/242156806 shutdown_connections 2023-12-02T00:29:18.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.913+0000 7fb2ae461640 1 --2- 172.21.15.46:0/242156806 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb28806ae40 0x7fb28806d300 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:18.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.913+0000 7fb2ae461640 1 --2- 172.21.15.46:0/242156806 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb2a815af70 0x7fb2a81573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:18.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.913+0000 7fb2ae461640 1 --2- 172.21.15.46:0/242156806 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2a80cf000 0x7fb2a81f1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:18.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.913+0000 7fb2ae461640 1 --2- 172.21.15.46:0/242156806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2a806bbf0 0x7fb2a80c4690 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:18.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.913+0000 7fb2ae461640 1 -- 172.21.15.46:0/242156806 >> 172.21.15.46:0/242156806 conn(0x7fb2a8070ec0 msgr2=0x7fb2a815d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:18.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.914+0000 7fb2ae461640 1 -- 172.21.15.46:0/242156806 shutdown_connections 2023-12-02T00:29:18.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:18.914+0000 7fb2ae461640 1 -- 172.21.15.46:0/242156806 wait complete. 2023-12-02T00:29:18.926 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-02T00:28:37.213645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-02T00:28:47.273337+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-02T00:28:48.060115+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-02T00:29:18.927 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-02T00:29:19.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.035+0000 7f292074c640 1 Processor -- start 2023-12-02T00:29:19.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.035+0000 7f292074c640 1 -- start start 2023-12-02T00:29:19.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.036+0000 7f292074c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f291806ea50 0x7f291806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:19.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.036+0000 7f292074c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2918160fb0 0x7f2918161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:19.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.036+0000 7f292074c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f29180b3ff0 0x7f29180b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:19.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.036+0000 7f292074c640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2918161d20 con 0x7f29180b3ff0 2023-12-02T00:29:19.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.036+0000 7f292074c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f291806f370 con 0x7f2918160fb0 2023-12-02T00:29:19.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.036+0000 7f292074c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f291815e760 con 0x7f291806ea50 2023-12-02T00:29:19.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.036+0000 7f291ece9640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f29180b3ff0 0x7f29180b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:19.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.036+0000 7f291ece9640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f29180b3ff0 0x7f29180b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44840/0 (socket says 172.21.15.46:44840) 2023-12-02T00:29:19.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.036+0000 7f291ece9640 1 -- 172.21.15.46:0/1433613579 learned_addr learned my addr 172.21.15.46:0/1433613579 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:19.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.037+0000 7f291ece9640 1 -- 172.21.15.46:0/1433613579 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f291806ea50 msgr2=0x7f291806ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:19.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.037+0000 7f291ece9640 1 --2- 172.21.15.46:0/1433613579 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f291806ea50 0x7f291806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:19.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.037+0000 7f291ece9640 1 -- 172.21.15.46:0/1433613579 >> v2:172.21.15.162:3300/0 conn(0x7f2918160fb0 msgr2=0x7f2918161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:19.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.037+0000 7f291dce7640 1 --2- 172.21.15.46:0/1433613579 >> v2:172.21.15.162:3300/0 conn(0x7f2918160fb0 0x7f2918161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:19.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.037+0000 7f291ece9640 1 --2- 172.21.15.46:0/1433613579 >> v2:172.21.15.162:3300/0 conn(0x7f2918160fb0 0x7f2918161410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:19.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.037+0000 7f291ece9640 1 -- 172.21.15.46:0/1433613579 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f291815d9a0 con 0x7f29180b3ff0 2023-12-02T00:29:19.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.037+0000 7f291dce7640 1 --2- 172.21.15.46:0/1433613579 >> v2:172.21.15.162:3300/0 conn(0x7f2918160fb0 0x7f2918161410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:29:19.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.037+0000 7f291ece9640 1 --2- 172.21.15.46:0/1433613579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29180b3ff0 0x7f29180b4460 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f29180d2120 tx=0x7f290802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24df918210163eb3 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:19.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.038+0000 7f291d4e6640 1 -- 172.21.15.46:0/1433613579 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290802c9e0 con 0x7f29180b3ff0 2023-12-02T00:29:19.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.038+0000 7f291d4e6640 1 -- 172.21.15.46:0/1433613579 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f290802cb40 con 0x7f29180b3ff0 2023-12-02T00:29:19.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.038+0000 7f291d4e6640 1 -- 172.21.15.46:0/1433613579 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29080343f0 con 0x7f29180b3ff0 2023-12-02T00:29:19.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.038+0000 7f292074c640 1 -- 172.21.15.46:0/1433613579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29180b3ff0 msgr2=0x7f29180b4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:19.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.038+0000 7f292074c640 1 --2- 172.21.15.46:0/1433613579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29180b3ff0 0x7f29180b4460 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f29180d2120 tx=0x7f290802be40 comp rx=0 tx=0).stop 2023-12-02T00:29:19.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.038+0000 7f292074c640 1 -- 172.21.15.46:0/1433613579 shutdown_connections 2023-12-02T00:29:19.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.038+0000 7f292074c640 1 --2- 172.21.15.46:0/1433613579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29180b3ff0 0x7f29180b4460 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:19.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.038+0000 7f292074c640 1 --2- 172.21.15.46:0/1433613579 >> v2:172.21.15.162:3300/0 conn(0x7f2918160fb0 0x7f2918161410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:19.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.038+0000 7f292074c640 1 --2- 172.21.15.46:0/1433613579 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f291806ea50 0x7f291806ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:19.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.038+0000 7f292074c640 1 -- 172.21.15.46:0/1433613579 >> 172.21.15.46:0/1433613579 conn(0x7f291806a6b0 msgr2=0x7f291806aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:19.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.038+0000 7f292074c640 1 -- 172.21.15.46:0/1433613579 shutdown_connections 2023-12-02T00:29:19.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.038+0000 7f292074c640 1 -- 172.21.15.46:0/1433613579 wait complete. 2023-12-02T00:29:19.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.039+0000 7f292074c640 1 Processor -- start 2023-12-02T00:29:19.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.040+0000 7f292074c640 1 -- start start 2023-12-02T00:29:19.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.040+0000 7f292074c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f291806ea50 0x7f2918131c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:19.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.040+0000 7f292074c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29180b3ff0 0x7f29181f6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:19.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.040+0000 7f292074c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2918160fb0 0x7f29181f2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:19.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.040+0000 7f292074c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29181f6560 con 0x7f2918160fb0 2023-12-02T00:29:19.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.040+0000 7f292074c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29181f67a0 con 0x7f29180b3ff0 2023-12-02T00:29:19.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.040+0000 7f292074c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29181f6a80 con 0x7f291806ea50 2023-12-02T00:29:19.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.040+0000 7f291ece9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2918160fb0 0x7f29181f2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:19.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.040+0000 7f291ece9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2918160fb0 0x7f29181f2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44848/0 (socket says 172.21.15.46:44848) 2023-12-02T00:29:19.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.040+0000 7f291ece9640 1 -- 172.21.15.46:0/3592160848 learned_addr learned my addr 172.21.15.46:0/3592160848 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:19.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.040+0000 7f291e4e8640 1 --2- 172.21.15.46:0/3592160848 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f291806ea50 0x7f2918131c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:19.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.040+0000 7f291dce7640 1 --2- 172.21.15.46:0/3592160848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29180b3ff0 0x7f29181f6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:19.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.041+0000 7f291ece9640 1 -- 172.21.15.46:0/3592160848 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f291806ea50 msgr2=0x7f2918131c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:19.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.041+0000 7f291ece9640 1 --2- 172.21.15.46:0/3592160848 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f291806ea50 0x7f2918131c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:19.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.041+0000 7f291ece9640 1 -- 172.21.15.46:0/3592160848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29180b3ff0 msgr2=0x7f29181f6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:19.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.041+0000 7f291ece9640 1 --2- 172.21.15.46:0/3592160848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29180b3ff0 0x7f29181f6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:19.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.041+0000 7f291ece9640 1 -- 172.21.15.46:0/3592160848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2908009b30 con 0x7f2918160fb0 2023-12-02T00:29:19.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.041+0000 7f291e4e8640 1 --2- 172.21.15.46:0/3592160848 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f291806ea50 0x7f2918131c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:19.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.041+0000 7f291dce7640 1 --2- 172.21.15.46:0/3592160848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29180b3ff0 0x7f29181f6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:19.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.041+0000 7f291ece9640 1 --2- 172.21.15.46:0/3592160848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2918160fb0 0x7f29181f2330 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2908036040 tx=0x7f2908004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:19.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.041+0000 7f2916ffd640 1 -- 172.21.15.46:0/3592160848 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2908009510 con 0x7f2918160fb0 2023-12-02T00:29:19.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.041+0000 7f2916ffd640 1 -- 172.21.15.46:0/3592160848 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f290803e450 con 0x7f2918160fb0 2023-12-02T00:29:19.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.041+0000 7f292074c640 1 -- 172.21.15.46:0/3592160848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29181f6d60 con 0x7f2918160fb0 2023-12-02T00:29:19.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.041+0000 7f2916ffd640 1 -- 172.21.15.46:0/3592160848 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290803e800 con 0x7f2918160fb0 2023-12-02T00:29:19.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.041+0000 7f292074c640 1 -- 172.21.15.46:0/3592160848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29181f2870 con 0x7f2918160fb0 2023-12-02T00:29:19.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.042+0000 7f292074c640 1 -- 172.21.15.46:0/3592160848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2918159bf0 con 0x7f2918160fb0 2023-12-02T00:29:19.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.042+0000 7f2916ffd640 1 -- 172.21.15.46:0/3592160848 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f290803e960 con 0x7f2918160fb0 2023-12-02T00:29:19.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.043+0000 7f2916ffd640 1 --2- 172.21.15.46:0/3592160848 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f290406ae40 0x7f290406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:19.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.043+0000 7f2916ffd640 1 -- 172.21.15.46:0/3592160848 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f29080ac3e0 con 0x7f2918160fb0 2023-12-02T00:29:19.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.043+0000 7f291e4e8640 1 --2- 172.21.15.46:0/3592160848 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f290406ae40 0x7f290406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:19.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.044+0000 7f291e4e8640 1 --2- 172.21.15.46:0/3592160848 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f290406ae40 0x7f290406d300 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2910009890 tx=0x7f2910008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:19.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.047+0000 7f2916ffd640 1 -- 172.21.15.46:0/3592160848 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2918159bf0 con 0x7f2918160fb0 2023-12-02T00:29:19.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.222+0000 7f292074c640 1 -- 172.21.15.46:0/3592160848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2918159bf0 con 0x7f2918160fb0 2023-12-02T00:29:19.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.222+0000 7f2916ffd640 1 -- 172.21.15.46:0/3592160848 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2918159bf0 con 0x7f2918160fb0 2023-12-02T00:29:19.226 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-12-02T00:29:19.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.224+0000 7f292074c640 1 -- 172.21.15.46:0/3592160848 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f290406ae40 msgr2=0x7f290406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:19.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.224+0000 7f292074c640 1 --2- 172.21.15.46:0/3592160848 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f290406ae40 0x7f290406d300 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2910009890 tx=0x7f2910008040 comp rx=0 tx=0).stop 2023-12-02T00:29:19.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.224+0000 7f292074c640 1 -- 172.21.15.46:0/3592160848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2918160fb0 msgr2=0x7f29181f2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:19.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.224+0000 7f292074c640 1 --2- 172.21.15.46:0/3592160848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2918160fb0 0x7f29181f2330 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2908036040 tx=0x7f2908004a40 comp rx=0 tx=0).stop 2023-12-02T00:29:19.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.224+0000 7f292074c640 1 -- 172.21.15.46:0/3592160848 shutdown_connections 2023-12-02T00:29:19.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.224+0000 7f292074c640 1 --2- 172.21.15.46:0/3592160848 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f290406ae40 0x7f290406d300 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:19.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.224+0000 7f292074c640 1 --2- 172.21.15.46:0/3592160848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2918160fb0 0x7f29181f2330 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:19.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.224+0000 7f292074c640 1 --2- 172.21.15.46:0/3592160848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29180b3ff0 0x7f29181f6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:19.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.224+0000 7f292074c640 1 --2- 172.21.15.46:0/3592160848 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f291806ea50 0x7f2918131c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:19.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.224+0000 7f292074c640 1 -- 172.21.15.46:0/3592160848 >> 172.21.15.46:0/3592160848 conn(0x7f291806a6b0 msgr2=0x7f29180b4b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:19.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.224+0000 7f292074c640 1 -- 172.21.15.46:0/3592160848 shutdown_connections 2023-12-02T00:29:19.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.224+0000 7f292074c640 1 -- 172.21.15.46:0/3592160848 wait complete. 2023-12-02T00:29:19.236 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-02T00:29:19.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.345+0000 7fd5218b4640 1 Processor -- start 2023-12-02T00:29:19.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.345+0000 7fd5218b4640 1 -- start start 2023-12-02T00:29:19.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.346+0000 7fd5218b4640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd51c15d330 0x7fd51c0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:19.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.346+0000 7fd5218b4640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd51c0d2910 0x7fd51c0d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:19.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.346+0000 7fd5218b4640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd51c0b2b10 con 0x7fd51c0d3730 2023-12-02T00:29:19.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.346+0000 7fd5218b4640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd51c0d3bc0 con 0x7fd51c0d2910 2023-12-02T00:29:19.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.346+0000 7fd5218b4640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd51c0d1590 con 0x7fd51c15d330 2023-12-02T00:29:19.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.346+0000 7fd51a7fc640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd51c0d2910 0x7fd51c0d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:19.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.346+0000 7fd51a7fc640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd51c0d2910 0x7fd51c0d31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44862/0 (socket says 172.21.15.46:44862) 2023-12-02T00:29:19.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.346+0000 7fd51a7fc640 1 -- 172.21.15.46:0/3245338216 learned_addr learned my addr 172.21.15.46:0/3245338216 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:19.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.346+0000 7fd51affd640 1 --2- 172.21.15.46:0/3245338216 >> v2:172.21.15.162:3300/0 conn(0x7fd51c15d330 0x7fd51c0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:19.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.347+0000 7fd51a7fc640 1 -- 172.21.15.46:0/3245338216 >> v2:172.21.15.162:3300/0 conn(0x7fd51c15d330 msgr2=0x7fd51c0b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:19.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.347+0000 7fd51a7fc640 1 --2- 172.21.15.46:0/3245338216 >> v2:172.21.15.162:3300/0 conn(0x7fd51c15d330 0x7fd51c0b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:19.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.347+0000 7fd51a7fc640 1 -- 172.21.15.46:0/3245338216 >> v1:172.21.15.162:6789/0 conn(0x7fd51c0d3730 legacy=0x7fd51c0b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:19.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.347+0000 7fd51a7fc640 1 -- 172.21.15.46:0/3245338216 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd51c0d1870 con 0x7fd51c0d2910 2023-12-02T00:29:19.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.347+0000 7fd51a7fc640 1 --2- 172.21.15.46:0/3245338216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd51c0d2910 0x7fd51c0d31f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd51c0d0d10 tx=0x7fd50402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b669200485e8e88 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:19.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.347+0000 7fd519ffb640 1 -- 172.21.15.46:0/3245338216 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5040317e0 con 0x7fd51c0d2910 2023-12-02T00:29:19.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.347+0000 7fd519ffb640 1 -- 172.21.15.46:0/3245338216 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd504031940 con 0x7fd51c0d2910 2023-12-02T00:29:19.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.347+0000 7fd519ffb640 1 -- 172.21.15.46:0/3245338216 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd504031cd0 con 0x7fd51c0d2910 2023-12-02T00:29:19.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.348+0000 7fd5218b4640 1 -- 172.21.15.46:0/3245338216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd51c0d2910 msgr2=0x7fd51c0d31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:19.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.348+0000 7fd5218b4640 1 --2- 172.21.15.46:0/3245338216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd51c0d2910 0x7fd51c0d31f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd51c0d0d10 tx=0x7fd50402b880 comp rx=0 tx=0).stop 2023-12-02T00:29:19.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.348+0000 7fd5218b4640 1 -- 172.21.15.46:0/3245338216 shutdown_connections 2023-12-02T00:29:19.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.348+0000 7fd5218b4640 1 --2- 172.21.15.46:0/3245338216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd51c0d2910 0x7fd51c0d31f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:19.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.348+0000 7fd5218b4640 1 --2- 172.21.15.46:0/3245338216 >> v2:172.21.15.162:3300/0 conn(0x7fd51c15d330 0x7fd51c0b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:19.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.348+0000 7fd5218b4640 1 -- 172.21.15.46:0/3245338216 >> 172.21.15.46:0/3245338216 conn(0x7fd51c072020 msgr2=0x7fd51c072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:19.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.348+0000 7fd5218b4640 1 -- 172.21.15.46:0/3245338216 shutdown_connections 2023-12-02T00:29:19.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.348+0000 7fd5218b4640 1 -- 172.21.15.46:0/3245338216 wait complete. 2023-12-02T00:29:19.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.349+0000 7fd5218b4640 1 Processor -- start 2023-12-02T00:29:19.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.349+0000 7fd5218b4640 1 -- start start 2023-12-02T00:29:19.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.350+0000 7fd5218b4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd51c0d2910 0x7fd51c14f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:19.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.350+0000 7fd5218b4640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd51c0d3730 0x7fd51c15d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:19.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.350+0000 7fd5218b4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd51c15d330 0x7fd51c0cc760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:19.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.350+0000 7fd5218b4640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51c1f3790 con 0x7fd51c15d330 2023-12-02T00:29:19.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.350+0000 7fd5218b4640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51c0cb410 con 0x7fd51c0d3730 2023-12-02T00:29:19.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.350+0000 7fd5218b4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd51c0cb6f0 con 0x7fd51c0d2910 2023-12-02T00:29:19.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.350+0000 7fd51b7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd51c15d330 0x7fd51c0cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:19.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.350+0000 7fd51b7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd51c15d330 0x7fd51c0cc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44876/0 (socket says 172.21.15.46:44876) 2023-12-02T00:29:19.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.350+0000 7fd51b7fe640 1 -- 172.21.15.46:0/1763779205 learned_addr learned my addr 172.21.15.46:0/1763779205 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:19.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.350+0000 7fd51a7fc640 1 --2- 172.21.15.46:0/1763779205 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd51c0d3730 0x7fd51c15d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:19.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.350+0000 7fd51affd640 1 --2- 172.21.15.46:0/1763779205 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd51c0d2910 0x7fd51c14f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:19.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.351+0000 7fd51b7fe640 1 -- 172.21.15.46:0/1763779205 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd51c0d2910 msgr2=0x7fd51c14f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:19.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.351+0000 7fd51b7fe640 1 --2- 172.21.15.46:0/1763779205 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd51c0d2910 0x7fd51c14f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:19.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.351+0000 7fd51b7fe640 1 -- 172.21.15.46:0/1763779205 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd51c0d3730 msgr2=0x7fd51c15d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:19.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.351+0000 7fd51b7fe640 1 --2- 172.21.15.46:0/1763779205 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd51c0d3730 0x7fd51c15d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:19.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.351+0000 7fd51b7fe640 1 -- 172.21.15.46:0/1763779205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5040094d0 con 0x7fd51c15d330 2023-12-02T00:29:19.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.351+0000 7fd51b7fe640 1 --2- 172.21.15.46:0/1763779205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd51c15d330 0x7fd51c0cc760 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd50c00c780 tx=0x7fd50c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:19.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.351+0000 7fd513fff640 1 -- 172.21.15.46:0/1763779205 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50c004b60 con 0x7fd51c15d330 2023-12-02T00:29:19.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.351+0000 7fd513fff640 1 -- 172.21.15.46:0/1763779205 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd50c00ce90 con 0x7fd51c15d330 2023-12-02T00:29:19.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.351+0000 7fd513fff640 1 -- 172.21.15.46:0/1763779205 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50c00a850 con 0x7fd51c15d330 2023-12-02T00:29:19.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.351+0000 7fd5218b4640 1 -- 172.21.15.46:0/1763779205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd51c0cb9d0 con 0x7fd51c15d330 2023-12-02T00:29:19.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.351+0000 7fd5218b4640 1 -- 172.21.15.46:0/1763779205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd51c0cb100 con 0x7fd51c15d330 2023-12-02T00:29:19.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.352+0000 7fd5218b4640 1 -- 172.21.15.46:0/1763779205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd51c0b1eb0 con 0x7fd51c15d330 2023-12-02T00:29:19.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.352+0000 7fd513fff640 1 -- 172.21.15.46:0/1763779205 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd50c00a9b0 con 0x7fd51c15d330 2023-12-02T00:29:19.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.353+0000 7fd513fff640 1 --2- 172.21.15.46:0/1763779205 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd4fc06ae40 0x7fd4fc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:19.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.353+0000 7fd513fff640 1 -- 172.21.15.46:0/1763779205 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd50c0896d0 con 0x7fd51c15d330 2023-12-02T00:29:19.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.353+0000 7fd51affd640 1 --2- 172.21.15.46:0/1763779205 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd4fc06ae40 0x7fd4fc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:19.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.354+0000 7fd51affd640 1 --2- 172.21.15.46:0/1763779205 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd4fc06ae40 0x7fd4fc06d300 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd5140079a0 tx=0x7fd514008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:19.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.357+0000 7fd513fff640 1 -- 172.21.15.46:0/1763779205 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd50c0547a0 con 0x7fd51c15d330 2023-12-02T00:29:19.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.533+0000 7fd5218b4640 1 -- 172.21.15.46:0/1763779205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd51c0d3170 con 0x7fd51c15d330 2023-12-02T00:29:19.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.533+0000 7fd513fff640 1 -- 172.21.15.46:0/1763779205 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd50c057dc0 con 0x7fd51c15d330 2023-12-02T00:29:19.535 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-12-02T00:29:19.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.535+0000 7fd5218b4640 1 -- 172.21.15.46:0/1763779205 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd4fc06ae40 msgr2=0x7fd4fc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:19.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.535+0000 7fd5218b4640 1 --2- 172.21.15.46:0/1763779205 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd4fc06ae40 0x7fd4fc06d300 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd5140079a0 tx=0x7fd514008040 comp rx=0 tx=0).stop 2023-12-02T00:29:19.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.535+0000 7fd5218b4640 1 -- 172.21.15.46:0/1763779205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd51c15d330 msgr2=0x7fd51c0cc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:19.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.535+0000 7fd5218b4640 1 --2- 172.21.15.46:0/1763779205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd51c15d330 0x7fd51c0cc760 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd50c00c780 tx=0x7fd50c00cc40 comp rx=0 tx=0).stop 2023-12-02T00:29:19.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.535+0000 7fd5218b4640 1 -- 172.21.15.46:0/1763779205 shutdown_connections 2023-12-02T00:29:19.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.535+0000 7fd5218b4640 1 --2- 172.21.15.46:0/1763779205 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd4fc06ae40 0x7fd4fc06d300 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:19.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.535+0000 7fd5218b4640 1 --2- 172.21.15.46:0/1763779205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd51c15d330 0x7fd51c0cc760 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:19.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.535+0000 7fd5218b4640 1 --2- 172.21.15.46:0/1763779205 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd51c0d3730 0x7fd51c15d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:19.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.535+0000 7fd5218b4640 1 --2- 172.21.15.46:0/1763779205 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd51c0d2910 0x7fd51c14f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:19.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.535+0000 7fd5218b4640 1 -- 172.21.15.46:0/1763779205 >> 172.21.15.46:0/1763779205 conn(0x7fd51c072020 msgr2=0x7fd51c072400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:19.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.535+0000 7fd5218b4640 1 -- 172.21.15.46:0/1763779205 shutdown_connections 2023-12-02T00:29:19.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.535+0000 7fd5218b4640 1 -- 172.21.15.46:0/1763779205 wait complete. 2023-12-02T00:29:19.547 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-02T00:29:19.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.656+0000 7f65ffa36640 1 Processor -- start 2023-12-02T00:29:19.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.656+0000 7f65ffa36640 1 -- start start 2023-12-02T00:29:19.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.657+0000 7f65ffa36640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f65f8160fb0 0x7f65f8161440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:19.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.657+0000 7f65ffa36640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f65f80d0ed0 0x7f65f80d1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:19.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.657+0000 7f65ffa36640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65f8161d20 con 0x7f65f806ea50 2023-12-02T00:29:19.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.657+0000 7f65ffa36640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65f815e760 con 0x7f65f8160fb0 2023-12-02T00:29:19.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.657+0000 7f65ffa36640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65f815d9a0 con 0x7f65f80d0ed0 2023-12-02T00:29:19.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.657+0000 7f65fd7d2640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f65f806ea50 0x7f65f806ee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38630/0 (socket says 172.21.15.46:38630) 2023-12-02T00:29:19.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.657+0000 7f65fd7d2640 1 -- 172.21.15.46:0/2959909508 learned_addr learned my addr 172.21.15.46:0/2959909508 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:19.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.657+0000 7f65effff640 1 -- 172.21.15.46:0/2959909508 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2843548917 0 0) 0x7f65f8161d20 con 0x7f65f806ea50 2023-12-02T00:29:19.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.657+0000 7f65fdfd3640 1 --2- 172.21.15.46:0/2959909508 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f65f80d0ed0 0x7f65f80d1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:19.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.657+0000 7f65fcfd1640 1 --2- 172.21.15.46:0/2959909508 >> v2:172.21.15.162:3300/0 conn(0x7f65f8160fb0 0x7f65f8161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:19.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.657+0000 7f65effff640 1 -- 172.21.15.46:0/2959909508 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65e4003520 con 0x7f65f806ea50 2023-12-02T00:29:19.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.657+0000 7f65effff640 1 -- 172.21.15.46:0/2959909508 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1585974916 0 0) 0x7f65e4003520 con 0x7f65f806ea50 2023-12-02T00:29:19.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.657+0000 7f65effff640 1 -- 172.21.15.46:0/2959909508 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f65f80d0ed0 msgr2=0x7f65f80d1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:19.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.657+0000 7f65effff640 1 --2- 172.21.15.46:0/2959909508 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f65f80d0ed0 0x7f65f80d1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:19.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.657+0000 7f65effff640 1 -- 172.21.15.46:0/2959909508 >> v2:172.21.15.162:3300/0 conn(0x7f65f8160fb0 msgr2=0x7f65f8161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:19.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.657+0000 7f65effff640 1 --2- 172.21.15.46:0/2959909508 >> v2:172.21.15.162:3300/0 conn(0x7f65f8160fb0 0x7f65f8161440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:19.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.657+0000 7f65effff640 1 -- 172.21.15.46:0/2959909508 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f815dc80 con 0x7f65f806ea50 2023-12-02T00:29:19.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.657+0000 7f65effff640 1 -- 172.21.15.46:0/2959909508 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7f65f4001670 con 0x7f65f806ea50 2023-12-02T00:29:19.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.657+0000 7f65effff640 1 -- 172.21.15.46:0/2959909508 >> v1:172.21.15.46:6789/0 conn(0x7f65f806ea50 legacy=0x7f65f806ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:19.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.657+0000 7f65effff640 1 --2- 172.21.15.46:0/2959909508 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65e4003970 0x7f65e4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:19.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.657+0000 7f65effff640 1 --2- 172.21.15.46:0/2959909508 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f65f80d0ed0 0x7f65e40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:19.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.657+0000 7f65effff640 1 --2- 172.21.15.46:0/2959909508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65e4004d00 0x7f65e4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:19.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.657+0000 7f65effff640 1 -- 172.21.15.46:0/2959909508 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f815e760 con 0x7f65e4004d00 2023-12-02T00:29:19.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.657+0000 7f65effff640 1 -- 172.21.15.46:0/2959909508 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f815d9a0 con 0x7f65e4003970 2023-12-02T00:29:19.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.657+0000 7f65effff640 1 -- 172.21.15.46:0/2959909508 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e400b980 con 0x7f65f80d0ed0 2023-12-02T00:29:19.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.658+0000 7f65fdfd3640 1 --2- 172.21.15.46:0/2959909508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65e4004d00 0x7f65e4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:19.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.658+0000 7f65fd7d2640 1 --2- 172.21.15.46:0/2959909508 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65e4003970 0x7f65e4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:19.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.658+0000 7f65fcfd1640 1 --2- 172.21.15.46:0/2959909508 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f65f80d0ed0 0x7f65e40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:19.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.658+0000 7f65fdfd3640 1 -- 172.21.15.46:0/2959909508 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f65f80d0ed0 msgr2=0x7f65e40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:19.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.658+0000 7f65fdfd3640 1 --2- 172.21.15.46:0/2959909508 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f65f80d0ed0 0x7f65e40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:19.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.658+0000 7f65fdfd3640 1 -- 172.21.15.46:0/2959909508 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65e4003970 msgr2=0x7f65e4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:19.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.658+0000 7f65fdfd3640 1 --2- 172.21.15.46:0/2959909508 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65e4003970 0x7f65e4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:19.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.658+0000 7f65fdfd3640 1 -- 172.21.15.46:0/2959909508 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e4003520 con 0x7f65e4004d00 2023-12-02T00:29:19.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.659+0000 7f65fdfd3640 1 --2- 172.21.15.46:0/2959909508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65e4004d00 0x7f65e4005180 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f65f0009fc0 tx=0x7f65e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd8fe40c0d4cef9 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:19.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.659+0000 7f65effff640 1 -- 172.21.15.46:0/2959909508 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f002cc20 con 0x7f65e4004d00 2023-12-02T00:29:19.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.659+0000 7f65effff640 1 -- 172.21.15.46:0/2959909508 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65f000f040 con 0x7f65e4004d00 2023-12-02T00:29:19.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.659+0000 7f65effff640 1 -- 172.21.15.46:0/2959909508 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f0034440 con 0x7f65e4004d00 2023-12-02T00:29:19.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.659+0000 7f65ffa36640 1 -- 172.21.15.46:0/2959909508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65e4004d00 msgr2=0x7f65e4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:19.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.659+0000 7f65ffa36640 1 --2- 172.21.15.46:0/2959909508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65e4004d00 0x7f65e4005180 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f65f0009fc0 tx=0x7f65e40057f0 comp rx=0 tx=0).stop 2023-12-02T00:29:19.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.659+0000 7f65fdfd3640 1 -- 172.21.15.46:0/2959909508 reap_dead start 2023-12-02T00:29:19.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.659+0000 7f65ffa36640 1 -- 172.21.15.46:0/2959909508 shutdown_connections 2023-12-02T00:29:19.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.659+0000 7f65ffa36640 1 -- 172.21.15.46:0/2959909508 >> 172.21.15.46:0/2959909508 conn(0x7f65f806a6b0 msgr2=0x7f65f806aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:19.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.659+0000 7f65ffa36640 1 -- 172.21.15.46:0/2959909508 shutdown_connections 2023-12-02T00:29:19.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.659+0000 7f65ffa36640 1 -- 172.21.15.46:0/2959909508 wait complete. 2023-12-02T00:29:19.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.660+0000 7f65ffa36640 1 Processor -- start 2023-12-02T00:29:19.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.661+0000 7f65ffa36640 1 -- start start 2023-12-02T00:29:19.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.661+0000 7f65ffa36640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f65e4004d00 0x7f65f8159f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:19.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.661+0000 7f65ffa36640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65f81595e0 0x7f65f8156ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:19.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.661+0000 7f65ffa36640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65f8157420 0x7f65f80b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:19.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.661+0000 7f65ffa36640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f80ccf30 con 0x7f65f81595e0 2023-12-02T00:29:19.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.661+0000 7f65ffa36640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f8132780 con 0x7f65f8157420 2023-12-02T00:29:19.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.661+0000 7f65ffa36640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65f8132a60 con 0x7f65e4004d00 2023-12-02T00:29:19.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.661+0000 7f65fcfd1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65f81595e0 0x7f65f8156ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:19.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.661+0000 7f65fcfd1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65f81595e0 0x7f65f8156ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44880/0 (socket says 172.21.15.46:44880) 2023-12-02T00:29:19.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.661+0000 7f65fcfd1640 1 -- 172.21.15.46:0/844430114 learned_addr learned my addr 172.21.15.46:0/844430114 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:19.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.662+0000 7f65fd7d2640 1 --2- 172.21.15.46:0/844430114 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f65e4004d00 0x7f65f8159f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:19.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.662+0000 7f65fdfd3640 1 --2- 172.21.15.46:0/844430114 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65f8157420 0x7f65f80b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:19.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.662+0000 7f65fcfd1640 1 -- 172.21.15.46:0/844430114 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f65e4004d00 msgr2=0x7f65f8159f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:19.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.662+0000 7f65fcfd1640 1 --2- 172.21.15.46:0/844430114 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f65e4004d00 0x7f65f8159f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:19.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.662+0000 7f65fcfd1640 1 -- 172.21.15.46:0/844430114 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65f8157420 msgr2=0x7f65f80b1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:19.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.662+0000 7f65fcfd1640 1 --2- 172.21.15.46:0/844430114 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65f8157420 0x7f65f80b1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:19.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.662+0000 7f65fcfd1640 1 -- 172.21.15.46:0/844430114 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f8132d40 con 0x7f65f81595e0 2023-12-02T00:29:19.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.662+0000 7f65fd7d2640 1 --2- 172.21.15.46:0/844430114 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f65e4004d00 0x7f65f8159f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:19.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.662+0000 7f65fdfd3640 1 --2- 172.21.15.46:0/844430114 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65f8157420 0x7f65f80b1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:19.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.662+0000 7f65fcfd1640 1 --2- 172.21.15.46:0/844430114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65f81595e0 0x7f65f8156ee0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f65e8011a70 tx=0x7f65e8011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:19.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.662+0000 7f65edffb640 1 -- 172.21.15.46:0/844430114 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e8009680 con 0x7f65f81595e0 2023-12-02T00:29:19.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.662+0000 7f65edffb640 1 -- 172.21.15.46:0/844430114 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65e800fb50 con 0x7f65f81595e0 2023-12-02T00:29:19.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.662+0000 7f65edffb640 1 -- 172.21.15.46:0/844430114 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e800fee0 con 0x7f65f81595e0 2023-12-02T00:29:19.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.662+0000 7f65ffa36640 1 -- 172.21.15.46:0/844430114 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65f8133020 con 0x7f65f81595e0 2023-12-02T00:29:19.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.662+0000 7f65ffa36640 1 -- 172.21.15.46:0/844430114 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65f80b2090 con 0x7f65f81595e0 2023-12-02T00:29:19.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.663+0000 7f65ffa36640 1 -- 172.21.15.46:0/844430114 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65f8069400 con 0x7f65f81595e0 2023-12-02T00:29:19.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.664+0000 7f65edffb640 1 -- 172.21.15.46:0/844430114 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f65e8014440 con 0x7f65f81595e0 2023-12-02T00:29:19.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.664+0000 7f65edffb640 1 --2- 172.21.15.46:0/844430114 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f65dc06ae70 0x7f65dc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:19.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.664+0000 7f65edffb640 1 -- 172.21.15.46:0/844430114 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f65e8089eb0 con 0x7f65f81595e0 2023-12-02T00:29:19.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.664+0000 7f65fd7d2640 1 --2- 172.21.15.46:0/844430114 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f65dc06ae70 0x7f65dc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:19.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.665+0000 7f65fd7d2640 1 --2- 172.21.15.46:0/844430114 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f65dc06ae70 0x7f65dc06d330 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f65f4007dd0 tx=0x7f65f4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:19.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.668+0000 7f65edffb640 1 -- 172.21.15.46:0/844430114 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f65e8054f60 con 0x7f65f81595e0 2023-12-02T00:29:19.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.846+0000 7f65ffa36640 1 -- 172.21.15.46:0/844430114 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f65f8070d30 con 0x7f65f81595e0 2023-12-02T00:29:19.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.847+0000 7f65edffb640 1 -- 172.21.15.46:0/844430114 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f65e8022020 con 0x7f65f81595e0 2023-12-02T00:29:19.848 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-12-02T00:29:19.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.849+0000 7f65ffa36640 1 -- 172.21.15.46:0/844430114 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f65dc06ae70 msgr2=0x7f65dc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:19.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.849+0000 7f65ffa36640 1 --2- 172.21.15.46:0/844430114 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f65dc06ae70 0x7f65dc06d330 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f65f4007dd0 tx=0x7f65f4009040 comp rx=0 tx=0).stop 2023-12-02T00:29:19.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.849+0000 7f65ffa36640 1 -- 172.21.15.46:0/844430114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65f81595e0 msgr2=0x7f65f8156ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:19.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.849+0000 7f65ffa36640 1 --2- 172.21.15.46:0/844430114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65f81595e0 0x7f65f8156ee0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f65e8011a70 tx=0x7f65e8011f30 comp rx=0 tx=0).stop 2023-12-02T00:29:19.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.849+0000 7f65ffa36640 1 -- 172.21.15.46:0/844430114 shutdown_connections 2023-12-02T00:29:19.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.849+0000 7f65ffa36640 1 --2- 172.21.15.46:0/844430114 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f65dc06ae70 0x7f65dc06d330 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:19.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.849+0000 7f65ffa36640 1 --2- 172.21.15.46:0/844430114 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65f8157420 0x7f65f80b1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:19.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.849+0000 7f65ffa36640 1 --2- 172.21.15.46:0/844430114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65f81595e0 0x7f65f8156ee0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:19.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.849+0000 7f65ffa36640 1 --2- 172.21.15.46:0/844430114 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f65e4004d00 0x7f65f8159f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:19.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.849+0000 7f65ffa36640 1 -- 172.21.15.46:0/844430114 >> 172.21.15.46:0/844430114 conn(0x7f65f806a6b0 msgr2=0x7f65f8140a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:19.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.849+0000 7f65ffa36640 1 -- 172.21.15.46:0/844430114 shutdown_connections 2023-12-02T00:29:19.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.849+0000 7f65ffa36640 1 -- 172.21.15.46:0/844430114 wait complete. 2023-12-02T00:29:19.881 INFO:tasks.cephfs_test_runner:Starting test: test_forbidden_modification (tasks.cephfs.test_pool_perm.TestPoolPerm) That a client who does not have the capability for setting 2023-12-02T00:29:19.881 INFO:tasks.cephfs_test_runner:test_forbidden_modification (tasks.cephfs.test_pool_perm.TestPoolPerm) 2023-12-02T00:29:19.881 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:29:19.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.987+0000 7fea68a75640 1 Processor -- start 2023-12-02T00:29:19.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.987+0000 7fea68a75640 1 -- start start 2023-12-02T00:29:19.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.988+0000 7fea68a75640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea6415d2e0 0x7fea640d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:19.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.988+0000 7fea68a75640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fea6415d6c0 0x7fea640d1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:19.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.988+0000 7fea68a75640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea6406ce00 con 0x7fea6406b4c0 2023-12-02T00:29:19.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.988+0000 7fea68a75640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea6406c1f0 con 0x7fea6415d6c0 2023-12-02T00:29:19.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.988+0000 7fea68a75640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea6406c4d0 con 0x7fea6415d2e0 2023-12-02T00:29:19.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.989+0000 7fea62d9d640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fea6406b4c0 0x7fea6406b910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38640/0 (socket says 172.21.15.46:38640) 2023-12-02T00:29:19.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.989+0000 7fea62d9d640 1 -- 172.21.15.46:0/4093668342 learned_addr learned my addr 172.21.15.46:0/4093668342 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:19.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.989+0000 7fea61d9b640 1 --2- 172.21.15.46:0/4093668342 >> v2:172.21.15.162:3300/0 conn(0x7fea6415d6c0 0x7fea640d1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:19.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.989+0000 7fea6259c640 1 --2- 172.21.15.46:0/4093668342 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea6415d2e0 0x7fea640d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:19.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.989+0000 7fea6159a640 1 -- 172.21.15.46:0/4093668342 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 446828803 0 0) 0x7fea6406ce00 con 0x7fea6406b4c0 2023-12-02T00:29:19.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.989+0000 7fea6159a640 1 -- 172.21.15.46:0/4093668342 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea50003520 con 0x7fea6406b4c0 2023-12-02T00:29:19.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.989+0000 7fea6159a640 1 -- 172.21.15.46:0/4093668342 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1516783067 0 0) 0x7fea50003520 con 0x7fea6406b4c0 2023-12-02T00:29:19.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.989+0000 7fea6159a640 1 -- 172.21.15.46:0/4093668342 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea6415d2e0 msgr2=0x7fea640d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:19.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.989+0000 7fea6159a640 1 --2- 172.21.15.46:0/4093668342 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea6415d2e0 0x7fea640d1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:19.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.989+0000 7fea6159a640 1 -- 172.21.15.46:0/4093668342 >> v2:172.21.15.162:3300/0 conn(0x7fea6415d6c0 msgr2=0x7fea640d1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:19.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.989+0000 7fea6159a640 1 --2- 172.21.15.46:0/4093668342 >> v2:172.21.15.162:3300/0 conn(0x7fea6415d6c0 0x7fea640d1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:19.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.989+0000 7fea6159a640 1 -- 172.21.15.46:0/4093668342 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea6406c7b0 con 0x7fea6406b4c0 2023-12-02T00:29:19.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.989+0000 7fea6159a640 1 -- 172.21.15.46:0/4093668342 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7fea5c001670 con 0x7fea6406b4c0 2023-12-02T00:29:19.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.989+0000 7fea6159a640 1 -- 172.21.15.46:0/4093668342 >> v1:172.21.15.46:6789/0 conn(0x7fea6406b4c0 legacy=0x7fea6406b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:19.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.989+0000 7fea6159a640 1 --2- 172.21.15.46:0/4093668342 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea6415d2e0 0x7fea50003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:19.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.989+0000 7fea6159a640 1 --2- 172.21.15.46:0/4093668342 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea50003ef0 0x7fea500047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:19.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.989+0000 7fea6159a640 1 --2- 172.21.15.46:0/4093668342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea50004d00 0x7fea50005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:19.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.989+0000 7fea6159a640 1 -- 172.21.15.46:0/4093668342 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6406c1f0 con 0x7fea50004d00 2023-12-02T00:29:19.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.989+0000 7fea6159a640 1 -- 172.21.15.46:0/4093668342 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6406c4d0 con 0x7fea50003ef0 2023-12-02T00:29:19.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.989+0000 7fea6159a640 1 -- 172.21.15.46:0/4093668342 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea5000b9d0 con 0x7fea6415d2e0 2023-12-02T00:29:19.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.989+0000 7fea62d9d640 1 --2- 172.21.15.46:0/4093668342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea50004d00 0x7fea50005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:19.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.989+0000 7fea6259c640 1 --2- 172.21.15.46:0/4093668342 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea6415d2e0 0x7fea50003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:19.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.989+0000 7fea61d9b640 1 --2- 172.21.15.46:0/4093668342 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea50003ef0 0x7fea500047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:19.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.990+0000 7fea62d9d640 1 -- 172.21.15.46:0/4093668342 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea6415d2e0 msgr2=0x7fea50003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:19.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.990+0000 7fea62d9d640 1 --2- 172.21.15.46:0/4093668342 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea6415d2e0 0x7fea50003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:19.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.990+0000 7fea62d9d640 1 -- 172.21.15.46:0/4093668342 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea50003ef0 msgr2=0x7fea500047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:19.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.990+0000 7fea62d9d640 1 --2- 172.21.15.46:0/4093668342 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea50003ef0 0x7fea500047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:19.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.990+0000 7fea62d9d640 1 -- 172.21.15.46:0/4093668342 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea50003520 con 0x7fea50004d00 2023-12-02T00:29:19.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.990+0000 7fea62d9d640 1 --2- 172.21.15.46:0/4093668342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea50004d00 0x7fea50005180 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fea5c00c240 tx=0x7fea500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1f4a855a2df2cf1 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:19.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.990+0000 7fea6159a640 1 -- 172.21.15.46:0/4093668342 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5c02cae0 con 0x7fea50004d00 2023-12-02T00:29:19.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.990+0000 7fea6159a640 1 -- 172.21.15.46:0/4093668342 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea5c02cc40 con 0x7fea50004d00 2023-12-02T00:29:19.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.990+0000 7fea6159a640 1 -- 172.21.15.46:0/4093668342 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5c0058d0 con 0x7fea50004d00 2023-12-02T00:29:19.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.991+0000 7fea68a75640 1 -- 172.21.15.46:0/4093668342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea50004d00 msgr2=0x7fea50005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:19.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.991+0000 7fea68a75640 1 --2- 172.21.15.46:0/4093668342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea50004d00 0x7fea50005180 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fea5c00c240 tx=0x7fea500057f0 comp rx=0 tx=0).stop 2023-12-02T00:29:19.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.991+0000 7fea62d9d640 1 -- 172.21.15.46:0/4093668342 reap_dead start 2023-12-02T00:29:19.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.991+0000 7fea68a75640 1 -- 172.21.15.46:0/4093668342 shutdown_connections 2023-12-02T00:29:19.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.991+0000 7fea68a75640 1 -- 172.21.15.46:0/4093668342 >> 172.21.15.46:0/4093668342 conn(0x7fea6409c580 msgr2=0x7fea640d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:19.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.991+0000 7fea68a75640 1 -- 172.21.15.46:0/4093668342 shutdown_connections 2023-12-02T00:29:19.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.991+0000 7fea68a75640 1 -- 172.21.15.46:0/4093668342 wait complete. 2023-12-02T00:29:19.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.992+0000 7fea68a75640 1 Processor -- start 2023-12-02T00:29:19.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.993+0000 7fea68a75640 1 -- start start 2023-12-02T00:29:20.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.993+0000 7fea68a75640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea50004d00 0x7fea641587e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:20.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.993+0000 7fea68a75640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea641f6620 0x7fea6415f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:20.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.993+0000 7fea68a75640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea641f6aa0 0x7fea6415e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:20.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.993+0000 7fea68a75640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea64158d20 con 0x7fea641f6aa0 2023-12-02T00:29:20.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.993+0000 7fea68a75640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea641f75c0 con 0x7fea50004d00 2023-12-02T00:29:20.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.993+0000 7fea68a75640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea6415dfe0 con 0x7fea641f6620 2023-12-02T00:29:20.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.993+0000 7fea62d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea641f6aa0 0x7fea6415e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:20.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.993+0000 7fea62d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea641f6aa0 0x7fea6415e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44898/0 (socket says 172.21.15.46:44898) 2023-12-02T00:29:20.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.993+0000 7fea62d9d640 1 -- 172.21.15.46:0/3208547124 learned_addr learned my addr 172.21.15.46:0/3208547124 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:20.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.993+0000 7fea6259c640 1 --2- 172.21.15.46:0/3208547124 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea50004d00 0x7fea641587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:20.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.994+0000 7fea61d9b640 1 --2- 172.21.15.46:0/3208547124 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea641f6620 0x7fea6415f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:20.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.994+0000 7fea62d9d640 1 -- 172.21.15.46:0/3208547124 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea641f6620 msgr2=0x7fea6415f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:20.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.994+0000 7fea62d9d640 1 --2- 172.21.15.46:0/3208547124 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea641f6620 0x7fea6415f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:20.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.994+0000 7fea62d9d640 1 -- 172.21.15.46:0/3208547124 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea50004d00 msgr2=0x7fea641587e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:20.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.994+0000 7fea62d9d640 1 --2- 172.21.15.46:0/3208547124 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea50004d00 0x7fea641587e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:20.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.994+0000 7fea62d9d640 1 -- 172.21.15.46:0/3208547124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea641f7000 con 0x7fea641f6aa0 2023-12-02T00:29:20.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.994+0000 7fea61d9b640 1 --2- 172.21.15.46:0/3208547124 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea641f6620 0x7fea6415f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:20.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.994+0000 7fea62d9d640 1 --2- 172.21.15.46:0/3208547124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea641f6aa0 0x7fea6415e470 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fea5c00c240 tx=0x7fea5c001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:20.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.994+0000 7fea5affd640 1 -- 172.21.15.46:0/3208547124 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5c03c070 con 0x7fea641f6aa0 2023-12-02T00:29:20.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.994+0000 7fea5affd640 1 -- 172.21.15.46:0/3208547124 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea5c001430 con 0x7fea641f6aa0 2023-12-02T00:29:20.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.994+0000 7fea68a75640 1 -- 172.21.15.46:0/3208547124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea641f72e0 con 0x7fea641f6aa0 2023-12-02T00:29:20.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.994+0000 7fea5affd640 1 -- 172.21.15.46:0/3208547124 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5c007070 con 0x7fea641f6aa0 2023-12-02T00:29:20.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.994+0000 7fea68a75640 1 -- 172.21.15.46:0/3208547124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea640ccea0 con 0x7fea641f6aa0 2023-12-02T00:29:20.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.995+0000 7fea68a75640 1 -- 172.21.15.46:0/3208547124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea6415dad0 con 0x7fea641f6aa0 2023-12-02T00:29:20.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.996+0000 7fea5affd640 1 -- 172.21.15.46:0/3208547124 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fea5c032070 con 0x7fea641f6aa0 2023-12-02T00:29:20.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.996+0000 7fea5affd640 1 --2- 172.21.15.46:0/3208547124 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fea4406ae40 0x7fea4406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:20.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.996+0000 7fea5affd640 1 -- 172.21.15.46:0/3208547124 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fea5c0aa220 con 0x7fea641f6aa0 2023-12-02T00:29:20.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.996+0000 7fea6259c640 1 --2- 172.21.15.46:0/3208547124 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fea4406ae40 0x7fea4406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:20.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:19.997+0000 7fea6259c640 1 --2- 172.21.15.46:0/3208547124 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fea4406ae40 0x7fea4406d300 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fea6415ba10 tx=0x7fea5400b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:20.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.000+0000 7fea5affd640 1 -- 172.21.15.46:0/3208547124 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fea6415dad0 con 0x7fea641f6aa0 2023-12-02T00:29:20.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.177+0000 7fea68a75640 1 -- 172.21.15.46:0/3208547124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fea6415dad0 con 0x7fea641f6aa0 2023-12-02T00:29:20.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.178+0000 7fea5affd640 1 -- 172.21.15.46:0/3208547124 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16481 (secure 0 0 0) 0x7fea6415dad0 con 0x7fea641f6aa0 2023-12-02T00:29:20.180 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:29:20.180 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":21,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:28:50.700112+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-02T00:28:47.273337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-02T00:28:48.060115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:29:20.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.180+0000 7fea68a75640 1 -- 172.21.15.46:0/3208547124 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fea4406ae40 msgr2=0x7fea4406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:20.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.180+0000 7fea68a75640 1 --2- 172.21.15.46:0/3208547124 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fea4406ae40 0x7fea4406d300 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fea6415ba10 tx=0x7fea5400b040 comp rx=0 tx=0).stop 2023-12-02T00:29:20.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.180+0000 7fea68a75640 1 -- 172.21.15.46:0/3208547124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea641f6aa0 msgr2=0x7fea6415e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:20.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.180+0000 7fea68a75640 1 --2- 172.21.15.46:0/3208547124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea641f6aa0 0x7fea6415e470 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fea5c00c240 tx=0x7fea5c001a00 comp rx=0 tx=0).stop 2023-12-02T00:29:20.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.180+0000 7fea68a75640 1 -- 172.21.15.46:0/3208547124 shutdown_connections 2023-12-02T00:29:20.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.180+0000 7fea68a75640 1 --2- 172.21.15.46:0/3208547124 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fea4406ae40 0x7fea4406d300 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:20.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.180+0000 7fea68a75640 1 --2- 172.21.15.46:0/3208547124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea641f6aa0 0x7fea6415e470 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:20.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.180+0000 7fea68a75640 1 --2- 172.21.15.46:0/3208547124 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea641f6620 0x7fea6415f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:20.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.180+0000 7fea68a75640 1 --2- 172.21.15.46:0/3208547124 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea50004d00 0x7fea641587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:20.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.180+0000 7fea68a75640 1 -- 172.21.15.46:0/3208547124 >> 172.21.15.46:0/3208547124 conn(0x7fea6409c580 msgr2=0x7fea6415bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:20.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.180+0000 7fea68a75640 1 -- 172.21.15.46:0/3208547124 shutdown_connections 2023-12-02T00:29:20.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.180+0000 7fea68a75640 1 -- 172.21.15.46:0/3208547124 wait complete. 2023-12-02T00:29:20.193 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-02T00:28:37.213645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-02T00:28:47.273337+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-02T00:28:48.060115+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-02T00:29:20.193 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-02T00:29:20.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.302+0000 7f310af68640 1 Processor -- start 2023-12-02T00:29:20.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.302+0000 7f310af68640 1 -- start start 2023-12-02T00:29:20.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.303+0000 7f310af68640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f310406c4c0 0x7f31040b34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:20.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.303+0000 7f310af68640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f310406d090 0x7f310406b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:20.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.303+0000 7f310af68640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f310406d510 con 0x7f3104099a70 2023-12-02T00:29:20.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.303+0000 7f310af68640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31040cf000 con 0x7f310406c4c0 2023-12-02T00:29:20.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.303+0000 7f310af68640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f310406ef70 con 0x7f310406d090 2023-12-02T00:29:20.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.303+0000 7f3109505640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f3104099a70 0x7f31040cf730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38642/0 (socket says 172.21.15.46:38642) 2023-12-02T00:29:20.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.303+0000 7f3109505640 1 -- 172.21.15.46:0/904322949 learned_addr learned my addr 172.21.15.46:0/904322949 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:20.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.303+0000 7f3103fff640 1 --2- 172.21.15.46:0/904322949 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f310406d090 0x7f310406b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:20.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.303+0000 7f3108d04640 1 --2- 172.21.15.46:0/904322949 >> v2:172.21.15.162:3300/0 conn(0x7f310406c4c0 0x7f31040b34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:20.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.303+0000 7f31037fe640 1 -- 172.21.15.46:0/904322949 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2356152534 0 0) 0x7f310406d510 con 0x7f3104099a70 2023-12-02T00:29:20.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.303+0000 7f31037fe640 1 -- 172.21.15.46:0/904322949 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30f4003520 con 0x7f3104099a70 2023-12-02T00:29:20.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.303+0000 7f31037fe640 1 -- 172.21.15.46:0/904322949 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2912736332 0 0) 0x7f30f4003520 con 0x7f3104099a70 2023-12-02T00:29:20.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.303+0000 7f3103fff640 1 -- 172.21.15.46:0/904322949 >> v2:172.21.15.162:3300/0 conn(0x7f310406c4c0 msgr2=0x7f31040b34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:20.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.303+0000 7f3103fff640 1 --2- 172.21.15.46:0/904322949 >> v2:172.21.15.162:3300/0 conn(0x7f310406c4c0 0x7f31040b34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:20.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.303+0000 7f3103fff640 1 -- 172.21.15.46:0/904322949 >> v1:172.21.15.46:6789/0 conn(0x7f3104099a70 legacy=0x7f31040cf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:20.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.303+0000 7f3103fff640 1 -- 172.21.15.46:0/904322949 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f310406f250 con 0x7f310406d090 2023-12-02T00:29:20.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.303+0000 7f3108d04640 1 --2- 172.21.15.46:0/904322949 >> v2:172.21.15.162:3300/0 conn(0x7f310406c4c0 0x7f31040b34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:29:20.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.304+0000 7f3103fff640 1 --2- 172.21.15.46:0/904322949 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f310406d090 0x7f310406b5e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f30f00097d0 tx=0x7f30f002b770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1090a924d08c8a2 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:20.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.304+0000 7f31037fe640 1 -- 172.21.15.46:0/904322949 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30fc0019f0 con 0x7f310406d090 2023-12-02T00:29:20.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.304+0000 7f31037fe640 1 -- 172.21.15.46:0/904322949 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30f002e050 con 0x7f310406d090 2023-12-02T00:29:20.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.304+0000 7f31037fe640 1 -- 172.21.15.46:0/904322949 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f0030730 con 0x7f310406d090 2023-12-02T00:29:20.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.305+0000 7f310af68640 1 -- 172.21.15.46:0/904322949 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f310406d090 msgr2=0x7f310406b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:20.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.305+0000 7f310af68640 1 --2- 172.21.15.46:0/904322949 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f310406d090 0x7f310406b5e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f30f00097d0 tx=0x7f30f002b770 comp rx=0 tx=0).stop 2023-12-02T00:29:20.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.305+0000 7f310af68640 1 -- 172.21.15.46:0/904322949 shutdown_connections 2023-12-02T00:29:20.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.305+0000 7f310af68640 1 --2- 172.21.15.46:0/904322949 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f310406d090 0x7f310406b5e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:20.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.305+0000 7f310af68640 1 --2- 172.21.15.46:0/904322949 >> v2:172.21.15.162:3300/0 conn(0x7f310406c4c0 0x7f31040b34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:20.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.305+0000 7f310af68640 1 -- 172.21.15.46:0/904322949 >> 172.21.15.46:0/904322949 conn(0x7f31040b29d0 msgr2=0x7f31040b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:20.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.305+0000 7f310af68640 1 -- 172.21.15.46:0/904322949 shutdown_connections 2023-12-02T00:29:20.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.305+0000 7f310af68640 1 -- 172.21.15.46:0/904322949 wait complete. 2023-12-02T00:29:20.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.306+0000 7f310af68640 1 Processor -- start 2023-12-02T00:29:20.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.306+0000 7f310af68640 1 -- start start 2023-12-02T00:29:20.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.307+0000 7f310af68640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f310406c4c0 0x7f31041f5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:20.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.307+0000 7f310af68640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f310406d090 0x7f31040b1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:20.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.307+0000 7f310af68640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3104099a70 0x7f31040b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:20.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.307+0000 7f310af68640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310415dad0 con 0x7f310406c4c0 2023-12-02T00:29:20.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.307+0000 7f310af68640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31041f3a20 con 0x7f3104099a70 2023-12-02T00:29:20.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.307+0000 7f310af68640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f310415ed20 con 0x7f310406d090 2023-12-02T00:29:20.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.307+0000 7f3108d04640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f310406c4c0 0x7f31041f5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:20.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.307+0000 7f3108d04640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f310406c4c0 0x7f31041f5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44910/0 (socket says 172.21.15.46:44910) 2023-12-02T00:29:20.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.307+0000 7f3108d04640 1 -- 172.21.15.46:0/1581185688 learned_addr learned my addr 172.21.15.46:0/1581185688 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:20.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.307+0000 7f3109505640 1 --2- 172.21.15.46:0/1581185688 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3104099a70 0x7f31040b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:20.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.307+0000 7f3103fff640 1 --2- 172.21.15.46:0/1581185688 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f310406d090 0x7f31040b1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:20.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.307+0000 7f3108d04640 1 -- 172.21.15.46:0/1581185688 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f310406d090 msgr2=0x7f31040b1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:20.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.308+0000 7f3108d04640 1 --2- 172.21.15.46:0/1581185688 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f310406d090 0x7f31040b1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:20.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.308+0000 7f3108d04640 1 -- 172.21.15.46:0/1581185688 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3104099a70 msgr2=0x7f31040b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:20.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.308+0000 7f3108d04640 1 --2- 172.21.15.46:0/1581185688 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3104099a70 0x7f31040b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:20.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.308+0000 7f3108d04640 1 -- 172.21.15.46:0/1581185688 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f8009400 con 0x7f310406c4c0 2023-12-02T00:29:20.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.308+0000 7f3109505640 1 --2- 172.21.15.46:0/1581185688 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3104099a70 0x7f31040b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:29:20.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.308+0000 7f3108d04640 1 --2- 172.21.15.46:0/1581185688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f310406c4c0 0x7f31041f5ce0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f30f800e960 tx=0x7f30f800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:20.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.308+0000 7f31017fa640 1 -- 172.21.15.46:0/1581185688 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f800ca00 con 0x7f310406c4c0 2023-12-02T00:29:20.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.308+0000 7f31017fa640 1 -- 172.21.15.46:0/1581185688 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30f800cb60 con 0x7f310406c4c0 2023-12-02T00:29:20.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.308+0000 7f310af68640 1 -- 172.21.15.46:0/1581185688 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30f0009400 con 0x7f310406c4c0 2023-12-02T00:29:20.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.308+0000 7f31017fa640 1 -- 172.21.15.46:0/1581185688 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f8013470 con 0x7f310406c4c0 2023-12-02T00:29:20.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.308+0000 7f310af68640 1 -- 172.21.15.46:0/1581185688 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f310415f2e0 con 0x7f310406c4c0 2023-12-02T00:29:20.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.309+0000 7f310af68640 1 -- 172.21.15.46:0/1581185688 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31040b6d10 con 0x7f310406c4c0 2023-12-02T00:29:20.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.309+0000 7f31017fa640 1 -- 172.21.15.46:0/1581185688 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f30f80135d0 con 0x7f310406c4c0 2023-12-02T00:29:20.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.310+0000 7f31017fa640 1 --2- 172.21.15.46:0/1581185688 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f30e806ae40 0x7f30e806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:20.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.310+0000 7f31017fa640 1 -- 172.21.15.46:0/1581185688 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f30f8089820 con 0x7f310406c4c0 2023-12-02T00:29:20.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.310+0000 7f3103fff640 1 --2- 172.21.15.46:0/1581185688 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f30e806ae40 0x7f30e806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:20.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.310+0000 7f3103fff640 1 --2- 172.21.15.46:0/1581185688 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f30e806ae40 0x7f30e806d300 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f30f0034000 tx=0x7f30f0008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:20.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.314+0000 7f31017fa640 1 -- 172.21.15.46:0/1581185688 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f30f8054960 con 0x7f310406c4c0 2023-12-02T00:29:20.387 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-02T00:29:20.390 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-02T00:29:20.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.491+0000 7f310af68640 1 -- 172.21.15.46:0/1581185688 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3104069320 con 0x7f310406c4c0 2023-12-02T00:29:20.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.491+0000 7f31017fa640 1 -- 172.21.15.46:0/1581185688 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f30f8089020 con 0x7f310406c4c0 2023-12-02T00:29:20.493 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-12-02T00:29:20.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.493+0000 7f310af68640 1 -- 172.21.15.46:0/1581185688 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f30e806ae40 msgr2=0x7f30e806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:20.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.493+0000 7f310af68640 1 --2- 172.21.15.46:0/1581185688 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f30e806ae40 0x7f30e806d300 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f30f0034000 tx=0x7f30f0008030 comp rx=0 tx=0).stop 2023-12-02T00:29:20.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.493+0000 7f310af68640 1 -- 172.21.15.46:0/1581185688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f310406c4c0 msgr2=0x7f31041f5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:20.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.493+0000 7f310af68640 1 --2- 172.21.15.46:0/1581185688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f310406c4c0 0x7f31041f5ce0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f30f800e960 tx=0x7f30f800ee20 comp rx=0 tx=0).stop 2023-12-02T00:29:20.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.493+0000 7f310af68640 1 -- 172.21.15.46:0/1581185688 shutdown_connections 2023-12-02T00:29:20.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.493+0000 7f310af68640 1 --2- 172.21.15.46:0/1581185688 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f30e806ae40 0x7f30e806d300 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:20.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.493+0000 7f310af68640 1 --2- 172.21.15.46:0/1581185688 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3104099a70 0x7f31040b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:20.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.493+0000 7f310af68640 1 --2- 172.21.15.46:0/1581185688 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f310406d090 0x7f31040b1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:20.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.493+0000 7f310af68640 1 --2- 172.21.15.46:0/1581185688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f310406c4c0 0x7f31041f5ce0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:20.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.493+0000 7f310af68640 1 -- 172.21.15.46:0/1581185688 >> 172.21.15.46:0/1581185688 conn(0x7f31040b29d0 msgr2=0x7f31040adc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:20.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.493+0000 7f310af68640 1 -- 172.21.15.46:0/1581185688 shutdown_connections 2023-12-02T00:29:20.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.493+0000 7f310af68640 1 -- 172.21.15.46:0/1581185688 wait complete. 2023-12-02T00:29:20.505 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-02T00:29:20.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.614+0000 7fe0cdee1640 1 Processor -- start 2023-12-02T00:29:20.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.614+0000 7fe0cdee1640 1 -- start start 2023-12-02T00:29:20.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.614+0000 7fe0cdee1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0c815d330 0x7fe0c80b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:20.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.614+0000 7fe0cdee1640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe0c80d2d90 0x7fe0c80b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:20.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.614+0000 7fe0cdee1640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0c80b2b10 con 0x7fe0c80d2910 2023-12-02T00:29:20.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.614+0000 7fe0cdee1640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0c80d1590 con 0x7fe0c80d2d90 2023-12-02T00:29:20.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.614+0000 7fe0cdee1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c80d1870 con 0x7fe0c815d330 2023-12-02T00:29:20.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.615+0000 7fe0c6ffd640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe0c80d2910 0x7fe0c80d31f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38654/0 (socket says 172.21.15.46:38654) 2023-12-02T00:29:20.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.615+0000 7fe0c7fff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe0c80d2d90 0x7fe0c80b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:20.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.615+0000 7fe0c6ffd640 1 -- 172.21.15.46:0/353246506 learned_addr learned my addr 172.21.15.46:0/353246506 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:20.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.615+0000 7fe0c77fe640 1 --2- 172.21.15.46:0/353246506 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0c815d330 0x7fe0c80b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:20.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.615+0000 7fe0c7fff640 1 -- 172.21.15.46:0/353246506 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0c815d330 msgr2=0x7fe0c80b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:20.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.615+0000 7fe0c7fff640 1 --2- 172.21.15.46:0/353246506 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0c815d330 0x7fe0c80b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:20.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.615+0000 7fe0c7fff640 1 -- 172.21.15.46:0/353246506 >> v1:172.21.15.46:6789/0 conn(0x7fe0c80d2910 legacy=0x7fe0c80d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:20.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.615+0000 7fe0c7fff640 1 -- 172.21.15.46:0/353246506 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c80d1b50 con 0x7fe0c80d2d90 2023-12-02T00:29:20.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.615+0000 7fe0c67fc640 1 -- 172.21.15.46:0/353246506 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 750979953 0 0) 0x7fe0c80b2b10 con 0x7fe0c80d2910 2023-12-02T00:29:20.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.616+0000 7fe0c7fff640 1 --2- 172.21.15.46:0/353246506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0c80d2d90 0x7fe0c80b6390 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0009d90 tx=0x7fe0c002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=449791f4e3819b21 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:20.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.616+0000 7fe0c67fc640 1 -- 172.21.15.46:0/353246506 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c002cbc0 con 0x7fe0c80d2d90 2023-12-02T00:29:20.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.616+0000 7fe0c67fc640 1 -- 172.21.15.46:0/353246506 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0c002cd20 con 0x7fe0c80d2d90 2023-12-02T00:29:20.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.616+0000 7fe0c67fc640 1 -- 172.21.15.46:0/353246506 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c0034660 con 0x7fe0c80d2d90 2023-12-02T00:29:20.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.616+0000 7fe0cdee1640 1 -- 172.21.15.46:0/353246506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0c80d2d90 msgr2=0x7fe0c80b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:20.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.616+0000 7fe0cdee1640 1 --2- 172.21.15.46:0/353246506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0c80d2d90 0x7fe0c80b6390 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0009d90 tx=0x7fe0c002be10 comp rx=0 tx=0).stop 2023-12-02T00:29:20.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.616+0000 7fe0cdee1640 1 -- 172.21.15.46:0/353246506 shutdown_connections 2023-12-02T00:29:20.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.616+0000 7fe0cdee1640 1 --2- 172.21.15.46:0/353246506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0c80d2d90 0x7fe0c80b6390 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:20.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.616+0000 7fe0cdee1640 1 --2- 172.21.15.46:0/353246506 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0c815d330 0x7fe0c80b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:20.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.616+0000 7fe0cdee1640 1 -- 172.21.15.46:0/353246506 >> 172.21.15.46:0/353246506 conn(0x7fe0c8072020 msgr2=0x7fe0c8072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:20.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.616+0000 7fe0cdee1640 1 -- 172.21.15.46:0/353246506 shutdown_connections 2023-12-02T00:29:20.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.616+0000 7fe0cdee1640 1 -- 172.21.15.46:0/353246506 wait complete. 2023-12-02T00:29:20.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.618+0000 7fe0cdee1640 1 Processor -- start 2023-12-02T00:29:20.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.618+0000 7fe0cdee1640 1 -- start start 2023-12-02T00:29:20.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.618+0000 7fe0cdee1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0c80d2910 0x7fe0c80cc470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:20.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.618+0000 7fe0cdee1640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0c80d2d90 0x7fe0c815e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:20.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.618+0000 7fe0cdee1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0c815d330 0x7fe0c8159310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:20.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.618+0000 7fe0cdee1640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c8160730 con 0x7fe0c815d330 2023-12-02T00:29:20.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.618+0000 7fe0cdee1640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c815e8d0 con 0x7fe0c80d2d90 2023-12-02T00:29:20.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.618+0000 7fe0cdee1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c815ebb0 con 0x7fe0c80d2910 2023-12-02T00:29:20.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.619+0000 7fe0c7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0c815d330 0x7fe0c8159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:20.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.619+0000 7fe0c7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0c815d330 0x7fe0c8159310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44936/0 (socket says 172.21.15.46:44936) 2023-12-02T00:29:20.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.619+0000 7fe0c7fff640 1 -- 172.21.15.46:0/1435334987 learned_addr learned my addr 172.21.15.46:0/1435334987 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:20.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.619+0000 7fe0c77fe640 1 --2- 172.21.15.46:0/1435334987 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0c80d2910 0x7fe0c80cc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:20.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.619+0000 7fe0c6ffd640 1 --2- 172.21.15.46:0/1435334987 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0c80d2d90 0x7fe0c815e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:20.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.619+0000 7fe0c7fff640 1 -- 172.21.15.46:0/1435334987 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0c80d2910 msgr2=0x7fe0c80cc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:20.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.619+0000 7fe0c7fff640 1 --2- 172.21.15.46:0/1435334987 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0c80d2910 0x7fe0c80cc470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:20.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.619+0000 7fe0c7fff640 1 -- 172.21.15.46:0/1435334987 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0c80d2d90 msgr2=0x7fe0c815e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:20.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.619+0000 7fe0c7fff640 1 --2- 172.21.15.46:0/1435334987 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0c80d2d90 0x7fe0c815e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:20.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.619+0000 7fe0c7fff640 1 -- 172.21.15.46:0/1435334987 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c0009a70 con 0x7fe0c815d330 2023-12-02T00:29:20.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.619+0000 7fe0c6ffd640 1 --2- 172.21.15.46:0/1435334987 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0c80d2d90 0x7fe0c815e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:20.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.619+0000 7fe0c7fff640 1 --2- 172.21.15.46:0/1435334987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0c815d330 0x7fe0c8159310 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0009d90 tx=0x7fe0c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:20.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.619+0000 7fe0affff640 1 -- 172.21.15.46:0/1435334987 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c0040650 con 0x7fe0c815d330 2023-12-02T00:29:20.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.619+0000 7fe0affff640 1 -- 172.21.15.46:0/1435334987 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0c002ce90 con 0x7fe0c815d330 2023-12-02T00:29:20.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.619+0000 7fe0affff640 1 -- 172.21.15.46:0/1435334987 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c0040a10 con 0x7fe0c815d330 2023-12-02T00:29:20.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.619+0000 7fe0cdee1640 1 -- 172.21.15.46:0/1435334987 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0c815ee90 con 0x7fe0c815d330 2023-12-02T00:29:20.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.619+0000 7fe0cdee1640 1 -- 172.21.15.46:0/1435334987 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0c815f450 con 0x7fe0c815d330 2023-12-02T00:29:20.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.620+0000 7fe0cdee1640 1 -- 172.21.15.46:0/1435334987 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0c8069400 con 0x7fe0c815d330 2023-12-02T00:29:20.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.621+0000 7fe0affff640 1 -- 172.21.15.46:0/1435334987 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe0c0040b70 con 0x7fe0c815d330 2023-12-02T00:29:20.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.621+0000 7fe0affff640 1 --2- 172.21.15.46:0/1435334987 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe0b406ae40 0x7fe0b406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:20.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.621+0000 7fe0affff640 1 -- 172.21.15.46:0/1435334987 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe0c00ac240 con 0x7fe0c815d330 2023-12-02T00:29:20.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.621+0000 7fe0c77fe640 1 --2- 172.21.15.46:0/1435334987 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe0b406ae40 0x7fe0b406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:20.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.622+0000 7fe0c77fe640 1 --2- 172.21.15.46:0/1435334987 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe0b406ae40 0x7fe0b406d300 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8006fd0 tx=0x7fe0b8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:20.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.625+0000 7fe0affff640 1 -- 172.21.15.46:0/1435334987 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe0c00772f0 con 0x7fe0c815d330 2023-12-02T00:29:20.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.801+0000 7fe0cdee1640 1 -- 172.21.15.46:0/1435334987 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe0c80cde90 con 0x7fe0c815d330 2023-12-02T00:29:20.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.802+0000 7fe0affff640 1 -- 172.21.15.46:0/1435334987 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe0c007a910 con 0x7fe0c815d330 2023-12-02T00:29:20.803 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-12-02T00:29:20.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.803+0000 7fe0cdee1640 1 -- 172.21.15.46:0/1435334987 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe0b406ae40 msgr2=0x7fe0b406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:20.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.803+0000 7fe0cdee1640 1 --2- 172.21.15.46:0/1435334987 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe0b406ae40 0x7fe0b406d300 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8006fd0 tx=0x7fe0b8008040 comp rx=0 tx=0).stop 2023-12-02T00:29:20.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.803+0000 7fe0cdee1640 1 -- 172.21.15.46:0/1435334987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0c815d330 msgr2=0x7fe0c8159310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:20.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.803+0000 7fe0cdee1640 1 --2- 172.21.15.46:0/1435334987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0c815d330 0x7fe0c8159310 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0009d90 tx=0x7fe0c0004a40 comp rx=0 tx=0).stop 2023-12-02T00:29:20.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.804+0000 7fe0cdee1640 1 -- 172.21.15.46:0/1435334987 shutdown_connections 2023-12-02T00:29:20.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.804+0000 7fe0cdee1640 1 --2- 172.21.15.46:0/1435334987 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe0b406ae40 0x7fe0b406d300 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:20.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.804+0000 7fe0cdee1640 1 --2- 172.21.15.46:0/1435334987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0c815d330 0x7fe0c8159310 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:20.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.804+0000 7fe0cdee1640 1 --2- 172.21.15.46:0/1435334987 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0c80d2d90 0x7fe0c815e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:20.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.804+0000 7fe0cdee1640 1 --2- 172.21.15.46:0/1435334987 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0c80d2910 0x7fe0c80cc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:20.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.804+0000 7fe0cdee1640 1 -- 172.21.15.46:0/1435334987 >> 172.21.15.46:0/1435334987 conn(0x7fe0c8072020 msgr2=0x7fe0c80b6350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:20.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.804+0000 7fe0cdee1640 1 -- 172.21.15.46:0/1435334987 shutdown_connections 2023-12-02T00:29:20.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.804+0000 7fe0cdee1640 1 -- 172.21.15.46:0/1435334987 wait complete. 2023-12-02T00:29:20.820 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-02T00:29:20.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.930+0000 7fa422b05640 1 Processor -- start 2023-12-02T00:29:20.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.930+0000 7fa422b05640 1 -- start start 2023-12-02T00:29:20.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.930+0000 7fa422b05640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa41c0d3900 0x7fa41c0b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:20.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.930+0000 7fa422b05640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa41c0b2b10 con 0x7fa41c0d2910 2023-12-02T00:29:20.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.930+0000 7fa422b05640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa41c0d1590 con 0x7fa41c15d330 2023-12-02T00:29:20.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.930+0000 7fa422b05640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa41c0d1870 con 0x7fa41c0d3900 2023-12-02T00:29:20.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.931+0000 7fa41bfff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa41c0d2910 0x7fa41c0d31f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38660/0 (socket says 172.21.15.46:38660) 2023-12-02T00:29:20.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.931+0000 7fa41bfff640 1 -- 172.21.15.46:0/2481663694 learned_addr learned my addr 172.21.15.46:0/2481663694 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:20.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.931+0000 7fa41b7fe640 1 -- 172.21.15.46:0/2481663694 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 403606989 0 0) 0x7fa41c0b2b10 con 0x7fa41c0d2910 2023-12-02T00:29:20.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.931+0000 7fa4210a2640 1 --2- 172.21.15.46:0/2481663694 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa41c0d3900 0x7fa41c0b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:20.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.931+0000 7fa41b7fe640 1 -- 172.21.15.46:0/2481663694 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa40c003520 con 0x7fa41c0d2910 2023-12-02T00:29:20.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.931+0000 7fa41b7fe640 1 -- 172.21.15.46:0/2481663694 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3492206109 0 0) 0x7fa40c003520 con 0x7fa41c0d2910 2023-12-02T00:29:20.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.931+0000 7fa41b7fe640 1 -- 172.21.15.46:0/2481663694 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa41c0d3900 msgr2=0x7fa41c0b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:20.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.931+0000 7fa41b7fe640 1 --2- 172.21.15.46:0/2481663694 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa41c0d3900 0x7fa41c0b6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:20.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.931+0000 7fa41b7fe640 1 -- 172.21.15.46:0/2481663694 >> v1:172.21.15.162:6789/0 conn(0x7fa41c15d330 legacy=0x7fa41c0b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:20.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.931+0000 7fa41b7fe640 1 -- 172.21.15.46:0/2481663694 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa41c0d1b50 con 0x7fa41c0d2910 2023-12-02T00:29:20.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.931+0000 7fa41b7fe640 1 -- 172.21.15.46:0/2481663694 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7fa408001670 con 0x7fa41c0d2910 2023-12-02T00:29:20.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.931+0000 7fa41b7fe640 1 -- 172.21.15.46:0/2481663694 >> v1:172.21.15.46:6789/0 conn(0x7fa41c0d2910 legacy=0x7fa41c0d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:20.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.931+0000 7fa41b7fe640 1 --2- 172.21.15.46:0/2481663694 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa40c0039a0 0x7fa40c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:20.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.931+0000 7fa41b7fe640 1 --2- 172.21.15.46:0/2481663694 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa41c0d3900 0x7fa40c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:20.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.931+0000 7fa41b7fe640 1 --2- 172.21.15.46:0/2481663694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa40c004d00 0x7fa40c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:20.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.931+0000 7fa41b7fe640 1 -- 172.21.15.46:0/2481663694 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa41c0d1870 con 0x7fa40c004d00 2023-12-02T00:29:20.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.931+0000 7fa41b7fe640 1 -- 172.21.15.46:0/2481663694 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c00bab0 con 0x7fa40c0039a0 2023-12-02T00:29:20.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.931+0000 7fa41b7fe640 1 -- 172.21.15.46:0/2481663694 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c00bc70 con 0x7fa41c0d3900 2023-12-02T00:29:20.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.931+0000 7fa4210a2640 1 --2- 172.21.15.46:0/2481663694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa40c004d00 0x7fa40c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:20.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.931+0000 7fa4208a1640 1 --2- 172.21.15.46:0/2481663694 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa40c0039a0 0x7fa40c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:20.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.931+0000 7fa41bfff640 1 --2- 172.21.15.46:0/2481663694 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa41c0d3900 0x7fa40c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:20.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.932+0000 7fa4210a2640 1 -- 172.21.15.46:0/2481663694 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa41c0d3900 msgr2=0x7fa40c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:20.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.932+0000 7fa4210a2640 1 --2- 172.21.15.46:0/2481663694 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa41c0d3900 0x7fa40c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:20.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.932+0000 7fa4210a2640 1 -- 172.21.15.46:0/2481663694 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa40c0039a0 msgr2=0x7fa40c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:20.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.932+0000 7fa4210a2640 1 --2- 172.21.15.46:0/2481663694 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa40c0039a0 0x7fa40c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:20.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.932+0000 7fa4210a2640 1 -- 172.21.15.46:0/2481663694 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa40c003520 con 0x7fa40c004d00 2023-12-02T00:29:20.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.932+0000 7fa4210a2640 1 --2- 172.21.15.46:0/2481663694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa40c004d00 0x7fa40c005180 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa414009fc0 tx=0x7fa40c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e27e0817f697a1b server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:20.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.933+0000 7fa41b7fe640 1 -- 172.21.15.46:0/2481663694 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41402cc20 con 0x7fa40c004d00 2023-12-02T00:29:20.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.933+0000 7fa41b7fe640 1 -- 172.21.15.46:0/2481663694 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa41400f040 con 0x7fa40c004d00 2023-12-02T00:29:20.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.933+0000 7fa41b7fe640 1 -- 172.21.15.46:0/2481663694 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa414034440 con 0x7fa40c004d00 2023-12-02T00:29:20.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.933+0000 7fa422b05640 1 -- 172.21.15.46:0/2481663694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa40c004d00 msgr2=0x7fa40c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:20.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.933+0000 7fa422b05640 1 --2- 172.21.15.46:0/2481663694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa40c004d00 0x7fa40c005180 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa414009fc0 tx=0x7fa40c0057f0 comp rx=0 tx=0).stop 2023-12-02T00:29:20.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.933+0000 7fa4210a2640 1 -- 172.21.15.46:0/2481663694 reap_dead start 2023-12-02T00:29:20.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.933+0000 7fa422b05640 1 -- 172.21.15.46:0/2481663694 shutdown_connections 2023-12-02T00:29:20.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.933+0000 7fa422b05640 1 -- 172.21.15.46:0/2481663694 >> 172.21.15.46:0/2481663694 conn(0x7fa41c072020 msgr2=0x7fa41c072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:20.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.933+0000 7fa422b05640 1 -- 172.21.15.46:0/2481663694 shutdown_connections 2023-12-02T00:29:20.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.933+0000 7fa422b05640 1 -- 172.21.15.46:0/2481663694 wait complete. 2023-12-02T00:29:20.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.934+0000 7fa422b05640 1 Processor -- start 2023-12-02T00:29:20.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.934+0000 7fa422b05640 1 -- start start 2023-12-02T00:29:20.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.935+0000 7fa422b05640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa40c004d00 0x7fa41c1f2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:20.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.935+0000 7fa422b05640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa41c0c9000 0x7fa41c0c9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:20.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.935+0000 7fa422b05640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa41c0c7640 0x7fa41c0c7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:20.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.935+0000 7fa422b05640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa41c0c99c0 con 0x7fa40c004d00 2023-12-02T00:29:20.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.935+0000 7fa422b05640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa41c0c86c0 con 0x7fa41c0c9000 2023-12-02T00:29:20.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.935+0000 7fa422b05640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa41c0c83e0 con 0x7fa41c0c7640 2023-12-02T00:29:20.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.935+0000 7fa4208a1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa40c004d00 0x7fa41c1f2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:20.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.935+0000 7fa4208a1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa40c004d00 0x7fa41c1f2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44952/0 (socket says 172.21.15.46:44952) 2023-12-02T00:29:20.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.935+0000 7fa4208a1640 1 -- 172.21.15.46:0/3670238981 learned_addr learned my addr 172.21.15.46:0/3670238981 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:20.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.935+0000 7fa41bfff640 1 --2- 172.21.15.46:0/3670238981 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa41c0c9000 0x7fa41c0c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:20.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.935+0000 7fa4210a2640 1 --2- 172.21.15.46:0/3670238981 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa41c0c7640 0x7fa41c0c7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:20.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.936+0000 7fa4208a1640 1 -- 172.21.15.46:0/3670238981 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa41c0c7640 msgr2=0x7fa41c0c7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:20.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.936+0000 7fa4208a1640 1 --2- 172.21.15.46:0/3670238981 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa41c0c7640 0x7fa41c0c7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:20.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.936+0000 7fa4208a1640 1 -- 172.21.15.46:0/3670238981 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa41c0c9000 msgr2=0x7fa41c0c9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:20.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.936+0000 7fa4208a1640 1 --2- 172.21.15.46:0/3670238981 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa41c0c9000 0x7fa41c0c9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:20.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.936+0000 7fa4208a1640 1 -- 172.21.15.46:0/3670238981 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa41c159680 con 0x7fa40c004d00 2023-12-02T00:29:20.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.936+0000 7fa4208a1640 1 --2- 172.21.15.46:0/3670238981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa40c004d00 0x7fa41c1f2310 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa41000f820 tx=0x7fa41000fc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:20.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.936+0000 7fa4197fa640 1 -- 172.21.15.46:0/3670238981 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4100050d0 con 0x7fa40c004d00 2023-12-02T00:29:20.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.936+0000 7fa4197fa640 1 -- 172.21.15.46:0/3670238981 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa410005230 con 0x7fa40c004d00 2023-12-02T00:29:20.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.936+0000 7fa422b05640 1 -- 172.21.15.46:0/3670238981 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa41c159960 con 0x7fa40c004d00 2023-12-02T00:29:20.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.936+0000 7fa4197fa640 1 -- 172.21.15.46:0/3670238981 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41000d730 con 0x7fa40c004d00 2023-12-02T00:29:20.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.936+0000 7fa422b05640 1 -- 172.21.15.46:0/3670238981 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa41c1f2850 con 0x7fa40c004d00 2023-12-02T00:29:20.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.937+0000 7fa422b05640 1 -- 172.21.15.46:0/3670238981 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa41c1f29e0 con 0x7fa40c004d00 2023-12-02T00:29:20.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.938+0000 7fa4197fa640 1 -- 172.21.15.46:0/3670238981 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa41000d890 con 0x7fa40c004d00 2023-12-02T00:29:20.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.938+0000 7fa4197fa640 1 --2- 172.21.15.46:0/3670238981 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa40006ae70 0x7fa40006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:20.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.938+0000 7fa4197fa640 1 -- 172.21.15.46:0/3670238981 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa410089420 con 0x7fa40c004d00 2023-12-02T00:29:20.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.938+0000 7fa41bfff640 1 --2- 172.21.15.46:0/3670238981 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa40006ae70 0x7fa40006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:20.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.939+0000 7fa41bfff640 1 --2- 172.21.15.46:0/3670238981 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa40006ae70 0x7fa40006d330 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa41c0c8f90 tx=0x7fa408001170 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:20.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:20.943+0000 7fa4197fa640 1 -- 172.21.15.46:0/3670238981 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa4100544d0 con 0x7fa40c004d00 2023-12-02T00:29:21.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.119+0000 7fa422b05640 1 -- 172.21.15.46:0/3670238981 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa41c069400 con 0x7fa40c004d00 2023-12-02T00:29:21.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.119+0000 7fa4197fa640 1 -- 172.21.15.46:0/3670238981 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa410057af0 con 0x7fa40c004d00 2023-12-02T00:29:21.121 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-12-02T00:29:21.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.121+0000 7fa422b05640 1 -- 172.21.15.46:0/3670238981 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa40006ae70 msgr2=0x7fa40006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:21.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.121+0000 7fa422b05640 1 --2- 172.21.15.46:0/3670238981 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa40006ae70 0x7fa40006d330 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa41c0c8f90 tx=0x7fa408001170 comp rx=0 tx=0).stop 2023-12-02T00:29:21.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.121+0000 7fa422b05640 1 -- 172.21.15.46:0/3670238981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa40c004d00 msgr2=0x7fa41c1f2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:21.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.121+0000 7fa422b05640 1 --2- 172.21.15.46:0/3670238981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa40c004d00 0x7fa41c1f2310 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa41000f820 tx=0x7fa41000fc20 comp rx=0 tx=0).stop 2023-12-02T00:29:21.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.121+0000 7fa422b05640 1 -- 172.21.15.46:0/3670238981 shutdown_connections 2023-12-02T00:29:21.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.121+0000 7fa422b05640 1 --2- 172.21.15.46:0/3670238981 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa40006ae70 0x7fa40006d330 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:21.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.121+0000 7fa422b05640 1 --2- 172.21.15.46:0/3670238981 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa41c0c7640 0x7fa41c0c7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:21.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.122+0000 7fa422b05640 1 --2- 172.21.15.46:0/3670238981 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa41c0c9000 0x7fa41c0c9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:21.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.122+0000 7fa422b05640 1 --2- 172.21.15.46:0/3670238981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa40c004d00 0x7fa41c1f2310 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:21.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.122+0000 7fa422b05640 1 -- 172.21.15.46:0/3670238981 >> 172.21.15.46:0/3670238981 conn(0x7fa41c072020 msgr2=0x7fa41c0b2d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:21.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.122+0000 7fa422b05640 1 -- 172.21.15.46:0/3670238981 shutdown_connections 2023-12-02T00:29:21.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.122+0000 7fa422b05640 1 -- 172.21.15.46:0/3670238981 wait complete. 2023-12-02T00:29:21.134 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification' 2023-12-02T00:29:21.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.239+0000 7f00a1db6640 1 Processor -- start 2023-12-02T00:29:21.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.239+0000 7f00a1db6640 1 -- start start 2023-12-02T00:29:21.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.240+0000 7f00a1db6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f009c160fb0 0x7f009c161440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:21.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.240+0000 7f00a1db6640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f009c0d0ed0 0x7f009c0d1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:21.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.240+0000 7f00a1db6640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f009c161d20 con 0x7f009c06ea50 2023-12-02T00:29:21.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.240+0000 7f00a1db6640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f009c15e760 con 0x7f009c0d0ed0 2023-12-02T00:29:21.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.240+0000 7f00a1db6640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f009c15d9a0 con 0x7f009c160fb0 2023-12-02T00:29:21.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.240+0000 7f009bfff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f009c0d0ed0 0x7f009c0d1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:21.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.240+0000 7f009bfff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f009c0d0ed0 0x7f009c0d1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44960/0 (socket says 172.21.15.46:44960) 2023-12-02T00:29:21.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.240+0000 7f009bfff640 1 -- 172.21.15.46:0/801303914 learned_addr learned my addr 172.21.15.46:0/801303914 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:21.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.240+0000 7f009affd640 1 --2- 172.21.15.46:0/801303914 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f009c160fb0 0x7f009c161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:21.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.240+0000 7f009bfff640 1 -- 172.21.15.46:0/801303914 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f009c160fb0 msgr2=0x7f009c161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:21.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.240+0000 7f009bfff640 1 --2- 172.21.15.46:0/801303914 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f009c160fb0 0x7f009c161440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:21.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.240+0000 7f009bfff640 1 -- 172.21.15.46:0/801303914 >> v1:172.21.15.162:6789/0 conn(0x7f009c06ea50 legacy=0x7f009c06ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:21.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.240+0000 7f009bfff640 1 -- 172.21.15.46:0/801303914 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f009c15dc80 con 0x7f009c0d0ed0 2023-12-02T00:29:21.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.240+0000 7f009affd640 1 --2- 172.21.15.46:0/801303914 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f009c160fb0 0x7f009c161440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:29:21.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.241+0000 7f009bfff640 1 --2- 172.21.15.46:0/801303914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f009c0d0ed0 0x7f009c0d1360 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0094009e50 tx=0x7f009402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e47271ba994b0be0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:21.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.241+0000 7f009a7fc640 1 -- 172.21.15.46:0/801303914 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009402ca70 con 0x7f009c0d0ed0 2023-12-02T00:29:21.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.241+0000 7f009a7fc640 1 -- 172.21.15.46:0/801303914 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f009402cbd0 con 0x7f009c0d0ed0 2023-12-02T00:29:21.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.241+0000 7f009a7fc640 1 -- 172.21.15.46:0/801303914 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00940343f0 con 0x7f009c0d0ed0 2023-12-02T00:29:21.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.242+0000 7f00a1db6640 1 -- 172.21.15.46:0/801303914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f009c0d0ed0 msgr2=0x7f009c0d1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:21.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.242+0000 7f00a1db6640 1 --2- 172.21.15.46:0/801303914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f009c0d0ed0 0x7f009c0d1360 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0094009e50 tx=0x7f009402bed0 comp rx=0 tx=0).stop 2023-12-02T00:29:21.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.242+0000 7f00a1db6640 1 -- 172.21.15.46:0/801303914 shutdown_connections 2023-12-02T00:29:21.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.242+0000 7f00a1db6640 1 --2- 172.21.15.46:0/801303914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f009c0d0ed0 0x7f009c0d1360 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:21.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.242+0000 7f00a1db6640 1 --2- 172.21.15.46:0/801303914 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f009c160fb0 0x7f009c161440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:21.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.242+0000 7f00a1db6640 1 -- 172.21.15.46:0/801303914 >> 172.21.15.46:0/801303914 conn(0x7f009c06a6b0 msgr2=0x7f009c06aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:21.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.242+0000 7f00a1db6640 1 -- 172.21.15.46:0/801303914 shutdown_connections 2023-12-02T00:29:21.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.242+0000 7f00a1db6640 1 -- 172.21.15.46:0/801303914 wait complete. 2023-12-02T00:29:21.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.243+0000 7f00a1db6640 1 Processor -- start 2023-12-02T00:29:21.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.243+0000 7f00a1db6640 1 -- start start 2023-12-02T00:29:21.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.244+0000 7f00a1db6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f009c06ea50 0x7f009c156a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:21.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.244+0000 7f00a1db6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f009c0d0ed0 0x7f009c157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:21.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.244+0000 7f00a1db6640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f009c160fb0 0x7f009c06be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:21.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.244+0000 7f00a1db6640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f009c1f30e0 con 0x7f009c06ea50 2023-12-02T00:29:21.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.244+0000 7f00a1db6640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f009c15a770 con 0x7f009c160fb0 2023-12-02T00:29:21.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.244+0000 7f00a1db6640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f009c158710 con 0x7f009c0d0ed0 2023-12-02T00:29:21.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.244+0000 7f009b7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f009c06ea50 0x7f009c156a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:21.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.244+0000 7f009b7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f009c06ea50 0x7f009c156a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44968/0 (socket says 172.21.15.46:44968) 2023-12-02T00:29:21.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.244+0000 7f009b7fe640 1 -- 172.21.15.46:0/1246569728 learned_addr learned my addr 172.21.15.46:0/1246569728 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:21.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.244+0000 7f009bfff640 1 --2- 172.21.15.46:0/1246569728 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f009c160fb0 0x7f009c06be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:21.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.244+0000 7f009affd640 1 --2- 172.21.15.46:0/1246569728 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f009c0d0ed0 0x7f009c157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:21.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.244+0000 7f009b7fe640 1 -- 172.21.15.46:0/1246569728 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f009c0d0ed0 msgr2=0x7f009c157ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:21.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.244+0000 7f009b7fe640 1 --2- 172.21.15.46:0/1246569728 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f009c0d0ed0 0x7f009c157ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:21.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.244+0000 7f009b7fe640 1 -- 172.21.15.46:0/1246569728 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f009c160fb0 msgr2=0x7f009c06be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:21.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.244+0000 7f009b7fe640 1 --2- 172.21.15.46:0/1246569728 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f009c160fb0 0x7f009c06be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:21.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.245+0000 7f009b7fe640 1 -- 172.21.15.46:0/1246569728 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0094009b30 con 0x7f009c06ea50 2023-12-02T00:29:21.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.245+0000 7f009b7fe640 1 --2- 172.21.15.46:0/1246569728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f009c06ea50 0x7f009c156a70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f008c00b3f0 tx=0x7f008c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:21.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.245+0000 7f0083fff640 1 -- 172.21.15.46:0/1246569728 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008c011830 con 0x7f009c06ea50 2023-12-02T00:29:21.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.245+0000 7f0083fff640 1 -- 172.21.15.46:0/1246569728 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f008c011990 con 0x7f009c06ea50 2023-12-02T00:29:21.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.245+0000 7f0083fff640 1 -- 172.21.15.46:0/1246569728 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008c011d40 con 0x7f009c06ea50 2023-12-02T00:29:21.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.245+0000 7f00a1db6640 1 -- 172.21.15.46:0/1246569728 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f009c1589f0 con 0x7f009c06ea50 2023-12-02T00:29:21.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.245+0000 7f00a1db6640 1 -- 172.21.15.46:0/1246569728 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f009c158fb0 con 0x7f009c06ea50 2023-12-02T00:29:21.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.246+0000 7f00a1db6640 1 -- 172.21.15.46:0/1246569728 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f009c069400 con 0x7f009c06ea50 2023-12-02T00:29:21.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.246+0000 7f0083fff640 1 -- 172.21.15.46:0/1246569728 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f008c01a400 con 0x7f009c06ea50 2023-12-02T00:29:21.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.247+0000 7f0083fff640 1 --2- 172.21.15.46:0/1246569728 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f008806ae40 0x7f008806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:21.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.247+0000 7f0083fff640 1 -- 172.21.15.46:0/1246569728 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f008c0894e0 con 0x7f009c06ea50 2023-12-02T00:29:21.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.247+0000 7f009affd640 1 --2- 172.21.15.46:0/1246569728 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f008806ae40 0x7f008806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:21.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.248+0000 7f009affd640 1 --2- 172.21.15.46:0/1246569728 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f008806ae40 0x7f008806d300 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f009c13ec40 tx=0x7f0090008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:21.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.251+0000 7f0083fff640 1 -- 172.21.15.46:0/1246569728 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f008c0544f0 con 0x7f009c06ea50 2023-12-02T00:29:21.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.500+0000 7f00a1db6640 1 -- 172.21.15.46:0/1246569728 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]} v 0) v1 -- 0x7f009c070d30 con 0x7f009c06ea50 2023-12-02T00:29:21.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.846+0000 7f0083fff640 1 -- 172.21.15.46:0/1246569728 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]}]=0 v55) v1 ==== 150+0+0 (secure 0 0 0) 0x7f008c057b10 con 0x7f009c06ea50 2023-12-02T00:29:21.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.848+0000 7f00a1db6640 1 -- 172.21.15.46:0/1246569728 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f008806ae40 msgr2=0x7f008806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:21.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.848+0000 7f00a1db6640 1 --2- 172.21.15.46:0/1246569728 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f008806ae40 0x7f008806d300 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f009c13ec40 tx=0x7f0090008040 comp rx=0 tx=0).stop 2023-12-02T00:29:21.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.848+0000 7f00a1db6640 1 -- 172.21.15.46:0/1246569728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f009c06ea50 msgr2=0x7f009c156a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:21.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.848+0000 7f00a1db6640 1 --2- 172.21.15.46:0/1246569728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f009c06ea50 0x7f009c156a70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f008c00b3f0 tx=0x7f008c00b8b0 comp rx=0 tx=0).stop 2023-12-02T00:29:21.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.849+0000 7f00a1db6640 1 -- 172.21.15.46:0/1246569728 shutdown_connections 2023-12-02T00:29:21.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.849+0000 7f00a1db6640 1 --2- 172.21.15.46:0/1246569728 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f008806ae40 0x7f008806d300 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:21.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.849+0000 7f00a1db6640 1 --2- 172.21.15.46:0/1246569728 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f009c160fb0 0x7f009c06be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:21.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.849+0000 7f00a1db6640 1 --2- 172.21.15.46:0/1246569728 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f009c0d0ed0 0x7f009c157ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:21.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.849+0000 7f00a1db6640 1 --2- 172.21.15.46:0/1246569728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f009c06ea50 0x7f009c156a70 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:21.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.849+0000 7f00a1db6640 1 -- 172.21.15.46:0/1246569728 >> 172.21.15.46:0/1246569728 conn(0x7f009c06a6b0 msgr2=0x7f009c06aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:21.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.849+0000 7f00a1db6640 1 -- 172.21.15.46:0/1246569728 shutdown_connections 2023-12-02T00:29:21.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.849+0000 7f00a1db6640 1 -- 172.21.15.46:0/1246569728 wait complete. 2023-12-02T00:29:21.862 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-02T00:29:21.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.973+0000 7f70558da640 1 Processor -- start 2023-12-02T00:29:21.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.973+0000 7f70558da640 1 -- start start 2023-12-02T00:29:21.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.974+0000 7f70558da640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f705006ea50 0x7f705006ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:21.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.974+0000 7f70558da640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7050161d20 con 0x7f7050160fb0 2023-12-02T00:29:21.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.974+0000 7f70558da640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f705006f370 con 0x7f70500d0ed0 2023-12-02T00:29:21.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.974+0000 7f70558da640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7050161b20 con 0x7f705006ea50 2023-12-02T00:29:21.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.974+0000 7f70467fc640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7050160fb0 0x7f7050161410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38664/0 (socket says 172.21.15.46:38664) 2023-12-02T00:29:21.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.974+0000 7f70467fc640 1 -- 172.21.15.46:0/3544603732 learned_addr learned my addr 172.21.15.46:0/3544603732 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:21.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.974+0000 7f704effd640 1 --2- 172.21.15.46:0/3544603732 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f705006ea50 0x7f705006ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:21.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.974+0000 7f704e7fc640 1 -- 172.21.15.46:0/3544603732 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1759700731 0 0) 0x7f7050161d20 con 0x7f7050160fb0 2023-12-02T00:29:21.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.974+0000 7f704e7fc640 1 -- 172.21.15.46:0/3544603732 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f703c003520 con 0x7f7050160fb0 2023-12-02T00:29:21.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.974+0000 7f704e7fc640 1 -- 172.21.15.46:0/3544603732 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3919909718 0 0) 0x7f703c003520 con 0x7f7050160fb0 2023-12-02T00:29:21.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.974+0000 7f704e7fc640 1 -- 172.21.15.46:0/3544603732 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f705006ea50 msgr2=0x7f705006ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:21.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.974+0000 7f704e7fc640 1 --2- 172.21.15.46:0/3544603732 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f705006ea50 0x7f705006ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:21.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.974+0000 7f704e7fc640 1 -- 172.21.15.46:0/3544603732 >> v1:172.21.15.162:6789/0 conn(0x7f70500d0ed0 legacy=0x7f70500d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:21.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.974+0000 7f704e7fc640 1 -- 172.21.15.46:0/3544603732 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f705015e760 con 0x7f7050160fb0 2023-12-02T00:29:21.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.974+0000 7f704e7fc640 1 -- 172.21.15.46:0/3544603732 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7f7038001670 con 0x7f7050160fb0 2023-12-02T00:29:21.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.974+0000 7f704e7fc640 1 -- 172.21.15.46:0/3544603732 >> v1:172.21.15.46:6789/0 conn(0x7f7050160fb0 legacy=0x7f7050161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:21.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.974+0000 7f704e7fc640 1 --2- 172.21.15.46:0/3544603732 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f705006ea50 0x7f703c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:21.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.974+0000 7f704e7fc640 1 --2- 172.21.15.46:0/3544603732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f703c003ee0 0x7f703c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:21.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.974+0000 7f704e7fc640 1 --2- 172.21.15.46:0/3544603732 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f703c004d00 0x7f703c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:21.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.974+0000 7f704e7fc640 1 -- 172.21.15.46:0/3544603732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7050161b20 con 0x7f703c003ee0 2023-12-02T00:29:21.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.974+0000 7f704e7fc640 1 -- 172.21.15.46:0/3544603732 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f703c00b9e0 con 0x7f703c004d00 2023-12-02T00:29:21.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.974+0000 7f704e7fc640 1 -- 172.21.15.46:0/3544603732 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f703c00bba0 con 0x7f705006ea50 2023-12-02T00:29:21.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.974+0000 7f70467fc640 1 --2- 172.21.15.46:0/3544603732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f703c003ee0 0x7f703c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:21.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.975+0000 7f704effd640 1 --2- 172.21.15.46:0/3544603732 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f705006ea50 0x7f703c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:21.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.975+0000 7f704f7fe640 1 --2- 172.21.15.46:0/3544603732 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f703c004d00 0x7f703c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:21.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.975+0000 7f70467fc640 1 -- 172.21.15.46:0/3544603732 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f705006ea50 msgr2=0x7f703c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:21.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.975+0000 7f70467fc640 1 --2- 172.21.15.46:0/3544603732 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f705006ea50 0x7f703c0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:21.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.975+0000 7f70467fc640 1 -- 172.21.15.46:0/3544603732 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f703c004d00 msgr2=0x7f703c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:21.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.975+0000 7f70467fc640 1 --2- 172.21.15.46:0/3544603732 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f703c004d00 0x7f703c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:21.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.975+0000 7f70467fc640 1 -- 172.21.15.46:0/3544603732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f703c003520 con 0x7f703c003ee0 2023-12-02T00:29:21.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.976+0000 7f70467fc640 1 --2- 172.21.15.46:0/3544603732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f703c003ee0 0x7f703c0047c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f703800c210 tx=0x7f703c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39b5acc9a5f00ee3 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:21.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.976+0000 7f704e7fc640 1 -- 172.21.15.46:0/3544603732 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7038005ad0 con 0x7f703c003ee0 2023-12-02T00:29:21.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.976+0000 7f704e7fc640 1 -- 172.21.15.46:0/3544603732 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7038005c50 con 0x7f703c003ee0 2023-12-02T00:29:21.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.976+0000 7f704e7fc640 1 -- 172.21.15.46:0/3544603732 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703802c810 con 0x7f703c003ee0 2023-12-02T00:29:21.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.976+0000 7f70558da640 1 -- 172.21.15.46:0/3544603732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f703c003ee0 msgr2=0x7f703c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:21.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.976+0000 7f70558da640 1 --2- 172.21.15.46:0/3544603732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f703c003ee0 0x7f703c0047c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f703800c210 tx=0x7f703c0057f0 comp rx=0 tx=0).stop 2023-12-02T00:29:21.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.976+0000 7f704f7fe640 1 -- 172.21.15.46:0/3544603732 reap_dead start 2023-12-02T00:29:21.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.976+0000 7f70558da640 1 -- 172.21.15.46:0/3544603732 shutdown_connections 2023-12-02T00:29:21.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.976+0000 7f70558da640 1 -- 172.21.15.46:0/3544603732 >> 172.21.15.46:0/3544603732 conn(0x7f705006a6b0 msgr2=0x7f705006aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:21.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.976+0000 7f70558da640 1 -- 172.21.15.46:0/3544603732 shutdown_connections 2023-12-02T00:29:21.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.976+0000 7f70558da640 1 -- 172.21.15.46:0/3544603732 wait complete. 2023-12-02T00:29:21.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.977+0000 7f70558da640 1 Processor -- start 2023-12-02T00:29:21.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.978+0000 7f70558da640 1 -- start start 2023-12-02T00:29:21.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.978+0000 7f70558da640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f703c003ee0 0x7f70501552e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:21.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.978+0000 7f70558da640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f705006c8a0 0x7f7050156e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:21.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.978+0000 7f70558da640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f705006cd20 0x7f705013d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:21.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.978+0000 7f70558da640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7050155820 con 0x7f703c003ee0 2023-12-02T00:29:21.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.978+0000 7f70558da640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70501f6400 con 0x7f705006cd20 2023-12-02T00:29:21.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.978+0000 7f70558da640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70501f66a0 con 0x7f705006c8a0 2023-12-02T00:29:21.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.978+0000 7f704effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f703c003ee0 0x7f70501552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:21.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.978+0000 7f704effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f703c003ee0 0x7f70501552e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44980/0 (socket says 172.21.15.46:44980) 2023-12-02T00:29:21.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.978+0000 7f704effd640 1 -- 172.21.15.46:0/1941998125 learned_addr learned my addr 172.21.15.46:0/1941998125 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:21.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.979+0000 7f70467fc640 1 --2- 172.21.15.46:0/1941998125 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f705006c8a0 0x7f7050156e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:21.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.979+0000 7f704f7fe640 1 --2- 172.21.15.46:0/1941998125 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f705006cd20 0x7f705013d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:21.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.979+0000 7f704effd640 1 -- 172.21.15.46:0/1941998125 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f705006c8a0 msgr2=0x7f7050156e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:21.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.979+0000 7f704effd640 1 --2- 172.21.15.46:0/1941998125 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f705006c8a0 0x7f7050156e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:21.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.979+0000 7f704effd640 1 -- 172.21.15.46:0/1941998125 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f705006cd20 msgr2=0x7f705013d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:21.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.979+0000 7f704effd640 1 --2- 172.21.15.46:0/1941998125 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f705006cd20 0x7f705013d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:21.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.979+0000 7f704effd640 1 -- 172.21.15.46:0/1941998125 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70501f6980 con 0x7f703c003ee0 2023-12-02T00:29:21.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.979+0000 7f70467fc640 1 --2- 172.21.15.46:0/1941998125 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f705006c8a0 0x7f7050156e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:21.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.979+0000 7f704f7fe640 1 --2- 172.21.15.46:0/1941998125 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f705006cd20 0x7f705013d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:21.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.979+0000 7f704effd640 1 --2- 172.21.15.46:0/1941998125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f703c003ee0 0x7f70501552e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f70400119a0 tx=0x7f7040011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:21.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.979+0000 7f704cff9640 1 -- 172.21.15.46:0/1941998125 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704000faf0 con 0x7f703c003ee0 2023-12-02T00:29:21.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.979+0000 7f704cff9640 1 -- 172.21.15.46:0/1941998125 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f704000fc50 con 0x7f703c003ee0 2023-12-02T00:29:21.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.979+0000 7f70558da640 1 -- 172.21.15.46:0/1941998125 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70501f6c60 con 0x7f703c003ee0 2023-12-02T00:29:21.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.979+0000 7f704cff9640 1 -- 172.21.15.46:0/1941998125 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7040014440 con 0x7f703c003ee0 2023-12-02T00:29:21.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.979+0000 7f70558da640 1 -- 172.21.15.46:0/1941998125 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70501f7220 con 0x7f703c003ee0 2023-12-02T00:29:21.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.980+0000 7f70558da640 1 -- 172.21.15.46:0/1941998125 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7050069400 con 0x7f703c003ee0 2023-12-02T00:29:21.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.981+0000 7f704cff9640 1 -- 172.21.15.46:0/1941998125 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f70400145a0 con 0x7f703c003ee0 2023-12-02T00:29:21.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.981+0000 7f704cff9640 1 --2- 172.21.15.46:0/1941998125 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f703006ae40 0x7f703006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:21.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.981+0000 7f704cff9640 1 -- 172.21.15.46:0/1941998125 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7040089ed0 con 0x7f703c003ee0 2023-12-02T00:29:21.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.981+0000 7f70467fc640 1 --2- 172.21.15.46:0/1941998125 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f703006ae40 0x7f703006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:21.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.982+0000 7f70467fc640 1 --2- 172.21.15.46:0/1941998125 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f703006ae40 0x7f703006d300 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7038034000 tx=0x7f7038009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:21.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:21.985+0000 7f704cff9640 1 -- 172.21.15.46:0/1941998125 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7040054f80 con 0x7f703c003ee0 2023-12-02T00:29:22.164 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.163+0000 7f70558da640 1 -- 172.21.15.46:0/1941998125 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f7050070d30 con 0x7f703c003ee0 2023-12-02T00:29:22.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.166+0000 7f704cff9640 1 -- 172.21.15.46:0/1941998125 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f70400220c0 con 0x7f703c003ee0 2023-12-02T00:29:22.170 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.168+0000 7f70558da640 1 -- 172.21.15.46:0/1941998125 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f703006ae40 msgr2=0x7f703006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:22.170 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.168+0000 7f70558da640 1 --2- 172.21.15.46:0/1941998125 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f703006ae40 0x7f703006d300 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7038034000 tx=0x7f7038009030 comp rx=0 tx=0).stop 2023-12-02T00:29:22.170 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.168+0000 7f70558da640 1 -- 172.21.15.46:0/1941998125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f703c003ee0 msgr2=0x7f70501552e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:22.170 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.168+0000 7f70558da640 1 --2- 172.21.15.46:0/1941998125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f703c003ee0 0x7f70501552e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f70400119a0 tx=0x7f7040011e60 comp rx=0 tx=0).stop 2023-12-02T00:29:22.170 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.168+0000 7f70558da640 1 -- 172.21.15.46:0/1941998125 shutdown_connections 2023-12-02T00:29:22.170 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.168+0000 7f70558da640 1 --2- 172.21.15.46:0/1941998125 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f703006ae40 0x7f703006d300 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:22.170 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.168+0000 7f70558da640 1 --2- 172.21.15.46:0/1941998125 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f705006cd20 0x7f705013d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:22.170 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.168+0000 7f70558da640 1 --2- 172.21.15.46:0/1941998125 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f705006c8a0 0x7f7050156e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:22.170 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.168+0000 7f70558da640 1 --2- 172.21.15.46:0/1941998125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f703c003ee0 0x7f70501552e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:22.170 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.168+0000 7f70558da640 1 -- 172.21.15.46:0/1941998125 >> 172.21.15.46:0/1941998125 conn(0x7f705006a6b0 msgr2=0x7f705015fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:22.170 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.168+0000 7f70558da640 1 -- 172.21.15.46:0/1941998125 shutdown_connections 2023-12-02T00:29:22.170 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.168+0000 7f70558da640 1 -- 172.21.15.46:0/1941998125 wait complete. 2023-12-02T00:29:22.180 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-02T00:29:22.181 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-02T00:29:22.231 DEBUG:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-02T00:29:22.283 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-02T00:29:22.284 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-02T00:29:22.284 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-02T00:29:22.300 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-02T00:29:22.301 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-02T00:29:22.356 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:29:22.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.502+0000 7f2db1b61640 1 Processor -- start 2023-12-02T00:29:22.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.502+0000 7f2db1b61640 1 -- start start 2023-12-02T00:29:22.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.503+0000 7f2db1b61640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2dac06f210 0x7f2dac140800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:22.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.503+0000 7f2db1b61640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2dac140d40 0x7f2dac13f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:22.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.503+0000 7f2db1b61640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dac15ba40 con 0x7f2dac15c2f0 2023-12-02T00:29:22.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.503+0000 7f2db1b61640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dac15c6d0 con 0x7f2dac140d40 2023-12-02T00:29:22.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.503+0000 7f2db1b61640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dac13e530 con 0x7f2dac06f210 2023-12-02T00:29:22.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.503+0000 7f2dabfff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2dac140d40 0x7f2dac13f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:22.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.503+0000 7f2dabfff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2dac140d40 0x7f2dac13f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44996/0 (socket says 172.21.15.46:44996) 2023-12-02T00:29:22.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.503+0000 7f2dabfff640 1 -- 172.21.15.46:0/1353737094 learned_addr learned my addr 172.21.15.46:0/1353737094 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:22.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.503+0000 7f2daaffd640 1 --2- 172.21.15.46:0/1353737094 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2dac06f210 0x7f2dac140800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:22.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.503+0000 7f2dabfff640 1 -- 172.21.15.46:0/1353737094 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2dac06f210 msgr2=0x7f2dac140800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:22.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.503+0000 7f2dabfff640 1 --2- 172.21.15.46:0/1353737094 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2dac06f210 0x7f2dac140800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:22.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.503+0000 7f2dabfff640 1 -- 172.21.15.46:0/1353737094 >> v1:172.21.15.162:6789/0 conn(0x7f2dac15c2f0 legacy=0x7f2dac06eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:22.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.503+0000 7f2dabfff640 1 -- 172.21.15.46:0/1353737094 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dac1411d0 con 0x7f2dac140d40 2023-12-02T00:29:22.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.504+0000 7f2dabfff640 1 --2- 172.21.15.46:0/1353737094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dac140d40 0x7f2dac13f330 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2da4009eb0 tx=0x7f2da402bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d2744794568d5 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:22.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.504+0000 7f2daa7fc640 1 -- 172.21.15.46:0/1353737094 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da402cad0 con 0x7f2dac140d40 2023-12-02T00:29:22.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.504+0000 7f2daa7fc640 1 -- 172.21.15.46:0/1353737094 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2da402cc30 con 0x7f2dac140d40 2023-12-02T00:29:22.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.504+0000 7f2daa7fc640 1 -- 172.21.15.46:0/1353737094 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da40343f0 con 0x7f2dac140d40 2023-12-02T00:29:22.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.505+0000 7f2db1b61640 1 -- 172.21.15.46:0/1353737094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dac140d40 msgr2=0x7f2dac13f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:22.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.505+0000 7f2db1b61640 1 --2- 172.21.15.46:0/1353737094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dac140d40 0x7f2dac13f330 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2da4009eb0 tx=0x7f2da402bf30 comp rx=0 tx=0).stop 2023-12-02T00:29:22.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.505+0000 7f2db1b61640 1 -- 172.21.15.46:0/1353737094 shutdown_connections 2023-12-02T00:29:22.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.505+0000 7f2db1b61640 1 --2- 172.21.15.46:0/1353737094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dac140d40 0x7f2dac13f330 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:22.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.505+0000 7f2db1b61640 1 --2- 172.21.15.46:0/1353737094 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2dac06f210 0x7f2dac140800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:22.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.505+0000 7f2db1b61640 1 -- 172.21.15.46:0/1353737094 >> 172.21.15.46:0/1353737094 conn(0x7f2dac06a6b0 msgr2=0x7f2dac06aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:22.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.505+0000 7f2db1b61640 1 -- 172.21.15.46:0/1353737094 shutdown_connections 2023-12-02T00:29:22.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.505+0000 7f2db1b61640 1 -- 172.21.15.46:0/1353737094 wait complete. 2023-12-02T00:29:22.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.506+0000 7f2db1b61640 1 Processor -- start 2023-12-02T00:29:22.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.506+0000 7f2db1b61640 1 -- start start 2023-12-02T00:29:22.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.507+0000 7f2db1b61640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2dac06f210 0x7f2dac13ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:22.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.507+0000 7f2db1b61640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dac140d40 0x7f2dac15a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:22.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.507+0000 7f2db1b61640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2dac15c2f0 0x7f2dac15a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:22.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.507+0000 7f2db1b61640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dac06c390 con 0x7f2dac140d40 2023-12-02T00:29:22.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.507+0000 7f2db1b61640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dac06b780 con 0x7f2dac15c2f0 2023-12-02T00:29:22.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.507+0000 7f2db1b61640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dac06ba60 con 0x7f2dac06f210 2023-12-02T00:29:22.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.507+0000 7f2daaffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dac140d40 0x7f2dac15a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:22.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.507+0000 7f2daaffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dac140d40 0x7f2dac15a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45004/0 (socket says 172.21.15.46:45004) 2023-12-02T00:29:22.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.507+0000 7f2daaffd640 1 -- 172.21.15.46:0/694995851 learned_addr learned my addr 172.21.15.46:0/694995851 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:22.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.507+0000 7f2dab7fe640 1 --2- 172.21.15.46:0/694995851 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2dac06f210 0x7f2dac13ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:22.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.507+0000 7f2dabfff640 1 --2- 172.21.15.46:0/694995851 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2dac15c2f0 0x7f2dac15a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:22.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.507+0000 7f2daaffd640 1 -- 172.21.15.46:0/694995851 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2dac06f210 msgr2=0x7f2dac13ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:22.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.507+0000 7f2daaffd640 1 --2- 172.21.15.46:0/694995851 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2dac06f210 0x7f2dac13ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:22.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.507+0000 7f2daaffd640 1 -- 172.21.15.46:0/694995851 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2dac15c2f0 msgr2=0x7f2dac15a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:22.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.507+0000 7f2daaffd640 1 --2- 172.21.15.46:0/694995851 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2dac15c2f0 0x7f2dac15a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:22.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.507+0000 7f2daaffd640 1 -- 172.21.15.46:0/694995851 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da4009b30 con 0x7f2dac140d40 2023-12-02T00:29:22.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.507+0000 7f2dabfff640 1 --2- 172.21.15.46:0/694995851 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2dac15c2f0 0x7f2dac15a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:22.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.508+0000 7f2daaffd640 1 --2- 172.21.15.46:0/694995851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dac140d40 0x7f2dac15a0c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2da000e9b0 tx=0x7f2da000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:22.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.508+0000 7f2d93fff640 1 -- 172.21.15.46:0/694995851 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da0009670 con 0x7f2dac140d40 2023-12-02T00:29:22.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.508+0000 7f2d93fff640 1 -- 172.21.15.46:0/694995851 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2da000c9a0 con 0x7f2dac140d40 2023-12-02T00:29:22.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.508+0000 7f2d93fff640 1 -- 172.21.15.46:0/694995851 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da000cd50 con 0x7f2dac140d40 2023-12-02T00:29:22.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.508+0000 7f2db1b61640 1 -- 172.21.15.46:0/694995851 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dac06bd40 con 0x7f2dac140d40 2023-12-02T00:29:22.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.508+0000 7f2db1b61640 1 -- 172.21.15.46:0/694995851 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dac13b380 con 0x7f2dac140d40 2023-12-02T00:29:22.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.509+0000 7f2db1b61640 1 -- 172.21.15.46:0/694995851 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dac069400 con 0x7f2dac140d40 2023-12-02T00:29:22.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.509+0000 7f2d93fff640 1 -- 172.21.15.46:0/694995851 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2da0013470 con 0x7f2dac140d40 2023-12-02T00:29:22.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.510+0000 7f2d93fff640 1 --2- 172.21.15.46:0/694995851 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2d9806ae40 0x7f2d9806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:22.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.510+0000 7f2d93fff640 1 -- 172.21.15.46:0/694995851 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2da0089810 con 0x7f2dac140d40 2023-12-02T00:29:22.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.510+0000 7f2dab7fe640 1 --2- 172.21.15.46:0/694995851 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2d9806ae40 0x7f2d9806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:22.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.510+0000 7f2dab7fe640 1 --2- 172.21.15.46:0/694995851 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2d9806ae40 0x7f2d9806d300 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c000f60 tx=0x7f2d9c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:22.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.514+0000 7f2d93fff640 1 -- 172.21.15.46:0/694995851 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2da0054970 con 0x7f2dac140d40 2023-12-02T00:29:22.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.692+0000 7f2db1b61640 1 -- 172.21.15.46:0/694995851 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2dac070d30 con 0x7f2dac140d40 2023-12-02T00:29:22.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.693+0000 7f2d93fff640 1 -- 172.21.15.46:0/694995851 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f2da0017090 con 0x7f2dac140d40 2023-12-02T00:29:22.695 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:29:22.695 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":21,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:28:50.700112+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-02T00:28:47.273337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-02T00:28:48.060115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:29:22.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.695+0000 7f2db1b61640 1 -- 172.21.15.46:0/694995851 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2d9806ae40 msgr2=0x7f2d9806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:22.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.695+0000 7f2db1b61640 1 --2- 172.21.15.46:0/694995851 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2d9806ae40 0x7f2d9806d300 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c000f60 tx=0x7f2d9c009040 comp rx=0 tx=0).stop 2023-12-02T00:29:22.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.695+0000 7f2db1b61640 1 -- 172.21.15.46:0/694995851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dac140d40 msgr2=0x7f2dac15a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:22.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.695+0000 7f2db1b61640 1 --2- 172.21.15.46:0/694995851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dac140d40 0x7f2dac15a0c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2da000e9b0 tx=0x7f2da000ee70 comp rx=0 tx=0).stop 2023-12-02T00:29:22.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.695+0000 7f2db1b61640 1 -- 172.21.15.46:0/694995851 shutdown_connections 2023-12-02T00:29:22.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.695+0000 7f2db1b61640 1 --2- 172.21.15.46:0/694995851 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2d9806ae40 0x7f2d9806d300 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:22.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.695+0000 7f2db1b61640 1 --2- 172.21.15.46:0/694995851 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2dac15c2f0 0x7f2dac15a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:22.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.695+0000 7f2db1b61640 1 --2- 172.21.15.46:0/694995851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dac140d40 0x7f2dac15a0c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:22.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.695+0000 7f2db1b61640 1 --2- 172.21.15.46:0/694995851 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2dac06f210 0x7f2dac13ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:22.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.695+0000 7f2db1b61640 1 -- 172.21.15.46:0/694995851 >> 172.21.15.46:0/694995851 conn(0x7f2dac06a6b0 msgr2=0x7f2dac15ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:22.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.695+0000 7f2db1b61640 1 -- 172.21.15.46:0/694995851 shutdown_connections 2023-12-02T00:29:22.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.695+0000 7f2db1b61640 1 -- 172.21.15.46:0/694995851 wait complete. 2023-12-02T00:29:22.708 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-02T00:28:37.213645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-02T00:28:47.273337+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-02T00:28:48.060115+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-02T00:29:22.709 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-02T00:29:22.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.816+0000 7fcd1d4c2640 1 Processor -- start 2023-12-02T00:29:22.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.816+0000 7fcd1d4c2640 1 -- start start 2023-12-02T00:29:22.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.817+0000 7fcd1d4c2640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcd1815d2d0 0x7fcd180b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:22.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.817+0000 7fcd1d4c2640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fcd18160f70 0x7fcd180cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:22.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.817+0000 7fcd1d4c2640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd181614e0 con 0x7fcd180ce480 2023-12-02T00:29:22.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.817+0000 7fcd1d4c2640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd181617c0 con 0x7fcd1815d2d0 2023-12-02T00:29:22.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.817+0000 7fcd1d4c2640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd18161aa0 con 0x7fcd18160f70 2023-12-02T00:29:22.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.817+0000 7fcd16ffd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcd1815d2d0 0x7fcd180b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:22.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.817+0000 7fcd177fe640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fcd180ce480 0x7fcd1806b570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50356/0 (socket says 172.21.15.46:50356) 2023-12-02T00:29:22.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.817+0000 7fcd16ffd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcd1815d2d0 0x7fcd180b8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52526/0 (socket says 172.21.15.46:52526) 2023-12-02T00:29:22.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.817+0000 7fcd177fe640 1 -- 172.21.15.46:0/4194000140 learned_addr learned my addr 172.21.15.46:0/4194000140 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:22.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.817+0000 7fcd0ffff640 1 --2- 172.21.15.46:0/4194000140 >> v2:172.21.15.162:3300/0 conn(0x7fcd18160f70 0x7fcd180cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:22.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.817+0000 7fcd16ffd640 1 -- 172.21.15.46:0/4194000140 >> v2:172.21.15.162:3300/0 conn(0x7fcd18160f70 msgr2=0x7fcd180cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:22.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.817+0000 7fcd16ffd640 1 --2- 172.21.15.46:0/4194000140 >> v2:172.21.15.162:3300/0 conn(0x7fcd18160f70 0x7fcd180cdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:22.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.817+0000 7fcd16ffd640 1 -- 172.21.15.46:0/4194000140 >> v1:172.21.15.46:6789/0 conn(0x7fcd180ce480 legacy=0x7fcd1806b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:22.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.817+0000 7fcd167fc640 1 -- 172.21.15.46:0/4194000140 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1084639793 0 0) 0x7fcd181614e0 con 0x7fcd180ce480 2023-12-02T00:29:22.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.817+0000 7fcd16ffd640 1 -- 172.21.15.46:0/4194000140 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd18161d80 con 0x7fcd1815d2d0 2023-12-02T00:29:22.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.817+0000 7fcd0ffff640 1 --2- 172.21.15.46:0/4194000140 >> v2:172.21.15.162:3300/0 conn(0x7fcd18160f70 0x7fcd180cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:29:22.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.818+0000 7fcd16ffd640 1 --2- 172.21.15.46:0/4194000140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd1815d2d0 0x7fcd180b8b90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcd180cdb80 tx=0x7fcd0802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50bcbd306b5fdaef server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:22.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.818+0000 7fcd167fc640 1 -- 172.21.15.46:0/4194000140 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd08031830 con 0x7fcd1815d2d0 2023-12-02T00:29:22.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.818+0000 7fcd167fc640 1 -- 172.21.15.46:0/4194000140 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd08031990 con 0x7fcd1815d2d0 2023-12-02T00:29:22.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.818+0000 7fcd167fc640 1 -- 172.21.15.46:0/4194000140 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd08031d20 con 0x7fcd1815d2d0 2023-12-02T00:29:22.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.819+0000 7fcd1d4c2640 1 -- 172.21.15.46:0/4194000140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd1815d2d0 msgr2=0x7fcd180b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:22.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.819+0000 7fcd1d4c2640 1 --2- 172.21.15.46:0/4194000140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd1815d2d0 0x7fcd180b8b90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcd180cdb80 tx=0x7fcd0802b8d0 comp rx=0 tx=0).stop 2023-12-02T00:29:22.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.819+0000 7fcd1d4c2640 1 -- 172.21.15.46:0/4194000140 shutdown_connections 2023-12-02T00:29:22.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.819+0000 7fcd1d4c2640 1 --2- 172.21.15.46:0/4194000140 >> v2:172.21.15.162:3300/0 conn(0x7fcd18160f70 0x7fcd180cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:22.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.819+0000 7fcd1d4c2640 1 --2- 172.21.15.46:0/4194000140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd1815d2d0 0x7fcd180b8b90 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:22.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.819+0000 7fcd1d4c2640 1 -- 172.21.15.46:0/4194000140 >> 172.21.15.46:0/4194000140 conn(0x7fcd18072230 msgr2=0x7fcd180cfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:22.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.819+0000 7fcd1d4c2640 1 -- 172.21.15.46:0/4194000140 shutdown_connections 2023-12-02T00:29:22.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.819+0000 7fcd1d4c2640 1 -- 172.21.15.46:0/4194000140 wait complete. 2023-12-02T00:29:22.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.820+0000 7fcd1d4c2640 1 Processor -- start 2023-12-02T00:29:22.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.820+0000 7fcd1d4c2640 1 -- start start 2023-12-02T00:29:22.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.821+0000 7fcd1d4c2640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd180ce480 0x7fcd180b41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:22.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.821+0000 7fcd1d4c2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd1815d2d0 0x7fcd1814b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:22.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.821+0000 7fcd1d4c2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcd18160f70 0x7fcd1815ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:22.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.821+0000 7fcd1d4c2640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd18155b00 con 0x7fcd1815d2d0 2023-12-02T00:29:22.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.821+0000 7fcd1d4c2640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd180b3100 con 0x7fcd180ce480 2023-12-02T00:29:22.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.821+0000 7fcd1d4c2640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd180b33e0 con 0x7fcd18160f70 2023-12-02T00:29:22.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.821+0000 7fcd0ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd1815d2d0 0x7fcd1814b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:22.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.821+0000 7fcd0ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd1815d2d0 0x7fcd1814b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52540/0 (socket says 172.21.15.46:52540) 2023-12-02T00:29:22.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.821+0000 7fcd0ffff640 1 -- 172.21.15.46:0/3729394474 learned_addr learned my addr 172.21.15.46:0/3729394474 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:22.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.821+0000 7fcd16ffd640 1 --2- 172.21.15.46:0/3729394474 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd180ce480 0x7fcd180b41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:22.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.821+0000 7fcd177fe640 1 --2- 172.21.15.46:0/3729394474 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcd18160f70 0x7fcd1815ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:22.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.821+0000 7fcd0ffff640 1 -- 172.21.15.46:0/3729394474 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcd18160f70 msgr2=0x7fcd1815ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:22.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.821+0000 7fcd0ffff640 1 --2- 172.21.15.46:0/3729394474 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcd18160f70 0x7fcd1815ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:22.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.821+0000 7fcd0ffff640 1 -- 172.21.15.46:0/3729394474 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd180ce480 msgr2=0x7fcd180b41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:22.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.821+0000 7fcd0ffff640 1 --2- 172.21.15.46:0/3729394474 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd180ce480 0x7fcd180b41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:22.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.821+0000 7fcd0ffff640 1 -- 172.21.15.46:0/3729394474 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd08009400 con 0x7fcd1815d2d0 2023-12-02T00:29:22.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.821+0000 7fcd0ffff640 1 --2- 172.21.15.46:0/3729394474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd1815d2d0 0x7fcd1814b4f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcd0000c7f0 tx=0x7fcd0000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:22.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.821+0000 7fcd0f7fe640 1 -- 172.21.15.46:0/3729394474 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd00007820 con 0x7fcd1815d2d0 2023-12-02T00:29:22.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.821+0000 7fcd0f7fe640 1 -- 172.21.15.46:0/3729394474 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd00007980 con 0x7fcd1815d2d0 2023-12-02T00:29:22.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.821+0000 7fcd0f7fe640 1 -- 172.21.15.46:0/3729394474 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd00007cf0 con 0x7fcd1815d2d0 2023-12-02T00:29:22.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.821+0000 7fcd1d4c2640 1 -- 172.21.15.46:0/3729394474 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd180b36c0 con 0x7fcd1815d2d0 2023-12-02T00:29:22.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.822+0000 7fcd1d4c2640 1 -- 172.21.15.46:0/3729394474 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd180b3c80 con 0x7fcd1815d2d0 2023-12-02T00:29:22.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.822+0000 7fcd1d4c2640 1 -- 172.21.15.46:0/3729394474 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd180b3e40 con 0x7fcd1815d2d0 2023-12-02T00:29:22.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.823+0000 7fcd0f7fe640 1 -- 172.21.15.46:0/3729394474 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcd000113f0 con 0x7fcd1815d2d0 2023-12-02T00:29:22.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.823+0000 7fcd0f7fe640 1 --2- 172.21.15.46:0/3729394474 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fccf806ae40 0x7fccf806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:22.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.823+0000 7fcd0f7fe640 1 -- 172.21.15.46:0/3729394474 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd00089700 con 0x7fcd1815d2d0 2023-12-02T00:29:22.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.823+0000 7fcd16ffd640 1 --2- 172.21.15.46:0/3729394474 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fccf806ae40 0x7fccf806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:22.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.824+0000 7fcd16ffd640 1 --2- 172.21.15.46:0/3729394474 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fccf806ae40 0x7fccf806d300 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcd08036000 tx=0x7fcd0803c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:22.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:22.827+0000 7fcd0f7fe640 1 -- 172.21.15.46:0/3729394474 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcd000547b0 con 0x7fcd1815d2d0 2023-12-02T00:29:23.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.001+0000 7fcd1d4c2640 1 -- 172.21.15.46:0/3729394474 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcd18069400 con 0x7fcd1815d2d0 2023-12-02T00:29:23.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.002+0000 7fcd0f7fe640 1 -- 172.21.15.46:0/3729394474 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcd0001f0a0 con 0x7fcd1815d2d0 2023-12-02T00:29:23.004 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-12-02T00:29:23.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.003+0000 7fcd1d4c2640 1 -- 172.21.15.46:0/3729394474 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fccf806ae40 msgr2=0x7fccf806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:23.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.003+0000 7fcd1d4c2640 1 --2- 172.21.15.46:0/3729394474 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fccf806ae40 0x7fccf806d300 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcd08036000 tx=0x7fcd0803c040 comp rx=0 tx=0).stop 2023-12-02T00:29:23.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.004+0000 7fcd1d4c2640 1 -- 172.21.15.46:0/3729394474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd1815d2d0 msgr2=0x7fcd1814b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:23.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.004+0000 7fcd1d4c2640 1 --2- 172.21.15.46:0/3729394474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd1815d2d0 0x7fcd1814b4f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcd0000c7f0 tx=0x7fcd0000ccb0 comp rx=0 tx=0).stop 2023-12-02T00:29:23.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.004+0000 7fcd1d4c2640 1 -- 172.21.15.46:0/3729394474 shutdown_connections 2023-12-02T00:29:23.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.004+0000 7fcd1d4c2640 1 --2- 172.21.15.46:0/3729394474 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fccf806ae40 0x7fccf806d300 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:23.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.004+0000 7fcd1d4c2640 1 --2- 172.21.15.46:0/3729394474 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcd18160f70 0x7fcd1815ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:23.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.004+0000 7fcd1d4c2640 1 --2- 172.21.15.46:0/3729394474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd1815d2d0 0x7fcd1814b4f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:23.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.004+0000 7fcd1d4c2640 1 --2- 172.21.15.46:0/3729394474 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd180ce480 0x7fcd180b41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:23.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.004+0000 7fcd1d4c2640 1 -- 172.21.15.46:0/3729394474 >> 172.21.15.46:0/3729394474 conn(0x7fcd18072230 msgr2=0x7fcd1815db20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:23.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.004+0000 7fcd1d4c2640 1 -- 172.21.15.46:0/3729394474 shutdown_connections 2023-12-02T00:29:23.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.004+0000 7fcd1d4c2640 1 -- 172.21.15.46:0/3729394474 wait complete. 2023-12-02T00:29:23.021 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-02T00:29:23.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.133+0000 7f55508a6640 1 Processor -- start 2023-12-02T00:29:23.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.133+0000 7f55508a6640 1 -- start start 2023-12-02T00:29:23.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.134+0000 7f55508a6640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f554c143490 0x7f554c0b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:23.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.134+0000 7f55508a6640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f554c15e2a0 con 0x7f554c15abf0 2023-12-02T00:29:23.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.134+0000 7f55508a6640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f554c15b240 con 0x7f554c15e730 2023-12-02T00:29:23.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.134+0000 7f55508a6640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f554c15b520 con 0x7f554c143490 2023-12-02T00:29:23.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.134+0000 7f554ad9d640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f554c15abf0 0x7f554c0b2f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50360/0 (socket says 172.21.15.46:50360) 2023-12-02T00:29:23.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.134+0000 7f554ad9d640 1 -- 172.21.15.46:0/3766747738 learned_addr learned my addr 172.21.15.46:0/3766747738 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:23.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.134+0000 7f554a59c640 1 --2- 172.21.15.46:0/3766747738 >> v2:172.21.15.162:3300/0 conn(0x7f554c143490 0x7f554c0b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:23.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.134+0000 7f5549d9b640 1 -- 172.21.15.46:0/3766747738 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2623032503 0 0) 0x7f554c15e2a0 con 0x7f554c15abf0 2023-12-02T00:29:23.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.134+0000 7f5549d9b640 1 -- 172.21.15.46:0/3766747738 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5538005730 con 0x7f554c15abf0 2023-12-02T00:29:23.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.134+0000 7f5549d9b640 1 -- 172.21.15.46:0/3766747738 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1889376244 0 0) 0x7f5538005730 con 0x7f554c15abf0 2023-12-02T00:29:23.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.134+0000 7f5549d9b640 1 -- 172.21.15.46:0/3766747738 >> v2:172.21.15.162:3300/0 conn(0x7f554c143490 msgr2=0x7f554c0b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:23.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.134+0000 7f5549d9b640 1 --2- 172.21.15.46:0/3766747738 >> v2:172.21.15.162:3300/0 conn(0x7f554c143490 0x7f554c0b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:23.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.134+0000 7f5549d9b640 1 -- 172.21.15.46:0/3766747738 >> v1:172.21.15.162:6789/0 conn(0x7f554c15e730 legacy=0x7f554c15a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:23.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.134+0000 7f5549d9b640 1 -- 172.21.15.46:0/3766747738 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f554c15dc50 con 0x7f554c15abf0 2023-12-02T00:29:23.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.134+0000 7f5549d9b640 1 -- 172.21.15.46:0/3766747738 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7f55440019f0 con 0x7f554c15abf0 2023-12-02T00:29:23.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.134+0000 7f5549d9b640 1 -- 172.21.15.46:0/3766747738 >> v1:172.21.15.46:6789/0 conn(0x7f554c15abf0 legacy=0x7f554c0b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:23.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.135+0000 7f5549d9b640 1 --2- 172.21.15.46:0/3766747738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5538005b80 0x7f5538005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:23.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.135+0000 7f5549d9b640 1 --2- 172.21.15.46:0/3766747738 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55380069d0 0x7f5538006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:23.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.135+0000 7f5549d9b640 1 --2- 172.21.15.46:0/3766747738 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5538007390 0x7f553800db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:23.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.135+0000 7f5549d9b640 1 -- 172.21.15.46:0/3766747738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554c15b520 con 0x7f5538005b80 2023-12-02T00:29:23.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.135+0000 7f5549d9b640 1 -- 172.21.15.46:0/3766747738 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553800e230 con 0x7f5538007390 2023-12-02T00:29:23.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.135+0000 7f5549d9b640 1 -- 172.21.15.46:0/3766747738 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f553800e3f0 con 0x7f55380069d0 2023-12-02T00:29:23.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.135+0000 7f554a59c640 1 --2- 172.21.15.46:0/3766747738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5538005b80 0x7f5538005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:23.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.135+0000 7f5541d9b640 1 --2- 172.21.15.46:0/3766747738 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55380069d0 0x7f5538006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:23.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.135+0000 7f554ad9d640 1 --2- 172.21.15.46:0/3766747738 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5538007390 0x7f553800db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:23.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.135+0000 7f554a59c640 1 -- 172.21.15.46:0/3766747738 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55380069d0 msgr2=0x7f5538006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:23.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.135+0000 7f554a59c640 1 --2- 172.21.15.46:0/3766747738 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55380069d0 0x7f5538006e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:23.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.135+0000 7f554a59c640 1 -- 172.21.15.46:0/3766747738 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5538007390 msgr2=0x7f553800db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:23.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.135+0000 7f554a59c640 1 --2- 172.21.15.46:0/3766747738 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5538007390 0x7f553800db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:23.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.135+0000 7f554a59c640 1 -- 172.21.15.46:0/3766747738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5538005730 con 0x7f5538005b80 2023-12-02T00:29:23.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.135+0000 7f554ad9d640 1 --2- 172.21.15.46:0/3766747738 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5538007390 0x7f553800db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:23.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.135+0000 7f5541d9b640 1 --2- 172.21.15.46:0/3766747738 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55380069d0 0x7f5538006e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:23.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.135+0000 7f554ad9d640 1 -- 172.21.15.46:0/3766747738 reap_dead start 2023-12-02T00:29:23.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.136+0000 7f554a59c640 1 --2- 172.21.15.46:0/3766747738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5538005b80 0x7f5538005fa0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f553c009ac0 tx=0x7f5538007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3026cbc3457ca5b4 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:23.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.136+0000 7f5549d9b640 1 -- 172.21.15.46:0/3766747738 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553c02c6b0 con 0x7f5538005b80 2023-12-02T00:29:23.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.136+0000 7f5549d9b640 1 -- 172.21.15.46:0/3766747738 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f553c02c810 con 0x7f5538005b80 2023-12-02T00:29:23.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.136+0000 7f5549d9b640 1 -- 172.21.15.46:0/3766747738 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553c02cba0 con 0x7f5538005b80 2023-12-02T00:29:23.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.136+0000 7f55508a6640 1 -- 172.21.15.46:0/3766747738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5538005b80 msgr2=0x7f5538005fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:23.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.136+0000 7f55508a6640 1 --2- 172.21.15.46:0/3766747738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5538005b80 0x7f5538005fa0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f553c009ac0 tx=0x7f5538007a00 comp rx=0 tx=0).stop 2023-12-02T00:29:23.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.136+0000 7f55508a6640 1 -- 172.21.15.46:0/3766747738 shutdown_connections 2023-12-02T00:29:23.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.136+0000 7f55508a6640 1 --2- 172.21.15.46:0/3766747738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5538005b80 0x7f5538005fa0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:23.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.136+0000 7f55508a6640 1 -- 172.21.15.46:0/3766747738 >> 172.21.15.46:0/3766747738 conn(0x7f554c06a4d0 msgr2=0x7f554c06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:23.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.136+0000 7f55508a6640 1 -- 172.21.15.46:0/3766747738 shutdown_connections 2023-12-02T00:29:23.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.136+0000 7f55508a6640 1 -- 172.21.15.46:0/3766747738 wait complete. 2023-12-02T00:29:23.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.138+0000 7f55508a6640 1 Processor -- start 2023-12-02T00:29:23.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.138+0000 7f55508a6640 1 -- start start 2023-12-02T00:29:23.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.139+0000 7f55508a6640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5538005b80 0x7f554c13fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:23.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.139+0000 7f55508a6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f554c13d7f0 0x7f554c0b36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:23.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.139+0000 7f55508a6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f554c0b3c00 0x7f554c15b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:23.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.139+0000 7f55508a6640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554c0b4080 con 0x7f554c13d7f0 2023-12-02T00:29:23.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.139+0000 7f55508a6640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554c13f0c0 con 0x7f5538005b80 2023-12-02T00:29:23.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.139+0000 7f55508a6640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f554c13ede0 con 0x7f554c0b3c00 2023-12-02T00:29:23.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.139+0000 7f5541d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f554c13d7f0 0x7f554c0b36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:23.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.139+0000 7f5541d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f554c13d7f0 0x7f554c0b36c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52556/0 (socket says 172.21.15.46:52556) 2023-12-02T00:29:23.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.139+0000 7f5541d9b640 1 -- 172.21.15.46:0/2430175684 learned_addr learned my addr 172.21.15.46:0/2430175684 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:23.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.139+0000 7f554a59c640 1 --2- 172.21.15.46:0/2430175684 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5538005b80 0x7f554c13fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:23.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.139+0000 7f554ad9d640 1 --2- 172.21.15.46:0/2430175684 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f554c0b3c00 0x7f554c15b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:23.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.139+0000 7f5541d9b640 1 -- 172.21.15.46:0/2430175684 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f554c0b3c00 msgr2=0x7f554c15b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:23.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.139+0000 7f5541d9b640 1 --2- 172.21.15.46:0/2430175684 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f554c0b3c00 0x7f554c15b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:23.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.139+0000 7f5541d9b640 1 -- 172.21.15.46:0/2430175684 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5538005b80 msgr2=0x7f554c13fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:23.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.139+0000 7f5541d9b640 1 --2- 172.21.15.46:0/2430175684 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5538005b80 0x7f554c13fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:23.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.139+0000 7f5541d9b640 1 -- 172.21.15.46:0/2430175684 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f554c0a8570 con 0x7f554c13d7f0 2023-12-02T00:29:23.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.139+0000 7f554ad9d640 1 --2- 172.21.15.46:0/2430175684 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f554c0b3c00 0x7f554c15b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:29:23.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.139+0000 7f5541d9b640 1 --2- 172.21.15.46:0/2430175684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f554c13d7f0 0x7f554c0b36c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f553400e480 tx=0x7f553400e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:23.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.139+0000 7f55437fe640 1 -- 172.21.15.46:0/2430175684 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5534017070 con 0x7f554c13d7f0 2023-12-02T00:29:23.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.139+0000 7f55437fe640 1 -- 172.21.15.46:0/2430175684 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55340052e0 con 0x7f554c13d7f0 2023-12-02T00:29:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.139+0000 7f55508a6640 1 -- 172.21.15.46:0/2430175684 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f554c0a8850 con 0x7f554c13d7f0 2023-12-02T00:29:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.139+0000 7f55437fe640 1 -- 172.21.15.46:0/2430175684 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5534005490 con 0x7f554c13d7f0 2023-12-02T00:29:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.140+0000 7f55508a6640 1 -- 172.21.15.46:0/2430175684 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f554c13eab0 con 0x7f554c13d7f0 2023-12-02T00:29:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.140+0000 7f55508a6640 1 -- 172.21.15.46:0/2430175684 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f554c069400 con 0x7f554c13d7f0 2023-12-02T00:29:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.141+0000 7f55437fe640 1 -- 172.21.15.46:0/2430175684 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5534007020 con 0x7f554c13d7f0 2023-12-02T00:29:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.141+0000 7f55437fe640 1 --2- 172.21.15.46:0/2430175684 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f552c06ae70 0x7f552c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.141+0000 7f55437fe640 1 -- 172.21.15.46:0/2430175684 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5534088ba0 con 0x7f554c13d7f0 2023-12-02T00:29:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.141+0000 7f554a59c640 1 --2- 172.21.15.46:0/2430175684 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f552c06ae70 0x7f552c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.142+0000 7f554a59c640 1 --2- 172.21.15.46:0/2430175684 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f552c06ae70 0x7f552c06d330 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f553c036000 tx=0x7f553c030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.145+0000 7f55437fe640 1 -- 172.21.15.46:0/2430175684 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5534053ce0 con 0x7f554c13d7f0 2023-12-02T00:29:23.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.322+0000 7f55508a6640 1 -- 172.21.15.46:0/2430175684 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f554c0b2820 con 0x7f554c13d7f0 2023-12-02T00:29:23.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.323+0000 7f55437fe640 1 -- 172.21.15.46:0/2430175684 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5534057300 con 0x7f554c13d7f0 2023-12-02T00:29:23.325 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-12-02T00:29:23.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.325+0000 7f55508a6640 1 -- 172.21.15.46:0/2430175684 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f552c06ae70 msgr2=0x7f552c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:23.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.325+0000 7f55508a6640 1 --2- 172.21.15.46:0/2430175684 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f552c06ae70 0x7f552c06d330 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f553c036000 tx=0x7f553c030000 comp rx=0 tx=0).stop 2023-12-02T00:29:23.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.325+0000 7f55508a6640 1 -- 172.21.15.46:0/2430175684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f554c13d7f0 msgr2=0x7f554c0b36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:23.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.325+0000 7f55508a6640 1 --2- 172.21.15.46:0/2430175684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f554c13d7f0 0x7f554c0b36c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f553400e480 tx=0x7f553400e940 comp rx=0 tx=0).stop 2023-12-02T00:29:23.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.325+0000 7f55508a6640 1 -- 172.21.15.46:0/2430175684 shutdown_connections 2023-12-02T00:29:23.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.325+0000 7f55508a6640 1 --2- 172.21.15.46:0/2430175684 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f552c06ae70 0x7f552c06d330 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:23.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.325+0000 7f55508a6640 1 --2- 172.21.15.46:0/2430175684 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f554c0b3c00 0x7f554c15b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:23.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.325+0000 7f55508a6640 1 --2- 172.21.15.46:0/2430175684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f554c13d7f0 0x7f554c0b36c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:23.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.325+0000 7f55508a6640 1 --2- 172.21.15.46:0/2430175684 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5538005b80 0x7f554c13fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:23.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.325+0000 7f55508a6640 1 -- 172.21.15.46:0/2430175684 >> 172.21.15.46:0/2430175684 conn(0x7f554c06a4d0 msgr2=0x7f554c140d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:23.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.325+0000 7f55508a6640 1 -- 172.21.15.46:0/2430175684 shutdown_connections 2023-12-02T00:29:23.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.325+0000 7f55508a6640 1 -- 172.21.15.46:0/2430175684 wait complete. 2023-12-02T00:29:23.338 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-02T00:29:23.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.448+0000 7f8f97297640 1 Processor -- start 2023-12-02T00:29:23.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.448+0000 7f8f97297640 1 -- start start 2023-12-02T00:29:23.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.449+0000 7f8f97297640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8f9015d2d0 0x7f8f900b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:23.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.449+0000 7f8f97297640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8f90160f70 0x7f8f900cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:23.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.449+0000 7f8f97297640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f900ce480 0x7f8f9006b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:23.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.449+0000 7f8f97297640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f901614e0 con 0x7f8f9015d2d0 2023-12-02T00:29:23.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.449+0000 7f8f97297640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f901617c0 con 0x7f8f90160f70 2023-12-02T00:29:23.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.449+0000 7f8f97297640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f90161aa0 con 0x7f8f900ce480 2023-12-02T00:29:23.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.449+0000 7f8f95033640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8f9015d2d0 0x7f8f900b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:23.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.449+0000 7f8f95033640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8f9015d2d0 0x7f8f900b8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52572/0 (socket says 172.21.15.46:52572) 2023-12-02T00:29:23.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.449+0000 7f8f95033640 1 -- 172.21.15.46:0/1210976200 learned_addr learned my addr 172.21.15.46:0/1210976200 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:23.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.449+0000 7f8f94832640 1 --2- 172.21.15.46:0/1210976200 >> v2:172.21.15.162:3300/0 conn(0x7f8f90160f70 0x7f8f900cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:23.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.449+0000 7f8f95834640 1 --2- 172.21.15.46:0/1210976200 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f900ce480 0x7f8f9006b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:23.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.449+0000 7f8f95033640 1 -- 172.21.15.46:0/1210976200 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f900ce480 msgr2=0x7f8f9006b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:23.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.449+0000 7f8f95033640 1 --2- 172.21.15.46:0/1210976200 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f900ce480 0x7f8f9006b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:23.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.449+0000 7f8f95033640 1 -- 172.21.15.46:0/1210976200 >> v2:172.21.15.162:3300/0 conn(0x7f8f90160f70 msgr2=0x7f8f900cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:23.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.449+0000 7f8f95033640 1 --2- 172.21.15.46:0/1210976200 >> v2:172.21.15.162:3300/0 conn(0x7f8f90160f70 0x7f8f900cdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:23.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.449+0000 7f8f95033640 1 -- 172.21.15.46:0/1210976200 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f90161d80 con 0x7f8f9015d2d0 2023-12-02T00:29:23.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.449+0000 7f8f95834640 1 --2- 172.21.15.46:0/1210976200 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f900ce480 0x7f8f9006b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:29:23.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.450+0000 7f8f95033640 1 --2- 172.21.15.46:0/1210976200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f9015d2d0 0x7f8f900b8b90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8f900cdb80 tx=0x7f8f8c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f4f0a183f0438d2 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:23.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.450+0000 7f8f87fff640 1 -- 172.21.15.46:0/1210976200 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c031870 con 0x7f8f9015d2d0 2023-12-02T00:29:23.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.450+0000 7f8f87fff640 1 -- 172.21.15.46:0/1210976200 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f8c0319d0 con 0x7f8f9015d2d0 2023-12-02T00:29:23.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.450+0000 7f8f87fff640 1 -- 172.21.15.46:0/1210976200 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c031d60 con 0x7f8f9015d2d0 2023-12-02T00:29:23.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.450+0000 7f8f97297640 1 -- 172.21.15.46:0/1210976200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f9015d2d0 msgr2=0x7f8f900b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:23.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.450+0000 7f8f97297640 1 --2- 172.21.15.46:0/1210976200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f9015d2d0 0x7f8f900b8b90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8f900cdb80 tx=0x7f8f8c02b910 comp rx=0 tx=0).stop 2023-12-02T00:29:23.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.451+0000 7f8f97297640 1 -- 172.21.15.46:0/1210976200 shutdown_connections 2023-12-02T00:29:23.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.451+0000 7f8f97297640 1 --2- 172.21.15.46:0/1210976200 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f900ce480 0x7f8f9006b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:23.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.451+0000 7f8f97297640 1 --2- 172.21.15.46:0/1210976200 >> v2:172.21.15.162:3300/0 conn(0x7f8f90160f70 0x7f8f900cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:23.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.451+0000 7f8f97297640 1 --2- 172.21.15.46:0/1210976200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f9015d2d0 0x7f8f900b8b90 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:23.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.451+0000 7f8f97297640 1 -- 172.21.15.46:0/1210976200 >> 172.21.15.46:0/1210976200 conn(0x7f8f90072230 msgr2=0x7f8f900cfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:23.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.451+0000 7f8f97297640 1 -- 172.21.15.46:0/1210976200 shutdown_connections 2023-12-02T00:29:23.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.451+0000 7f8f97297640 1 -- 172.21.15.46:0/1210976200 wait complete. 2023-12-02T00:29:23.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.452+0000 7f8f97297640 1 Processor -- start 2023-12-02T00:29:23.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.452+0000 7f8f97297640 1 -- start start 2023-12-02T00:29:23.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.453+0000 7f8f97297640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f900ce480 0x7f8f9014e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:23.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.453+0000 7f8f97297640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f9015d2d0 0x7f8f90160340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:23.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.453+0000 7f8f97297640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f90160f70 0x7f8f9015eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:23.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.453+0000 7f8f97297640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9015ca40 con 0x7f8f9015d2d0 2023-12-02T00:29:23.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.453+0000 7f8f97297640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9015daa0 con 0x7f8f90160f70 2023-12-02T00:29:23.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.453+0000 7f8f97297640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9015dd80 con 0x7f8f900ce480 2023-12-02T00:29:23.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.453+0000 7f8f94832640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f9015d2d0 0x7f8f90160340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:23.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.453+0000 7f8f94832640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f9015d2d0 0x7f8f90160340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52580/0 (socket says 172.21.15.46:52580) 2023-12-02T00:29:23.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.453+0000 7f8f94832640 1 -- 172.21.15.46:0/2390405452 learned_addr learned my addr 172.21.15.46:0/2390405452 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:23.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.453+0000 7f8f95033640 1 --2- 172.21.15.46:0/2390405452 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f900ce480 0x7f8f9014e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:23.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.453+0000 7f8f95834640 1 --2- 172.21.15.46:0/2390405452 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f90160f70 0x7f8f9015eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:23.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.453+0000 7f8f94832640 1 -- 172.21.15.46:0/2390405452 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f900ce480 msgr2=0x7f8f9014e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:23.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.453+0000 7f8f94832640 1 --2- 172.21.15.46:0/2390405452 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f900ce480 0x7f8f9014e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:23.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.453+0000 7f8f94832640 1 -- 172.21.15.46:0/2390405452 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f90160f70 msgr2=0x7f8f9015eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:23.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.453+0000 7f8f94832640 1 --2- 172.21.15.46:0/2390405452 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f90160f70 0x7f8f9015eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:23.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.453+0000 7f8f94832640 1 -- 172.21.15.46:0/2390405452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f8c0094d0 con 0x7f8f9015d2d0 2023-12-02T00:29:23.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.453+0000 7f8f95033640 1 --2- 172.21.15.46:0/2390405452 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f900ce480 0x7f8f9014e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:23.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.453+0000 7f8f94832640 1 --2- 172.21.15.46:0/2390405452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f9015d2d0 0x7f8f90160340 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8f8000d5f0 tx=0x7f8f8000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:23.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.453+0000 7f8f85ffb640 1 -- 172.21.15.46:0/2390405452 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f800125f0 con 0x7f8f9015d2d0 2023-12-02T00:29:23.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.453+0000 7f8f85ffb640 1 -- 172.21.15.46:0/2390405452 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f80012750 con 0x7f8f9015d2d0 2023-12-02T00:29:23.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.453+0000 7f8f97297640 1 -- 172.21.15.46:0/2390405452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f9015e060 con 0x7f8f9015d2d0 2023-12-02T00:29:23.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.453+0000 7f8f85ffb640 1 -- 172.21.15.46:0/2390405452 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f80012b00 con 0x7f8f9015d2d0 2023-12-02T00:29:23.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.454+0000 7f8f97297640 1 -- 172.21.15.46:0/2390405452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f9015e620 con 0x7f8f9015d2d0 2023-12-02T00:29:23.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.454+0000 7f8f97297640 1 -- 172.21.15.46:0/2390405452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f9015e7e0 con 0x7f8f9015d2d0 2023-12-02T00:29:23.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.455+0000 7f8f85ffb640 1 -- 172.21.15.46:0/2390405452 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8f80012c60 con 0x7f8f9015d2d0 2023-12-02T00:29:23.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.456+0000 7f8f85ffb640 1 --2- 172.21.15.46:0/2390405452 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8f7406ae40 0x7f8f7406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:23.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.456+0000 7f8f85ffb640 1 -- 172.21.15.46:0/2390405452 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8f80089a50 con 0x7f8f9015d2d0 2023-12-02T00:29:23.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.456+0000 7f8f95033640 1 --2- 172.21.15.46:0/2390405452 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8f7406ae40 0x7f8f7406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:23.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.456+0000 7f8f95033640 1 --2- 172.21.15.46:0/2390405452 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8f7406ae40 0x7f8f7406d300 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c036000 tx=0x7f8f8c03c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:23.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.460+0000 7f8f85ffb640 1 -- 172.21.15.46:0/2390405452 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8f80054b00 con 0x7f8f9015d2d0 2023-12-02T00:29:23.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.636+0000 7f8f97297640 1 -- 172.21.15.46:0/2390405452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8f90069400 con 0x7f8f9015d2d0 2023-12-02T00:29:23.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.636+0000 7f8f85ffb640 1 -- 172.21.15.46:0/2390405452 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8f80058120 con 0x7f8f9015d2d0 2023-12-02T00:29:23.639 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-12-02T00:29:23.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.638+0000 7f8f97297640 1 -- 172.21.15.46:0/2390405452 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8f7406ae40 msgr2=0x7f8f7406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:23.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.638+0000 7f8f97297640 1 --2- 172.21.15.46:0/2390405452 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8f7406ae40 0x7f8f7406d300 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c036000 tx=0x7f8f8c03c040 comp rx=0 tx=0).stop 2023-12-02T00:29:23.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.638+0000 7f8f97297640 1 -- 172.21.15.46:0/2390405452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f9015d2d0 msgr2=0x7f8f90160340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:23.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.638+0000 7f8f97297640 1 --2- 172.21.15.46:0/2390405452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f9015d2d0 0x7f8f90160340 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8f8000d5f0 tx=0x7f8f8000dab0 comp rx=0 tx=0).stop 2023-12-02T00:29:23.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.638+0000 7f8f97297640 1 -- 172.21.15.46:0/2390405452 shutdown_connections 2023-12-02T00:29:23.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.638+0000 7f8f97297640 1 --2- 172.21.15.46:0/2390405452 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8f7406ae40 0x7f8f7406d300 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:23.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.638+0000 7f8f97297640 1 --2- 172.21.15.46:0/2390405452 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f90160f70 0x7f8f9015eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:23.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.638+0000 7f8f97297640 1 --2- 172.21.15.46:0/2390405452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f9015d2d0 0x7f8f90160340 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:23.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.638+0000 7f8f97297640 1 --2- 172.21.15.46:0/2390405452 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f900ce480 0x7f8f9014e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:23.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.638+0000 7f8f97297640 1 -- 172.21.15.46:0/2390405452 >> 172.21.15.46:0/2390405452 conn(0x7f8f90072230 msgr2=0x7f8f900c6d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:23.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.638+0000 7f8f97297640 1 -- 172.21.15.46:0/2390405452 shutdown_connections 2023-12-02T00:29:23.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.638+0000 7f8f97297640 1 -- 172.21.15.46:0/2390405452 wait complete. 2023-12-02T00:29:23.651 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-02T00:29:23.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.756+0000 7fb407d1d640 1 Processor -- start 2023-12-02T00:29:23.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.757+0000 7fb407d1d640 1 -- start start 2023-12-02T00:29:23.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.757+0000 7fb407d1d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb40015d330 0x7fb4000b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:23.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.757+0000 7fb407d1d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb4000d3900 0x7fb4000b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:23.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.757+0000 7fb407d1d640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4000b2b10 con 0x7fb4000d2910 2023-12-02T00:29:23.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.757+0000 7fb407d1d640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4000d1590 con 0x7fb40015d330 2023-12-02T00:29:23.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.757+0000 7fb407d1d640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4000d1870 con 0x7fb4000d3900 2023-12-02T00:29:23.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.757+0000 7fb405ab9640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb40015d330 0x7fb4000b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:23.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.757+0000 7fb4052b8640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb4000d2910 0x7fb4000d31f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50376/0 (socket says 172.21.15.46:50376) 2023-12-02T00:29:23.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.758+0000 7fb405ab9640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb40015d330 0x7fb4000b8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52588/0 (socket says 172.21.15.46:52588) 2023-12-02T00:29:23.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.758+0000 7fb4052b8640 1 -- 172.21.15.46:0/950558458 learned_addr learned my addr 172.21.15.46:0/950558458 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:23.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.758+0000 7fb4062ba640 1 --2- 172.21.15.46:0/950558458 >> v2:172.21.15.162:3300/0 conn(0x7fb4000d3900 0x7fb4000b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:23.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.758+0000 7fb405ab9640 1 -- 172.21.15.46:0/950558458 >> v2:172.21.15.162:3300/0 conn(0x7fb4000d3900 msgr2=0x7fb4000b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:23.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.758+0000 7fb405ab9640 1 --2- 172.21.15.46:0/950558458 >> v2:172.21.15.162:3300/0 conn(0x7fb4000d3900 0x7fb4000b6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:23.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.758+0000 7fb405ab9640 1 -- 172.21.15.46:0/950558458 >> v1:172.21.15.46:6789/0 conn(0x7fb4000d2910 legacy=0x7fb4000d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:23.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.758+0000 7fb405ab9640 1 -- 172.21.15.46:0/950558458 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4000d1b50 con 0x7fb40015d330 2023-12-02T00:29:23.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.758+0000 7fb404ab7640 1 -- 172.21.15.46:0/950558458 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3016144820 0 0) 0x7fb4000b2b10 con 0x7fb4000d2910 2023-12-02T00:29:23.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.759+0000 7fb405ab9640 1 --2- 172.21.15.46:0/950558458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb40015d330 0x7fb4000b8b80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb40006c500 tx=0x7fb3fc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0093b3e9bfedd53 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:23.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.759+0000 7fb404ab7640 1 -- 172.21.15.46:0/950558458 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc02e070 con 0x7fb40015d330 2023-12-02T00:29:23.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.759+0000 7fb404ab7640 1 -- 172.21.15.46:0/950558458 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3fc0315a0 con 0x7fb40015d330 2023-12-02T00:29:23.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.759+0000 7fb404ab7640 1 -- 172.21.15.46:0/950558458 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc031930 con 0x7fb40015d330 2023-12-02T00:29:23.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.759+0000 7fb407d1d640 1 -- 172.21.15.46:0/950558458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb40015d330 msgr2=0x7fb4000b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:23.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.759+0000 7fb407d1d640 1 --2- 172.21.15.46:0/950558458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb40015d330 0x7fb4000b8b80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb40006c500 tx=0x7fb3fc02b840 comp rx=0 tx=0).stop 2023-12-02T00:29:23.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.759+0000 7fb407d1d640 1 -- 172.21.15.46:0/950558458 shutdown_connections 2023-12-02T00:29:23.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.759+0000 7fb407d1d640 1 --2- 172.21.15.46:0/950558458 >> v2:172.21.15.162:3300/0 conn(0x7fb4000d3900 0x7fb4000b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:23.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.759+0000 7fb407d1d640 1 --2- 172.21.15.46:0/950558458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb40015d330 0x7fb4000b8b80 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:23.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.759+0000 7fb407d1d640 1 -- 172.21.15.46:0/950558458 >> 172.21.15.46:0/950558458 conn(0x7fb400072020 msgr2=0x7fb400072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:23.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.759+0000 7fb407d1d640 1 -- 172.21.15.46:0/950558458 shutdown_connections 2023-12-02T00:29:23.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.759+0000 7fb407d1d640 1 -- 172.21.15.46:0/950558458 wait complete. 2023-12-02T00:29:23.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.761+0000 7fb407d1d640 1 Processor -- start 2023-12-02T00:29:23.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.761+0000 7fb407d1d640 1 -- start start 2023-12-02T00:29:23.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.761+0000 7fb407d1d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4000d2910 0x7fb40014e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:23.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.761+0000 7fb407d1d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4000d3900 0x7fb4000cbac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:23.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.761+0000 7fb407d1d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb40015d330 0x7fb4000cc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:23.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.761+0000 7fb407d1d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb40015e0c0 con 0x7fb4000d3900 2023-12-02T00:29:23.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.761+0000 7fb407d1d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4000c8650 con 0x7fb4000d2910 2023-12-02T00:29:23.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.761+0000 7fb407d1d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4000c7ad0 con 0x7fb40015d330 2023-12-02T00:29:23.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.761+0000 7fb4052b8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4000d3900 0x7fb4000cbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:23.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.761+0000 7fb4052b8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4000d3900 0x7fb4000cbac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52600/0 (socket says 172.21.15.46:52600) 2023-12-02T00:29:23.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.761+0000 7fb4052b8640 1 -- 172.21.15.46:0/3030299833 learned_addr learned my addr 172.21.15.46:0/3030299833 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:23.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.762+0000 7fb405ab9640 1 --2- 172.21.15.46:0/3030299833 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4000d2910 0x7fb40014e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:23.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.762+0000 7fb4062ba640 1 --2- 172.21.15.46:0/3030299833 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb40015d330 0x7fb4000cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:23.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.762+0000 7fb4052b8640 1 -- 172.21.15.46:0/3030299833 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb40015d330 msgr2=0x7fb4000cc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:23.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.762+0000 7fb4052b8640 1 --2- 172.21.15.46:0/3030299833 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb40015d330 0x7fb4000cc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:23.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.762+0000 7fb4052b8640 1 -- 172.21.15.46:0/3030299833 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4000d2910 msgr2=0x7fb40014e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:23.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.762+0000 7fb4052b8640 1 --2- 172.21.15.46:0/3030299833 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4000d2910 0x7fb40014e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:23.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.762+0000 7fb4052b8640 1 -- 172.21.15.46:0/3030299833 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3fc009400 con 0x7fb4000d3900 2023-12-02T00:29:23.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.762+0000 7fb405ab9640 1 --2- 172.21.15.46:0/3030299833 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4000d2910 0x7fb40014e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:23.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.762+0000 7fb4052b8640 1 --2- 172.21.15.46:0/3030299833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4000d3900 0x7fb4000cbac0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb3f000c430 tx=0x7fb3f000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:23.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.762+0000 7fb3f67fc640 1 -- 172.21.15.46:0/3030299833 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f00074e0 con 0x7fb4000d3900 2023-12-02T00:29:23.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.762+0000 7fb3f67fc640 1 -- 172.21.15.46:0/3030299833 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3f0007660 con 0x7fb4000d3900 2023-12-02T00:29:23.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.762+0000 7fb3f67fc640 1 -- 172.21.15.46:0/3030299833 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f0004a20 con 0x7fb4000d3900 2023-12-02T00:29:23.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.762+0000 7fb407d1d640 1 -- 172.21.15.46:0/3030299833 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4000c7db0 con 0x7fb4000d3900 2023-12-02T00:29:23.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.762+0000 7fb407d1d640 1 -- 172.21.15.46:0/3030299833 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4000c8370 con 0x7fb4000d3900 2023-12-02T00:29:23.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.763+0000 7fb407d1d640 1 -- 172.21.15.46:0/3030299833 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb40015e250 con 0x7fb4000d3900 2023-12-02T00:29:23.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.763+0000 7fb3f67fc640 1 -- 172.21.15.46:0/3030299833 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb3f0004b80 con 0x7fb4000d3900 2023-12-02T00:29:23.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.764+0000 7fb3f67fc640 1 --2- 172.21.15.46:0/3030299833 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb3e406ae40 0x7fb3e406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:23.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.764+0000 7fb3f67fc640 1 -- 172.21.15.46:0/3030299833 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb3f0089420 con 0x7fb4000d3900 2023-12-02T00:29:23.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.764+0000 7fb405ab9640 1 --2- 172.21.15.46:0/3030299833 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb3e406ae40 0x7fb3e406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:23.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.765+0000 7fb405ab9640 1 --2- 172.21.15.46:0/3030299833 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb3e406ae40 0x7fb3e406d300 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc036000 tx=0x7fb3fc03d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:23.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.768+0000 7fb3f67fc640 1 -- 172.21.15.46:0/3030299833 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb40015e250 con 0x7fb4000d3900 2023-12-02T00:29:23.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.958+0000 7fb407d1d640 1 -- 172.21.15.46:0/3030299833 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb40015e200 con 0x7fb4000d3900 2023-12-02T00:29:23.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.959+0000 7fb3f67fc640 1 -- 172.21.15.46:0/3030299833 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb3f00544d0 con 0x7fb4000d3900 2023-12-02T00:29:23.961 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2023-12-02T00:29:23.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.961+0000 7fb407d1d640 1 -- 172.21.15.46:0/3030299833 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb3e406ae40 msgr2=0x7fb3e406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:23.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.961+0000 7fb407d1d640 1 --2- 172.21.15.46:0/3030299833 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb3e406ae40 0x7fb3e406d300 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc036000 tx=0x7fb3fc03d040 comp rx=0 tx=0).stop 2023-12-02T00:29:23.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.961+0000 7fb407d1d640 1 -- 172.21.15.46:0/3030299833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4000d3900 msgr2=0x7fb4000cbac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:23.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.961+0000 7fb407d1d640 1 --2- 172.21.15.46:0/3030299833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4000d3900 0x7fb4000cbac0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb3f000c430 tx=0x7fb3f000c8f0 comp rx=0 tx=0).stop 2023-12-02T00:29:23.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.961+0000 7fb407d1d640 1 -- 172.21.15.46:0/3030299833 shutdown_connections 2023-12-02T00:29:23.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.961+0000 7fb407d1d640 1 --2- 172.21.15.46:0/3030299833 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb3e406ae40 0x7fb3e406d300 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:23.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.961+0000 7fb407d1d640 1 --2- 172.21.15.46:0/3030299833 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb40015d330 0x7fb4000cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:23.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.961+0000 7fb407d1d640 1 --2- 172.21.15.46:0/3030299833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4000d3900 0x7fb4000cbac0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:23.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.961+0000 7fb407d1d640 1 --2- 172.21.15.46:0/3030299833 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4000d2910 0x7fb40014e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:23.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.961+0000 7fb407d1d640 1 -- 172.21.15.46:0/3030299833 >> 172.21.15.46:0/3030299833 conn(0x7fb400072020 msgr2=0x7fb400072400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:23.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.961+0000 7fb407d1d640 1 -- 172.21.15.46:0/3030299833 shutdown_connections 2023-12-02T00:29:23.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:23.961+0000 7fb407d1d640 1 -- 172.21.15.46:0/3030299833 wait complete. 2023-12-02T00:29:23.973 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-02T00:29:23.974 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-02T00:29:23.990 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:29:24.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.133+0000 7f7c35bd0640 1 Processor -- start 2023-12-02T00:29:24.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.134+0000 7f7c35bd0640 1 -- start start 2023-12-02T00:29:24.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.134+0000 7f7c35bd0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c3015d2e0 0x7f7c300d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:24.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.134+0000 7f7c35bd0640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7c3015d6c0 0x7f7c300d1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:24.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.134+0000 7f7c35bd0640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c3006ce00 con 0x7f7c3006b4c0 2023-12-02T00:29:24.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.134+0000 7f7c35bd0640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c3006c1f0 con 0x7f7c3015d6c0 2023-12-02T00:29:24.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.134+0000 7f7c35bd0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3006c4d0 con 0x7f7c3015d2e0 2023-12-02T00:29:24.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.134+0000 7f7c2effd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7c3015d6c0 0x7f7c300d1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:24.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.135+0000 7f7c2effd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7c3015d6c0 0x7f7c300d1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52608/0 (socket says 172.21.15.46:52608) 2023-12-02T00:29:24.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.135+0000 7f7c2effd640 1 -- 172.21.15.46:0/4187459252 learned_addr learned my addr 172.21.15.46:0/4187459252 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:24.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.135+0000 7f7c2ffff640 1 --1- 172.21.15.46:0/4187459252 >> v1:172.21.15.46:6789/0 conn(0x7f7c3006b4c0 0x7f7c3006b910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50380/0 (socket says 172.21.15.46:50380) 2023-12-02T00:29:24.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.135+0000 7f7c2f7fe640 1 --2- 172.21.15.46:0/4187459252 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c3015d2e0 0x7f7c300d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:24.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.135+0000 7f7c2effd640 1 -- 172.21.15.46:0/4187459252 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c3015d2e0 msgr2=0x7f7c300d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:24.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.135+0000 7f7c2effd640 1 --2- 172.21.15.46:0/4187459252 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c3015d2e0 0x7f7c300d1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:24.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.135+0000 7f7c2effd640 1 -- 172.21.15.46:0/4187459252 >> v1:172.21.15.46:6789/0 conn(0x7f7c3006b4c0 legacy=0x7f7c3006b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:24.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.135+0000 7f7c2effd640 1 -- 172.21.15.46:0/4187459252 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c3006c7b0 con 0x7f7c3015d6c0 2023-12-02T00:29:24.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.135+0000 7f7c2e7fc640 1 -- 172.21.15.46:0/4187459252 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1347649205 0 0) 0x7f7c3006ce00 con 0x7f7c3006b4c0 2023-12-02T00:29:24.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.136+0000 7f7c2effd640 1 --2- 172.21.15.46:0/4187459252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c3015d6c0 0x7f7c300d1b80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7c24009870 tx=0x7f7c2402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28ff02fbd83e3c8c server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:24.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.136+0000 7f7c2e7fc640 1 -- 172.21.15.46:0/4187459252 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c24031640 con 0x7f7c3015d6c0 2023-12-02T00:29:24.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.136+0000 7f7c2e7fc640 1 -- 172.21.15.46:0/4187459252 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c240317a0 con 0x7f7c3015d6c0 2023-12-02T00:29:24.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.136+0000 7f7c2e7fc640 1 -- 172.21.15.46:0/4187459252 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c24031b30 con 0x7f7c3015d6c0 2023-12-02T00:29:24.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.136+0000 7f7c35bd0640 1 -- 172.21.15.46:0/4187459252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c3015d6c0 msgr2=0x7f7c300d1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:24.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.136+0000 7f7c35bd0640 1 --2- 172.21.15.46:0/4187459252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c3015d6c0 0x7f7c300d1b80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7c24009870 tx=0x7f7c2402b8f0 comp rx=0 tx=0).stop 2023-12-02T00:29:24.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.136+0000 7f7c35bd0640 1 -- 172.21.15.46:0/4187459252 shutdown_connections 2023-12-02T00:29:24.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.136+0000 7f7c35bd0640 1 --2- 172.21.15.46:0/4187459252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c3015d6c0 0x7f7c300d1b80 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:24.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.136+0000 7f7c35bd0640 1 --2- 172.21.15.46:0/4187459252 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c3015d2e0 0x7f7c300d1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:24.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.136+0000 7f7c35bd0640 1 -- 172.21.15.46:0/4187459252 >> 172.21.15.46:0/4187459252 conn(0x7f7c3009c580 msgr2=0x7f7c300d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:24.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.136+0000 7f7c35bd0640 1 -- 172.21.15.46:0/4187459252 shutdown_connections 2023-12-02T00:29:24.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.136+0000 7f7c35bd0640 1 -- 172.21.15.46:0/4187459252 wait complete. 2023-12-02T00:29:24.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.137+0000 7f7c35bd0640 1 Processor -- start 2023-12-02T00:29:24.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.138+0000 7f7c35bd0640 1 -- start start 2023-12-02T00:29:24.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.138+0000 7f7c35bd0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c3006b4c0 0x7f7c300b6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:24.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.138+0000 7f7c35bd0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c3015d2e0 0x7f7c300b2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:24.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.138+0000 7f7c35bd0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c3015d6c0 0x7f7c301510f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:24.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.138+0000 7f7c35bd0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c300b3bc0 con 0x7f7c3015d6c0 2023-12-02T00:29:24.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.138+0000 7f7c35bd0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c300b2f40 con 0x7f7c3015d2e0 2023-12-02T00:29:24.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.138+0000 7f7c35bd0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c300b3220 con 0x7f7c3006b4c0 2023-12-02T00:29:24.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.139+0000 7f7c2ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c3015d6c0 0x7f7c301510f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:24.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.139+0000 7f7c2ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c3015d6c0 0x7f7c301510f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52622/0 (socket says 172.21.15.46:52622) 2023-12-02T00:29:24.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.139+0000 7f7c2ffff640 1 -- 172.21.15.46:0/1618521563 learned_addr learned my addr 172.21.15.46:0/1618521563 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:24.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.139+0000 7f7c2f7fe640 1 --2- 172.21.15.46:0/1618521563 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c3006b4c0 0x7f7c300b6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:24.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.139+0000 7f7c2effd640 1 --2- 172.21.15.46:0/1618521563 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c3015d2e0 0x7f7c300b2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:24.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.139+0000 7f7c2ffff640 1 -- 172.21.15.46:0/1618521563 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c3006b4c0 msgr2=0x7f7c300b6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:24.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.139+0000 7f7c2ffff640 1 --2- 172.21.15.46:0/1618521563 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c3006b4c0 0x7f7c300b6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:24.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.139+0000 7f7c2ffff640 1 -- 172.21.15.46:0/1618521563 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c3015d2e0 msgr2=0x7f7c300b2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:24.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.139+0000 7f7c2ffff640 1 --2- 172.21.15.46:0/1618521563 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c3015d2e0 0x7f7c300b2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:24.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.139+0000 7f7c2ffff640 1 -- 172.21.15.46:0/1618521563 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c240094d0 con 0x7f7c3015d6c0 2023-12-02T00:29:24.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.139+0000 7f7c2effd640 1 --2- 172.21.15.46:0/1618521563 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c3015d2e0 0x7f7c300b2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:24.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.139+0000 7f7c2ffff640 1 --2- 172.21.15.46:0/1618521563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c3015d6c0 0x7f7c301510f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7c2800c7e0 tx=0x7f7c2800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:24.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.139+0000 7f7c17fff640 1 -- 172.21.15.46:0/1618521563 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2800a640 con 0x7f7c3015d6c0 2023-12-02T00:29:24.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.139+0000 7f7c17fff640 1 -- 172.21.15.46:0/1618521563 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c2800a7c0 con 0x7f7c3015d6c0 2023-12-02T00:29:24.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.139+0000 7f7c35bd0640 1 -- 172.21.15.46:0/1618521563 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c300b3500 con 0x7f7c3015d6c0 2023-12-02T00:29:24.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.139+0000 7f7c17fff640 1 -- 172.21.15.46:0/1618521563 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2800ab70 con 0x7f7c3015d6c0 2023-12-02T00:29:24.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.139+0000 7f7c35bd0640 1 -- 172.21.15.46:0/1618521563 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c301f6820 con 0x7f7c3015d6c0 2023-12-02T00:29:24.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.140+0000 7f7c35bd0640 1 -- 172.21.15.46:0/1618521563 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c30067420 con 0x7f7c3015d6c0 2023-12-02T00:29:24.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.141+0000 7f7c17fff640 1 -- 172.21.15.46:0/1618521563 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7c2800acd0 con 0x7f7c3015d6c0 2023-12-02T00:29:24.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.141+0000 7f7c17fff640 1 --2- 172.21.15.46:0/1618521563 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7c1c06ae40 0x7f7c1c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:24.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.141+0000 7f7c17fff640 1 -- 172.21.15.46:0/1618521563 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7c28088d70 con 0x7f7c3015d6c0 2023-12-02T00:29:24.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.141+0000 7f7c2f7fe640 1 --2- 172.21.15.46:0/1618521563 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7c1c06ae40 0x7f7c1c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:24.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.142+0000 7f7c2f7fe640 1 --2- 172.21.15.46:0/1618521563 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7c1c06ae40 0x7f7c1c06d300 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7c20006fd0 tx=0x7f7c20008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:24.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.145+0000 7f7c17fff640 1 -- 172.21.15.46:0/1618521563 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7c28053f60 con 0x7f7c3015d6c0 2023-12-02T00:29:24.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.321+0000 7f7c35bd0640 1 -- 172.21.15.46:0/1618521563 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7c300d0950 con 0x7f7c3015d6c0 2023-12-02T00:29:24.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.322+0000 7f7c17fff640 1 -- 172.21.15.46:0/1618521563 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f7c28018020 con 0x7f7c3015d6c0 2023-12-02T00:29:24.324 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:29:24.324 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":21,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:28:50.700112+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-02T00:28:47.273337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-02T00:28:48.060115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:29:24.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.324+0000 7f7c35bd0640 1 -- 172.21.15.46:0/1618521563 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7c1c06ae40 msgr2=0x7f7c1c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:24.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.324+0000 7f7c35bd0640 1 --2- 172.21.15.46:0/1618521563 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7c1c06ae40 0x7f7c1c06d300 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7c20006fd0 tx=0x7f7c20008040 comp rx=0 tx=0).stop 2023-12-02T00:29:24.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.324+0000 7f7c35bd0640 1 -- 172.21.15.46:0/1618521563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c3015d6c0 msgr2=0x7f7c301510f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:24.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.324+0000 7f7c35bd0640 1 --2- 172.21.15.46:0/1618521563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c3015d6c0 0x7f7c301510f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7c2800c7e0 tx=0x7f7c2800cca0 comp rx=0 tx=0).stop 2023-12-02T00:29:24.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.324+0000 7f7c35bd0640 1 -- 172.21.15.46:0/1618521563 shutdown_connections 2023-12-02T00:29:24.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.324+0000 7f7c35bd0640 1 --2- 172.21.15.46:0/1618521563 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7c1c06ae40 0x7f7c1c06d300 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:24.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.324+0000 7f7c35bd0640 1 --2- 172.21.15.46:0/1618521563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c3015d6c0 0x7f7c301510f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:24.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.324+0000 7f7c35bd0640 1 --2- 172.21.15.46:0/1618521563 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c3015d2e0 0x7f7c300b2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:24.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.324+0000 7f7c35bd0640 1 --2- 172.21.15.46:0/1618521563 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c3006b4c0 0x7f7c300b6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:24.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.324+0000 7f7c35bd0640 1 -- 172.21.15.46:0/1618521563 >> 172.21.15.46:0/1618521563 conn(0x7f7c3009c580 msgr2=0x7f7c300cdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:24.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.324+0000 7f7c35bd0640 1 -- 172.21.15.46:0/1618521563 shutdown_connections 2023-12-02T00:29:24.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.324+0000 7f7c35bd0640 1 -- 172.21.15.46:0/1618521563 wait complete. 2023-12-02T00:29:24.338 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-02T00:28:37.213645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-02T00:28:47.273337+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-02T00:28:48.060115+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-02T00:29:24.338 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-02T00:29:24.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.446+0000 7fe68addc640 1 Processor -- start 2023-12-02T00:29:24.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.446+0000 7fe68addc640 1 -- start start 2023-12-02T00:29:24.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.447+0000 7fe68addc640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe684158690 0x7fe6840b44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:24.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.447+0000 7fe68addc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe6840b0cf0 0x7fe6840b1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:24.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.447+0000 7fe68addc640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe684162d80 con 0x7fe6841651b0 2023-12-02T00:29:24.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.447+0000 7fe68addc640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe684070390 con 0x7fe684158690 2023-12-02T00:29:24.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.447+0000 7fe68addc640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6840705a0 con 0x7fe6840b0cf0 2023-12-02T00:29:24.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.447+0000 7fe689379640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe6841651b0 0x7fe684165640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50384/0 (socket says 172.21.15.46:50384) 2023-12-02T00:29:24.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.447+0000 7fe689379640 1 -- 172.21.15.46:0/802625115 learned_addr learned my addr 172.21.15.46:0/802625115 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:24.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.447+0000 7fe688b78640 1 --2- 172.21.15.46:0/802625115 >> v2:172.21.15.162:3300/0 conn(0x7fe684158690 0x7fe6840b44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:24.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.447+0000 7fe683fff640 1 --2- 172.21.15.46:0/802625115 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe6840b0cf0 0x7fe6840b1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:24.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.447+0000 7fe6837fe640 1 -- 172.21.15.46:0/802625115 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2104825092 0 0) 0x7fe684162d80 con 0x7fe6841651b0 2023-12-02T00:29:24.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.447+0000 7fe6837fe640 1 -- 172.21.15.46:0/802625115 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe674003520 con 0x7fe6841651b0 2023-12-02T00:29:24.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.447+0000 7fe6837fe640 1 -- 172.21.15.46:0/802625115 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1898734499 0 0) 0x7fe674003520 con 0x7fe6841651b0 2023-12-02T00:29:24.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.447+0000 7fe6837fe640 1 -- 172.21.15.46:0/802625115 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe6840b0cf0 msgr2=0x7fe6840b1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:24.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.447+0000 7fe6837fe640 1 --2- 172.21.15.46:0/802625115 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe6840b0cf0 0x7fe6840b1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:24.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.447+0000 7fe6837fe640 1 -- 172.21.15.46:0/802625115 >> v2:172.21.15.162:3300/0 conn(0x7fe684158690 msgr2=0x7fe6840b44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:24.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.447+0000 7fe6837fe640 1 --2- 172.21.15.46:0/802625115 >> v2:172.21.15.162:3300/0 conn(0x7fe684158690 0x7fe6840b44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:24.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.447+0000 7fe6837fe640 1 -- 172.21.15.46:0/802625115 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6840b1690 con 0x7fe6841651b0 2023-12-02T00:29:24.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.447+0000 7fe6837fe640 1 -- 172.21.15.46:0/802625115 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7fe67c0019f0 con 0x7fe6841651b0 2023-12-02T00:29:24.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.447+0000 7fe688b78640 1 --2- 172.21.15.46:0/802625115 >> v2:172.21.15.162:3300/0 conn(0x7fe684158690 0x7fe6840b44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:29:24.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.447+0000 7fe6837fe640 1 -- 172.21.15.46:0/802625115 >> v1:172.21.15.46:6789/0 conn(0x7fe6841651b0 legacy=0x7fe684165640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:24.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.447+0000 7fe6837fe640 1 --2- 172.21.15.46:0/802625115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe674003970 0x7fe674003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:24.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.447+0000 7fe6837fe640 1 --2- 172.21.15.46:0/802625115 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe6840b0cf0 0x7fe6740047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:24.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.448+0000 7fe6837fe640 1 --2- 172.21.15.46:0/802625115 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe674004d00 0x7fe674005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:24.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.448+0000 7fe6837fe640 1 -- 172.21.15.46:0/802625115 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe684070390 con 0x7fe674003970 2023-12-02T00:29:24.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.448+0000 7fe6837fe640 1 -- 172.21.15.46:0/802625115 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6840705a0 con 0x7fe674004d00 2023-12-02T00:29:24.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.448+0000 7fe6837fe640 1 -- 172.21.15.46:0/802625115 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe67400b980 con 0x7fe6840b0cf0 2023-12-02T00:29:24.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.448+0000 7fe688b78640 1 --2- 172.21.15.46:0/802625115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe674003970 0x7fe674003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:24.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.448+0000 7fe683fff640 1 --2- 172.21.15.46:0/802625115 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe6840b0cf0 0x7fe6740047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:24.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.448+0000 7fe689379640 1 --2- 172.21.15.46:0/802625115 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe674004d00 0x7fe674005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:24.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.448+0000 7fe688b78640 1 -- 172.21.15.46:0/802625115 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe6840b0cf0 msgr2=0x7fe6740047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:24.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.448+0000 7fe688b78640 1 --2- 172.21.15.46:0/802625115 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe6840b0cf0 0x7fe6740047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:24.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.448+0000 7fe688b78640 1 -- 172.21.15.46:0/802625115 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe674004d00 msgr2=0x7fe674005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:24.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.448+0000 7fe688b78640 1 --2- 172.21.15.46:0/802625115 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe674004d00 0x7fe674005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:24.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.448+0000 7fe688b78640 1 -- 172.21.15.46:0/802625115 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe674003520 con 0x7fe674003970 2023-12-02T00:29:24.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.448+0000 7fe683fff640 1 --2- 172.21.15.46:0/802625115 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe6840b0cf0 0x7fe6740047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:24.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.448+0000 7fe689379640 1 --2- 172.21.15.46:0/802625115 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe674004d00 0x7fe674005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:24.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.449+0000 7fe688b78640 1 --2- 172.21.15.46:0/802625115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe674003970 0x7fe674003d90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe678009cc0 tx=0x7fe6740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f21213e5396bfb48 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:24.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.449+0000 7fe6837fe640 1 -- 172.21.15.46:0/802625115 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67802c960 con 0x7fe674003970 2023-12-02T00:29:24.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.449+0000 7fe6837fe640 1 -- 172.21.15.46:0/802625115 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe67802cac0 con 0x7fe674003970 2023-12-02T00:29:24.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.449+0000 7fe6837fe640 1 -- 172.21.15.46:0/802625115 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67802ce70 con 0x7fe674003970 2023-12-02T00:29:24.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.449+0000 7fe68addc640 1 -- 172.21.15.46:0/802625115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe674003970 msgr2=0x7fe674003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:24.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.449+0000 7fe68addc640 1 --2- 172.21.15.46:0/802625115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe674003970 0x7fe674003d90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe678009cc0 tx=0x7fe6740057f0 comp rx=0 tx=0).stop 2023-12-02T00:29:24.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.449+0000 7fe689379640 1 -- 172.21.15.46:0/802625115 reap_dead start 2023-12-02T00:29:24.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.449+0000 7fe68addc640 1 -- 172.21.15.46:0/802625115 shutdown_connections 2023-12-02T00:29:24.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.449+0000 7fe68addc640 1 -- 172.21.15.46:0/802625115 >> 172.21.15.46:0/802625115 conn(0x7fe68406a4d0 msgr2=0x7fe68406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:24.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.449+0000 7fe68addc640 1 -- 172.21.15.46:0/802625115 shutdown_connections 2023-12-02T00:29:24.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.449+0000 7fe68addc640 1 -- 172.21.15.46:0/802625115 wait complete. 2023-12-02T00:29:24.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.451+0000 7fe68addc640 1 Processor -- start 2023-12-02T00:29:24.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.451+0000 7fe68addc640 1 -- start start 2023-12-02T00:29:24.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.451+0000 7fe68addc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe674003970 0x7fe684160eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:24.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.451+0000 7fe68addc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe6841570e0 0x7fe684157540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:24.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.451+0000 7fe68addc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe684163650 0x7fe684163ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:24.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.451+0000 7fe68addc640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6841521d0 con 0x7fe684163650 2023-12-02T00:29:24.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.451+0000 7fe68addc640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe684157a80 con 0x7fe6841570e0 2023-12-02T00:29:24.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.451+0000 7fe68addc640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe684164010 con 0x7fe674003970 2023-12-02T00:29:24.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.452+0000 7fe689379640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe684163650 0x7fe684163ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:24.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.452+0000 7fe689379640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe684163650 0x7fe684163ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52648/0 (socket says 172.21.15.46:52648) 2023-12-02T00:29:24.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.452+0000 7fe689379640 1 -- 172.21.15.46:0/2287816067 learned_addr learned my addr 172.21.15.46:0/2287816067 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:24.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.452+0000 7fe683fff640 1 --2- 172.21.15.46:0/2287816067 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe6841570e0 0x7fe684157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:24.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.452+0000 7fe688b78640 1 --2- 172.21.15.46:0/2287816067 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe674003970 0x7fe684160eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:24.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.452+0000 7fe689379640 1 -- 172.21.15.46:0/2287816067 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe674003970 msgr2=0x7fe684160eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:24.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.452+0000 7fe689379640 1 --2- 172.21.15.46:0/2287816067 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe674003970 0x7fe684160eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:24.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.452+0000 7fe689379640 1 -- 172.21.15.46:0/2287816067 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe6841570e0 msgr2=0x7fe684157540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:24.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.452+0000 7fe689379640 1 --2- 172.21.15.46:0/2287816067 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe6841570e0 0x7fe684157540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:24.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.452+0000 7fe689379640 1 -- 172.21.15.46:0/2287816067 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe684164580 con 0x7fe684163650 2023-12-02T00:29:24.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.452+0000 7fe689379640 1 --2- 172.21.15.46:0/2287816067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe684163650 0x7fe684163ad0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe6840cc000 tx=0x7fe67c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:24.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.452+0000 7fe6817fa640 1 -- 172.21.15.46:0/2287816067 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67c007ae0 con 0x7fe684163650 2023-12-02T00:29:24.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.452+0000 7fe6817fa640 1 -- 172.21.15.46:0/2287816067 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe67c00eda0 con 0x7fe684163650 2023-12-02T00:29:24.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.452+0000 7fe68addc640 1 -- 172.21.15.46:0/2287816067 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6841642a0 con 0x7fe684163650 2023-12-02T00:29:24.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.452+0000 7fe6817fa640 1 -- 172.21.15.46:0/2287816067 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67c013660 con 0x7fe684163650 2023-12-02T00:29:24.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.452+0000 7fe68addc640 1 -- 172.21.15.46:0/2287816067 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6841421e0 con 0x7fe684163650 2023-12-02T00:29:24.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.453+0000 7fe68addc640 1 -- 172.21.15.46:0/2287816067 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe684069400 con 0x7fe684163650 2023-12-02T00:29:24.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.454+0000 7fe6817fa640 1 -- 172.21.15.46:0/2287816067 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe67c0137c0 con 0x7fe684163650 2023-12-02T00:29:24.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.454+0000 7fe6817fa640 1 --2- 172.21.15.46:0/2287816067 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe66806ae40 0x7fe66806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:24.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.454+0000 7fe6817fa640 1 -- 172.21.15.46:0/2287816067 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe67c0899b0 con 0x7fe684163650 2023-12-02T00:29:24.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.454+0000 7fe688b78640 1 --2- 172.21.15.46:0/2287816067 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe66806ae40 0x7fe66806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:24.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.455+0000 7fe688b78640 1 --2- 172.21.15.46:0/2287816067 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe66806ae40 0x7fe66806d300 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe678036000 tx=0x7fe678030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:24.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.458+0000 7fe6817fa640 1 -- 172.21.15.46:0/2287816067 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe67c054b10 con 0x7fe684163650 2023-12-02T00:29:24.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.632+0000 7fe68addc640 1 -- 172.21.15.46:0/2287816067 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe684069400 con 0x7fe684163650 2023-12-02T00:29:24.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.633+0000 7fe6817fa640 1 -- 172.21.15.46:0/2287816067 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe67c058130 con 0x7fe684163650 2023-12-02T00:29:24.635 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-12-02T00:29:24.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.634+0000 7fe68addc640 1 -- 172.21.15.46:0/2287816067 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe66806ae40 msgr2=0x7fe66806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:24.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.634+0000 7fe68addc640 1 --2- 172.21.15.46:0/2287816067 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe66806ae40 0x7fe66806d300 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe678036000 tx=0x7fe678030000 comp rx=0 tx=0).stop 2023-12-02T00:29:24.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.634+0000 7fe68addc640 1 -- 172.21.15.46:0/2287816067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe684163650 msgr2=0x7fe684163ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:24.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.634+0000 7fe68addc640 1 --2- 172.21.15.46:0/2287816067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe684163650 0x7fe684163ad0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe6840cc000 tx=0x7fe67c00e530 comp rx=0 tx=0).stop 2023-12-02T00:29:24.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.634+0000 7fe68addc640 1 -- 172.21.15.46:0/2287816067 shutdown_connections 2023-12-02T00:29:24.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.634+0000 7fe68addc640 1 --2- 172.21.15.46:0/2287816067 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe66806ae40 0x7fe66806d300 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:24.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.634+0000 7fe68addc640 1 --2- 172.21.15.46:0/2287816067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe684163650 0x7fe684163ad0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:24.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.634+0000 7fe68addc640 1 --2- 172.21.15.46:0/2287816067 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe6841570e0 0x7fe684157540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:24.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.634+0000 7fe68addc640 1 --2- 172.21.15.46:0/2287816067 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe674003970 0x7fe684160eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:24.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.634+0000 7fe68addc640 1 -- 172.21.15.46:0/2287816067 >> 172.21.15.46:0/2287816067 conn(0x7fe68406a4d0 msgr2=0x7fe684152fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:24.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.634+0000 7fe68addc640 1 -- 172.21.15.46:0/2287816067 shutdown_connections 2023-12-02T00:29:24.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.634+0000 7fe68addc640 1 -- 172.21.15.46:0/2287816067 wait complete. 2023-12-02T00:29:24.647 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-02T00:29:24.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.755+0000 7fa0e524f640 1 Processor -- start 2023-12-02T00:29:24.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.755+0000 7fa0e524f640 1 -- start start 2023-12-02T00:29:24.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.756+0000 7fa0e524f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa0e015d330 0x7fa0e00b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:24.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.756+0000 7fa0e524f640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa0e00d2910 0x7fa0e00d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:24.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.756+0000 7fa0e524f640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0e00b2b10 con 0x7fa0e00d2d90 2023-12-02T00:29:24.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.756+0000 7fa0e524f640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0e00d1590 con 0x7fa0e00d2910 2023-12-02T00:29:24.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.756+0000 7fa0e524f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e00d1870 con 0x7fa0e015d330 2023-12-02T00:29:24.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.756+0000 7fa0df59e640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa0e00d2d90 0x7fa0e00b6390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50390/0 (socket says 172.21.15.46:50390) 2023-12-02T00:29:24.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.756+0000 7fa0df59e640 1 -- 172.21.15.46:0/1530300554 learned_addr learned my addr 172.21.15.46:0/1530300554 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:24.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.756+0000 7fa0de59c640 1 --2- 172.21.15.46:0/1530300554 >> v2:172.21.15.162:3300/0 conn(0x7fa0e00d2910 0x7fa0e00d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:24.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.756+0000 7fa0ded9d640 1 --2- 172.21.15.46:0/1530300554 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa0e015d330 0x7fa0e00b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:24.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.756+0000 7fa0ddd9b640 1 -- 172.21.15.46:0/1530300554 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2708949973 0 0) 0x7fa0e00b2b10 con 0x7fa0e00d2d90 2023-12-02T00:29:24.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.756+0000 7fa0ddd9b640 1 -- 172.21.15.46:0/1530300554 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0c8003520 con 0x7fa0e00d2d90 2023-12-02T00:29:24.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.757+0000 7fa0ddd9b640 1 -- 172.21.15.46:0/1530300554 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4265004031 0 0) 0x7fa0c8003520 con 0x7fa0e00d2d90 2023-12-02T00:29:24.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.757+0000 7fa0ddd9b640 1 -- 172.21.15.46:0/1530300554 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa0e015d330 msgr2=0x7fa0e00b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:24.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.757+0000 7fa0ddd9b640 1 --2- 172.21.15.46:0/1530300554 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa0e015d330 0x7fa0e00b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:24.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.757+0000 7fa0ddd9b640 1 -- 172.21.15.46:0/1530300554 >> v2:172.21.15.162:3300/0 conn(0x7fa0e00d2910 msgr2=0x7fa0e00d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:24.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.757+0000 7fa0ddd9b640 1 --2- 172.21.15.46:0/1530300554 >> v2:172.21.15.162:3300/0 conn(0x7fa0e00d2910 0x7fa0e00d31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:24.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.757+0000 7fa0ddd9b640 1 -- 172.21.15.46:0/1530300554 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e00d1b50 con 0x7fa0e00d2d90 2023-12-02T00:29:24.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.757+0000 7fa0ddd9b640 1 -- 172.21.15.46:0/1530300554 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7fa0d00019f0 con 0x7fa0e00d2d90 2023-12-02T00:29:24.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.757+0000 7fa0ddd9b640 1 -- 172.21.15.46:0/1530300554 >> v1:172.21.15.46:6789/0 conn(0x7fa0e00d2d90 legacy=0x7fa0e00b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:24.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.757+0000 7fa0ddd9b640 1 --2- 172.21.15.46:0/1530300554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0c8003970 0x7fa0c8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:24.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.757+0000 7fa0ddd9b640 1 --2- 172.21.15.46:0/1530300554 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa0e015d330 0x7fa0c80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:24.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.757+0000 7fa0ddd9b640 1 --2- 172.21.15.46:0/1530300554 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa0c8004d00 0x7fa0c8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:24.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.757+0000 7fa0ddd9b640 1 -- 172.21.15.46:0/1530300554 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e00d1590 con 0x7fa0c8003970 2023-12-02T00:29:24.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.757+0000 7fa0ddd9b640 1 -- 172.21.15.46:0/1530300554 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e00d1870 con 0x7fa0c8004d00 2023-12-02T00:29:24.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.757+0000 7fa0ddd9b640 1 -- 172.21.15.46:0/1530300554 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c800b980 con 0x7fa0e015d330 2023-12-02T00:29:24.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.757+0000 7fa0ded9d640 1 --2- 172.21.15.46:0/1530300554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0c8003970 0x7fa0c8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:24.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.757+0000 7fa0de59c640 1 --2- 172.21.15.46:0/1530300554 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa0e015d330 0x7fa0c80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:24.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.757+0000 7fa0df59e640 1 --2- 172.21.15.46:0/1530300554 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa0c8004d00 0x7fa0c8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:24.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.757+0000 7fa0ded9d640 1 -- 172.21.15.46:0/1530300554 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa0e015d330 msgr2=0x7fa0c80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:24.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.757+0000 7fa0ded9d640 1 --2- 172.21.15.46:0/1530300554 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa0e015d330 0x7fa0c80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:24.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.757+0000 7fa0ded9d640 1 -- 172.21.15.46:0/1530300554 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa0c8004d00 msgr2=0x7fa0c8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:24.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.757+0000 7fa0ded9d640 1 --2- 172.21.15.46:0/1530300554 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa0c8004d00 0x7fa0c8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:24.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.757+0000 7fa0ded9d640 1 -- 172.21.15.46:0/1530300554 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c8003520 con 0x7fa0c8003970 2023-12-02T00:29:24.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.757+0000 7fa0df59e640 1 --2- 172.21.15.46:0/1530300554 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa0c8004d00 0x7fa0c8005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:29:24.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.758+0000 7fa0ded9d640 1 --2- 172.21.15.46:0/1530300554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0c8003970 0x7fa0c8003d90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8009ac0 tx=0x7fa0c80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a41fdc3bbf0be5f server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:24.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.758+0000 7fa0ddd9b640 1 -- 172.21.15.46:0/1530300554 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d802c6b0 con 0x7fa0c8003970 2023-12-02T00:29:24.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.758+0000 7fa0ddd9b640 1 -- 172.21.15.46:0/1530300554 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0d802c810 con 0x7fa0c8003970 2023-12-02T00:29:24.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.758+0000 7fa0ddd9b640 1 -- 172.21.15.46:0/1530300554 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d802cba0 con 0x7fa0c8003970 2023-12-02T00:29:24.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.759+0000 7fa0e524f640 1 -- 172.21.15.46:0/1530300554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0c8003970 msgr2=0x7fa0c8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:24.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.759+0000 7fa0e524f640 1 --2- 172.21.15.46:0/1530300554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0c8003970 0x7fa0c8003d90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8009ac0 tx=0x7fa0c80057f0 comp rx=0 tx=0).stop 2023-12-02T00:29:24.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.759+0000 7fa0df59e640 1 -- 172.21.15.46:0/1530300554 reap_dead start 2023-12-02T00:29:24.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.759+0000 7fa0e524f640 1 -- 172.21.15.46:0/1530300554 shutdown_connections 2023-12-02T00:29:24.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.759+0000 7fa0e524f640 1 -- 172.21.15.46:0/1530300554 >> 172.21.15.46:0/1530300554 conn(0x7fa0e0072020 msgr2=0x7fa0e0072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:24.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.759+0000 7fa0e524f640 1 -- 172.21.15.46:0/1530300554 shutdown_connections 2023-12-02T00:29:24.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.759+0000 7fa0e524f640 1 -- 172.21.15.46:0/1530300554 wait complete. 2023-12-02T00:29:24.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.760+0000 7fa0e524f640 1 Processor -- start 2023-12-02T00:29:24.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.760+0000 7fa0e524f640 1 -- start start 2023-12-02T00:29:24.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.761+0000 7fa0e524f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa0c8003970 0x7fa0e01f3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:24.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.761+0000 7fa0e524f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0e00c95c0 0x7fa0e01430d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:24.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.761+0000 7fa0e524f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa0e0143610 0x7fa0e00cc580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:24.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.761+0000 7fa0e524f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e01f3c70 con 0x7fa0e00c95c0 2023-12-02T00:29:24.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.761+0000 7fa0e524f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e00cb500 con 0x7fa0c8003970 2023-12-02T00:29:24.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.761+0000 7fa0e524f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e00cb7c0 con 0x7fa0e0143610 2023-12-02T00:29:24.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.761+0000 7fa0de59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0e00c95c0 0x7fa0e01430d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:24.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.761+0000 7fa0de59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0e00c95c0 0x7fa0e01430d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52668/0 (socket says 172.21.15.46:52668) 2023-12-02T00:29:24.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.761+0000 7fa0de59c640 1 -- 172.21.15.46:0/1822675244 learned_addr learned my addr 172.21.15.46:0/1822675244 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:24.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.761+0000 7fa0df59e640 1 --2- 172.21.15.46:0/1822675244 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa0e0143610 0x7fa0e00cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:24.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.761+0000 7fa0ded9d640 1 --2- 172.21.15.46:0/1822675244 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa0c8003970 0x7fa0e01f3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:24.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.761+0000 7fa0de59c640 1 -- 172.21.15.46:0/1822675244 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa0e0143610 msgr2=0x7fa0e00cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:24.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.761+0000 7fa0de59c640 1 --2- 172.21.15.46:0/1822675244 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa0e0143610 0x7fa0e00cc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:24.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.761+0000 7fa0de59c640 1 -- 172.21.15.46:0/1822675244 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa0c8003970 msgr2=0x7fa0e01f3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:24.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.761+0000 7fa0de59c640 1 --2- 172.21.15.46:0/1822675244 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa0c8003970 0x7fa0e01f3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:24.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.761+0000 7fa0de59c640 1 -- 172.21.15.46:0/1822675244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e00cbaa0 con 0x7fa0e00c95c0 2023-12-02T00:29:24.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.761+0000 7fa0ded9d640 1 --2- 172.21.15.46:0/1822675244 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa0c8003970 0x7fa0e01f3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:24.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.762+0000 7fa0de59c640 1 --2- 172.21.15.46:0/1822675244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0e00c95c0 0x7fa0e01430d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa0d40116d0 tx=0x7fa0d4011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:24.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.762+0000 7fa0c77fe640 1 -- 172.21.15.46:0/1822675244 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d400f9c0 con 0x7fa0e00c95c0 2023-12-02T00:29:24.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.762+0000 7fa0c77fe640 1 -- 172.21.15.46:0/1822675244 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0d400fb20 con 0x7fa0e00c95c0 2023-12-02T00:29:24.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.762+0000 7fa0e524f640 1 -- 172.21.15.46:0/1822675244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0e00cbd80 con 0x7fa0e00c95c0 2023-12-02T00:29:24.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.762+0000 7fa0c77fe640 1 -- 172.21.15.46:0/1822675244 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d400fe90 con 0x7fa0e00c95c0 2023-12-02T00:29:24.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.762+0000 7fa0e524f640 1 -- 172.21.15.46:0/1822675244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0e01f2eb0 con 0x7fa0e00c95c0 2023-12-02T00:29:24.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.763+0000 7fa0e524f640 1 -- 172.21.15.46:0/1822675244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0e0069400 con 0x7fa0e00c95c0 2023-12-02T00:29:24.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.763+0000 7fa0c77fe640 1 -- 172.21.15.46:0/1822675244 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa0d4015460 con 0x7fa0e00c95c0 2023-12-02T00:29:24.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.764+0000 7fa0c77fe640 1 --2- 172.21.15.46:0/1822675244 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa0cc06ae70 0x7fa0cc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:24.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.764+0000 7fa0c77fe640 1 -- 172.21.15.46:0/1822675244 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa0d4089d70 con 0x7fa0e00c95c0 2023-12-02T00:29:24.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.764+0000 7fa0ded9d640 1 --2- 172.21.15.46:0/1822675244 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa0cc06ae70 0x7fa0cc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:24.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.764+0000 7fa0ded9d640 1 --2- 172.21.15.46:0/1822675244 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa0cc06ae70 0x7fa0cc06d330 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8036000 tx=0x7fa0d8030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:24.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.768+0000 7fa0c77fe640 1 -- 172.21.15.46:0/1822675244 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa0d4054e20 con 0x7fa0e00c95c0 2023-12-02T00:29:24.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.944+0000 7fa0e524f640 1 -- 172.21.15.46:0/1822675244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa0e00b8a20 con 0x7fa0e00c95c0 2023-12-02T00:29:24.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.944+0000 7fa0c77fe640 1 -- 172.21.15.46:0/1822675244 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa0d4058440 con 0x7fa0e00c95c0 2023-12-02T00:29:24.946 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-12-02T00:29:24.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.946+0000 7fa0e524f640 1 -- 172.21.15.46:0/1822675244 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa0cc06ae70 msgr2=0x7fa0cc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:24.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.946+0000 7fa0e524f640 1 --2- 172.21.15.46:0/1822675244 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa0cc06ae70 0x7fa0cc06d330 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8036000 tx=0x7fa0d8030000 comp rx=0 tx=0).stop 2023-12-02T00:29:24.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.946+0000 7fa0e524f640 1 -- 172.21.15.46:0/1822675244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0e00c95c0 msgr2=0x7fa0e01430d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:24.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.946+0000 7fa0e524f640 1 --2- 172.21.15.46:0/1822675244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0e00c95c0 0x7fa0e01430d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa0d40116d0 tx=0x7fa0d4011b90 comp rx=0 tx=0).stop 2023-12-02T00:29:24.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.946+0000 7fa0e524f640 1 -- 172.21.15.46:0/1822675244 shutdown_connections 2023-12-02T00:29:24.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.946+0000 7fa0e524f640 1 --2- 172.21.15.46:0/1822675244 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa0cc06ae70 0x7fa0cc06d330 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:24.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.946+0000 7fa0e524f640 1 --2- 172.21.15.46:0/1822675244 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa0e0143610 0x7fa0e00cc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:24.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.946+0000 7fa0e524f640 1 --2- 172.21.15.46:0/1822675244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0e00c95c0 0x7fa0e01430d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:24.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.946+0000 7fa0e524f640 1 --2- 172.21.15.46:0/1822675244 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa0c8003970 0x7fa0e01f3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:24.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.946+0000 7fa0e524f640 1 -- 172.21.15.46:0/1822675244 >> 172.21.15.46:0/1822675244 conn(0x7fa0e0072020 msgr2=0x7fa0e00b64f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:24.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.946+0000 7fa0e524f640 1 -- 172.21.15.46:0/1822675244 shutdown_connections 2023-12-02T00:29:24.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:24.946+0000 7fa0e524f640 1 -- 172.21.15.46:0/1822675244 wait complete. 2023-12-02T00:29:24.960 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-02T00:29:25.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.067+0000 7f1994210640 1 Processor -- start 2023-12-02T00:29:25.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.067+0000 7f1994210640 1 -- start start 2023-12-02T00:29:25.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.068+0000 7f1994210640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f198c140310 0x7f198c1618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:25.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.068+0000 7f1994210640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f198c1409d0 0x7f198c06a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:25.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.068+0000 7f1994210640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f198c1406f0 con 0x7f198c1428d0 2023-12-02T00:29:25.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.068+0000 7f1994210640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f198c161e20 con 0x7f198c140310 2023-12-02T00:29:25.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.068+0000 7f1994210640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f198c143960 con 0x7f198c1409d0 2023-12-02T00:29:25.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.068+0000 7f1991fac640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f198c140310 0x7f198c1618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:25.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.068+0000 7f1991fac640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f198c140310 0x7f198c1618e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52674/0 (socket says 172.21.15.46:52674) 2023-12-02T00:29:25.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.068+0000 7f1991fac640 1 -- 172.21.15.46:0/4223073364 learned_addr learned my addr 172.21.15.46:0/4223073364 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:25.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.068+0000 7f19917ab640 1 --2- 172.21.15.46:0/4223073364 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f198c1409d0 0x7f198c06a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:25.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.068+0000 7f1991fac640 1 -- 172.21.15.46:0/4223073364 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f198c1409d0 msgr2=0x7f198c06a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:25.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.068+0000 7f1991fac640 1 --2- 172.21.15.46:0/4223073364 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f198c1409d0 0x7f198c06a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:25.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.068+0000 7f1991fac640 1 -- 172.21.15.46:0/4223073364 >> v1:172.21.15.162:6789/0 conn(0x7f198c1428d0 legacy=0x7f198c0714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:25.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.068+0000 7f1991fac640 1 -- 172.21.15.46:0/4223073364 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f198c142de0 con 0x7f198c140310 2023-12-02T00:29:25.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.069+0000 7f1991fac640 1 --2- 172.21.15.46:0/4223073364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f198c140310 0x7f198c1618e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1984009870 tx=0x7f198402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac4eec6fb1d024aa server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:25.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.069+0000 7f1990faa640 1 -- 172.21.15.46:0/4223073364 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19840317c0 con 0x7f198c140310 2023-12-02T00:29:25.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.069+0000 7f1990faa640 1 -- 172.21.15.46:0/4223073364 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1984031920 con 0x7f198c140310 2023-12-02T00:29:25.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.069+0000 7f1990faa640 1 -- 172.21.15.46:0/4223073364 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1984031cb0 con 0x7f198c140310 2023-12-02T00:29:25.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.070+0000 7f1994210640 1 -- 172.21.15.46:0/4223073364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f198c140310 msgr2=0x7f198c1618e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:25.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.070+0000 7f1994210640 1 --2- 172.21.15.46:0/4223073364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f198c140310 0x7f198c1618e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1984009870 tx=0x7f198402b860 comp rx=0 tx=0).stop 2023-12-02T00:29:25.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.070+0000 7f1994210640 1 -- 172.21.15.46:0/4223073364 shutdown_connections 2023-12-02T00:29:25.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.070+0000 7f1994210640 1 --2- 172.21.15.46:0/4223073364 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f198c1409d0 0x7f198c06a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:25.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.070+0000 7f1994210640 1 --2- 172.21.15.46:0/4223073364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f198c140310 0x7f198c1618e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:25.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.070+0000 7f1994210640 1 -- 172.21.15.46:0/4223073364 >> 172.21.15.46:0/4223073364 conn(0x7f198c0b1e50 msgr2=0x7f198c0b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:25.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.070+0000 7f1994210640 1 -- 172.21.15.46:0/4223073364 shutdown_connections 2023-12-02T00:29:25.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.070+0000 7f1994210640 1 -- 172.21.15.46:0/4223073364 wait complete. 2023-12-02T00:29:25.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.071+0000 7f1994210640 1 Processor -- start 2023-12-02T00:29:25.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.071+0000 7f1994210640 1 -- start start 2023-12-02T00:29:25.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.072+0000 7f1994210640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f198c140310 0x7f198c153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:25.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.072+0000 7f1994210640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f198c1409d0 0x7f198c0aec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:25.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.072+0000 7f1994210640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f198c1428d0 0x7f198c0add90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:25.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.072+0000 7f1994210640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198c154010 con 0x7f198c1428d0 2023-12-02T00:29:25.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.072+0000 7f1994210640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198c0ad900 con 0x7f198c1409d0 2023-12-02T00:29:25.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.072+0000 7f1994210640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f198c0ad600 con 0x7f198c140310 2023-12-02T00:29:25.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.072+0000 7f19927ad640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f198c1428d0 0x7f198c0add90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:25.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.072+0000 7f19927ad640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f198c1428d0 0x7f198c0add90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52690/0 (socket says 172.21.15.46:52690) 2023-12-02T00:29:25.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.072+0000 7f19927ad640 1 -- 172.21.15.46:0/3846613455 learned_addr learned my addr 172.21.15.46:0/3846613455 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:25.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.072+0000 7f19917ab640 1 --2- 172.21.15.46:0/3846613455 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f198c1409d0 0x7f198c0aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:25.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.072+0000 7f1991fac640 1 --2- 172.21.15.46:0/3846613455 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f198c140310 0x7f198c153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:25.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.072+0000 7f19927ad640 1 -- 172.21.15.46:0/3846613455 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f198c140310 msgr2=0x7f198c153ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:25.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.072+0000 7f19927ad640 1 --2- 172.21.15.46:0/3846613455 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f198c140310 0x7f198c153ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:25.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.072+0000 7f19927ad640 1 -- 172.21.15.46:0/3846613455 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f198c1409d0 msgr2=0x7f198c0aec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:25.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.072+0000 7f19927ad640 1 --2- 172.21.15.46:0/3846613455 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f198c1409d0 0x7f198c0aec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:25.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.072+0000 7f19927ad640 1 -- 172.21.15.46:0/3846613455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19840094d0 con 0x7f198c1428d0 2023-12-02T00:29:25.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.073+0000 7f19927ad640 1 --2- 172.21.15.46:0/3846613455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f198c1428d0 0x7f198c0add90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f197c00c7b0 tx=0x7f197c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:25.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.073+0000 7f198a7fc640 1 -- 172.21.15.46:0/3846613455 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197c004b60 con 0x7f198c1428d0 2023-12-02T00:29:25.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.073+0000 7f198a7fc640 1 -- 172.21.15.46:0/3846613455 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f197c004cc0 con 0x7f198c1428d0 2023-12-02T00:29:25.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.073+0000 7f1994210640 1 -- 172.21.15.46:0/3846613455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f198c0af520 con 0x7f198c1428d0 2023-12-02T00:29:25.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.073+0000 7f198a7fc640 1 -- 172.21.15.46:0/3846613455 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197c00aa90 con 0x7f198c1428d0 2023-12-02T00:29:25.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.073+0000 7f1994210640 1 -- 172.21.15.46:0/3846613455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f198c0afae0 con 0x7f198c1428d0 2023-12-02T00:29:25.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.074+0000 7f1994210640 1 -- 172.21.15.46:0/3846613455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f198c0adab0 con 0x7f198c1428d0 2023-12-02T00:29:25.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.074+0000 7f198a7fc640 1 -- 172.21.15.46:0/3846613455 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f197c00abf0 con 0x7f198c1428d0 2023-12-02T00:29:25.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.075+0000 7f198a7fc640 1 --2- 172.21.15.46:0/3846613455 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f197806ae40 0x7f197806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:25.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.075+0000 7f198a7fc640 1 -- 172.21.15.46:0/3846613455 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f197c089660 con 0x7f198c1428d0 2023-12-02T00:29:25.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.075+0000 7f1991fac640 1 --2- 172.21.15.46:0/3846613455 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f197806ae40 0x7f197806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:25.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.075+0000 7f1991fac640 1 --2- 172.21.15.46:0/3846613455 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f197806ae40 0x7f197806d300 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1984036000 tx=0x7f198403d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:25.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.079+0000 7f198a7fc640 1 -- 172.21.15.46:0/3846613455 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f198c0adab0 con 0x7f198c1428d0 2023-12-02T00:29:25.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.255+0000 7f1994210640 1 -- 172.21.15.46:0/3846613455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f198c0adab0 con 0x7f198c1428d0 2023-12-02T00:29:25.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.255+0000 7f198a7fc640 1 -- 172.21.15.46:0/3846613455 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f198c0adab0 con 0x7f198c1428d0 2023-12-02T00:29:25.258 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-12-02T00:29:25.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.257+0000 7f1994210640 1 -- 172.21.15.46:0/3846613455 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f197806ae40 msgr2=0x7f197806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:25.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.257+0000 7f1994210640 1 --2- 172.21.15.46:0/3846613455 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f197806ae40 0x7f197806d300 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1984036000 tx=0x7f198403d040 comp rx=0 tx=0).stop 2023-12-02T00:29:25.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.257+0000 7f1994210640 1 -- 172.21.15.46:0/3846613455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f198c1428d0 msgr2=0x7f198c0add90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:25.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.257+0000 7f1994210640 1 --2- 172.21.15.46:0/3846613455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f198c1428d0 0x7f198c0add90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f197c00c7b0 tx=0x7f197c00cc70 comp rx=0 tx=0).stop 2023-12-02T00:29:25.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.257+0000 7f1994210640 1 -- 172.21.15.46:0/3846613455 shutdown_connections 2023-12-02T00:29:25.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.257+0000 7f1994210640 1 --2- 172.21.15.46:0/3846613455 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f197806ae40 0x7f197806d300 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:25.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.257+0000 7f1994210640 1 --2- 172.21.15.46:0/3846613455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f198c1428d0 0x7f198c0add90 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:25.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.257+0000 7f1994210640 1 --2- 172.21.15.46:0/3846613455 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f198c1409d0 0x7f198c0aec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:25.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.257+0000 7f1994210640 1 --2- 172.21.15.46:0/3846613455 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f198c140310 0x7f198c153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:25.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.257+0000 7f1994210640 1 -- 172.21.15.46:0/3846613455 >> 172.21.15.46:0/3846613455 conn(0x7f198c0b1e50 msgr2=0x7f198c15eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:25.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.257+0000 7f1994210640 1 -- 172.21.15.46:0/3846613455 shutdown_connections 2023-12-02T00:29:25.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.257+0000 7f1994210640 1 -- 172.21.15.46:0/3846613455 wait complete. 2023-12-02T00:29:25.270 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-02T00:29:25.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.376+0000 7f950e083640 1 Processor -- start 2023-12-02T00:29:25.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.376+0000 7f950e083640 1 -- start start 2023-12-02T00:29:25.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.377+0000 7f950e083640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9508142270 0x7f95081426c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:25.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.377+0000 7f950e083640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9508161d20 con 0x7f9508160fb0 2023-12-02T00:29:25.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.377+0000 7f950e083640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f950815e760 con 0x7f950806ea50 2023-12-02T00:29:25.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.377+0000 7f950e083640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95080d1ee0 con 0x7f9508142270 2023-12-02T00:29:25.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.377+0000 7f9506ffd640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9508160fb0 0x7f9508161440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50394/0 (socket says 172.21.15.46:50394) 2023-12-02T00:29:25.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.377+0000 7f9506ffd640 1 -- 172.21.15.46:0/1766149611 learned_addr learned my addr 172.21.15.46:0/1766149611 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:25.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.377+0000 7f9507fff640 1 --2- 172.21.15.46:0/1766149611 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9508142270 0x7f95081426c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:25.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.377+0000 7f95067fc640 1 -- 172.21.15.46:0/1766149611 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 35795029 0 0) 0x7f9508161d20 con 0x7f9508160fb0 2023-12-02T00:29:25.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.377+0000 7f95067fc640 1 -- 172.21.15.46:0/1766149611 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94f0003520 con 0x7f9508160fb0 2023-12-02T00:29:25.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.377+0000 7f95067fc640 1 -- 172.21.15.46:0/1766149611 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1317225034 0 0) 0x7f950815e760 con 0x7f950806ea50 2023-12-02T00:29:25.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.377+0000 7f95067fc640 1 -- 172.21.15.46:0/1766149611 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9508161d20 con 0x7f950806ea50 2023-12-02T00:29:25.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.378+0000 7f9507fff640 1 -- 172.21.15.46:0/1766149611 >> v1:172.21.15.162:6789/0 conn(0x7f950806ea50 legacy=0x7f950806ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:25.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.378+0000 7f9507fff640 1 -- 172.21.15.46:0/1766149611 >> v1:172.21.15.46:6789/0 conn(0x7f9508160fb0 legacy=0x7f9508161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:25.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.378+0000 7f9507fff640 1 -- 172.21.15.46:0/1766149611 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f950815d9a0 con 0x7f9508142270 2023-12-02T00:29:25.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.378+0000 7f9507fff640 1 --2- 172.21.15.46:0/1766149611 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9508142270 0x7f95081426c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9500009dc0 tx=0x7f950002bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d2511b0e4c50fb6 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:25.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.379+0000 7f95067fc640 1 -- 172.21.15.46:0/1766149611 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950002ccd0 con 0x7f9508142270 2023-12-02T00:29:25.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.379+0000 7f95067fc640 1 -- 172.21.15.46:0/1766149611 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f950002ce30 con 0x7f9508142270 2023-12-02T00:29:25.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.379+0000 7f95067fc640 1 -- 172.21.15.46:0/1766149611 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9500034660 con 0x7f9508142270 2023-12-02T00:29:25.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.379+0000 7f950e083640 1 -- 172.21.15.46:0/1766149611 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9508142270 msgr2=0x7f95081426c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:25.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.379+0000 7f950e083640 1 --2- 172.21.15.46:0/1766149611 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9508142270 0x7f95081426c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9500009dc0 tx=0x7f950002bf20 comp rx=0 tx=0).stop 2023-12-02T00:29:25.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.379+0000 7f950e083640 1 -- 172.21.15.46:0/1766149611 shutdown_connections 2023-12-02T00:29:25.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.379+0000 7f950e083640 1 --2- 172.21.15.46:0/1766149611 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9508142270 0x7f95081426c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:25.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.379+0000 7f950e083640 1 -- 172.21.15.46:0/1766149611 >> 172.21.15.46:0/1766149611 conn(0x7f950806a6b0 msgr2=0x7f950806aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:25.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.379+0000 7f950e083640 1 -- 172.21.15.46:0/1766149611 shutdown_connections 2023-12-02T00:29:25.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.379+0000 7f950e083640 1 -- 172.21.15.46:0/1766149611 wait complete. 2023-12-02T00:29:25.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.380+0000 7f950e083640 1 Processor -- start 2023-12-02T00:29:25.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.381+0000 7f950e083640 1 -- start start 2023-12-02T00:29:25.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.381+0000 7f950e083640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f950806ea50 0x7f95081598b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:25.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.381+0000 7f950e083640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9508142270 0x7f950815c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:25.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.381+0000 7f950e083640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9508160fb0 0x7f95081567c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:25.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.381+0000 7f950e083640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95081f7200 con 0x7f9508160fb0 2023-12-02T00:29:25.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.381+0000 7f950e083640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95081f6350 con 0x7f950806ea50 2023-12-02T00:29:25.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.381+0000 7f950e083640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95081f65f0 con 0x7f9508142270 2023-12-02T00:29:25.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.381+0000 7f9507fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9508160fb0 0x7f95081567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:25.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.381+0000 7f9507fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9508160fb0 0x7f95081567c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52706/0 (socket says 172.21.15.46:52706) 2023-12-02T00:29:25.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.381+0000 7f9507fff640 1 -- 172.21.15.46:0/1138612524 learned_addr learned my addr 172.21.15.46:0/1138612524 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:25.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.381+0000 7f9506ffd640 1 --2- 172.21.15.46:0/1138612524 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9508142270 0x7f950815c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:25.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.381+0000 7f95077fe640 1 --2- 172.21.15.46:0/1138612524 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f950806ea50 0x7f95081598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:25.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.382+0000 7f9507fff640 1 -- 172.21.15.46:0/1138612524 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9508142270 msgr2=0x7f950815c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:25.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.382+0000 7f9507fff640 1 --2- 172.21.15.46:0/1138612524 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9508142270 0x7f950815c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:25.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.382+0000 7f9507fff640 1 -- 172.21.15.46:0/1138612524 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f950806ea50 msgr2=0x7f95081598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:25.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.382+0000 7f9507fff640 1 --2- 172.21.15.46:0/1138612524 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f950806ea50 0x7f95081598b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:25.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.382+0000 7f9507fff640 1 -- 172.21.15.46:0/1138612524 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9500009a70 con 0x7f9508160fb0 2023-12-02T00:29:25.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.382+0000 7f9507fff640 1 --2- 172.21.15.46:0/1138612524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9508160fb0 0x7f95081567c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f950002dfd0 tx=0x7f9500004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:25.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.382+0000 7f94effff640 1 -- 172.21.15.46:0/1138612524 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950003d600 con 0x7f9508160fb0 2023-12-02T00:29:25.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.382+0000 7f94effff640 1 -- 172.21.15.46:0/1138612524 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f950003d760 con 0x7f9508160fb0 2023-12-02T00:29:25.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.382+0000 7f950e083640 1 -- 172.21.15.46:0/1138612524 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95081f68d0 con 0x7f9508160fb0 2023-12-02T00:29:25.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.382+0000 7f94effff640 1 -- 172.21.15.46:0/1138612524 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950003db10 con 0x7f9508160fb0 2023-12-02T00:29:25.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.382+0000 7f950e083640 1 -- 172.21.15.46:0/1138612524 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95081f6e90 con 0x7f9508160fb0 2023-12-02T00:29:25.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.383+0000 7f950e083640 1 -- 172.21.15.46:0/1138612524 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9508157ea0 con 0x7f9508160fb0 2023-12-02T00:29:25.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.383+0000 7f94effff640 1 -- 172.21.15.46:0/1138612524 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f950003b040 con 0x7f9508160fb0 2023-12-02T00:29:25.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.384+0000 7f94effff640 1 --2- 172.21.15.46:0/1138612524 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f94f406ae70 0x7f94f406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:25.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.384+0000 7f94effff640 1 -- 172.21.15.46:0/1138612524 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f95000ab570 con 0x7f9508160fb0 2023-12-02T00:29:25.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.384+0000 7f95077fe640 1 --2- 172.21.15.46:0/1138612524 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f94f406ae70 0x7f94f406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:25.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.385+0000 7f95077fe640 1 --2- 172.21.15.46:0/1138612524 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f94f406ae70 0x7f94f406d330 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f94f80011e0 tx=0x7f94f8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:25.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.388+0000 7f94effff640 1 -- 172.21.15.46:0/1138612524 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9500076760 con 0x7f9508160fb0 2023-12-02T00:29:25.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.580+0000 7f950e083640 1 -- 172.21.15.46:0/1138612524 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9508069400 con 0x7f9508160fb0 2023-12-02T00:29:25.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.580+0000 7f94effff640 1 -- 172.21.15.46:0/1138612524 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9500009450 con 0x7f9508160fb0 2023-12-02T00:29:25.582 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2023-12-02T00:29:25.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.582+0000 7f950e083640 1 -- 172.21.15.46:0/1138612524 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f94f406ae70 msgr2=0x7f94f406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:25.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.582+0000 7f950e083640 1 --2- 172.21.15.46:0/1138612524 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f94f406ae70 0x7f94f406d330 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f94f80011e0 tx=0x7f94f8009040 comp rx=0 tx=0).stop 2023-12-02T00:29:25.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.582+0000 7f950e083640 1 -- 172.21.15.46:0/1138612524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9508160fb0 msgr2=0x7f95081567c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:25.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.582+0000 7f950e083640 1 --2- 172.21.15.46:0/1138612524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9508160fb0 0x7f95081567c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f950002dfd0 tx=0x7f9500004a40 comp rx=0 tx=0).stop 2023-12-02T00:29:25.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.582+0000 7f950e083640 1 -- 172.21.15.46:0/1138612524 shutdown_connections 2023-12-02T00:29:25.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.582+0000 7f950e083640 1 --2- 172.21.15.46:0/1138612524 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f94f406ae70 0x7f94f406d330 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:25.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.582+0000 7f950e083640 1 --2- 172.21.15.46:0/1138612524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9508160fb0 0x7f95081567c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:25.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.582+0000 7f950e083640 1 --2- 172.21.15.46:0/1138612524 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9508142270 0x7f950815c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:25.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.582+0000 7f950e083640 1 --2- 172.21.15.46:0/1138612524 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f950806ea50 0x7f95081598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:25.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.582+0000 7f950e083640 1 -- 172.21.15.46:0/1138612524 >> 172.21.15.46:0/1138612524 conn(0x7f950806a6b0 msgr2=0x7f9508140850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:25.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.582+0000 7f950e083640 1 -- 172.21.15.46:0/1138612524 shutdown_connections 2023-12-02T00:29:25.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.582+0000 7f950e083640 1 -- 172.21.15.46:0/1138612524 wait complete. 2023-12-02T00:29:25.595 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi046.front.sepia.ceph.com... 2023-12-02T00:29:25.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:25.596 DEBUG:teuthology.orchestra.run.smithi046:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-12-02T00:29:25.627 INFO:teuthology.orchestra.run:waiting for 300 2023-12-02T00:29:25.627 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2023-12-02T00:29:25.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:25.627 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-02T00:29:25.695 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-12-02T00:29:25.695 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-02T00:29:25.715 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-12-02T00:29:25.716 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-02T00:29:25.774 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:29:25.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.878+0000 7f61b6ae5640 1 Processor -- start 2023-12-02T00:29:25.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.878+0000 7f61b6ae5640 1 -- start start 2023-12-02T00:29:25.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.879+0000 7f61b6ae5640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f61b00aebf0 0x7f61b015e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:25.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.879+0000 7f61b6ae5640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f61b006b4e0 0x7f61b006b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:25.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.879+0000 7f61b6ae5640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61b00cb910 con 0x7f61b00b3930 2023-12-02T00:29:25.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.879+0000 7f61b6ae5640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61b006bed0 con 0x7f61b006b4e0 2023-12-02T00:29:25.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.879+0000 7f61b6ae5640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61b015eaa0 con 0x7f61b00aebf0 2023-12-02T00:29:25.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.879+0000 7f61affff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f61b006b4e0 0x7f61b006b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:25.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.879+0000 7f61b5082640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f61b00b3930 0x7f61b00b3da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50402/0 (socket says 172.21.15.46:50402) 2023-12-02T00:29:25.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.879+0000 7f61affff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f61b006b4e0 0x7f61b006b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52722/0 (socket says 172.21.15.46:52722) 2023-12-02T00:29:25.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.879+0000 7f61b5082640 1 -- 172.21.15.46:0/2589717474 learned_addr learned my addr 172.21.15.46:0/2589717474 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:25.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.880+0000 7f61b4881640 1 --2- 172.21.15.46:0/2589717474 >> v2:172.21.15.162:3300/0 conn(0x7f61b00aebf0 0x7f61b015e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:25.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.880+0000 7f61affff640 1 -- 172.21.15.46:0/2589717474 >> v2:172.21.15.162:3300/0 conn(0x7f61b00aebf0 msgr2=0x7f61b015e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:25.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.880+0000 7f61affff640 1 --2- 172.21.15.46:0/2589717474 >> v2:172.21.15.162:3300/0 conn(0x7f61b00aebf0 0x7f61b015e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:25.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.880+0000 7f61affff640 1 -- 172.21.15.46:0/2589717474 >> v1:172.21.15.46:6789/0 conn(0x7f61b00b3930 legacy=0x7f61b00b3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:25.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.880+0000 7f61affff640 1 -- 172.21.15.46:0/2589717474 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b00cbbb0 con 0x7f61b006b4e0 2023-12-02T00:29:25.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.880+0000 7f61af7fe640 1 -- 172.21.15.46:0/2589717474 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2773618549 0 0) 0x7f61b00cb910 con 0x7f61b00b3930 2023-12-02T00:29:25.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.880+0000 7f61affff640 1 --2- 172.21.15.46:0/2589717474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61b006b4e0 0x7f61b006b990 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f61b0071fb0 tx=0x7f61a402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc972c975f883f86 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:25.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.881+0000 7f61af7fe640 1 -- 172.21.15.46:0/2589717474 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a4031830 con 0x7f61b006b4e0 2023-12-02T00:29:25.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.881+0000 7f61af7fe640 1 -- 172.21.15.46:0/2589717474 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61a4031990 con 0x7f61b006b4e0 2023-12-02T00:29:25.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.881+0000 7f61af7fe640 1 -- 172.21.15.46:0/2589717474 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a4031d20 con 0x7f61b006b4e0 2023-12-02T00:29:25.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.881+0000 7f61b6ae5640 1 -- 172.21.15.46:0/2589717474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61b006b4e0 msgr2=0x7f61b006b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:25.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.881+0000 7f61b6ae5640 1 --2- 172.21.15.46:0/2589717474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61b006b4e0 0x7f61b006b990 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f61b0071fb0 tx=0x7f61a402b8d0 comp rx=0 tx=0).stop 2023-12-02T00:29:25.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.881+0000 7f61b6ae5640 1 -- 172.21.15.46:0/2589717474 shutdown_connections 2023-12-02T00:29:25.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.881+0000 7f61b6ae5640 1 --2- 172.21.15.46:0/2589717474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61b006b4e0 0x7f61b006b990 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:25.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.881+0000 7f61b6ae5640 1 --2- 172.21.15.46:0/2589717474 >> v2:172.21.15.162:3300/0 conn(0x7f61b00aebf0 0x7f61b015e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:25.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.881+0000 7f61b6ae5640 1 -- 172.21.15.46:0/2589717474 >> 172.21.15.46:0/2589717474 conn(0x7f61b0070ec0 msgr2=0x7f61b00712a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:25.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.881+0000 7f61b6ae5640 1 -- 172.21.15.46:0/2589717474 shutdown_connections 2023-12-02T00:29:25.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.881+0000 7f61b6ae5640 1 -- 172.21.15.46:0/2589717474 wait complete. 2023-12-02T00:29:25.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.882+0000 7f61b6ae5640 1 Processor -- start 2023-12-02T00:29:25.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.882+0000 7f61b6ae5640 1 -- start start 2023-12-02T00:29:25.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.883+0000 7f61b6ae5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61b006b4e0 0x7f61b0154c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:25.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.883+0000 7f61b6ae5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61b00aebf0 0x7f61b0143fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:25.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.883+0000 7f61b6ae5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61b00b3930 0x7f61b0156410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:25.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.883+0000 7f61b6ae5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b0155ff0 con 0x7f61b006b4e0 2023-12-02T00:29:25.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.883+0000 7f61b6ae5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b0150bb0 con 0x7f61b00aebf0 2023-12-02T00:29:25.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.883+0000 7f61b6ae5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61b0155160 con 0x7f61b00b3930 2023-12-02T00:29:25.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.883+0000 7f61b4881640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61b006b4e0 0x7f61b0154c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:25.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.883+0000 7f61b4881640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61b006b4e0 0x7f61b0154c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52730/0 (socket says 172.21.15.46:52730) 2023-12-02T00:29:25.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.883+0000 7f61b4881640 1 -- 172.21.15.46:0/2812113973 learned_addr learned my addr 172.21.15.46:0/2812113973 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:25.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.884+0000 7f61affff640 1 --2- 172.21.15.46:0/2812113973 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61b00aebf0 0x7f61b0143fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:25.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.884+0000 7f61b4881640 1 -- 172.21.15.46:0/2812113973 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61b00b3930 msgr2=0x7f61b0156410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:25.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.884+0000 7f61b4881640 1 --2- 172.21.15.46:0/2812113973 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61b00b3930 0x7f61b0156410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:25.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.884+0000 7f61b4881640 1 -- 172.21.15.46:0/2812113973 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61b00aebf0 msgr2=0x7f61b0143fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:25.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.884+0000 7f61b4881640 1 --2- 172.21.15.46:0/2812113973 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61b00aebf0 0x7f61b0143fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:25.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.884+0000 7f61b4881640 1 -- 172.21.15.46:0/2812113973 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a4009400 con 0x7f61b006b4e0 2023-12-02T00:29:25.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.884+0000 7f61b4881640 1 --2- 172.21.15.46:0/2812113973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61b006b4e0 0x7f61b0154c20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f61a000d800 tx=0x7f61a000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:25.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.884+0000 7f61ad7fa640 1 -- 172.21.15.46:0/2812113973 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a000b6c0 con 0x7f61b006b4e0 2023-12-02T00:29:25.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.884+0000 7f61ad7fa640 1 -- 172.21.15.46:0/2812113973 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61a000b820 con 0x7f61b006b4e0 2023-12-02T00:29:25.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.884+0000 7f61ad7fa640 1 -- 172.21.15.46:0/2812113973 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a000bbb0 con 0x7f61b006b4e0 2023-12-02T00:29:25.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.884+0000 7f61b6ae5640 1 -- 172.21.15.46:0/2812113973 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61b01444e0 con 0x7f61b006b4e0 2023-12-02T00:29:25.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.884+0000 7f61b6ae5640 1 -- 172.21.15.46:0/2812113973 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61b0155650 con 0x7f61b006b4e0 2023-12-02T00:29:25.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.885+0000 7f61b6ae5640 1 -- 172.21.15.46:0/2812113973 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61b0067420 con 0x7f61b006b4e0 2023-12-02T00:29:25.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.885+0000 7f61ad7fa640 1 -- 172.21.15.46:0/2812113973 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f61a000bd10 con 0x7f61b006b4e0 2023-12-02T00:29:25.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.886+0000 7f61ad7fa640 1 --2- 172.21.15.46:0/2812113973 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f619c06ae40 0x7f619c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:25.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.886+0000 7f61ad7fa640 1 -- 172.21.15.46:0/2812113973 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f61a0089bb0 con 0x7f61b006b4e0 2023-12-02T00:29:25.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.886+0000 7f61affff640 1 --2- 172.21.15.46:0/2812113973 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f619c06ae40 0x7f619c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:25.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.887+0000 7f61affff640 1 --2- 172.21.15.46:0/2812113973 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f619c06ae40 0x7f619c06d300 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f61a4036000 tx=0x7f61a403c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:25.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:25.890+0000 7f61ad7fa640 1 -- 172.21.15.46:0/2812113973 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f61a0054c60 con 0x7f61b006b4e0 2023-12-02T00:29:26.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.065+0000 7f61b6ae5640 1 -- 172.21.15.46:0/2812113973 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f61b006cdd0 con 0x7f61b006b4e0 2023-12-02T00:29:26.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.066+0000 7f61ad7fa640 1 -- 172.21.15.46:0/2812113973 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f61a0058280 con 0x7f61b006b4e0 2023-12-02T00:29:26.069 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:29:26.069 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":21,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:28:50.700112+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-02T00:28:47.273337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-02T00:28:48.060115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:29:26.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.068+0000 7f61b6ae5640 1 -- 172.21.15.46:0/2812113973 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f619c06ae40 msgr2=0x7f619c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:26.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.068+0000 7f61b6ae5640 1 --2- 172.21.15.46:0/2812113973 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f619c06ae40 0x7f619c06d300 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f61a4036000 tx=0x7f61a403c040 comp rx=0 tx=0).stop 2023-12-02T00:29:26.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.068+0000 7f61b6ae5640 1 -- 172.21.15.46:0/2812113973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61b006b4e0 msgr2=0x7f61b0154c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:26.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.068+0000 7f61b6ae5640 1 --2- 172.21.15.46:0/2812113973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61b006b4e0 0x7f61b0154c20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f61a000d800 tx=0x7f61a000dcc0 comp rx=0 tx=0).stop 2023-12-02T00:29:26.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.068+0000 7f61b6ae5640 1 -- 172.21.15.46:0/2812113973 shutdown_connections 2023-12-02T00:29:26.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.068+0000 7f61b6ae5640 1 --2- 172.21.15.46:0/2812113973 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f619c06ae40 0x7f619c06d300 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:26.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.068+0000 7f61b6ae5640 1 --2- 172.21.15.46:0/2812113973 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61b00b3930 0x7f61b0156410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:26.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.068+0000 7f61b6ae5640 1 --2- 172.21.15.46:0/2812113973 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61b00aebf0 0x7f61b0143fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:26.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.068+0000 7f61b6ae5640 1 --2- 172.21.15.46:0/2812113973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61b006b4e0 0x7f61b0154c20 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:26.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.068+0000 7f61b6ae5640 1 -- 172.21.15.46:0/2812113973 >> 172.21.15.46:0/2812113973 conn(0x7f61b0070ec0 msgr2=0x7f61b0153b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:26.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.068+0000 7f61b6ae5640 1 -- 172.21.15.46:0/2812113973 shutdown_connections 2023-12-02T00:29:26.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.068+0000 7f61b6ae5640 1 -- 172.21.15.46:0/2812113973 wait complete. 2023-12-02T00:29:26.081 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-02T00:28:37.213645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-02T00:28:47.273337+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-02T00:28:48.060115+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-02T00:29:26.082 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-02T00:29:26.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.190+0000 7f9b8659e640 1 Processor -- start 2023-12-02T00:29:26.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.190+0000 7f9b8659e640 1 -- start start 2023-12-02T00:29:26.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.191+0000 7f9b8659e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9b8015d1b0 0x7f9b800c92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:26.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.191+0000 7f9b8659e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9b800c7b00 0x7f9b800c7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:26.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.191+0000 7f9b8659e640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b800c8490 con 0x7f9b800d14f0 2023-12-02T00:29:26.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.191+0000 7f9b8659e640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b800d1a20 con 0x7f9b800c7b00 2023-12-02T00:29:26.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.191+0000 7f9b8659e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b800d1d00 con 0x7f9b8015d1b0 2023-12-02T00:29:26.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.191+0000 7f9b85d9d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9b800c7b00 0x7f9b800c7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:26.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.191+0000 7f9b84d9b640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9b800d14f0 0x7f9b800ce5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50410/0 (socket says 172.21.15.46:50410) 2023-12-02T00:29:26.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.191+0000 7f9b85d9d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9b800c7b00 0x7f9b800c7f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52736/0 (socket says 172.21.15.46:52736) 2023-12-02T00:29:26.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.191+0000 7f9b84d9b640 1 -- 172.21.15.46:0/962669674 learned_addr learned my addr 172.21.15.46:0/962669674 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:26.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.191+0000 7f9b8559c640 1 --2- 172.21.15.46:0/962669674 >> v2:172.21.15.162:3300/0 conn(0x7f9b8015d1b0 0x7f9b800c92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:26.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.191+0000 7f9b85d9d640 1 -- 172.21.15.46:0/962669674 >> v2:172.21.15.162:3300/0 conn(0x7f9b8015d1b0 msgr2=0x7f9b800c92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:26.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.191+0000 7f9b85d9d640 1 --2- 172.21.15.46:0/962669674 >> v2:172.21.15.162:3300/0 conn(0x7f9b8015d1b0 0x7f9b800c92f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:26.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.191+0000 7f9b85d9d640 1 -- 172.21.15.46:0/962669674 >> v1:172.21.15.46:6789/0 conn(0x7f9b800d14f0 legacy=0x7f9b800ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:26.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.191+0000 7f9b85d9d640 1 -- 172.21.15.46:0/962669674 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b800cddb0 con 0x7f9b800c7b00 2023-12-02T00:29:26.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.191+0000 7f9b7ffff640 1 -- 172.21.15.46:0/962669674 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 726038295 0 0) 0x7f9b74001110 con 0x7f9b800d14f0 2023-12-02T00:29:26.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.192+0000 7f9b85d9d640 1 --2- 172.21.15.46:0/962669674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b800c7b00 0x7f9b800c7f50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9b8008d6e0 tx=0x7f9b7002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d445cc6213c6a48e server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:26.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.192+0000 7f9b7ffff640 1 -- 172.21.15.46:0/962669674 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7002cbf0 con 0x7f9b800c7b00 2023-12-02T00:29:26.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.192+0000 7f9b7ffff640 1 -- 172.21.15.46:0/962669674 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b7002cd50 con 0x7f9b800c7b00 2023-12-02T00:29:26.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.192+0000 7f9b7ffff640 1 -- 172.21.15.46:0/962669674 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b70034660 con 0x7f9b800c7b00 2023-12-02T00:29:26.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.192+0000 7f9b8659e640 1 -- 172.21.15.46:0/962669674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b800c7b00 msgr2=0x7f9b800c7f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:26.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.192+0000 7f9b8659e640 1 --2- 172.21.15.46:0/962669674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b800c7b00 0x7f9b800c7f50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9b8008d6e0 tx=0x7f9b7002be40 comp rx=0 tx=0).stop 2023-12-02T00:29:26.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.193+0000 7f9b8659e640 1 -- 172.21.15.46:0/962669674 shutdown_connections 2023-12-02T00:29:26.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.193+0000 7f9b8659e640 1 --2- 172.21.15.46:0/962669674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b800c7b00 0x7f9b800c7f50 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:26.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.193+0000 7f9b8659e640 1 --2- 172.21.15.46:0/962669674 >> v2:172.21.15.162:3300/0 conn(0x7f9b8015d1b0 0x7f9b800c92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:26.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.193+0000 7f9b8659e640 1 -- 172.21.15.46:0/962669674 >> 172.21.15.46:0/962669674 conn(0x7f9b8009c300 msgr2=0x7f9b800d2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:26.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.193+0000 7f9b8659e640 1 -- 172.21.15.46:0/962669674 shutdown_connections 2023-12-02T00:29:26.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.193+0000 7f9b8659e640 1 -- 172.21.15.46:0/962669674 wait complete. 2023-12-02T00:29:26.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.194+0000 7f9b8659e640 1 Processor -- start 2023-12-02T00:29:26.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.194+0000 7f9b8659e640 1 -- start start 2023-12-02T00:29:26.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.195+0000 7f9b8659e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b800c7b00 0x7f9b8015b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:26.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.195+0000 7f9b8659e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b800d14f0 0x7f9b800b1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:26.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.195+0000 7f9b8659e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b8015d1b0 0x7f9b8006c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:26.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.195+0000 7f9b8659e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b801f39e0 con 0x7f9b800d14f0 2023-12-02T00:29:26.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.195+0000 7f9b8659e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b801f3700 con 0x7f9b800c7b00 2023-12-02T00:29:26.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.195+0000 7f9b8659e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8015e0c0 con 0x7f9b8015d1b0 2023-12-02T00:29:26.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.195+0000 7f9b84d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b800d14f0 0x7f9b800b1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:26.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.195+0000 7f9b84d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b800d14f0 0x7f9b800b1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52748/0 (socket says 172.21.15.46:52748) 2023-12-02T00:29:26.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.195+0000 7f9b84d9b640 1 -- 172.21.15.46:0/1409878137 learned_addr learned my addr 172.21.15.46:0/1409878137 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:26.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.195+0000 7f9b85d9d640 1 --2- 172.21.15.46:0/1409878137 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b8015d1b0 0x7f9b8006c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:26.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.195+0000 7f9b8559c640 1 --2- 172.21.15.46:0/1409878137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b800c7b00 0x7f9b8015b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:26.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.195+0000 7f9b84d9b640 1 -- 172.21.15.46:0/1409878137 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b8015d1b0 msgr2=0x7f9b8006c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:26.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.195+0000 7f9b84d9b640 1 --2- 172.21.15.46:0/1409878137 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b8015d1b0 0x7f9b8006c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:26.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.195+0000 7f9b84d9b640 1 -- 172.21.15.46:0/1409878137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b800c7b00 msgr2=0x7f9b8015b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:26.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.195+0000 7f9b84d9b640 1 --2- 172.21.15.46:0/1409878137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b800c7b00 0x7f9b8015b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:26.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.195+0000 7f9b84d9b640 1 -- 172.21.15.46:0/1409878137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b70009a70 con 0x7f9b800d14f0 2023-12-02T00:29:26.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.195+0000 7f9b85d9d640 1 --2- 172.21.15.46:0/1409878137 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b8015d1b0 0x7f9b8006c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:26.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.195+0000 7f9b8559c640 1 --2- 172.21.15.46:0/1409878137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b800c7b00 0x7f9b8015b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:26.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.195+0000 7f9b84d9b640 1 --2- 172.21.15.46:0/1409878137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b800d14f0 0x7f9b800b1e10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9b7400c780 tx=0x7f9b7400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:26.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.195+0000 7f9b7dffb640 1 -- 172.21.15.46:0/1409878137 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b740072d0 con 0x7f9b800d14f0 2023-12-02T00:29:26.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.196+0000 7f9b7dffb640 1 -- 172.21.15.46:0/1409878137 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b74007430 con 0x7f9b800d14f0 2023-12-02T00:29:26.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.196+0000 7f9b8659e640 1 -- 172.21.15.46:0/1409878137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b8015e3a0 con 0x7f9b800d14f0 2023-12-02T00:29:26.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.196+0000 7f9b7dffb640 1 -- 172.21.15.46:0/1409878137 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b74001d10 con 0x7f9b800d14f0 2023-12-02T00:29:26.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.196+0000 7f9b8659e640 1 -- 172.21.15.46:0/1409878137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b800b2350 con 0x7f9b800d14f0 2023-12-02T00:29:26.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.197+0000 7f9b8659e640 1 -- 172.21.15.46:0/1409878137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b8015e8c0 con 0x7f9b800d14f0 2023-12-02T00:29:26.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.197+0000 7f9b7dffb640 1 -- 172.21.15.46:0/1409878137 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9b740047e0 con 0x7f9b800d14f0 2023-12-02T00:29:26.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.197+0000 7f9b7dffb640 1 --2- 172.21.15.46:0/1409878137 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9b6c06ae40 0x7f9b6c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:26.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.197+0000 7f9b7dffb640 1 -- 172.21.15.46:0/1409878137 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9b74088950 con 0x7f9b800d14f0 2023-12-02T00:29:26.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.198+0000 7f9b8559c640 1 --2- 172.21.15.46:0/1409878137 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9b6c06ae40 0x7f9b6c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:26.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.198+0000 7f9b8559c640 1 --2- 172.21.15.46:0/1409878137 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9b6c06ae40 0x7f9b6c06d300 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9b78006fd0 tx=0x7f9b78008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:26.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.202+0000 7f9b7dffb640 1 -- 172.21.15.46:0/1409878137 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9b8015e8c0 con 0x7f9b800d14f0 2023-12-02T00:29:26.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.377+0000 7f9b8659e640 1 -- 172.21.15.46:0/1409878137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9b800c8490 con 0x7f9b800d14f0 2023-12-02T00:29:26.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.378+0000 7f9b7dffb640 1 -- 172.21.15.46:0/1409878137 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9b800c8490 con 0x7f9b800d14f0 2023-12-02T00:29:26.380 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-12-02T00:29:26.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.379+0000 7f9b8659e640 1 -- 172.21.15.46:0/1409878137 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9b6c06ae40 msgr2=0x7f9b6c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:26.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.379+0000 7f9b8659e640 1 --2- 172.21.15.46:0/1409878137 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9b6c06ae40 0x7f9b6c06d300 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9b78006fd0 tx=0x7f9b78008040 comp rx=0 tx=0).stop 2023-12-02T00:29:26.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.379+0000 7f9b8659e640 1 -- 172.21.15.46:0/1409878137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b800d14f0 msgr2=0x7f9b800b1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:26.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.379+0000 7f9b8659e640 1 --2- 172.21.15.46:0/1409878137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b800d14f0 0x7f9b800b1e10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9b7400c780 tx=0x7f9b7400cc40 comp rx=0 tx=0).stop 2023-12-02T00:29:26.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.379+0000 7f9b8659e640 1 -- 172.21.15.46:0/1409878137 shutdown_connections 2023-12-02T00:29:26.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.379+0000 7f9b8659e640 1 --2- 172.21.15.46:0/1409878137 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9b6c06ae40 0x7f9b6c06d300 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:26.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.379+0000 7f9b8659e640 1 --2- 172.21.15.46:0/1409878137 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b8015d1b0 0x7f9b8006c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:26.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.379+0000 7f9b8659e640 1 --2- 172.21.15.46:0/1409878137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b800d14f0 0x7f9b800b1e10 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:26.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.379+0000 7f9b8659e640 1 --2- 172.21.15.46:0/1409878137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b800c7b00 0x7f9b8015b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:26.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.379+0000 7f9b8659e640 1 -- 172.21.15.46:0/1409878137 >> 172.21.15.46:0/1409878137 conn(0x7f9b8009c300 msgr2=0x7f9b800cf540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:26.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.380+0000 7f9b8659e640 1 -- 172.21.15.46:0/1409878137 shutdown_connections 2023-12-02T00:29:26.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.380+0000 7f9b8659e640 1 -- 172.21.15.46:0/1409878137 wait complete. 2023-12-02T00:29:26.393 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-02T00:29:26.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.501+0000 7fb9a54fc640 1 Processor -- start 2023-12-02T00:29:26.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.501+0000 7fb9a54fc640 1 -- start start 2023-12-02T00:29:26.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.502+0000 7fb9a54fc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9a00cf320 0x7fb9a00d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:26.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.502+0000 7fb9a54fc640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9a00b3f10 con 0x7fb9a015b130 2023-12-02T00:29:26.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.502+0000 7fb9a54fc640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9a00d0a50 con 0x7fb9a015f760 2023-12-02T00:29:26.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.502+0000 7fb9a54fc640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a015fd20 con 0x7fb9a00cf320 2023-12-02T00:29:26.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.502+0000 7fb99effd640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb9a015b130 0x7fb9a00cec10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50412/0 (socket says 172.21.15.46:50412) 2023-12-02T00:29:26.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.502+0000 7fb99effd640 1 -- 172.21.15.46:0/3831091436 learned_addr learned my addr 172.21.15.46:0/3831091436 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:26.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.502+0000 7fb99dffb640 1 -- 172.21.15.46:0/3831091436 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2093941775 0 0) 0x7fb9a00b3f10 con 0x7fb9a015b130 2023-12-02T00:29:26.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.503+0000 7fb99e7fc640 1 --2- 172.21.15.46:0/3831091436 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9a00cf320 0x7fb9a00d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:26.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.503+0000 7fb99dffb640 1 -- 172.21.15.46:0/3831091436 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb98c003520 con 0x7fb9a015b130 2023-12-02T00:29:26.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.503+0000 7fb99dffb640 1 -- 172.21.15.46:0/3831091436 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3036870362 0 0) 0x7fb98c003520 con 0x7fb9a015b130 2023-12-02T00:29:26.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.503+0000 7fb99dffb640 1 -- 172.21.15.46:0/3831091436 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9a00cf320 msgr2=0x7fb9a00d01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:26.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.503+0000 7fb99dffb640 1 --2- 172.21.15.46:0/3831091436 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9a00cf320 0x7fb9a00d01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:26.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.503+0000 7fb99dffb640 1 -- 172.21.15.46:0/3831091436 >> v1:172.21.15.162:6789/0 conn(0x7fb9a015f760 legacy=0x7fb9a00cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:26.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.503+0000 7fb99dffb640 1 -- 172.21.15.46:0/3831091436 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9a00d0d30 con 0x7fb9a015b130 2023-12-02T00:29:26.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.503+0000 7fb99dffb640 1 -- 172.21.15.46:0/3831091436 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7fb9900019f0 con 0x7fb9a015b130 2023-12-02T00:29:26.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.503+0000 7fb99dffb640 1 -- 172.21.15.46:0/3831091436 >> v1:172.21.15.46:6789/0 conn(0x7fb9a015b130 legacy=0x7fb9a00cec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:26.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.503+0000 7fb99dffb640 1 --2- 172.21.15.46:0/3831091436 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9a00cf320 0x7fb98c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:26.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.503+0000 7fb99dffb640 1 --2- 172.21.15.46:0/3831091436 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb98c003f20 0x7fb98c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:26.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.503+0000 7fb99dffb640 1 --2- 172.21.15.46:0/3831091436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb98c004d00 0x7fb98c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:26.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.503+0000 7fb99dffb640 1 -- 172.21.15.46:0/3831091436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a015fd20 con 0x7fb98c004d00 2023-12-02T00:29:26.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.503+0000 7fb99dffb640 1 -- 172.21.15.46:0/3831091436 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb98c00ba60 con 0x7fb98c003f20 2023-12-02T00:29:26.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.503+0000 7fb99dffb640 1 -- 172.21.15.46:0/3831091436 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb98c00bc20 con 0x7fb9a00cf320 2023-12-02T00:29:26.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.503+0000 7fb99f7fe640 1 --2- 172.21.15.46:0/3831091436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb98c004d00 0x7fb98c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:26.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.503+0000 7fb99effd640 1 --2- 172.21.15.46:0/3831091436 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9a00cf320 0x7fb98c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:26.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.503+0000 7fb99e7fc640 1 --2- 172.21.15.46:0/3831091436 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb98c003f20 0x7fb98c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:26.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.503+0000 7fb99f7fe640 1 -- 172.21.15.46:0/3831091436 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9a00cf320 msgr2=0x7fb98c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:26.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.503+0000 7fb99f7fe640 1 --2- 172.21.15.46:0/3831091436 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9a00cf320 0x7fb98c0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:26.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.503+0000 7fb99f7fe640 1 -- 172.21.15.46:0/3831091436 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb98c003f20 msgr2=0x7fb98c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:26.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.503+0000 7fb99f7fe640 1 --2- 172.21.15.46:0/3831091436 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb98c003f20 0x7fb98c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:26.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.503+0000 7fb99f7fe640 1 -- 172.21.15.46:0/3831091436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb98c003520 con 0x7fb98c004d00 2023-12-02T00:29:26.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.503+0000 7fb99e7fc640 1 --2- 172.21.15.46:0/3831091436 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb98c003f20 0x7fb98c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:26.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.503+0000 7fb99effd640 1 --2- 172.21.15.46:0/3831091436 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9a00cf320 0x7fb98c0039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:26.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.504+0000 7fb99f7fe640 1 --2- 172.21.15.46:0/3831091436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb98c004d00 0x7fb98c005180 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb998001750 tx=0x7fb98c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=410ee4756d373226 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:26.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.504+0000 7fb99dffb640 1 -- 172.21.15.46:0/3831091436 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9980052d0 con 0x7fb98c004d00 2023-12-02T00:29:26.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.504+0000 7fb99dffb640 1 -- 172.21.15.46:0/3831091436 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb99802e050 con 0x7fb98c004d00 2023-12-02T00:29:26.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.504+0000 7fb99dffb640 1 -- 172.21.15.46:0/3831091436 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb998003830 con 0x7fb98c004d00 2023-12-02T00:29:26.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.505+0000 7fb9a54fc640 1 -- 172.21.15.46:0/3831091436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb98c004d00 msgr2=0x7fb98c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:26.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.505+0000 7fb9a54fc640 1 --2- 172.21.15.46:0/3831091436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb98c004d00 0x7fb98c005180 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb998001750 tx=0x7fb98c0057f0 comp rx=0 tx=0).stop 2023-12-02T00:29:26.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.505+0000 7fb99f7fe640 1 -- 172.21.15.46:0/3831091436 reap_dead start 2023-12-02T00:29:26.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.505+0000 7fb9a54fc640 1 -- 172.21.15.46:0/3831091436 shutdown_connections 2023-12-02T00:29:26.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.505+0000 7fb9a54fc640 1 -- 172.21.15.46:0/3831091436 >> 172.21.15.46:0/3831091436 conn(0x7fb9a00b2c10 msgr2=0x7fb9a00b2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:26.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.505+0000 7fb9a54fc640 1 -- 172.21.15.46:0/3831091436 shutdown_connections 2023-12-02T00:29:26.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.505+0000 7fb9a54fc640 1 -- 172.21.15.46:0/3831091436 wait complete. 2023-12-02T00:29:26.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.506+0000 7fb9a54fc640 1 Processor -- start 2023-12-02T00:29:26.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.506+0000 7fb9a54fc640 1 -- start start 2023-12-02T00:29:26.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.507+0000 7fb9a54fc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb98c004d00 0x7fb9a0152880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:26.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.507+0000 7fb9a54fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9a006d0d0 0x7fb9a006b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:26.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.507+0000 7fb9a54fc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9a006bb20 0x7fb9a00b5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:26.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.507+0000 7fb9a54fc640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a006c030 con 0x7fb9a006d0d0 2023-12-02T00:29:26.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.507+0000 7fb9a54fc640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a006c310 con 0x7fb98c004d00 2023-12-02T00:29:26.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.507+0000 7fb9a54fc640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a006c5f0 con 0x7fb9a006bb20 2023-12-02T00:29:26.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.507+0000 7fb99e7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9a006d0d0 0x7fb9a006b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:26.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.507+0000 7fb99e7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9a006d0d0 0x7fb9a006b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52764/0 (socket says 172.21.15.46:52764) 2023-12-02T00:29:26.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.507+0000 7fb99e7fc640 1 -- 172.21.15.46:0/3271332042 learned_addr learned my addr 172.21.15.46:0/3271332042 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:26.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.507+0000 7fb99effd640 1 --2- 172.21.15.46:0/3271332042 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb98c004d00 0x7fb9a0152880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:26.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.507+0000 7fb99e7fc640 1 -- 172.21.15.46:0/3271332042 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9a006bb20 msgr2=0x7fb9a00b5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:26.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.507+0000 7fb99e7fc640 1 --2- 172.21.15.46:0/3271332042 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9a006bb20 0x7fb9a00b5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:26.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.507+0000 7fb99e7fc640 1 -- 172.21.15.46:0/3271332042 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb98c004d00 msgr2=0x7fb9a0152880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:26.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.507+0000 7fb99e7fc640 1 --2- 172.21.15.46:0/3271332042 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb98c004d00 0x7fb9a0152880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:26.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.507+0000 7fb99e7fc640 1 -- 172.21.15.46:0/3271332042 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9a006c8d0 con 0x7fb9a006d0d0 2023-12-02T00:29:26.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.507+0000 7fb99effd640 1 --2- 172.21.15.46:0/3271332042 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb98c004d00 0x7fb9a0152880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:29:26.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.507+0000 7fb99e7fc640 1 --2- 172.21.15.46:0/3271332042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9a006d0d0 0x7fb9a006b5e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb988011580 tx=0x7fb988011a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:26.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.508+0000 7fb9977fe640 1 -- 172.21.15.46:0/3271332042 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb988015870 con 0x7fb9a006d0d0 2023-12-02T00:29:26.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.508+0000 7fb9977fe640 1 -- 172.21.15.46:0/3271332042 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9880159d0 con 0x7fb9a006d0d0 2023-12-02T00:29:26.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.508+0000 7fb9a54fc640 1 -- 172.21.15.46:0/3271332042 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9a006cbb0 con 0x7fb9a006d0d0 2023-12-02T00:29:26.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.508+0000 7fb9977fe640 1 -- 172.21.15.46:0/3271332042 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb988015d60 con 0x7fb9a006d0d0 2023-12-02T00:29:26.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.508+0000 7fb9a54fc640 1 -- 172.21.15.46:0/3271332042 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9a00b6150 con 0x7fb9a006d0d0 2023-12-02T00:29:26.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.509+0000 7fb9a54fc640 1 -- 172.21.15.46:0/3271332042 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9a0067fc0 con 0x7fb9a006d0d0 2023-12-02T00:29:26.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.509+0000 7fb9977fe640 1 -- 172.21.15.46:0/3271332042 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb98800f460 con 0x7fb9a006d0d0 2023-12-02T00:29:26.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.510+0000 7fb9977fe640 1 --2- 172.21.15.46:0/3271332042 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb98006ae40 0x7fb98006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:26.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.510+0000 7fb9977fe640 1 -- 172.21.15.46:0/3271332042 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb988089eb0 con 0x7fb9a006d0d0 2023-12-02T00:29:26.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.510+0000 7fb99effd640 1 --2- 172.21.15.46:0/3271332042 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb98006ae40 0x7fb98006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:26.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.510+0000 7fb99effd640 1 --2- 172.21.15.46:0/3271332042 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb98006ae40 0x7fb98006d300 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb990001330 tx=0x7fb990001510 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:26.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.514+0000 7fb9977fe640 1 -- 172.21.15.46:0/3271332042 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb988054ed0 con 0x7fb9a006d0d0 2023-12-02T00:29:26.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.690+0000 7fb9a54fc640 1 -- 172.21.15.46:0/3271332042 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb9a0067420 con 0x7fb9a006d0d0 2023-12-02T00:29:26.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.690+0000 7fb9977fe640 1 -- 172.21.15.46:0/3271332042 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb9880584f0 con 0x7fb9a006d0d0 2023-12-02T00:29:26.692 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-12-02T00:29:26.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.692+0000 7fb9a54fc640 1 -- 172.21.15.46:0/3271332042 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb98006ae40 msgr2=0x7fb98006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:26.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.692+0000 7fb9a54fc640 1 --2- 172.21.15.46:0/3271332042 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb98006ae40 0x7fb98006d300 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb990001330 tx=0x7fb990001510 comp rx=0 tx=0).stop 2023-12-02T00:29:26.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.692+0000 7fb9a54fc640 1 -- 172.21.15.46:0/3271332042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9a006d0d0 msgr2=0x7fb9a006b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:26.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.692+0000 7fb9a54fc640 1 --2- 172.21.15.46:0/3271332042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9a006d0d0 0x7fb9a006b5e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb988011580 tx=0x7fb988011a40 comp rx=0 tx=0).stop 2023-12-02T00:29:26.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.692+0000 7fb9a54fc640 1 -- 172.21.15.46:0/3271332042 shutdown_connections 2023-12-02T00:29:26.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.692+0000 7fb9a54fc640 1 --2- 172.21.15.46:0/3271332042 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb98006ae40 0x7fb98006d300 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:26.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.692+0000 7fb9a54fc640 1 --2- 172.21.15.46:0/3271332042 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9a006bb20 0x7fb9a00b5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:26.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.692+0000 7fb9a54fc640 1 --2- 172.21.15.46:0/3271332042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9a006d0d0 0x7fb9a006b5e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:26.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.692+0000 7fb9a54fc640 1 --2- 172.21.15.46:0/3271332042 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb98c004d00 0x7fb9a0152880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:26.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.692+0000 7fb9a54fc640 1 -- 172.21.15.46:0/3271332042 >> 172.21.15.46:0/3271332042 conn(0x7fb9a00b2c10 msgr2=0x7fb9a00b47d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:26.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.692+0000 7fb9a54fc640 1 -- 172.21.15.46:0/3271332042 shutdown_connections 2023-12-02T00:29:26.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.692+0000 7fb9a54fc640 1 -- 172.21.15.46:0/3271332042 wait complete. 2023-12-02T00:29:26.705 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-02T00:29:26.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.812+0000 7f8a7503d640 1 Processor -- start 2023-12-02T00:29:26.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.812+0000 7f8a7503d640 1 -- start start 2023-12-02T00:29:26.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.812+0000 7f8a7503d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8a7006ea50 0x7f8a7006ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:26.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.812+0000 7f8a7503d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8a700d0ed0 0x7f8a700d1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:26.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.812+0000 7f8a7503d640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a70161d20 con 0x7f8a70160fb0 2023-12-02T00:29:26.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.812+0000 7f8a7503d640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a7006f370 con 0x7f8a700d0ed0 2023-12-02T00:29:26.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.812+0000 7f8a7503d640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a7015e760 con 0x7f8a7006ea50 2023-12-02T00:29:26.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.813+0000 7f8a6f59e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8a700d0ed0 0x7f8a700d1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:26.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.813+0000 7f8a6f59e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8a700d0ed0 0x7f8a700d1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52772/0 (socket says 172.21.15.46:52772) 2023-12-02T00:29:26.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.813+0000 7f8a6f59e640 1 -- 172.21.15.46:0/1053363419 learned_addr learned my addr 172.21.15.46:0/1053363419 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:26.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.813+0000 7f8a6ed9d640 1 --2- 172.21.15.46:0/1053363419 >> v2:172.21.15.162:3300/0 conn(0x7f8a7006ea50 0x7f8a7006ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:26.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.813+0000 7f8a6f59e640 1 -- 172.21.15.46:0/1053363419 >> v2:172.21.15.162:3300/0 conn(0x7f8a7006ea50 msgr2=0x7f8a7006ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:26.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.813+0000 7f8a6f59e640 1 --2- 172.21.15.46:0/1053363419 >> v2:172.21.15.162:3300/0 conn(0x7f8a7006ea50 0x7f8a7006ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:26.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.813+0000 7f8a6f59e640 1 -- 172.21.15.46:0/1053363419 >> v1:172.21.15.162:6789/0 conn(0x7f8a70160fb0 legacy=0x7f8a70161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:26.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.813+0000 7f8a6f59e640 1 -- 172.21.15.46:0/1053363419 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a7015d9a0 con 0x7f8a700d0ed0 2023-12-02T00:29:26.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.814+0000 7f8a6f59e640 1 --2- 172.21.15.46:0/1053363419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a700d0ed0 0x7f8a700d1320 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8a7006f680 tx=0x7f8a6802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8fee6672998ff7e server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:26.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.814+0000 7f8a6dd9b640 1 -- 172.21.15.46:0/1053363419 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6802ca80 con 0x7f8a700d0ed0 2023-12-02T00:29:26.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.814+0000 7f8a6dd9b640 1 -- 172.21.15.46:0/1053363419 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a6802cbe0 con 0x7f8a700d0ed0 2023-12-02T00:29:26.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.814+0000 7f8a6dd9b640 1 -- 172.21.15.46:0/1053363419 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a680343f0 con 0x7f8a700d0ed0 2023-12-02T00:29:26.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.814+0000 7f8a7503d640 1 -- 172.21.15.46:0/1053363419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a700d0ed0 msgr2=0x7f8a700d1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:26.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.814+0000 7f8a7503d640 1 --2- 172.21.15.46:0/1053363419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a700d0ed0 0x7f8a700d1320 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8a7006f680 tx=0x7f8a6802bee0 comp rx=0 tx=0).stop 2023-12-02T00:29:26.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.814+0000 7f8a7503d640 1 -- 172.21.15.46:0/1053363419 shutdown_connections 2023-12-02T00:29:26.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.814+0000 7f8a7503d640 1 --2- 172.21.15.46:0/1053363419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a700d0ed0 0x7f8a700d1320 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:26.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.814+0000 7f8a7503d640 1 --2- 172.21.15.46:0/1053363419 >> v2:172.21.15.162:3300/0 conn(0x7f8a7006ea50 0x7f8a7006ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:26.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.814+0000 7f8a7503d640 1 -- 172.21.15.46:0/1053363419 >> 172.21.15.46:0/1053363419 conn(0x7f8a7006a6b0 msgr2=0x7f8a7006aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:26.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.814+0000 7f8a7503d640 1 -- 172.21.15.46:0/1053363419 shutdown_connections 2023-12-02T00:29:26.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.815+0000 7f8a7503d640 1 -- 172.21.15.46:0/1053363419 wait complete. 2023-12-02T00:29:26.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.816+0000 7f8a7503d640 1 Processor -- start 2023-12-02T00:29:26.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.816+0000 7f8a7503d640 1 -- start start 2023-12-02T00:29:26.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.817+0000 7f8a7503d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a7006ea50 0x7f8a7013d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:26.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.817+0000 7f8a7503d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8a700d0ed0 0x7f8a7013f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:26.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.817+0000 7f8a7503d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a70160fb0 0x7f8a70156a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:26.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.817+0000 7f8a7503d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a70156f40 con 0x7f8a7006ea50 2023-12-02T00:29:26.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.817+0000 7f8a7503d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a70159b00 con 0x7f8a70160fb0 2023-12-02T00:29:26.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.817+0000 7f8a7503d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7013dc90 con 0x7f8a700d0ed0 2023-12-02T00:29:26.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.817+0000 7f8a6ed9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a7006ea50 0x7f8a7013d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:26.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.817+0000 7f8a6ed9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a7006ea50 0x7f8a7013d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52774/0 (socket says 172.21.15.46:52774) 2023-12-02T00:29:26.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.817+0000 7f8a6ed9d640 1 -- 172.21.15.46:0/2819043726 learned_addr learned my addr 172.21.15.46:0/2819043726 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:26.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.817+0000 7f8a6e59c640 1 --2- 172.21.15.46:0/2819043726 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8a700d0ed0 0x7f8a7013f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:26.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.817+0000 7f8a6f59e640 1 --2- 172.21.15.46:0/2819043726 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a70160fb0 0x7f8a70156a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:26.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.817+0000 7f8a6ed9d640 1 -- 172.21.15.46:0/2819043726 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8a700d0ed0 msgr2=0x7f8a7013f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:26.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.817+0000 7f8a6ed9d640 1 --2- 172.21.15.46:0/2819043726 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8a700d0ed0 0x7f8a7013f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:26.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.817+0000 7f8a6ed9d640 1 -- 172.21.15.46:0/2819043726 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a70160fb0 msgr2=0x7f8a70156a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:26.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.817+0000 7f8a6ed9d640 1 --2- 172.21.15.46:0/2819043726 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a70160fb0 0x7f8a70156a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:26.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.817+0000 7f8a6ed9d640 1 -- 172.21.15.46:0/2819043726 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a68009b30 con 0x7f8a7006ea50 2023-12-02T00:29:26.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.817+0000 7f8a6f59e640 1 --2- 172.21.15.46:0/2819043726 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a70160fb0 0x7f8a70156a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:26.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.817+0000 7f8a6ed9d640 1 --2- 172.21.15.46:0/2819043726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a7006ea50 0x7f8a7013d6b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000e870 tx=0x7f8a6000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:26.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.817+0000 7f8a577fe640 1 -- 172.21.15.46:0/2819043726 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a600094f0 con 0x7f8a7006ea50 2023-12-02T00:29:26.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.818+0000 7f8a577fe640 1 -- 172.21.15.46:0/2819043726 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a6000c940 con 0x7f8a7006ea50 2023-12-02T00:29:26.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.818+0000 7f8a7503d640 1 -- 172.21.15.46:0/2819043726 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a7013df70 con 0x7f8a7006ea50 2023-12-02T00:29:26.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.818+0000 7f8a577fe640 1 -- 172.21.15.46:0/2819043726 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6000ccf0 con 0x7f8a7006ea50 2023-12-02T00:29:26.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.818+0000 7f8a7503d640 1 -- 172.21.15.46:0/2819043726 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a7013e530 con 0x7f8a7006ea50 2023-12-02T00:29:26.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.818+0000 7f8a7503d640 1 -- 172.21.15.46:0/2819043726 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a70070ac0 con 0x7f8a7006ea50 2023-12-02T00:29:26.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.819+0000 7f8a577fe640 1 -- 172.21.15.46:0/2819043726 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8a60013470 con 0x7f8a7006ea50 2023-12-02T00:29:26.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.819+0000 7f8a577fe640 1 --2- 172.21.15.46:0/2819043726 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8a5c06ae40 0x7f8a5c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:26.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.820+0000 7f8a577fe640 1 -- 172.21.15.46:0/2819043726 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8a60089710 con 0x7f8a7006ea50 2023-12-02T00:29:26.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.820+0000 7f8a6e59c640 1 --2- 172.21.15.46:0/2819043726 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8a5c06ae40 0x7f8a5c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:26.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.820+0000 7f8a6e59c640 1 --2- 172.21.15.46:0/2819043726 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8a5c06ae40 0x7f8a5c06d300 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8a701f62e0 tx=0x7f8a64009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:26.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:26.824+0000 7f8a577fe640 1 -- 172.21.15.46:0/2819043726 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8a600547e0 con 0x7f8a7006ea50 2023-12-02T00:29:27.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.002+0000 7f8a7503d640 1 -- 172.21.15.46:0/2819043726 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8a70069400 con 0x7f8a7006ea50 2023-12-02T00:29:27.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.002+0000 7f8a577fe640 1 -- 172.21.15.46:0/2819043726 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8a60057e00 con 0x7f8a7006ea50 2023-12-02T00:29:27.004 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-12-02T00:29:27.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.004+0000 7f8a7503d640 1 -- 172.21.15.46:0/2819043726 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8a5c06ae40 msgr2=0x7f8a5c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:27.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.004+0000 7f8a7503d640 1 --2- 172.21.15.46:0/2819043726 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8a5c06ae40 0x7f8a5c06d300 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8a701f62e0 tx=0x7f8a64009040 comp rx=0 tx=0).stop 2023-12-02T00:29:27.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.004+0000 7f8a7503d640 1 -- 172.21.15.46:0/2819043726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a7006ea50 msgr2=0x7f8a7013d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:27.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.004+0000 7f8a7503d640 1 --2- 172.21.15.46:0/2819043726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a7006ea50 0x7f8a7013d6b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000e870 tx=0x7f8a6000ed30 comp rx=0 tx=0).stop 2023-12-02T00:29:27.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.004+0000 7f8a7503d640 1 -- 172.21.15.46:0/2819043726 shutdown_connections 2023-12-02T00:29:27.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.004+0000 7f8a7503d640 1 --2- 172.21.15.46:0/2819043726 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8a5c06ae40 0x7f8a5c06d300 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:27.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.004+0000 7f8a7503d640 1 --2- 172.21.15.46:0/2819043726 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a70160fb0 0x7f8a70156a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:27.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.004+0000 7f8a7503d640 1 --2- 172.21.15.46:0/2819043726 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8a700d0ed0 0x7f8a7013f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:27.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.004+0000 7f8a7503d640 1 --2- 172.21.15.46:0/2819043726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a7006ea50 0x7f8a7013d6b0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:27.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.004+0000 7f8a7503d640 1 -- 172.21.15.46:0/2819043726 >> 172.21.15.46:0/2819043726 conn(0x7f8a7006a6b0 msgr2=0x7f8a7006aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:27.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.004+0000 7f8a7503d640 1 -- 172.21.15.46:0/2819043726 shutdown_connections 2023-12-02T00:29:27.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.004+0000 7f8a7503d640 1 -- 172.21.15.46:0/2819043726 wait complete. 2023-12-02T00:29:27.017 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-02T00:29:27.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.122+0000 7f416e675640 1 Processor -- start 2023-12-02T00:29:27.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.123+0000 7f416e675640 1 -- start start 2023-12-02T00:29:27.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.123+0000 7f416e675640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f41680d2910 0x7f41680d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:27.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.123+0000 7f416e675640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41680b2b10 con 0x7f416815d330 2023-12-02T00:29:27.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.123+0000 7f416e675640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41680d1590 con 0x7f41680d3730 2023-12-02T00:29:27.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.123+0000 7f416e675640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41680d3bc0 con 0x7f41680d2910 2023-12-02T00:29:27.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.123+0000 7f41677fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f41680d2910 0x7f41680d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:27.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.123+0000 7f4167fff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f416815d330 0x7f41680b8b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50420/0 (socket says 172.21.15.46:50420) 2023-12-02T00:29:27.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.123+0000 7f4167fff640 1 -- 172.21.15.46:0/4085665308 learned_addr learned my addr 172.21.15.46:0/4085665308 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:27.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.124+0000 7f41677fe640 1 -- 172.21.15.46:0/4085665308 >> v1:172.21.15.162:6789/0 conn(0x7f41680d3730 legacy=0x7f41680b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:27.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.124+0000 7f41677fe640 1 -- 172.21.15.46:0/4085665308 >> v1:172.21.15.46:6789/0 conn(0x7f416815d330 legacy=0x7f41680b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:27.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.124+0000 7f4166ffd640 1 -- 172.21.15.46:0/4085665308 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1396858838 0 0) 0x7f41680b2b10 con 0x7f416815d330 2023-12-02T00:29:27.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.124+0000 7f41677fe640 1 -- 172.21.15.46:0/4085665308 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41680d1870 con 0x7f41680d2910 2023-12-02T00:29:27.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.125+0000 7f41677fe640 1 --2- 172.21.15.46:0/4085665308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41680d2910 0x7f41680d31f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f415c0097a0 tx=0x7f415c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cc7b301c2558cd1 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:27.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.125+0000 7f4166ffd640 1 -- 172.21.15.46:0/4085665308 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415c02e070 con 0x7f41680d2910 2023-12-02T00:29:27.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.125+0000 7f4166ffd640 1 -- 172.21.15.46:0/4085665308 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f415c0315a0 con 0x7f41680d2910 2023-12-02T00:29:27.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.125+0000 7f4166ffd640 1 -- 172.21.15.46:0/4085665308 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415c031930 con 0x7f41680d2910 2023-12-02T00:29:27.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.125+0000 7f416e675640 1 -- 172.21.15.46:0/4085665308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41680d2910 msgr2=0x7f41680d31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:27.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.125+0000 7f416e675640 1 --2- 172.21.15.46:0/4085665308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41680d2910 0x7f41680d31f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f415c0097a0 tx=0x7f415c02b840 comp rx=0 tx=0).stop 2023-12-02T00:29:27.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.125+0000 7f416e675640 1 -- 172.21.15.46:0/4085665308 shutdown_connections 2023-12-02T00:29:27.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.125+0000 7f416e675640 1 --2- 172.21.15.46:0/4085665308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41680d2910 0x7f41680d31f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:27.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.125+0000 7f416e675640 1 -- 172.21.15.46:0/4085665308 >> 172.21.15.46:0/4085665308 conn(0x7f4168072020 msgr2=0x7f4168072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:27.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.125+0000 7f416e675640 1 -- 172.21.15.46:0/4085665308 shutdown_connections 2023-12-02T00:29:27.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.125+0000 7f416e675640 1 -- 172.21.15.46:0/4085665308 wait complete. 2023-12-02T00:29:27.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.126+0000 7f416e675640 1 Processor -- start 2023-12-02T00:29:27.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.127+0000 7f416e675640 1 -- start start 2023-12-02T00:29:27.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.127+0000 7f416e675640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41680d2910 0x7f41681f2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:27.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.127+0000 7f416e675640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41680d3730 0x7f416815fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:27.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.127+0000 7f416e675640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f416815d330 0x7f41681600b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:27.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.127+0000 7f416e675640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41680c7bf0 con 0x7f41680d3730 2023-12-02T00:29:27.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.127+0000 7f416e675640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41681f3ad0 con 0x7f41680d2910 2023-12-02T00:29:27.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.127+0000 7f416e675640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41681f3db0 con 0x7f416815d330 2023-12-02T00:29:27.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.128+0000 7f41677fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41680d3730 0x7f416815fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:27.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.128+0000 7f41677fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41680d3730 0x7f416815fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52800/0 (socket says 172.21.15.46:52800) 2023-12-02T00:29:27.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.128+0000 7f41677fe640 1 -- 172.21.15.46:0/880443350 learned_addr learned my addr 172.21.15.46:0/880443350 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:27.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.128+0000 7f4167fff640 1 --2- 172.21.15.46:0/880443350 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41680d2910 0x7f41681f2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:27.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.128+0000 7f416cc12640 1 --2- 172.21.15.46:0/880443350 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f416815d330 0x7f41681600b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:27.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.128+0000 7f41677fe640 1 -- 172.21.15.46:0/880443350 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f416815d330 msgr2=0x7f41681600b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:27.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.128+0000 7f41677fe640 1 --2- 172.21.15.46:0/880443350 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f416815d330 0x7f41681600b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:27.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.128+0000 7f41677fe640 1 -- 172.21.15.46:0/880443350 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41680d2910 msgr2=0x7f41681f2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:27.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.128+0000 7f41677fe640 1 --2- 172.21.15.46:0/880443350 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41680d2910 0x7f41681f2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:27.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.128+0000 7f41677fe640 1 -- 172.21.15.46:0/880443350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f415c009400 con 0x7f41680d3730 2023-12-02T00:29:27.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.128+0000 7f416cc12640 1 --2- 172.21.15.46:0/880443350 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f416815d330 0x7f41681600b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:27.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.128+0000 7f41677fe640 1 --2- 172.21.15.46:0/880443350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41680d3730 0x7f416815fb70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f415c0097a0 tx=0x7f415c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:27.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.128+0000 7f4164ff9640 1 -- 172.21.15.46:0/880443350 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415c004230 con 0x7f41680d3730 2023-12-02T00:29:27.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.128+0000 7f4164ff9640 1 -- 172.21.15.46:0/880443350 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f415c0043b0 con 0x7f41680d3730 2023-12-02T00:29:27.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.128+0000 7f416e675640 1 -- 172.21.15.46:0/880443350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41680c7e40 con 0x7f41680d3730 2023-12-02T00:29:27.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.128+0000 7f4164ff9640 1 -- 172.21.15.46:0/880443350 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415c004520 con 0x7f41680d3730 2023-12-02T00:29:27.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.128+0000 7f416e675640 1 -- 172.21.15.46:0/880443350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41680b8f40 con 0x7f41680d3730 2023-12-02T00:29:27.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.129+0000 7f416e675640 1 -- 172.21.15.46:0/880443350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41680b90d0 con 0x7f41680d3730 2023-12-02T00:29:27.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.130+0000 7f4164ff9640 1 -- 172.21.15.46:0/880443350 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f415c032880 con 0x7f41680d3730 2023-12-02T00:29:27.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.130+0000 7f4164ff9640 1 --2- 172.21.15.46:0/880443350 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f415406ae40 0x7f415406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:27.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.130+0000 7f4164ff9640 1 -- 172.21.15.46:0/880443350 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f415c0ab680 con 0x7f41680d3730 2023-12-02T00:29:27.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.130+0000 7f4167fff640 1 --2- 172.21.15.46:0/880443350 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f415406ae40 0x7f415406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:27.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.131+0000 7f4167fff640 1 --2- 172.21.15.46:0/880443350 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f415406ae40 0x7f415406d300 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4158007fd0 tx=0x7f4158007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:27.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.134+0000 7f4164ff9640 1 -- 172.21.15.46:0/880443350 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f415c0767c0 con 0x7f41680d3730 2023-12-02T00:29:27.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.326+0000 7f416e675640 1 -- 172.21.15.46:0/880443350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f41680b90d0 con 0x7f41680d3730 2023-12-02T00:29:27.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.326+0000 7f4164ff9640 1 -- 172.21.15.46:0/880443350 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7f415c079de0 con 0x7f41680d3730 2023-12-02T00:29:27.329 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2023-12-02T00:29:27.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.328+0000 7f416e675640 1 -- 172.21.15.46:0/880443350 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f415406ae40 msgr2=0x7f415406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:27.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.328+0000 7f416e675640 1 --2- 172.21.15.46:0/880443350 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f415406ae40 0x7f415406d300 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4158007fd0 tx=0x7f4158007de0 comp rx=0 tx=0).stop 2023-12-02T00:29:27.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.328+0000 7f416e675640 1 -- 172.21.15.46:0/880443350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41680d3730 msgr2=0x7f416815fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:27.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.328+0000 7f416e675640 1 --2- 172.21.15.46:0/880443350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41680d3730 0x7f416815fb70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f415c0097a0 tx=0x7f415c0046c0 comp rx=0 tx=0).stop 2023-12-02T00:29:27.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.329+0000 7f416e675640 1 -- 172.21.15.46:0/880443350 shutdown_connections 2023-12-02T00:29:27.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.329+0000 7f416e675640 1 --2- 172.21.15.46:0/880443350 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f415406ae40 0x7f415406d300 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:27.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.329+0000 7f416e675640 1 --2- 172.21.15.46:0/880443350 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f416815d330 0x7f41681600b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:27.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.329+0000 7f416e675640 1 --2- 172.21.15.46:0/880443350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41680d3730 0x7f416815fb70 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:27.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.329+0000 7f416e675640 1 --2- 172.21.15.46:0/880443350 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41680d2910 0x7f41681f2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:27.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.329+0000 7f416e675640 1 -- 172.21.15.46:0/880443350 >> 172.21.15.46:0/880443350 conn(0x7f4168072020 msgr2=0x7f4168072400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:27.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.329+0000 7f416e675640 1 -- 172.21.15.46:0/880443350 shutdown_connections 2023-12-02T00:29:27.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.329+0000 7f416e675640 1 -- 172.21.15.46:0/880443350 wait complete. 2023-12-02T00:29:27.341 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-12-02T00:29:27.342 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-02T00:29:27.361 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:29:27.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.468+0000 7fc21470f640 1 Processor -- start 2023-12-02T00:29:27.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.468+0000 7fc21470f640 1 -- start start 2023-12-02T00:29:27.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.469+0000 7fc21470f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc20c1415f0 0x7fc20c141e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:27.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.469+0000 7fc21470f640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc20c0b2ef0 con 0x7fc20c06bbf0 2023-12-02T00:29:27.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.469+0000 7fc21470f640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc20c0b31d0 con 0x7fc20c144a00 2023-12-02T00:29:27.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.469+0000 7fc21470f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc20c0b3b70 con 0x7fc20c1415f0 2023-12-02T00:29:27.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.470+0000 7fc212cac640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc20c06bbf0 0x7fc20c0b3f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50424/0 (socket says 172.21.15.46:50424) 2023-12-02T00:29:27.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.470+0000 7fc212cac640 1 -- 172.21.15.46:0/2578016426 learned_addr learned my addr 172.21.15.46:0/2578016426 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:27.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.470+0000 7fc2124ab640 1 --2- 172.21.15.46:0/2578016426 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc20c1415f0 0x7fc20c141e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:27.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.470+0000 7fc2114a9640 1 -- 172.21.15.46:0/2578016426 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4282091858 0 0) 0x7fc20c0b2ef0 con 0x7fc20c06bbf0 2023-12-02T00:29:27.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.470+0000 7fc2114a9640 1 -- 172.21.15.46:0/2578016426 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1f4003520 con 0x7fc20c06bbf0 2023-12-02T00:29:27.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.470+0000 7fc2114a9640 1 -- 172.21.15.46:0/2578016426 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 77334057 0 0) 0x7fc1f4003520 con 0x7fc20c06bbf0 2023-12-02T00:29:27.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.470+0000 7fc2114a9640 1 -- 172.21.15.46:0/2578016426 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc20c1415f0 msgr2=0x7fc20c141e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:27.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.470+0000 7fc2114a9640 1 --2- 172.21.15.46:0/2578016426 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc20c1415f0 0x7fc20c141e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:27.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.470+0000 7fc2114a9640 1 -- 172.21.15.46:0/2578016426 >> v1:172.21.15.162:6789/0 conn(0x7fc20c144a00 legacy=0x7fc20c06b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:27.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.470+0000 7fc2114a9640 1 -- 172.21.15.46:0/2578016426 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc20c0b34b0 con 0x7fc20c06bbf0 2023-12-02T00:29:27.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.470+0000 7fc2114a9640 1 -- 172.21.15.46:0/2578016426 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7fc1fc0019f0 con 0x7fc20c06bbf0 2023-12-02T00:29:27.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.470+0000 7fc2114a9640 1 -- 172.21.15.46:0/2578016426 >> v1:172.21.15.46:6789/0 conn(0x7fc20c06bbf0 legacy=0x7fc20c0b3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:27.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.470+0000 7fc2114a9640 1 --2- 172.21.15.46:0/2578016426 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc20c1415f0 0x7fc1f40039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:27.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.470+0000 7fc2114a9640 1 --2- 172.21.15.46:0/2578016426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1f4003f20 0x7fc1f40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:27.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.470+0000 7fc2114a9640 1 --2- 172.21.15.46:0/2578016426 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1f4004d00 0x7fc1f4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:27.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.470+0000 7fc2114a9640 1 -- 172.21.15.46:0/2578016426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20c0b3b70 con 0x7fc1f4003f20 2023-12-02T00:29:27.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.470+0000 7fc2114a9640 1 -- 172.21.15.46:0/2578016426 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f400ba60 con 0x7fc1f4004d00 2023-12-02T00:29:27.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.470+0000 7fc2114a9640 1 -- 172.21.15.46:0/2578016426 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f400bc20 con 0x7fc20c1415f0 2023-12-02T00:29:27.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.470+0000 7fc211caa640 1 --2- 172.21.15.46:0/2578016426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1f4003f20 0x7fc1f40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:27.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.470+0000 7fc2124ab640 1 --2- 172.21.15.46:0/2578016426 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc20c1415f0 0x7fc1f40039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:27.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.470+0000 7fc212cac640 1 --2- 172.21.15.46:0/2578016426 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1f4004d00 0x7fc1f4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:27.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.471+0000 7fc211caa640 1 -- 172.21.15.46:0/2578016426 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc20c1415f0 msgr2=0x7fc1f40039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:27.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.471+0000 7fc211caa640 1 --2- 172.21.15.46:0/2578016426 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc20c1415f0 0x7fc1f40039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:27.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.471+0000 7fc211caa640 1 -- 172.21.15.46:0/2578016426 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1f4004d00 msgr2=0x7fc1f4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:27.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.471+0000 7fc211caa640 1 --2- 172.21.15.46:0/2578016426 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1f4004d00 0x7fc1f4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:27.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.471+0000 7fc211caa640 1 -- 172.21.15.46:0/2578016426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f4003520 con 0x7fc1f4003f20 2023-12-02T00:29:27.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.471+0000 7fc212cac640 1 --2- 172.21.15.46:0/2578016426 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1f4004d00 0x7fc1f4005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:27.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.471+0000 7fc211caa640 1 --2- 172.21.15.46:0/2578016426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1f4003f20 0x7fc1f40047c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc200001720 tx=0x7fc1f40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6250a660b075267c server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:27.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.471+0000 7fc2114a9640 1 -- 172.21.15.46:0/2578016426 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2000052d0 con 0x7fc1f4003f20 2023-12-02T00:29:27.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.471+0000 7fc2114a9640 1 -- 172.21.15.46:0/2578016426 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc20002e050 con 0x7fc1f4003f20 2023-12-02T00:29:27.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.472+0000 7fc2114a9640 1 -- 172.21.15.46:0/2578016426 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc200003830 con 0x7fc1f4003f20 2023-12-02T00:29:27.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.472+0000 7fc21470f640 1 -- 172.21.15.46:0/2578016426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1f4003f20 msgr2=0x7fc1f40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:27.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.472+0000 7fc21470f640 1 --2- 172.21.15.46:0/2578016426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1f4003f20 0x7fc1f40047c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc200001720 tx=0x7fc1f40057f0 comp rx=0 tx=0).stop 2023-12-02T00:29:27.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.472+0000 7fc212cac640 1 -- 172.21.15.46:0/2578016426 reap_dead start 2023-12-02T00:29:27.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.472+0000 7fc21470f640 1 -- 172.21.15.46:0/2578016426 shutdown_connections 2023-12-02T00:29:27.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.472+0000 7fc21470f640 1 -- 172.21.15.46:0/2578016426 >> 172.21.15.46:0/2578016426 conn(0x7fc20c070e70 msgr2=0x7fc20c071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:27.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.472+0000 7fc21470f640 1 -- 172.21.15.46:0/2578016426 shutdown_connections 2023-12-02T00:29:27.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.472+0000 7fc21470f640 1 -- 172.21.15.46:0/2578016426 wait complete. 2023-12-02T00:29:27.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.473+0000 7fc21470f640 1 Processor -- start 2023-12-02T00:29:27.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.473+0000 7fc21470f640 1 -- start start 2023-12-02T00:29:27.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.474+0000 7fc21470f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc1f4003f20 0x7fc20c135e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:27.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.474+0000 7fc21470f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc20c1564e0 0x7fc20c156960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:27.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.474+0000 7fc21470f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc20c1591c0 0x7fc20c159640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:27.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.474+0000 7fc21470f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20c159b80 con 0x7fc20c1564e0 2023-12-02T00:29:27.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.474+0000 7fc21470f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20c157ae0 con 0x7fc20c1591c0 2023-12-02T00:29:27.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.474+0000 7fc21470f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc20c157d20 con 0x7fc1f4003f20 2023-12-02T00:29:27.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.474+0000 7fc211caa640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc20c1564e0 0x7fc20c156960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:27.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.474+0000 7fc211caa640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc20c1564e0 0x7fc20c156960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52812/0 (socket says 172.21.15.46:52812) 2023-12-02T00:29:27.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.474+0000 7fc211caa640 1 -- 172.21.15.46:0/1938277476 learned_addr learned my addr 172.21.15.46:0/1938277476 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:27.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.474+0000 7fc2124ab640 1 --2- 172.21.15.46:0/1938277476 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc1f4003f20 0x7fc20c135e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:27.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.474+0000 7fc212cac640 1 --2- 172.21.15.46:0/1938277476 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc20c1591c0 0x7fc20c159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:27.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.474+0000 7fc211caa640 1 -- 172.21.15.46:0/1938277476 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc1f4003f20 msgr2=0x7fc20c135e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:27.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.474+0000 7fc211caa640 1 --2- 172.21.15.46:0/1938277476 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc1f4003f20 0x7fc20c135e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:27.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.474+0000 7fc211caa640 1 -- 172.21.15.46:0/1938277476 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc20c1591c0 msgr2=0x7fc20c159640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:27.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.474+0000 7fc211caa640 1 --2- 172.21.15.46:0/1938277476 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc20c1591c0 0x7fc20c159640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:27.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.474+0000 7fc211caa640 1 -- 172.21.15.46:0/1938277476 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc20c157fc0 con 0x7fc20c1564e0 2023-12-02T00:29:27.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.475+0000 7fc211caa640 1 --2- 172.21.15.46:0/1938277476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc20c1564e0 0x7fc20c156960 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc200005040 tx=0x7fc200001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:27.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.475+0000 7fc20affd640 1 -- 172.21.15.46:0/1938277476 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2000076c0 con 0x7fc20c1564e0 2023-12-02T00:29:27.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.475+0000 7fc20affd640 1 -- 172.21.15.46:0/1938277476 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc200007a30 con 0x7fc20c1564e0 2023-12-02T00:29:27.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.475+0000 7fc20affd640 1 -- 172.21.15.46:0/1938277476 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc200006be0 con 0x7fc20c1564e0 2023-12-02T00:29:27.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.475+0000 7fc21470f640 1 -- 172.21.15.46:0/1938277476 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc20c1582a0 con 0x7fc20c1564e0 2023-12-02T00:29:27.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.475+0000 7fc21470f640 1 -- 172.21.15.46:0/1938277476 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc20c158860 con 0x7fc20c1564e0 2023-12-02T00:29:27.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.476+0000 7fc21470f640 1 -- 172.21.15.46:0/1938277476 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc20c1589f0 con 0x7fc20c1564e0 2023-12-02T00:29:27.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.476+0000 7fc20affd640 1 -- 172.21.15.46:0/1938277476 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc200006d40 con 0x7fc20c1564e0 2023-12-02T00:29:27.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.477+0000 7fc20affd640 1 --2- 172.21.15.46:0/1938277476 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc1f806ae40 0x7fc1f806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:27.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.477+0000 7fc20affd640 1 -- 172.21.15.46:0/1938277476 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc2000ab320 con 0x7fc20c1564e0 2023-12-02T00:29:27.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.477+0000 7fc2124ab640 1 --2- 172.21.15.46:0/1938277476 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc1f806ae40 0x7fc1f806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:27.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.478+0000 7fc2124ab640 1 --2- 172.21.15.46:0/1938277476 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc1f806ae40 0x7fc1f806d300 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc2040099e0 tx=0x7fc204008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:27.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.481+0000 7fc20affd640 1 -- 172.21.15.46:0/1938277476 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc20007a010 con 0x7fc20c1564e0 2023-12-02T00:29:27.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.657+0000 7fc21470f640 1 -- 172.21.15.46:0/1938277476 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc20c069400 con 0x7fc20c1564e0 2023-12-02T00:29:27.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.658+0000 7fc20affd640 1 -- 172.21.15.46:0/1938277476 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16481 (secure 0 0 0) 0x7fc200079ba0 con 0x7fc20c1564e0 2023-12-02T00:29:27.660 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:29:27.660 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":21,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:28:50.700112+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-02T00:28:47.273337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-02T00:28:48.060115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:29:27.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.660+0000 7fc21470f640 1 -- 172.21.15.46:0/1938277476 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc1f806ae40 msgr2=0x7fc1f806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:27.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.660+0000 7fc21470f640 1 --2- 172.21.15.46:0/1938277476 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc1f806ae40 0x7fc1f806d300 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc2040099e0 tx=0x7fc204008040 comp rx=0 tx=0).stop 2023-12-02T00:29:27.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.660+0000 7fc21470f640 1 -- 172.21.15.46:0/1938277476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc20c1564e0 msgr2=0x7fc20c156960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:27.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.660+0000 7fc21470f640 1 --2- 172.21.15.46:0/1938277476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc20c1564e0 0x7fc20c156960 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc200005040 tx=0x7fc200001120 comp rx=0 tx=0).stop 2023-12-02T00:29:27.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.660+0000 7fc21470f640 1 -- 172.21.15.46:0/1938277476 shutdown_connections 2023-12-02T00:29:27.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.660+0000 7fc21470f640 1 --2- 172.21.15.46:0/1938277476 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc1f806ae40 0x7fc1f806d300 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:27.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.660+0000 7fc21470f640 1 --2- 172.21.15.46:0/1938277476 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc20c1591c0 0x7fc20c159640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:27.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.660+0000 7fc21470f640 1 --2- 172.21.15.46:0/1938277476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc20c1564e0 0x7fc20c156960 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:27.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.660+0000 7fc21470f640 1 --2- 172.21.15.46:0/1938277476 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc1f4003f20 0x7fc20c135e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:27.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.660+0000 7fc21470f640 1 -- 172.21.15.46:0/1938277476 >> 172.21.15.46:0/1938277476 conn(0x7fc20c070e70 msgr2=0x7fc20c141a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:27.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.660+0000 7fc21470f640 1 -- 172.21.15.46:0/1938277476 shutdown_connections 2023-12-02T00:29:27.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.660+0000 7fc21470f640 1 -- 172.21.15.46:0/1938277476 wait complete. 2023-12-02T00:29:27.673 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-02T00:28:37.213645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-02T00:28:47.273337+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-02T00:28:48.060115+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-02T00:29:27.674 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-02T00:29:27.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.781+0000 7f9afec36640 1 Processor -- start 2023-12-02T00:29:27.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.781+0000 7f9afec36640 1 -- start start 2023-12-02T00:29:27.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.782+0000 7f9afec36640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9af80d3900 0x7f9af80b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:27.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.782+0000 7f9afec36640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9af80b2b10 con 0x7f9af80d2910 2023-12-02T00:29:27.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.782+0000 7f9afec36640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9af80d1590 con 0x7f9af815d330 2023-12-02T00:29:27.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.782+0000 7f9afec36640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9af80d1870 con 0x7f9af80d3900 2023-12-02T00:29:27.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.782+0000 7f9af7fff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9af80d2910 0x7f9af80d31f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50434/0 (socket says 172.21.15.46:50434) 2023-12-02T00:29:27.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.782+0000 7f9af7fff640 1 -- 172.21.15.46:0/658906217 learned_addr learned my addr 172.21.15.46:0/658906217 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:27.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.783+0000 7f9afd1d3640 1 --2- 172.21.15.46:0/658906217 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9af80d3900 0x7f9af80b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:27.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.783+0000 7f9af77fe640 1 -- 172.21.15.46:0/658906217 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2334477068 0 0) 0x7f9af80b2b10 con 0x7f9af80d2910 2023-12-02T00:29:27.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.783+0000 7f9af77fe640 1 -- 172.21.15.46:0/658906217 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ae4003520 con 0x7f9af80d2910 2023-12-02T00:29:27.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.783+0000 7f9afd1d3640 1 -- 172.21.15.46:0/658906217 >> v1:172.21.15.162:6789/0 conn(0x7f9af815d330 legacy=0x7f9af80b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:27.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.783+0000 7f9afd1d3640 1 -- 172.21.15.46:0/658906217 >> v1:172.21.15.46:6789/0 conn(0x7f9af80d2910 legacy=0x7f9af80d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:27.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.783+0000 7f9af77fe640 1 -- 172.21.15.46:0/658906217 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1414295164 0 0) 0x7f9ae4003520 con 0x7f9af80d2910 2023-12-02T00:29:27.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.783+0000 7f9afd1d3640 1 -- 172.21.15.46:0/658906217 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af80d1b50 con 0x7f9af80d3900 2023-12-02T00:29:27.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.784+0000 7f9afd1d3640 1 --2- 172.21.15.46:0/658906217 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9af80d3900 0x7f9af80b6390 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9ae80097d0 tx=0x7f9ae802b770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=651cc342f23eb2a server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:27.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.784+0000 7f9af77fe640 1 -- 172.21.15.46:0/658906217 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae0001670 con 0x7f9af80d3900 2023-12-02T00:29:27.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.784+0000 7f9af77fe640 1 -- 172.21.15.46:0/658906217 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae802e050 con 0x7f9af80d3900 2023-12-02T00:29:27.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.784+0000 7f9af77fe640 1 -- 172.21.15.46:0/658906217 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae8030710 con 0x7f9af80d3900 2023-12-02T00:29:27.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.784+0000 7f9afec36640 1 -- 172.21.15.46:0/658906217 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9af80d3900 msgr2=0x7f9af80b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:27.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.784+0000 7f9afec36640 1 --2- 172.21.15.46:0/658906217 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9af80d3900 0x7f9af80b6390 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9ae80097d0 tx=0x7f9ae802b770 comp rx=0 tx=0).stop 2023-12-02T00:29:27.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.784+0000 7f9afec36640 1 -- 172.21.15.46:0/658906217 shutdown_connections 2023-12-02T00:29:27.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.784+0000 7f9afec36640 1 --2- 172.21.15.46:0/658906217 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9af80d3900 0x7f9af80b6390 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:27.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.784+0000 7f9afec36640 1 -- 172.21.15.46:0/658906217 >> 172.21.15.46:0/658906217 conn(0x7f9af8072020 msgr2=0x7f9af8072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:27.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.785+0000 7f9afec36640 1 -- 172.21.15.46:0/658906217 shutdown_connections 2023-12-02T00:29:27.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.785+0000 7f9afec36640 1 -- 172.21.15.46:0/658906217 wait complete. 2023-12-02T00:29:27.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.786+0000 7f9afec36640 1 Processor -- start 2023-12-02T00:29:27.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.786+0000 7f9afec36640 1 -- start start 2023-12-02T00:29:27.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.786+0000 7f9afec36640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9af80d2910 0x7f9af815d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:27.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.786+0000 7f9afec36640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9af80d3900 0x7f9af8153770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:27.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.786+0000 7f9afec36640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9af815d330 0x7f9af80b7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:27.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.787+0000 7f9afec36640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af815de60 con 0x7f9af80d2910 2023-12-02T00:29:27.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.787+0000 7f9afec36640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af80c9800 con 0x7f9af815d330 2023-12-02T00:29:27.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.787+0000 7f9afec36640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9af80c8bf0 con 0x7f9af80d3900 2023-12-02T00:29:27.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.787+0000 7f9afc9d2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9af80d2910 0x7f9af815d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:27.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.787+0000 7f9afc9d2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9af80d2910 0x7f9af815d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52828/0 (socket says 172.21.15.46:52828) 2023-12-02T00:29:27.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.787+0000 7f9afc9d2640 1 -- 172.21.15.46:0/3943944474 learned_addr learned my addr 172.21.15.46:0/3943944474 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:27.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.787+0000 7f9afd1d3640 1 --2- 172.21.15.46:0/3943944474 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9af815d330 0x7f9af80b7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:27.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.787+0000 7f9af7fff640 1 --2- 172.21.15.46:0/3943944474 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9af80d3900 0x7f9af8153770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:27.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.787+0000 7f9afc9d2640 1 -- 172.21.15.46:0/3943944474 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9af80d3900 msgr2=0x7f9af8153770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:27.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.787+0000 7f9afc9d2640 1 --2- 172.21.15.46:0/3943944474 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9af80d3900 0x7f9af8153770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:27.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.787+0000 7f9afc9d2640 1 -- 172.21.15.46:0/3943944474 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9af815d330 msgr2=0x7f9af80b7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:27.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.787+0000 7f9afc9d2640 1 --2- 172.21.15.46:0/3943944474 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9af815d330 0x7f9af80b7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:27.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.787+0000 7f9afc9d2640 1 -- 172.21.15.46:0/3943944474 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae8009400 con 0x7f9af80d2910 2023-12-02T00:29:27.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.787+0000 7f9af7fff640 1 --2- 172.21.15.46:0/3943944474 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9af80d3900 0x7f9af8153770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:29:27.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.787+0000 7f9afc9d2640 1 --2- 172.21.15.46:0/3943944474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9af80d2910 0x7f9af815d920 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9af000c870 tx=0x7f9af000cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:27.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.787+0000 7f9af57fa640 1 -- 172.21.15.46:0/3943944474 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af0003b70 con 0x7f9af80d2910 2023-12-02T00:29:27.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.787+0000 7f9af57fa640 1 -- 172.21.15.46:0/3943944474 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af0003cf0 con 0x7f9af80d2910 2023-12-02T00:29:27.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.787+0000 7f9afec36640 1 -- 172.21.15.46:0/3943944474 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9af80c8ed0 con 0x7f9af80d2910 2023-12-02T00:29:27.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.787+0000 7f9af57fa640 1 -- 172.21.15.46:0/3943944474 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af000aa90 con 0x7f9af80d2910 2023-12-02T00:29:27.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.787+0000 7f9afec36640 1 -- 172.21.15.46:0/3943944474 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9af80c9490 con 0x7f9af80d2910 2023-12-02T00:29:27.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.788+0000 7f9afec36640 1 -- 172.21.15.46:0/3943944474 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9af8069400 con 0x7f9af80d2910 2023-12-02T00:29:27.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.789+0000 7f9af57fa640 1 -- 172.21.15.46:0/3943944474 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9af000abf0 con 0x7f9af80d2910 2023-12-02T00:29:27.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.789+0000 7f9af57fa640 1 --2- 172.21.15.46:0/3943944474 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9ad806ae40 0x7f9ad806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:27.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.789+0000 7f9af57fa640 1 -- 172.21.15.46:0/3943944474 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9af0089430 con 0x7f9af80d2910 2023-12-02T00:29:27.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.790+0000 7f9af7fff640 1 --2- 172.21.15.46:0/3943944474 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9ad806ae40 0x7f9ad806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:27.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.790+0000 7f9af7fff640 1 --2- 172.21.15.46:0/3943944474 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9ad806ae40 0x7f9ad806d300 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9ae00011e0 tx=0x7f9ae0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:27.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.794+0000 7f9af57fa640 1 -- 172.21.15.46:0/3943944474 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9af0054570 con 0x7f9af80d2910 2023-12-02T00:29:27.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.970+0000 7f9afec36640 1 -- 172.21.15.46:0/3943944474 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9af80d3290 con 0x7f9af80d2910 2023-12-02T00:29:27.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.970+0000 7f9af57fa640 1 -- 172.21.15.46:0/3943944474 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9af0003e60 con 0x7f9af80d2910 2023-12-02T00:29:27.973 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-12-02T00:29:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.972+0000 7f9afec36640 1 -- 172.21.15.46:0/3943944474 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9ad806ae40 msgr2=0x7f9ad806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.972+0000 7f9afec36640 1 --2- 172.21.15.46:0/3943944474 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9ad806ae40 0x7f9ad806d300 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9ae00011e0 tx=0x7f9ae0009040 comp rx=0 tx=0).stop 2023-12-02T00:29:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.972+0000 7f9afec36640 1 -- 172.21.15.46:0/3943944474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9af80d2910 msgr2=0x7f9af815d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.972+0000 7f9afec36640 1 --2- 172.21.15.46:0/3943944474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9af80d2910 0x7f9af815d920 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9af000c870 tx=0x7f9af000cd30 comp rx=0 tx=0).stop 2023-12-02T00:29:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.972+0000 7f9afec36640 1 -- 172.21.15.46:0/3943944474 shutdown_connections 2023-12-02T00:29:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.972+0000 7f9afec36640 1 --2- 172.21.15.46:0/3943944474 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9ad806ae40 0x7f9ad806d300 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.972+0000 7f9afec36640 1 --2- 172.21.15.46:0/3943944474 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9af815d330 0x7f9af80b7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.972+0000 7f9afec36640 1 --2- 172.21.15.46:0/3943944474 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9af80d3900 0x7f9af8153770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.972+0000 7f9afec36640 1 --2- 172.21.15.46:0/3943944474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9af80d2910 0x7f9af815d920 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.972+0000 7f9afec36640 1 -- 172.21.15.46:0/3943944474 >> 172.21.15.46:0/3943944474 conn(0x7f9af8072020 msgr2=0x7f9af80b4630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.972+0000 7f9afec36640 1 -- 172.21.15.46:0/3943944474 shutdown_connections 2023-12-02T00:29:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:27.973+0000 7f9afec36640 1 -- 172.21.15.46:0/3943944474 wait complete. 2023-12-02T00:29:27.987 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-02T00:29:28.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.147+0000 7fd56ca7c640 1 Processor -- start 2023-12-02T00:29:28.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.147+0000 7fd56ca7c640 1 -- start start 2023-12-02T00:29:28.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.148+0000 7fd56ca7c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd56806ea50 0x7fd56806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:28.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.148+0000 7fd56ca7c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd5680d0ed0 0x7fd5680d1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:28.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.148+0000 7fd56ca7c640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd568161d20 con 0x7fd568160fb0 2023-12-02T00:29:28.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.148+0000 7fd56ca7c640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd56806f370 con 0x7fd56806ea50 2023-12-02T00:29:28.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.148+0000 7fd56ca7c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd56815e760 con 0x7fd5680d0ed0 2023-12-02T00:29:28.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.148+0000 7fd56659c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd56806ea50 0x7fd56806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:28.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.148+0000 7fd56659c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd56806ea50 0x7fd56806ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52838/0 (socket says 172.21.15.46:52838) 2023-12-02T00:29:28.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.148+0000 7fd56659c640 1 -- 172.21.15.46:0/1205323557 learned_addr learned my addr 172.21.15.46:0/1205323557 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:28.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.148+0000 7fd566d9d640 1 --2- 172.21.15.46:0/1205323557 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd5680d0ed0 0x7fd5680d1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:28.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.148+0000 7fd56659c640 1 -- 172.21.15.46:0/1205323557 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd5680d0ed0 msgr2=0x7fd5680d1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:28.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.148+0000 7fd56659c640 1 --2- 172.21.15.46:0/1205323557 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd5680d0ed0 0x7fd5680d1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:28.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.148+0000 7fd56659c640 1 -- 172.21.15.46:0/1205323557 >> v1:172.21.15.162:6789/0 conn(0x7fd568160fb0 legacy=0x7fd568161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:28.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.148+0000 7fd56659c640 1 -- 172.21.15.46:0/1205323557 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd56815d9a0 con 0x7fd56806ea50 2023-12-02T00:29:28.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.149+0000 7fd56659c640 1 --2- 172.21.15.46:0/1205323557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd56806ea50 0x7fd56806ee30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd560009870 tx=0x7fd56002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce1a3ff102523290 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:28.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.149+0000 7fd56559a640 1 -- 172.21.15.46:0/1205323557 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd560031640 con 0x7fd56806ea50 2023-12-02T00:29:28.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.149+0000 7fd56559a640 1 -- 172.21.15.46:0/1205323557 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5600317a0 con 0x7fd56806ea50 2023-12-02T00:29:28.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.149+0000 7fd56559a640 1 -- 172.21.15.46:0/1205323557 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd560031b30 con 0x7fd56806ea50 2023-12-02T00:29:28.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.150+0000 7fd56ca7c640 1 -- 172.21.15.46:0/1205323557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd56806ea50 msgr2=0x7fd56806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:28.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.150+0000 7fd56ca7c640 1 --2- 172.21.15.46:0/1205323557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd56806ea50 0x7fd56806ee30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd560009870 tx=0x7fd56002b8f0 comp rx=0 tx=0).stop 2023-12-02T00:29:28.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.150+0000 7fd56ca7c640 1 -- 172.21.15.46:0/1205323557 shutdown_connections 2023-12-02T00:29:28.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.150+0000 7fd56ca7c640 1 --2- 172.21.15.46:0/1205323557 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd5680d0ed0 0x7fd5680d1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:28.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.150+0000 7fd56ca7c640 1 --2- 172.21.15.46:0/1205323557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd56806ea50 0x7fd56806ee30 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:28.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.150+0000 7fd56ca7c640 1 -- 172.21.15.46:0/1205323557 >> 172.21.15.46:0/1205323557 conn(0x7fd56806a6b0 msgr2=0x7fd56806aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:28.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.150+0000 7fd56ca7c640 1 -- 172.21.15.46:0/1205323557 shutdown_connections 2023-12-02T00:29:28.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.150+0000 7fd56ca7c640 1 -- 172.21.15.46:0/1205323557 wait complete. 2023-12-02T00:29:28.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.151+0000 7fd56ca7c640 1 Processor -- start 2023-12-02T00:29:28.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.151+0000 7fd56ca7c640 1 -- start start 2023-12-02T00:29:28.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.152+0000 7fd56ca7c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd56806ea50 0x7fd568155fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:28.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.152+0000 7fd56ca7c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd5680d0ed0 0x7fd5681e9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:28.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.152+0000 7fd56ca7c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd568160fb0 0x7fd56813f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:28.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.152+0000 7fd56ca7c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5681e7f60 con 0x7fd56806ea50 2023-12-02T00:29:28.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.152+0000 7fd56ca7c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5681e8240 con 0x7fd568160fb0 2023-12-02T00:29:28.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.152+0000 7fd56ca7c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5681e8520 con 0x7fd5680d0ed0 2023-12-02T00:29:28.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.152+0000 7fd56659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd56806ea50 0x7fd568155fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:28.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.152+0000 7fd56659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd56806ea50 0x7fd568155fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52854/0 (socket says 172.21.15.46:52854) 2023-12-02T00:29:28.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.152+0000 7fd56659c640 1 -- 172.21.15.46:0/124068700 learned_addr learned my addr 172.21.15.46:0/124068700 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:28.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.152+0000 7fd56659c640 1 -- 172.21.15.46:0/124068700 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd5680d0ed0 msgr2=0x7fd5681e9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:28.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.152+0000 7fd56659c640 1 --2- 172.21.15.46:0/124068700 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd5680d0ed0 0x7fd5681e9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:28.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.152+0000 7fd56659c640 1 -- 172.21.15.46:0/124068700 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd568160fb0 msgr2=0x7fd56813f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:28.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.152+0000 7fd56659c640 1 --2- 172.21.15.46:0/124068700 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd568160fb0 0x7fd56813f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:28.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.152+0000 7fd56659c640 1 -- 172.21.15.46:0/124068700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5600094d0 con 0x7fd56806ea50 2023-12-02T00:29:28.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.153+0000 7fd56659c640 1 --2- 172.21.15.46:0/124068700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd56806ea50 0x7fd568155fc0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd560009870 tx=0x7fd5600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:28.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.153+0000 7fd556ffd640 1 -- 172.21.15.46:0/124068700 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56003ac60 con 0x7fd56806ea50 2023-12-02T00:29:28.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.153+0000 7fd556ffd640 1 -- 172.21.15.46:0/124068700 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd56003adc0 con 0x7fd56806ea50 2023-12-02T00:29:28.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.153+0000 7fd56ca7c640 1 -- 172.21.15.46:0/124068700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5681e8800 con 0x7fd56806ea50 2023-12-02T00:29:28.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.153+0000 7fd556ffd640 1 -- 172.21.15.46:0/124068700 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd560044630 con 0x7fd56806ea50 2023-12-02T00:29:28.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.153+0000 7fd56ca7c640 1 -- 172.21.15.46:0/124068700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd568156500 con 0x7fd56806ea50 2023-12-02T00:29:28.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.154+0000 7fd56ca7c640 1 -- 172.21.15.46:0/124068700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5681e8d20 con 0x7fd56806ea50 2023-12-02T00:29:28.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.154+0000 7fd556ffd640 1 -- 172.21.15.46:0/124068700 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd56002f050 con 0x7fd56806ea50 2023-12-02T00:29:28.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.155+0000 7fd556ffd640 1 --2- 172.21.15.46:0/124068700 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd54806ae40 0x7fd54806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:28.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.155+0000 7fd556ffd640 1 -- 172.21.15.46:0/124068700 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd5600ab530 con 0x7fd56806ea50 2023-12-02T00:29:28.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.155+0000 7fd565d9b640 1 --2- 172.21.15.46:0/124068700 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd54806ae40 0x7fd54806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:28.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.156+0000 7fd565d9b640 1 --2- 172.21.15.46:0/124068700 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd54806ae40 0x7fd54806d300 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd56815ab70 tx=0x7fd55c0079b0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:28.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.159+0000 7fd556ffd640 1 -- 172.21.15.46:0/124068700 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd5681e8d20 con 0x7fd56806ea50 2023-12-02T00:29:28.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.334+0000 7fd56ca7c640 1 -- 172.21.15.46:0/124068700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd5681e8d20 con 0x7fd56806ea50 2023-12-02T00:29:28.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.335+0000 7fd556ffd640 1 -- 172.21.15.46:0/124068700 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd5681e8d20 con 0x7fd56806ea50 2023-12-02T00:29:28.337 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-12-02T00:29:28.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.337+0000 7fd56ca7c640 1 -- 172.21.15.46:0/124068700 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd54806ae40 msgr2=0x7fd54806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:28.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.337+0000 7fd56ca7c640 1 --2- 172.21.15.46:0/124068700 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd54806ae40 0x7fd54806d300 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd56815ab70 tx=0x7fd55c0079b0 comp rx=0 tx=0).stop 2023-12-02T00:29:28.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.337+0000 7fd56ca7c640 1 -- 172.21.15.46:0/124068700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd56806ea50 msgr2=0x7fd568155fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:28.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.337+0000 7fd56ca7c640 1 --2- 172.21.15.46:0/124068700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd56806ea50 0x7fd568155fc0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd560009870 tx=0x7fd5600046c0 comp rx=0 tx=0).stop 2023-12-02T00:29:28.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.337+0000 7fd56ca7c640 1 -- 172.21.15.46:0/124068700 shutdown_connections 2023-12-02T00:29:28.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.337+0000 7fd56ca7c640 1 --2- 172.21.15.46:0/124068700 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd54806ae40 0x7fd54806d300 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:28.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.337+0000 7fd56ca7c640 1 --2- 172.21.15.46:0/124068700 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd568160fb0 0x7fd56813f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:28.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.337+0000 7fd56ca7c640 1 --2- 172.21.15.46:0/124068700 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd5680d0ed0 0x7fd5681e9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:28.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.337+0000 7fd56ca7c640 1 --2- 172.21.15.46:0/124068700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd56806ea50 0x7fd568155fc0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:28.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.337+0000 7fd56ca7c640 1 -- 172.21.15.46:0/124068700 >> 172.21.15.46:0/124068700 conn(0x7fd56806a6b0 msgr2=0x7fd56806aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:28.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.337+0000 7fd56ca7c640 1 -- 172.21.15.46:0/124068700 shutdown_connections 2023-12-02T00:29:28.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.337+0000 7fd56ca7c640 1 -- 172.21.15.46:0/124068700 wait complete. 2023-12-02T00:29:28.350 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-02T00:29:28.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.458+0000 7fadc279e640 1 Processor -- start 2023-12-02T00:29:28.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.459+0000 7fadc279e640 1 -- start start 2023-12-02T00:29:28.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.459+0000 7fadc279e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fadbc0c8210 0x7fadbc167e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:28.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.459+0000 7fadc279e640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadbc0d1a20 con 0x7fadbc0d14f0 2023-12-02T00:29:28.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.459+0000 7fadc279e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadbc0d1d00 con 0x7fadbc15d1b0 2023-12-02T00:29:28.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.459+0000 7fadc279e640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadbc0cddb0 con 0x7fadbc0c8210 2023-12-02T00:29:28.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.460+0000 7fadc1f9d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fadbc0c8210 0x7fadbc167e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:28.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.460+0000 7fadc0f9b640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fadbc0d14f0 0x7fadbc0c6af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50436/0 (socket says 172.21.15.46:50436) 2023-12-02T00:29:28.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.460+0000 7fadc0f9b640 1 -- 172.21.15.46:0/1839609325 learned_addr learned my addr 172.21.15.46:0/1839609325 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:28.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.460+0000 7fadc1f9d640 1 -- 172.21.15.46:0/1839609325 >> v1:172.21.15.162:6789/0 conn(0x7fadbc15d1b0 legacy=0x7fadbc0c92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:28.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.460+0000 7fadb3fff640 1 -- 172.21.15.46:0/1839609325 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 187914570 0 0) 0x7fadbc0d1a20 con 0x7fadbc0d14f0 2023-12-02T00:29:28.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.460+0000 7fadc1f9d640 1 -- 172.21.15.46:0/1839609325 >> v1:172.21.15.46:6789/0 conn(0x7fadbc0d14f0 legacy=0x7fadbc0c6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:28.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.460+0000 7fadc1f9d640 1 -- 172.21.15.46:0/1839609325 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadbc0ce090 con 0x7fadbc0c8210 2023-12-02T00:29:28.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.461+0000 7fadc1f9d640 1 --2- 172.21.15.46:0/1839609325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fadbc0c8210 0x7fadbc167e50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fadb4009d90 tx=0x7fadb402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a14f58913b447f91 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:28.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.461+0000 7fadb3fff640 1 -- 172.21.15.46:0/1839609325 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb402cc60 con 0x7fadbc0c8210 2023-12-02T00:29:28.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.461+0000 7fadb3fff640 1 -- 172.21.15.46:0/1839609325 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb402cdc0 con 0x7fadbc0c8210 2023-12-02T00:29:28.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.461+0000 7fadb3fff640 1 -- 172.21.15.46:0/1839609325 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb4034660 con 0x7fadbc0c8210 2023-12-02T00:29:28.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.461+0000 7fadc279e640 1 -- 172.21.15.46:0/1839609325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fadbc0c8210 msgr2=0x7fadbc167e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:28.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.461+0000 7fadc279e640 1 --2- 172.21.15.46:0/1839609325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fadbc0c8210 0x7fadbc167e50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fadb4009d90 tx=0x7fadb402beb0 comp rx=0 tx=0).stop 2023-12-02T00:29:28.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.461+0000 7fadc279e640 1 -- 172.21.15.46:0/1839609325 shutdown_connections 2023-12-02T00:29:28.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.461+0000 7fadc279e640 1 --2- 172.21.15.46:0/1839609325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fadbc0c8210 0x7fadbc167e50 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:28.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.461+0000 7fadc279e640 1 -- 172.21.15.46:0/1839609325 >> 172.21.15.46:0/1839609325 conn(0x7fadbc09c300 msgr2=0x7fadbc0d2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:28.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.462+0000 7fadc279e640 1 -- 172.21.15.46:0/1839609325 shutdown_connections 2023-12-02T00:29:28.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.462+0000 7fadc279e640 1 -- 172.21.15.46:0/1839609325 wait complete. 2023-12-02T00:29:28.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.463+0000 7fadc279e640 1 Processor -- start 2023-12-02T00:29:28.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.463+0000 7fadc279e640 1 -- start start 2023-12-02T00:29:28.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.464+0000 7fadc279e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fadbc0c8210 0x7fadbc149ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:28.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.464+0000 7fadc279e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fadbc0d14f0 0x7fadbc0ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:28.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.464+0000 7fadc279e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fadbc15d1b0 0x7fadbc0cd060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:28.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.464+0000 7fadc279e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc0b8550 con 0x7fadbc0d14f0 2023-12-02T00:29:28.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.464+0000 7fadc279e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc14adc0 con 0x7fadbc15d1b0 2023-12-02T00:29:28.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.464+0000 7fadc279e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc14a490 con 0x7fadbc0c8210 2023-12-02T00:29:28.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.464+0000 7fadc0f9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fadbc0d14f0 0x7fadbc0ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:28.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.464+0000 7fadc0f9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fadbc0d14f0 0x7fadbc0ccb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52876/0 (socket says 172.21.15.46:52876) 2023-12-02T00:29:28.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.464+0000 7fadc0f9b640 1 -- 172.21.15.46:0/469014623 learned_addr learned my addr 172.21.15.46:0/469014623 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:28.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.464+0000 7fadc179c640 1 --2- 172.21.15.46:0/469014623 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fadbc0c8210 0x7fadbc149ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:28.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.464+0000 7fadc1f9d640 1 --2- 172.21.15.46:0/469014623 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fadbc15d1b0 0x7fadbc0cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:28.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.464+0000 7fadc0f9b640 1 -- 172.21.15.46:0/469014623 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fadbc0c8210 msgr2=0x7fadbc149ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:28.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.464+0000 7fadc0f9b640 1 --2- 172.21.15.46:0/469014623 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fadbc0c8210 0x7fadbc149ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:28.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.464+0000 7fadc0f9b640 1 -- 172.21.15.46:0/469014623 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fadbc15d1b0 msgr2=0x7fadbc0cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:28.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.464+0000 7fadc0f9b640 1 --2- 172.21.15.46:0/469014623 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fadbc15d1b0 0x7fadbc0cd060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:28.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.464+0000 7fadc0f9b640 1 -- 172.21.15.46:0/469014623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb4009a70 con 0x7fadbc0d14f0 2023-12-02T00:29:28.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.464+0000 7fadc179c640 1 --2- 172.21.15.46:0/469014623 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fadbc0c8210 0x7fadbc149ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:29:28.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.464+0000 7fadc0f9b640 1 --2- 172.21.15.46:0/469014623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fadbc0d14f0 0x7fadbc0ccb20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fadac00c490 tx=0x7fadac00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:28.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.464+0000 7fadb1ffb640 1 -- 172.21.15.46:0/469014623 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadac0074e0 con 0x7fadbc0d14f0 2023-12-02T00:29:28.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.464+0000 7fadb1ffb640 1 -- 172.21.15.46:0/469014623 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadac007640 con 0x7fadbc0d14f0 2023-12-02T00:29:28.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.464+0000 7fadc279e640 1 -- 172.21.15.46:0/469014623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadbc14a770 con 0x7fadbc0d14f0 2023-12-02T00:29:28.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.464+0000 7fadb1ffb640 1 -- 172.21.15.46:0/469014623 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadac004a20 con 0x7fadbc0d14f0 2023-12-02T00:29:28.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.465+0000 7fadc279e640 1 -- 172.21.15.46:0/469014623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadbc14a000 con 0x7fadbc0d14f0 2023-12-02T00:29:28.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.466+0000 7fadc279e640 1 -- 172.21.15.46:0/469014623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadbc069320 con 0x7fadbc0d14f0 2023-12-02T00:29:28.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.466+0000 7fadb1ffb640 1 -- 172.21.15.46:0/469014623 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fadac004b80 con 0x7fadbc0d14f0 2023-12-02T00:29:28.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.466+0000 7fadb1ffb640 1 --2- 172.21.15.46:0/469014623 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fada006ae40 0x7fada006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:28.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.466+0000 7fadb1ffb640 1 -- 172.21.15.46:0/469014623 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fadac088a80 con 0x7fadbc0d14f0 2023-12-02T00:29:28.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.467+0000 7fadc179c640 1 --2- 172.21.15.46:0/469014623 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fada006ae40 0x7fada006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:28.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.467+0000 7fadc179c640 1 --2- 172.21.15.46:0/469014623 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fada006ae40 0x7fada006d300 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fadb8000f60 tx=0x7fadb8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:28.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.471+0000 7fadb1ffb640 1 -- 172.21.15.46:0/469014623 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fadac053bc0 con 0x7fadbc0d14f0 2023-12-02T00:29:28.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.648+0000 7fadc279e640 1 -- 172.21.15.46:0/469014623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fadbc14af00 con 0x7fadbc0d14f0 2023-12-02T00:29:28.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.648+0000 7fadb1ffb640 1 -- 172.21.15.46:0/469014623 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fadac018020 con 0x7fadbc0d14f0 2023-12-02T00:29:28.650 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-12-02T00:29:28.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.650+0000 7fadc279e640 1 -- 172.21.15.46:0/469014623 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fada006ae40 msgr2=0x7fada006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:28.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.650+0000 7fadc279e640 1 --2- 172.21.15.46:0/469014623 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fada006ae40 0x7fada006d300 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fadb8000f60 tx=0x7fadb8009040 comp rx=0 tx=0).stop 2023-12-02T00:29:28.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.650+0000 7fadc279e640 1 -- 172.21.15.46:0/469014623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fadbc0d14f0 msgr2=0x7fadbc0ccb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:28.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.650+0000 7fadc279e640 1 --2- 172.21.15.46:0/469014623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fadbc0d14f0 0x7fadbc0ccb20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fadac00c490 tx=0x7fadac00c950 comp rx=0 tx=0).stop 2023-12-02T00:29:28.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.650+0000 7fadc279e640 1 -- 172.21.15.46:0/469014623 shutdown_connections 2023-12-02T00:29:28.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.650+0000 7fadc279e640 1 --2- 172.21.15.46:0/469014623 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fada006ae40 0x7fada006d300 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:28.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.650+0000 7fadc279e640 1 --2- 172.21.15.46:0/469014623 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fadbc15d1b0 0x7fadbc0cd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:28.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.650+0000 7fadc279e640 1 --2- 172.21.15.46:0/469014623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fadbc0d14f0 0x7fadbc0ccb20 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:28.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.650+0000 7fadc279e640 1 --2- 172.21.15.46:0/469014623 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fadbc0c8210 0x7fadbc149ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:28.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.650+0000 7fadc279e640 1 -- 172.21.15.46:0/469014623 >> 172.21.15.46:0/469014623 conn(0x7fadbc09c300 msgr2=0x7fadbc06bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:28.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.650+0000 7fadc279e640 1 -- 172.21.15.46:0/469014623 shutdown_connections 2023-12-02T00:29:28.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.650+0000 7fadc279e640 1 -- 172.21.15.46:0/469014623 wait complete. 2023-12-02T00:29:28.663 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-02T00:29:28.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.769+0000 7fe019c83640 1 Processor -- start 2023-12-02T00:29:28.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.769+0000 7fe019c83640 1 -- start start 2023-12-02T00:29:28.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.770+0000 7fe019c83640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe014160fb0 0x7fe014161440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:28.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.770+0000 7fe019c83640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe014161d20 con 0x7fe0140d0ed0 2023-12-02T00:29:28.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.770+0000 7fe019c83640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe01415e760 con 0x7fe01406ea50 2023-12-02T00:29:28.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.770+0000 7fe019c83640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe01415d9a0 con 0x7fe014160fb0 2023-12-02T00:29:28.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.770+0000 7fe013fff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe0140d0ed0 0x7fe0140d1320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50442/0 (socket says 172.21.15.46:50442) 2023-12-02T00:29:28.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.770+0000 7fe013fff640 1 -- 172.21.15.46:0/2409701579 learned_addr learned my addr 172.21.15.46:0/2409701579 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:28.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.770+0000 7fe012ffd640 1 --2- 172.21.15.46:0/2409701579 >> v2:172.21.15.162:3300/0 conn(0x7fe014160fb0 0x7fe014161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:28.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.771+0000 7fe0127fc640 1 -- 172.21.15.46:0/2409701579 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3392342123 0 0) 0x7fe014161d20 con 0x7fe0140d0ed0 2023-12-02T00:29:28.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.771+0000 7fe0127fc640 1 -- 172.21.15.46:0/2409701579 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdffc003520 con 0x7fe0140d0ed0 2023-12-02T00:29:28.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.771+0000 7fe012ffd640 1 -- 172.21.15.46:0/2409701579 >> v1:172.21.15.162:6789/0 conn(0x7fe01406ea50 legacy=0x7fe01406ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:28.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.771+0000 7fe012ffd640 1 -- 172.21.15.46:0/2409701579 >> v1:172.21.15.46:6789/0 conn(0x7fe0140d0ed0 legacy=0x7fe0140d1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:28.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.771+0000 7fe012ffd640 1 -- 172.21.15.46:0/2409701579 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe01415dc80 con 0x7fe014160fb0 2023-12-02T00:29:28.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.771+0000 7fe012ffd640 1 --2- 172.21.15.46:0/2409701579 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe014160fb0 0x7fe014161440 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe0080097a0 tx=0x7fe00802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfa5e6018aaaf87a server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:28.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.772+0000 7fe0127fc640 1 -- 172.21.15.46:0/2409701579 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe008031830 con 0x7fe014160fb0 2023-12-02T00:29:28.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.772+0000 7fe0127fc640 1 -- 172.21.15.46:0/2409701579 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe008031990 con 0x7fe014160fb0 2023-12-02T00:29:28.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.772+0000 7fe0127fc640 1 -- 172.21.15.46:0/2409701579 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe008031d00 con 0x7fe014160fb0 2023-12-02T00:29:28.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.772+0000 7fe019c83640 1 -- 172.21.15.46:0/2409701579 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe014160fb0 msgr2=0x7fe014161440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:28.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.772+0000 7fe019c83640 1 --2- 172.21.15.46:0/2409701579 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe014160fb0 0x7fe014161440 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe0080097a0 tx=0x7fe00802b8d0 comp rx=0 tx=0).stop 2023-12-02T00:29:28.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.772+0000 7fe019c83640 1 -- 172.21.15.46:0/2409701579 shutdown_connections 2023-12-02T00:29:28.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.772+0000 7fe019c83640 1 --2- 172.21.15.46:0/2409701579 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe014160fb0 0x7fe014161440 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:28.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.772+0000 7fe019c83640 1 -- 172.21.15.46:0/2409701579 >> 172.21.15.46:0/2409701579 conn(0x7fe01406a6b0 msgr2=0x7fe01406aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:28.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.772+0000 7fe019c83640 1 -- 172.21.15.46:0/2409701579 shutdown_connections 2023-12-02T00:29:28.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.772+0000 7fe019c83640 1 -- 172.21.15.46:0/2409701579 wait complete. 2023-12-02T00:29:28.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.773+0000 7fe019c83640 1 Processor -- start 2023-12-02T00:29:28.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.774+0000 7fe019c83640 1 -- start start 2023-12-02T00:29:28.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.774+0000 7fe019c83640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe01406ea50 0x7fe0141568e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:28.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.774+0000 7fe019c83640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0140d0ed0 0x7fe014157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:28.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.774+0000 7fe019c83640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe014160fb0 0x7fe014158a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:28.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.774+0000 7fe019c83640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01406cfe0 con 0x7fe01406ea50 2023-12-02T00:29:28.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.774+0000 7fe019c83640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01406c360 con 0x7fe014160fb0 2023-12-02T00:29:28.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.774+0000 7fe019c83640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe01406c640 con 0x7fe0140d0ed0 2023-12-02T00:29:28.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.774+0000 7fe0137fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe01406ea50 0x7fe0141568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:28.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.774+0000 7fe0137fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe01406ea50 0x7fe0141568e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52878/0 (socket says 172.21.15.46:52878) 2023-12-02T00:29:28.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.774+0000 7fe0137fe640 1 -- 172.21.15.46:0/3136493040 learned_addr learned my addr 172.21.15.46:0/3136493040 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:28.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.775+0000 7fe012ffd640 1 --2- 172.21.15.46:0/3136493040 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0140d0ed0 0x7fe014157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:28.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.775+0000 7fe013fff640 1 --2- 172.21.15.46:0/3136493040 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe014160fb0 0x7fe014158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:28.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.775+0000 7fe0137fe640 1 -- 172.21.15.46:0/3136493040 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0140d0ed0 msgr2=0x7fe014157ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:28.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.775+0000 7fe0137fe640 1 --2- 172.21.15.46:0/3136493040 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0140d0ed0 0x7fe014157ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:28.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.775+0000 7fe0137fe640 1 -- 172.21.15.46:0/3136493040 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe014160fb0 msgr2=0x7fe014158a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:28.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.775+0000 7fe0137fe640 1 --2- 172.21.15.46:0/3136493040 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe014160fb0 0x7fe014158a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:28.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.775+0000 7fe0137fe640 1 -- 172.21.15.46:0/3136493040 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe008009400 con 0x7fe01406ea50 2023-12-02T00:29:28.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.775+0000 7fe012ffd640 1 --2- 172.21.15.46:0/3136493040 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0140d0ed0 0x7fe014157ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:28.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.775+0000 7fe013fff640 1 --2- 172.21.15.46:0/3136493040 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe014160fb0 0x7fe014158a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:28.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.775+0000 7fe0137fe640 1 --2- 172.21.15.46:0/3136493040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe01406ea50 0x7fe0141568e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe00400c3c0 tx=0x7fe00400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:28.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.775+0000 7fdffbfff640 1 -- 172.21.15.46:0/3136493040 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0040052d0 con 0x7fe01406ea50 2023-12-02T00:29:28.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.775+0000 7fdffbfff640 1 -- 172.21.15.46:0/3136493040 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe004005450 con 0x7fe01406ea50 2023-12-02T00:29:28.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.775+0000 7fe019c83640 1 -- 172.21.15.46:0/3136493040 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe01406c920 con 0x7fe01406ea50 2023-12-02T00:29:28.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.775+0000 7fdffbfff640 1 -- 172.21.15.46:0/3136493040 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe004007260 con 0x7fe01406ea50 2023-12-02T00:29:28.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.775+0000 7fe019c83640 1 -- 172.21.15.46:0/3136493040 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe014156e20 con 0x7fe01406ea50 2023-12-02T00:29:28.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.776+0000 7fe019c83640 1 -- 172.21.15.46:0/3136493040 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe01406d120 con 0x7fe01406ea50 2023-12-02T00:29:28.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.776+0000 7fdffbfff640 1 -- 172.21.15.46:0/3136493040 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe0040073c0 con 0x7fe01406ea50 2023-12-02T00:29:28.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.777+0000 7fdffbfff640 1 --2- 172.21.15.46:0/3136493040 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe00006ae40 0x7fe00006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:28.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.777+0000 7fdffbfff640 1 -- 172.21.15.46:0/3136493040 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe004088920 con 0x7fe01406ea50 2023-12-02T00:29:28.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.777+0000 7fe012ffd640 1 --2- 172.21.15.46:0/3136493040 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe00006ae40 0x7fe00006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:28.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.778+0000 7fe012ffd640 1 --2- 172.21.15.46:0/3136493040 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe00006ae40 0x7fe00006d300 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe008036000 tx=0x7fe008008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:28.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.781+0000 7fdffbfff640 1 -- 172.21.15.46:0/3136493040 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe01406d120 con 0x7fe01406ea50 2023-12-02T00:29:28.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.973+0000 7fe019c83640 1 -- 172.21.15.46:0/3136493040 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe01406d120 con 0x7fe01406ea50 2023-12-02T00:29:28.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.973+0000 7fdffbfff640 1 -- 172.21.15.46:0/3136493040 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe01406d120 con 0x7fe01406ea50 2023-12-02T00:29:28.975 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2023-12-02T00:29:28.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.975+0000 7fe019c83640 1 -- 172.21.15.46:0/3136493040 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe00006ae40 msgr2=0x7fe00006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:28.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.975+0000 7fe019c83640 1 --2- 172.21.15.46:0/3136493040 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe00006ae40 0x7fe00006d300 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe008036000 tx=0x7fe008008030 comp rx=0 tx=0).stop 2023-12-02T00:29:28.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.975+0000 7fe019c83640 1 -- 172.21.15.46:0/3136493040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe01406ea50 msgr2=0x7fe0141568e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:28.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.975+0000 7fe019c83640 1 --2- 172.21.15.46:0/3136493040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe01406ea50 0x7fe0141568e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe00400c3c0 tx=0x7fe00400c880 comp rx=0 tx=0).stop 2023-12-02T00:29:28.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.975+0000 7fe019c83640 1 -- 172.21.15.46:0/3136493040 shutdown_connections 2023-12-02T00:29:28.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.975+0000 7fe019c83640 1 --2- 172.21.15.46:0/3136493040 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe00006ae40 0x7fe00006d300 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:28.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.975+0000 7fe019c83640 1 --2- 172.21.15.46:0/3136493040 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe014160fb0 0x7fe014158a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:28.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.975+0000 7fe019c83640 1 --2- 172.21.15.46:0/3136493040 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0140d0ed0 0x7fe014157ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:28.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.975+0000 7fe019c83640 1 --2- 172.21.15.46:0/3136493040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe01406ea50 0x7fe0141568e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:28.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.975+0000 7fe019c83640 1 -- 172.21.15.46:0/3136493040 >> 172.21.15.46:0/3136493040 conn(0x7fe01406a6b0 msgr2=0x7fe01413e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:28.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.975+0000 7fe019c83640 1 -- 172.21.15.46:0/3136493040 shutdown_connections 2023-12-02T00:29:28.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:28.975+0000 7fe019c83640 1 -- 172.21.15.46:0/3136493040 wait complete. 2023-12-02T00:29:28.988 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi162.front.sepia.ceph.com... 2023-12-02T00:29:28.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:28.988 DEBUG:teuthology.orchestra.run.smithi162:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-12-02T00:29:29.021 INFO:teuthology.orchestra.run:waiting for 300 2023-12-02T00:29:29.021 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi162.front.sepia.ceph.com 2023-12-02T00:29:29.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:29.021 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-12-02T00:29:29.158 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-02T00:29:29.158 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-02T00:29:29.174 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-02T00:29:29.174 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-02T00:29:29.231 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:29:29.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.376+0000 7f8f56496640 1 Processor -- start 2023-12-02T00:29:29.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.376+0000 7f8f56496640 1 -- start start 2023-12-02T00:29:29.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.377+0000 7f8f56496640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8f5015d330 0x7f8f500b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:29.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.377+0000 7f8f56496640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8f500d2910 0x7f8f500d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:29.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.377+0000 7f8f56496640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f500b2b10 con 0x7f8f500d3730 2023-12-02T00:29:29.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.377+0000 7f8f56496640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f500d3bc0 con 0x7f8f5015d330 2023-12-02T00:29:29.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.377+0000 7f8f56496640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f500d1590 con 0x7f8f500d2910 2023-12-02T00:29:29.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.377+0000 7f8f4ffff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8f5015d330 0x7f8f500b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:29.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.377+0000 7f8f4ffff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8f5015d330 0x7f8f500b8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52888/0 (socket says 172.21.15.46:52888) 2023-12-02T00:29:29.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.377+0000 7f8f4ffff640 1 -- 172.21.15.46:0/4200819590 learned_addr learned my addr 172.21.15.46:0/4200819590 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:29.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.378+0000 7f8f4f7fe640 1 --2- 172.21.15.46:0/4200819590 >> v2:172.21.15.162:3300/0 conn(0x7f8f500d2910 0x7f8f500d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:29.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.378+0000 7f8f4ffff640 1 -- 172.21.15.46:0/4200819590 >> v2:172.21.15.162:3300/0 conn(0x7f8f500d2910 msgr2=0x7f8f500d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:29.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.378+0000 7f8f4ffff640 1 --2- 172.21.15.46:0/4200819590 >> v2:172.21.15.162:3300/0 conn(0x7f8f500d2910 0x7f8f500d31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:29.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.378+0000 7f8f4ffff640 1 -- 172.21.15.46:0/4200819590 >> v1:172.21.15.162:6789/0 conn(0x7f8f500d3730 legacy=0x7f8f500b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:29.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.378+0000 7f8f4ffff640 1 -- 172.21.15.46:0/4200819590 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f500d1870 con 0x7f8f5015d330 2023-12-02T00:29:29.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.378+0000 7f8f4ffff640 1 --2- 172.21.15.46:0/4200819590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f5015d330 0x7f8f500b8b80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8f500d1540 tx=0x7f8f4002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c7678edba22d8c6 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:29.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.379+0000 7f8f4effd640 1 -- 172.21.15.46:0/4200819590 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f400317e0 con 0x7f8f5015d330 2023-12-02T00:29:29.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.379+0000 7f8f4effd640 1 -- 172.21.15.46:0/4200819590 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f40031940 con 0x7f8f5015d330 2023-12-02T00:29:29.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.379+0000 7f8f4effd640 1 -- 172.21.15.46:0/4200819590 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f40031cd0 con 0x7f8f5015d330 2023-12-02T00:29:29.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.379+0000 7f8f56496640 1 -- 172.21.15.46:0/4200819590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f5015d330 msgr2=0x7f8f500b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:29.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.379+0000 7f8f56496640 1 --2- 172.21.15.46:0/4200819590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f5015d330 0x7f8f500b8b80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8f500d1540 tx=0x7f8f4002b880 comp rx=0 tx=0).stop 2023-12-02T00:29:29.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.379+0000 7f8f56496640 1 -- 172.21.15.46:0/4200819590 shutdown_connections 2023-12-02T00:29:29.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.379+0000 7f8f56496640 1 --2- 172.21.15.46:0/4200819590 >> v2:172.21.15.162:3300/0 conn(0x7f8f500d2910 0x7f8f500d31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:29.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.379+0000 7f8f56496640 1 --2- 172.21.15.46:0/4200819590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f5015d330 0x7f8f500b8b80 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:29.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.379+0000 7f8f56496640 1 -- 172.21.15.46:0/4200819590 >> 172.21.15.46:0/4200819590 conn(0x7f8f50072020 msgr2=0x7f8f50072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:29.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.379+0000 7f8f56496640 1 -- 172.21.15.46:0/4200819590 shutdown_connections 2023-12-02T00:29:29.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.379+0000 7f8f56496640 1 -- 172.21.15.46:0/4200819590 wait complete. 2023-12-02T00:29:29.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.380+0000 7f8f56496640 1 Processor -- start 2023-12-02T00:29:29.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.381+0000 7f8f56496640 1 -- start start 2023-12-02T00:29:29.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.381+0000 7f8f56496640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f500d2910 0x7f8f501f23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:29.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.381+0000 7f8f56496640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f500d3730 0x7f8f50157f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:29.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.381+0000 7f8f56496640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f5015d330 0x7f8f500c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:29.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.381+0000 7f8f56496640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f50156330 con 0x7f8f5015d330 2023-12-02T00:29:29.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.381+0000 7f8f56496640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f500c75b0 con 0x7f8f500d2910 2023-12-02T00:29:29.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.381+0000 7f8f56496640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f500c7870 con 0x7f8f500d3730 2023-12-02T00:29:29.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.381+0000 7f8f54a33640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f5015d330 0x7f8f500c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:29.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.381+0000 7f8f54a33640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f5015d330 0x7f8f500c8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52902/0 (socket says 172.21.15.46:52902) 2023-12-02T00:29:29.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.381+0000 7f8f54a33640 1 -- 172.21.15.46:0/3164314664 learned_addr learned my addr 172.21.15.46:0/3164314664 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:29.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.382+0000 7f8f4ffff640 1 --2- 172.21.15.46:0/3164314664 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f500d2910 0x7f8f501f23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:29.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.382+0000 7f8f4f7fe640 1 --2- 172.21.15.46:0/3164314664 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f500d3730 0x7f8f50157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:29.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.382+0000 7f8f54a33640 1 -- 172.21.15.46:0/3164314664 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f500d3730 msgr2=0x7f8f50157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:29.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.382+0000 7f8f54a33640 1 --2- 172.21.15.46:0/3164314664 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f500d3730 0x7f8f50157f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:29.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.382+0000 7f8f54a33640 1 -- 172.21.15.46:0/3164314664 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f500d2910 msgr2=0x7f8f501f23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:29.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.382+0000 7f8f54a33640 1 --2- 172.21.15.46:0/3164314664 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f500d2910 0x7f8f501f23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:29.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.382+0000 7f8f54a33640 1 -- 172.21.15.46:0/3164314664 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f400094d0 con 0x7f8f5015d330 2023-12-02T00:29:29.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.382+0000 7f8f4ffff640 1 --2- 172.21.15.46:0/3164314664 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f500d2910 0x7f8f501f23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:29:29.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.382+0000 7f8f54a33640 1 --2- 172.21.15.46:0/3164314664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f5015d330 0x7f8f500c8ce0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8f4800c780 tx=0x7f8f4800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:29.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.382+0000 7f8f4cff9640 1 -- 172.21.15.46:0/3164314664 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f48004b60 con 0x7f8f5015d330 2023-12-02T00:29:29.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.382+0000 7f8f4cff9640 1 -- 172.21.15.46:0/3164314664 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f4800ce90 con 0x7f8f5015d330 2023-12-02T00:29:29.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.382+0000 7f8f4cff9640 1 -- 172.21.15.46:0/3164314664 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4800a850 con 0x7f8f5015d330 2023-12-02T00:29:29.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.382+0000 7f8f56496640 1 -- 172.21.15.46:0/3164314664 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f500c7b50 con 0x7f8f5015d330 2023-12-02T00:29:29.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.382+0000 7f8f56496640 1 -- 172.21.15.46:0/3164314664 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f501f2930 con 0x7f8f5015d330 2023-12-02T00:29:29.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.383+0000 7f8f56496640 1 -- 172.21.15.46:0/3164314664 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f501564a0 con 0x7f8f5015d330 2023-12-02T00:29:29.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.384+0000 7f8f4cff9640 1 -- 172.21.15.46:0/3164314664 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8f4800a9b0 con 0x7f8f5015d330 2023-12-02T00:29:29.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.384+0000 7f8f4cff9640 1 --2- 172.21.15.46:0/3164314664 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8f3006ae40 0x7f8f3006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:29.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.384+0000 7f8f4cff9640 1 -- 172.21.15.46:0/3164314664 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8f480897d0 con 0x7f8f5015d330 2023-12-02T00:29:29.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.384+0000 7f8f4ffff640 1 --2- 172.21.15.46:0/3164314664 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8f3006ae40 0x7f8f3006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:29.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.385+0000 7f8f4ffff640 1 --2- 172.21.15.46:0/3164314664 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8f3006ae40 0x7f8f3006d300 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8f40036000 tx=0x7f8f4003d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:29.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.388+0000 7f8f4cff9640 1 -- 172.21.15.46:0/3164314664 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8f48054880 con 0x7f8f5015d330 2023-12-02T00:29:29.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.566+0000 7f8f56496640 1 -- 172.21.15.46:0/3164314664 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8f50069400 con 0x7f8f5015d330 2023-12-02T00:29:29.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.567+0000 7f8f4cff9640 1 -- 172.21.15.46:0/3164314664 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f8f48057ea0 con 0x7f8f5015d330 2023-12-02T00:29:29.569 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:29:29.570 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":21,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:28:50.700112+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-02T00:28:47.273337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-02T00:28:48.060115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:29:29.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.569+0000 7f8f56496640 1 -- 172.21.15.46:0/3164314664 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8f3006ae40 msgr2=0x7f8f3006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:29.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.569+0000 7f8f56496640 1 --2- 172.21.15.46:0/3164314664 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8f3006ae40 0x7f8f3006d300 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8f40036000 tx=0x7f8f4003d040 comp rx=0 tx=0).stop 2023-12-02T00:29:29.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.569+0000 7f8f56496640 1 -- 172.21.15.46:0/3164314664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f5015d330 msgr2=0x7f8f500c8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.569+0000 7f8f56496640 1 --2- 172.21.15.46:0/3164314664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f5015d330 0x7f8f500c8ce0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8f4800c780 tx=0x7f8f4800cc40 comp rx=0 tx=0).stop 2023-12-02T00:29:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.569+0000 7f8f56496640 1 -- 172.21.15.46:0/3164314664 shutdown_connections 2023-12-02T00:29:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.569+0000 7f8f56496640 1 --2- 172.21.15.46:0/3164314664 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8f3006ae40 0x7f8f3006d300 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.569+0000 7f8f56496640 1 --2- 172.21.15.46:0/3164314664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f5015d330 0x7f8f500c8ce0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.569+0000 7f8f56496640 1 --2- 172.21.15.46:0/3164314664 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f500d3730 0x7f8f50157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.569+0000 7f8f56496640 1 --2- 172.21.15.46:0/3164314664 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f500d2910 0x7f8f501f23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.569+0000 7f8f56496640 1 -- 172.21.15.46:0/3164314664 >> 172.21.15.46:0/3164314664 conn(0x7f8f50072020 msgr2=0x7f8f500b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.569+0000 7f8f56496640 1 -- 172.21.15.46:0/3164314664 shutdown_connections 2023-12-02T00:29:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.569+0000 7f8f56496640 1 -- 172.21.15.46:0/3164314664 wait complete. 2023-12-02T00:29:29.583 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-02T00:28:37.213645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-02T00:28:47.273337+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-02T00:28:48.060115+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-02T00:29:29.583 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-02T00:29:29.693 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.691+0000 7efcbd27c640 1 Processor -- start 2023-12-02T00:29:29.693 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.691+0000 7efcbd27c640 1 -- start start 2023-12-02T00:29:29.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.691+0000 7efcbd27c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcb806ea50 0x7efcb806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:29.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.691+0000 7efcbd27c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7efcb80d0ed0 0x7efcb80d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:29.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.691+0000 7efcbd27c640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcb8161d20 con 0x7efcb8160fb0 2023-12-02T00:29:29.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.691+0000 7efcbd27c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcb8161b20 con 0x7efcb80d0ed0 2023-12-02T00:29:29.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.691+0000 7efcbd27c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcb806f370 con 0x7efcb806ea50 2023-12-02T00:29:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.692+0000 7efcae7fc640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7efcb8160fb0 0x7efcb8161410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50456/0 (socket says 172.21.15.46:50456) 2023-12-02T00:29:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.692+0000 7efcae7fc640 1 -- 172.21.15.46:0/2272899848 learned_addr learned my addr 172.21.15.46:0/2272899848 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.692+0000 7efcb77fe640 1 --2- 172.21.15.46:0/2272899848 >> v2:172.21.15.162:3300/0 conn(0x7efcb80d0ed0 0x7efcb80d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.692+0000 7efcb67fc640 1 -- 172.21.15.46:0/2272899848 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3644543336 0 0) 0x7efcb8161d20 con 0x7efcb8160fb0 2023-12-02T00:29:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.692+0000 7efcb6ffd640 1 --2- 172.21.15.46:0/2272899848 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcb806ea50 0x7efcb806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.692+0000 7efcb67fc640 1 -- 172.21.15.46:0/2272899848 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efca4003520 con 0x7efcb8160fb0 2023-12-02T00:29:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.692+0000 7efcb67fc640 1 -- 172.21.15.46:0/2272899848 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4255993205 0 0) 0x7efca4003520 con 0x7efcb8160fb0 2023-12-02T00:29:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.692+0000 7efcb67fc640 1 -- 172.21.15.46:0/2272899848 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcb806ea50 msgr2=0x7efcb806ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.692+0000 7efcb67fc640 1 --2- 172.21.15.46:0/2272899848 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcb806ea50 0x7efcb806ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.692+0000 7efcb67fc640 1 -- 172.21.15.46:0/2272899848 >> v2:172.21.15.162:3300/0 conn(0x7efcb80d0ed0 msgr2=0x7efcb80d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.692+0000 7efcb67fc640 1 --2- 172.21.15.46:0/2272899848 >> v2:172.21.15.162:3300/0 conn(0x7efcb80d0ed0 0x7efcb80d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.692+0000 7efcb67fc640 1 -- 172.21.15.46:0/2272899848 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcb815e760 con 0x7efcb8160fb0 2023-12-02T00:29:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.692+0000 7efcb67fc640 1 -- 172.21.15.46:0/2272899848 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7efca0001670 con 0x7efcb8160fb0 2023-12-02T00:29:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.692+0000 7efcb67fc640 1 -- 172.21.15.46:0/2272899848 >> v1:172.21.15.46:6789/0 conn(0x7efcb8160fb0 legacy=0x7efcb8161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.692+0000 7efcb67fc640 1 --2- 172.21.15.46:0/2272899848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efca4003970 0x7efca4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.692+0000 7efcb67fc640 1 --2- 172.21.15.46:0/2272899848 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcb806ea50 0x7efca40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.692+0000 7efcb67fc640 1 --2- 172.21.15.46:0/2272899848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efca4004d00 0x7efca4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.692+0000 7efcb67fc640 1 -- 172.21.15.46:0/2272899848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb8161b20 con 0x7efca4003970 2023-12-02T00:29:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.692+0000 7efcb67fc640 1 -- 172.21.15.46:0/2272899848 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb806f370 con 0x7efca4004d00 2023-12-02T00:29:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.692+0000 7efcb67fc640 1 -- 172.21.15.46:0/2272899848 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efca400b980 con 0x7efcb806ea50 2023-12-02T00:29:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.692+0000 7efcb6ffd640 1 --2- 172.21.15.46:0/2272899848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efca4003970 0x7efca4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.692+0000 7efcae7fc640 1 --2- 172.21.15.46:0/2272899848 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcb806ea50 0x7efca40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.693+0000 7efcb77fe640 1 --2- 172.21.15.46:0/2272899848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efca4004d00 0x7efca4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.693+0000 7efcb6ffd640 1 -- 172.21.15.46:0/2272899848 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcb806ea50 msgr2=0x7efca40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.693+0000 7efcb6ffd640 1 --2- 172.21.15.46:0/2272899848 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcb806ea50 0x7efca40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.693+0000 7efcb6ffd640 1 -- 172.21.15.46:0/2272899848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efca4004d00 msgr2=0x7efca4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.693+0000 7efcb6ffd640 1 --2- 172.21.15.46:0/2272899848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efca4004d00 0x7efca4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.693+0000 7efcb6ffd640 1 -- 172.21.15.46:0/2272899848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca4003520 con 0x7efca4003970 2023-12-02T00:29:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.693+0000 7efcb6ffd640 1 --2- 172.21.15.46:0/2272899848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efca4003970 0x7efca4003d90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7efca8009ac0 tx=0x7efca40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceabd4072b2e4381 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:29.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.694+0000 7efcb67fc640 1 -- 172.21.15.46:0/2272899848 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca802c6b0 con 0x7efca4003970 2023-12-02T00:29:29.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.694+0000 7efcb67fc640 1 -- 172.21.15.46:0/2272899848 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efca802c810 con 0x7efca4003970 2023-12-02T00:29:29.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.694+0000 7efcb67fc640 1 -- 172.21.15.46:0/2272899848 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca802cba0 con 0x7efca4003970 2023-12-02T00:29:29.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.694+0000 7efcbd27c640 1 -- 172.21.15.46:0/2272899848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efca4003970 msgr2=0x7efca4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:29.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.694+0000 7efcbd27c640 1 --2- 172.21.15.46:0/2272899848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efca4003970 0x7efca4003d90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7efca8009ac0 tx=0x7efca40057f0 comp rx=0 tx=0).stop 2023-12-02T00:29:29.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.694+0000 7efcb77fe640 1 -- 172.21.15.46:0/2272899848 reap_dead start 2023-12-02T00:29:29.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.694+0000 7efcbd27c640 1 -- 172.21.15.46:0/2272899848 shutdown_connections 2023-12-02T00:29:29.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.694+0000 7efcbd27c640 1 -- 172.21.15.46:0/2272899848 >> 172.21.15.46:0/2272899848 conn(0x7efcb806a6b0 msgr2=0x7efcb806aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:29.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.694+0000 7efcbd27c640 1 -- 172.21.15.46:0/2272899848 shutdown_connections 2023-12-02T00:29:29.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.694+0000 7efcbd27c640 1 -- 172.21.15.46:0/2272899848 wait complete. 2023-12-02T00:29:29.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.695+0000 7efcbd27c640 1 Processor -- start 2023-12-02T00:29:29.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.696+0000 7efcbd27c640 1 -- start start 2023-12-02T00:29:29.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.696+0000 7efcbd27c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efca4003970 0x7efcb8156af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:29.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.696+0000 7efcbd27c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcb815ac10 0x7efcb813e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:29.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.696+0000 7efcbd27c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcb8149410 0x7efcb81498a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:29.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.696+0000 7efcbd27c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb813f440 con 0x7efca4003970 2023-12-02T00:29:29.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.696+0000 7efcbd27c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb8147800 con 0x7efcb815ac10 2023-12-02T00:29:29.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.696+0000 7efcbd27c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcb8149de0 con 0x7efcb8149410 2023-12-02T00:29:29.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.696+0000 7efcb6ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efca4003970 0x7efcb8156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:29.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.696+0000 7efcb6ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efca4003970 0x7efcb8156af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52918/0 (socket says 172.21.15.46:52918) 2023-12-02T00:29:29.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.696+0000 7efcb6ffd640 1 -- 172.21.15.46:0/3074015729 learned_addr learned my addr 172.21.15.46:0/3074015729 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:29.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.697+0000 7efcb77fe640 1 --2- 172.21.15.46:0/3074015729 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcb8149410 0x7efcb81498a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:29.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.697+0000 7efcae7fc640 1 --2- 172.21.15.46:0/3074015729 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcb815ac10 0x7efcb813e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:29.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.697+0000 7efcb6ffd640 1 -- 172.21.15.46:0/3074015729 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcb8149410 msgr2=0x7efcb81498a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:29.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.697+0000 7efcb6ffd640 1 --2- 172.21.15.46:0/3074015729 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcb8149410 0x7efcb81498a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:29.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.697+0000 7efcb6ffd640 1 -- 172.21.15.46:0/3074015729 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcb815ac10 msgr2=0x7efcb813e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:29.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.697+0000 7efcb6ffd640 1 --2- 172.21.15.46:0/3074015729 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcb815ac10 0x7efcb813e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:29.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.697+0000 7efcb6ffd640 1 -- 172.21.15.46:0/3074015729 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcb813ee80 con 0x7efca4003970 2023-12-02T00:29:29.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.697+0000 7efcb77fe640 1 --2- 172.21.15.46:0/3074015729 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcb8149410 0x7efcb81498a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:29.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.697+0000 7efcae7fc640 1 --2- 172.21.15.46:0/3074015729 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcb815ac10 0x7efcb813e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:29.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.697+0000 7efcb6ffd640 1 --2- 172.21.15.46:0/3074015729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efca4003970 0x7efcb8156af0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7efca8037040 tx=0x7efca8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:29.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.697+0000 7efcaffff640 1 -- 172.21.15.46:0/3074015729 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca8003810 con 0x7efca4003970 2023-12-02T00:29:29.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.697+0000 7efcaffff640 1 -- 172.21.15.46:0/3074015729 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efca8003970 con 0x7efca4003970 2023-12-02T00:29:29.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.697+0000 7efcbd27c640 1 -- 172.21.15.46:0/3074015729 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcb813f160 con 0x7efca4003970 2023-12-02T00:29:29.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.697+0000 7efcaffff640 1 -- 172.21.15.46:0/3074015729 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca8003d00 con 0x7efca4003970 2023-12-02T00:29:29.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.697+0000 7efcbd27c640 1 -- 172.21.15.46:0/3074015729 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcb8157030 con 0x7efca4003970 2023-12-02T00:29:29.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.698+0000 7efcbd27c640 1 -- 172.21.15.46:0/3074015729 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcb81571c0 con 0x7efca4003970 2023-12-02T00:29:29.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.699+0000 7efcaffff640 1 -- 172.21.15.46:0/3074015729 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efca8040450 con 0x7efca4003970 2023-12-02T00:29:29.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.699+0000 7efcaffff640 1 --2- 172.21.15.46:0/3074015729 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7efc9806ae70 0x7efc9806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:29.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.699+0000 7efcaffff640 1 -- 172.21.15.46:0/3074015729 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efca80ab4d0 con 0x7efca4003970 2023-12-02T00:29:29.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.699+0000 7efcae7fc640 1 --2- 172.21.15.46:0/3074015729 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7efc9806ae70 0x7efc9806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:29.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.700+0000 7efcae7fc640 1 --2- 172.21.15.46:0/3074015729 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7efc9806ae70 0x7efc9806d330 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7efcb813dab0 tx=0x7efca0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:29.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.703+0000 7efcaffff640 1 -- 172.21.15.46:0/3074015729 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efca8076610 con 0x7efca4003970 2023-12-02T00:29:29.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.882+0000 7efcbd27c640 1 -- 172.21.15.46:0/3074015729 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efcb815f9c0 con 0x7efca4003970 2023-12-02T00:29:29.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.882+0000 7efcaffff640 1 -- 172.21.15.46:0/3074015729 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7efca80044e0 con 0x7efca4003970 2023-12-02T00:29:29.884 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-12-02T00:29:29.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.884+0000 7efcbd27c640 1 -- 172.21.15.46:0/3074015729 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7efc9806ae70 msgr2=0x7efc9806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:29.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.884+0000 7efcbd27c640 1 --2- 172.21.15.46:0/3074015729 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7efc9806ae70 0x7efc9806d330 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7efcb813dab0 tx=0x7efca0009040 comp rx=0 tx=0).stop 2023-12-02T00:29:29.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.884+0000 7efcbd27c640 1 -- 172.21.15.46:0/3074015729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efca4003970 msgr2=0x7efcb8156af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:29.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.884+0000 7efcbd27c640 1 --2- 172.21.15.46:0/3074015729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efca4003970 0x7efcb8156af0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7efca8037040 tx=0x7efca8004730 comp rx=0 tx=0).stop 2023-12-02T00:29:29.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.884+0000 7efcbd27c640 1 -- 172.21.15.46:0/3074015729 shutdown_connections 2023-12-02T00:29:29.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.884+0000 7efcbd27c640 1 --2- 172.21.15.46:0/3074015729 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7efc9806ae70 0x7efc9806d330 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:29.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.884+0000 7efcbd27c640 1 --2- 172.21.15.46:0/3074015729 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcb8149410 0x7efcb81498a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:29.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.884+0000 7efcbd27c640 1 --2- 172.21.15.46:0/3074015729 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcb815ac10 0x7efcb813e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:29.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.884+0000 7efcbd27c640 1 --2- 172.21.15.46:0/3074015729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efca4003970 0x7efcb8156af0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:29.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.884+0000 7efcbd27c640 1 -- 172.21.15.46:0/3074015729 >> 172.21.15.46:0/3074015729 conn(0x7efcb806a6b0 msgr2=0x7efcb815f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:29.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.884+0000 7efcbd27c640 1 -- 172.21.15.46:0/3074015729 shutdown_connections 2023-12-02T00:29:29.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:29.884+0000 7efcbd27c640 1 -- 172.21.15.46:0/3074015729 wait complete. 2023-12-02T00:29:29.897 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-02T00:29:30.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.004+0000 7f4f06953640 1 Processor -- start 2023-12-02T00:29:30.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.004+0000 7f4f06953640 1 -- start start 2023-12-02T00:29:30.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.005+0000 7f4f06953640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4f0015d2d0 0x7f4f000b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:30.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.005+0000 7f4f06953640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4f000d2040 0x7f4f000b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:30.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.005+0000 7f4f06953640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f000cda40 0x7f4f000cded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:30.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.005+0000 7f4f06953640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f000d1bb0 con 0x7f4f000d2040 2023-12-02T00:29:30.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.005+0000 7f4f06953640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f0015d6b0 con 0x7f4f0015d2d0 2023-12-02T00:29:30.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.005+0000 7f4f06953640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f000ce700 con 0x7f4f000cda40 2023-12-02T00:29:30.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.005+0000 7f4eff7fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4f000d2040 0x7f4f000b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:30.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.005+0000 7f4eff7fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4f000d2040 0x7f4f000b51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52922/0 (socket says 172.21.15.46:52922) 2023-12-02T00:29:30.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.005+0000 7f4eff7fe640 1 -- 172.21.15.46:0/69225655 learned_addr learned my addr 172.21.15.46:0/69225655 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:30.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.005+0000 7f4f04ef0640 1 --2- 172.21.15.46:0/69225655 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f000cda40 0x7f4f000cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:30.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.005+0000 7f4efffff640 1 --2- 172.21.15.46:0/69225655 >> v2:172.21.15.162:3300/0 conn(0x7f4f0015d2d0 0x7f4f000b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:30.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.006+0000 7f4eff7fe640 1 -- 172.21.15.46:0/69225655 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f000cda40 msgr2=0x7f4f000cded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:30.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.006+0000 7f4eff7fe640 1 --2- 172.21.15.46:0/69225655 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f000cda40 0x7f4f000cded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:30.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.006+0000 7f4eff7fe640 1 -- 172.21.15.46:0/69225655 >> v2:172.21.15.162:3300/0 conn(0x7f4f0015d2d0 msgr2=0x7f4f000b4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:30.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.006+0000 7f4eff7fe640 1 --2- 172.21.15.46:0/69225655 >> v2:172.21.15.162:3300/0 conn(0x7f4f0015d2d0 0x7f4f000b4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:30.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.006+0000 7f4eff7fe640 1 -- 172.21.15.46:0/69225655 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f000d1280 con 0x7f4f000d2040 2023-12-02T00:29:30.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.006+0000 7f4eff7fe640 1 --2- 172.21.15.46:0/69225655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f000d2040 0x7f4f000b51a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4f0006c5d0 tx=0x7f4ef402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d309521b45b93769 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:30.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.007+0000 7f4efeffd640 1 -- 172.21.15.46:0/69225655 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef4031870 con 0x7f4f000d2040 2023-12-02T00:29:30.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.007+0000 7f4efeffd640 1 -- 172.21.15.46:0/69225655 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ef40319d0 con 0x7f4f000d2040 2023-12-02T00:29:30.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.007+0000 7f4efeffd640 1 -- 172.21.15.46:0/69225655 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef4031d60 con 0x7f4f000d2040 2023-12-02T00:29:30.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.007+0000 7f4f06953640 1 -- 172.21.15.46:0/69225655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f000d2040 msgr2=0x7f4f000b51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:30.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.007+0000 7f4f06953640 1 --2- 172.21.15.46:0/69225655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f000d2040 0x7f4f000b51a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4f0006c5d0 tx=0x7f4ef402b910 comp rx=0 tx=0).stop 2023-12-02T00:29:30.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.007+0000 7f4f06953640 1 -- 172.21.15.46:0/69225655 shutdown_connections 2023-12-02T00:29:30.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.007+0000 7f4f06953640 1 --2- 172.21.15.46:0/69225655 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f000cda40 0x7f4f000cded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:30.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.007+0000 7f4f06953640 1 --2- 172.21.15.46:0/69225655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f000d2040 0x7f4f000b51a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:30.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.007+0000 7f4f06953640 1 --2- 172.21.15.46:0/69225655 >> v2:172.21.15.162:3300/0 conn(0x7f4f0015d2d0 0x7f4f000b4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:30.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.007+0000 7f4f06953640 1 -- 172.21.15.46:0/69225655 >> 172.21.15.46:0/69225655 conn(0x7f4f00072050 msgr2=0x7f4f00072460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:30.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.007+0000 7f4f06953640 1 -- 172.21.15.46:0/69225655 shutdown_connections 2023-12-02T00:29:30.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.007+0000 7f4f06953640 1 -- 172.21.15.46:0/69225655 wait complete. 2023-12-02T00:29:30.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.008+0000 7f4f06953640 1 Processor -- start 2023-12-02T00:29:30.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.009+0000 7f4f06953640 1 -- start start 2023-12-02T00:29:30.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.009+0000 7f4f06953640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f000cda40 0x7f4f000cc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:30.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.009+0000 7f4f06953640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f000d2040 0x7f4f001e87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:30.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.009+0000 7f4f06953640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f0015d2d0 0x7f4f0014f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:30.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.009+0000 7f4f06953640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f000c7db0 con 0x7f4f0015d2d0 2023-12-02T00:29:30.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.009+0000 7f4f06953640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f001e9260 con 0x7f4f000cda40 2023-12-02T00:29:30.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.009+0000 7f4f06953640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f001e9540 con 0x7f4f000d2040 2023-12-02T00:29:30.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.010+0000 7f4f04ef0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f0015d2d0 0x7f4f0014f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:30.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.010+0000 7f4f04ef0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f0015d2d0 0x7f4f0014f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52938/0 (socket says 172.21.15.46:52938) 2023-12-02T00:29:30.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.010+0000 7f4f04ef0640 1 -- 172.21.15.46:0/3853410287 learned_addr learned my addr 172.21.15.46:0/3853410287 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:30.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.010+0000 7f4eff7fe640 1 --2- 172.21.15.46:0/3853410287 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f000d2040 0x7f4f001e87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:30.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.010+0000 7f4efffff640 1 --2- 172.21.15.46:0/3853410287 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f000cda40 0x7f4f000cc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:30.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.010+0000 7f4f04ef0640 1 -- 172.21.15.46:0/3853410287 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f000d2040 msgr2=0x7f4f001e87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:30.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.010+0000 7f4f04ef0640 1 --2- 172.21.15.46:0/3853410287 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f000d2040 0x7f4f001e87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:30.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.010+0000 7f4f04ef0640 1 -- 172.21.15.46:0/3853410287 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f000cda40 msgr2=0x7f4f000cc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:30.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.010+0000 7f4f04ef0640 1 --2- 172.21.15.46:0/3853410287 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f000cda40 0x7f4f000cc6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:30.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.010+0000 7f4f04ef0640 1 -- 172.21.15.46:0/3853410287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef40094d0 con 0x7f4f0015d2d0 2023-12-02T00:29:30.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.010+0000 7f4f04ef0640 1 --2- 172.21.15.46:0/3853410287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f0015d2d0 0x7f4f0014f230 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4ef800eed0 tx=0x7f4ef800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:30.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.010+0000 7f4efcff9640 1 -- 172.21.15.46:0/3853410287 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8010040 con 0x7f4f0015d2d0 2023-12-02T00:29:30.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.010+0000 7f4efcff9640 1 -- 172.21.15.46:0/3853410287 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ef8014400 con 0x7f4f0015d2d0 2023-12-02T00:29:30.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.010+0000 7f4f06953640 1 -- 172.21.15.46:0/3853410287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f000c8000 con 0x7f4f0015d2d0 2023-12-02T00:29:30.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.010+0000 7f4efcff9640 1 -- 172.21.15.46:0/3853410287 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8009ea0 con 0x7f4f0015d2d0 2023-12-02T00:29:30.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.010+0000 7f4f06953640 1 -- 172.21.15.46:0/3853410287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f000b4af0 con 0x7f4f0015d2d0 2023-12-02T00:29:30.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.011+0000 7f4f06953640 1 -- 172.21.15.46:0/3853410287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f000c8520 con 0x7f4f0015d2d0 2023-12-02T00:29:30.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.012+0000 7f4efcff9640 1 -- 172.21.15.46:0/3853410287 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4ef80147b0 con 0x7f4f0015d2d0 2023-12-02T00:29:30.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.012+0000 7f4efcff9640 1 --2- 172.21.15.46:0/3853410287 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f4eec06ae40 0x7f4eec06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:30.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.012+0000 7f4efcff9640 1 -- 172.21.15.46:0/3853410287 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f4ef8089c20 con 0x7f4f0015d2d0 2023-12-02T00:29:30.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.012+0000 7f4efffff640 1 --2- 172.21.15.46:0/3853410287 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f4eec06ae40 0x7f4eec06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:30.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.013+0000 7f4efffff640 1 --2- 172.21.15.46:0/3853410287 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f4eec06ae40 0x7f4eec06d300 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0009aa0 tx=0x7f4ef0009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:30.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.017+0000 7f4efcff9640 1 -- 172.21.15.46:0/3853410287 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4f000c8520 con 0x7f4f0015d2d0 2023-12-02T00:29:30.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.191+0000 7f4f06953640 1 -- 172.21.15.46:0/3853410287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4f000c8520 con 0x7f4f0015d2d0 2023-12-02T00:29:30.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.192+0000 7f4efcff9640 1 -- 172.21.15.46:0/3853410287 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4f000c8520 con 0x7f4f0015d2d0 2023-12-02T00:29:30.194 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-12-02T00:29:30.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.193+0000 7f4f06953640 1 -- 172.21.15.46:0/3853410287 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f4eec06ae40 msgr2=0x7f4eec06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:30.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.193+0000 7f4f06953640 1 --2- 172.21.15.46:0/3853410287 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f4eec06ae40 0x7f4eec06d300 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0009aa0 tx=0x7f4ef0009190 comp rx=0 tx=0).stop 2023-12-02T00:29:30.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.193+0000 7f4f06953640 1 -- 172.21.15.46:0/3853410287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f0015d2d0 msgr2=0x7f4f0014f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:30.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.193+0000 7f4f06953640 1 --2- 172.21.15.46:0/3853410287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f0015d2d0 0x7f4f0014f230 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4ef800eed0 tx=0x7f4ef800b480 comp rx=0 tx=0).stop 2023-12-02T00:29:30.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.193+0000 7f4f06953640 1 -- 172.21.15.46:0/3853410287 shutdown_connections 2023-12-02T00:29:30.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.193+0000 7f4f06953640 1 --2- 172.21.15.46:0/3853410287 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f4eec06ae40 0x7f4eec06d300 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:30.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.193+0000 7f4f06953640 1 --2- 172.21.15.46:0/3853410287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f0015d2d0 0x7f4f0014f230 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:30.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.193+0000 7f4f06953640 1 --2- 172.21.15.46:0/3853410287 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f000d2040 0x7f4f001e87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:30.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.193+0000 7f4f06953640 1 --2- 172.21.15.46:0/3853410287 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f000cda40 0x7f4f000cc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:30.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.194+0000 7f4f06953640 1 -- 172.21.15.46:0/3853410287 >> 172.21.15.46:0/3853410287 conn(0x7f4f00072050 msgr2=0x7f4f00072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:30.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.194+0000 7f4f06953640 1 -- 172.21.15.46:0/3853410287 shutdown_connections 2023-12-02T00:29:30.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.194+0000 7f4f06953640 1 -- 172.21.15.46:0/3853410287 wait complete. 2023-12-02T00:29:30.207 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-02T00:29:30.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.314+0000 7fa401b33640 1 Processor -- start 2023-12-02T00:29:30.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.314+0000 7fa401b33640 1 -- start start 2023-12-02T00:29:30.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.315+0000 7fa401b33640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa3fc160fb0 0x7fa3fc161440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:30.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.315+0000 7fa401b33640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa3fc0d0ed0 0x7fa3fc0d1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:30.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.315+0000 7fa401b33640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3fc161d20 con 0x7fa3fc06ea50 2023-12-02T00:29:30.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.315+0000 7fa401b33640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3fc15e760 con 0x7fa3fc160fb0 2023-12-02T00:29:30.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.315+0000 7fa401b33640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3fc15d9a0 con 0x7fa3fc0d0ed0 2023-12-02T00:29:30.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.315+0000 7fa3faffd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa3fc160fb0 0x7fa3fc161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:30.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.315+0000 7fa3faffd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa3fc160fb0 0x7fa3fc161440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52948/0 (socket says 172.21.15.46:52948) 2023-12-02T00:29:30.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.315+0000 7fa3faffd640 1 -- 172.21.15.46:0/37970676 learned_addr learned my addr 172.21.15.46:0/37970676 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:30.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.315+0000 7fa3fbfff640 1 --2- 172.21.15.46:0/37970676 >> v2:172.21.15.162:3300/0 conn(0x7fa3fc0d0ed0 0x7fa3fc0d1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:30.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.316+0000 7fa3faffd640 1 -- 172.21.15.46:0/37970676 >> v2:172.21.15.162:3300/0 conn(0x7fa3fc0d0ed0 msgr2=0x7fa3fc0d1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:30.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.316+0000 7fa3faffd640 1 --2- 172.21.15.46:0/37970676 >> v2:172.21.15.162:3300/0 conn(0x7fa3fc0d0ed0 0x7fa3fc0d1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:30.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.316+0000 7fa3faffd640 1 -- 172.21.15.46:0/37970676 >> v1:172.21.15.162:6789/0 conn(0x7fa3fc06ea50 legacy=0x7fa3fc06ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:30.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.316+0000 7fa3fa7fc640 1 -- 172.21.15.46:0/37970676 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3435670227 0 0) 0x7fa3fc161d20 con 0x7fa3fc06ea50 2023-12-02T00:29:30.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.316+0000 7fa3faffd640 1 -- 172.21.15.46:0/37970676 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3fc15dc80 con 0x7fa3fc160fb0 2023-12-02T00:29:30.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.316+0000 7fa3faffd640 1 --2- 172.21.15.46:0/37970676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3fc160fb0 0x7fa3fc161440 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc0b55a0 tx=0x7fa3f002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d254e82a3f5c4406 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:30.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.316+0000 7fa3fa7fc640 1 -- 172.21.15.46:0/37970676 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f0031870 con 0x7fa3fc160fb0 2023-12-02T00:29:30.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.317+0000 7fa3fa7fc640 1 -- 172.21.15.46:0/37970676 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3f00319d0 con 0x7fa3fc160fb0 2023-12-02T00:29:30.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.317+0000 7fa3fa7fc640 1 -- 172.21.15.46:0/37970676 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f0031d60 con 0x7fa3fc160fb0 2023-12-02T00:29:30.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.317+0000 7fa401b33640 1 -- 172.21.15.46:0/37970676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3fc160fb0 msgr2=0x7fa3fc161440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:30.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.317+0000 7fa401b33640 1 --2- 172.21.15.46:0/37970676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3fc160fb0 0x7fa3fc161440 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc0b55a0 tx=0x7fa3f002b910 comp rx=0 tx=0).stop 2023-12-02T00:29:30.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.317+0000 7fa401b33640 1 -- 172.21.15.46:0/37970676 shutdown_connections 2023-12-02T00:29:30.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.317+0000 7fa401b33640 1 --2- 172.21.15.46:0/37970676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3fc160fb0 0x7fa3fc161440 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:30.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.317+0000 7fa401b33640 1 --2- 172.21.15.46:0/37970676 >> v2:172.21.15.162:3300/0 conn(0x7fa3fc0d0ed0 0x7fa3fc0d1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:30.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.317+0000 7fa401b33640 1 -- 172.21.15.46:0/37970676 >> 172.21.15.46:0/37970676 conn(0x7fa3fc06a6b0 msgr2=0x7fa3fc06aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:30.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.317+0000 7fa401b33640 1 -- 172.21.15.46:0/37970676 shutdown_connections 2023-12-02T00:29:30.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.317+0000 7fa401b33640 1 -- 172.21.15.46:0/37970676 wait complete. 2023-12-02T00:29:30.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.318+0000 7fa401b33640 1 Processor -- start 2023-12-02T00:29:30.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.318+0000 7fa401b33640 1 -- start start 2023-12-02T00:29:30.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.319+0000 7fa401b33640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3fc06ea50 0x7fa3fc0b4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:30.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.319+0000 7fa401b33640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa3fc0d0ed0 0x7fa3fc14c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:30.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.319+0000 7fa401b33640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3fc160fb0 0x7fa3fc1594d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:30.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.319+0000 7fa401b33640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3fc14ccf0 con 0x7fa3fc06ea50 2023-12-02T00:29:30.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.319+0000 7fa401b33640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3fc158430 con 0x7fa3fc160fb0 2023-12-02T00:29:30.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.319+0000 7fa401b33640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3fc158710 con 0x7fa3fc0d0ed0 2023-12-02T00:29:30.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.319+0000 7fa3fb7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3fc06ea50 0x7fa3fc0b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:30.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.319+0000 7fa3fb7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3fc06ea50 0x7fa3fc0b4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52958/0 (socket says 172.21.15.46:52958) 2023-12-02T00:29:30.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.319+0000 7fa3fb7fe640 1 -- 172.21.15.46:0/2921194991 learned_addr learned my addr 172.21.15.46:0/2921194991 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:30.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.319+0000 7fa3fbfff640 1 --2- 172.21.15.46:0/2921194991 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3fc160fb0 0x7fa3fc1594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:30.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.319+0000 7fa3faffd640 1 --2- 172.21.15.46:0/2921194991 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa3fc0d0ed0 0x7fa3fc14c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:30.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.319+0000 7fa3fb7fe640 1 -- 172.21.15.46:0/2921194991 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa3fc0d0ed0 msgr2=0x7fa3fc14c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:30.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.319+0000 7fa3fb7fe640 1 --2- 172.21.15.46:0/2921194991 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa3fc0d0ed0 0x7fa3fc14c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:30.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.319+0000 7fa3fb7fe640 1 -- 172.21.15.46:0/2921194991 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3fc160fb0 msgr2=0x7fa3fc1594d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:30.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.319+0000 7fa3fb7fe640 1 --2- 172.21.15.46:0/2921194991 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3fc160fb0 0x7fa3fc1594d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:30.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.319+0000 7fa3fb7fe640 1 -- 172.21.15.46:0/2921194991 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f00094d0 con 0x7fa3fc06ea50 2023-12-02T00:29:30.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.319+0000 7fa3faffd640 1 --2- 172.21.15.46:0/2921194991 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa3fc0d0ed0 0x7fa3fc14c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:29:30.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.320+0000 7fa3fb7fe640 1 --2- 172.21.15.46:0/2921194991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3fc06ea50 0x7fa3fc0b4170 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec00c410 tx=0x7fa3ec00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:30.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.320+0000 7fa3e3fff640 1 -- 172.21.15.46:0/2921194991 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec0035f0 con 0x7fa3fc06ea50 2023-12-02T00:29:30.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.320+0000 7fa3e3fff640 1 -- 172.21.15.46:0/2921194991 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3ec003770 con 0x7fa3fc06ea50 2023-12-02T00:29:30.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.320+0000 7fa401b33640 1 -- 172.21.15.46:0/2921194991 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3fc1589f0 con 0x7fa3fc06ea50 2023-12-02T00:29:30.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.320+0000 7fa3e3fff640 1 -- 172.21.15.46:0/2921194991 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec003b20 con 0x7fa3fc06ea50 2023-12-02T00:29:30.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.320+0000 7fa401b33640 1 -- 172.21.15.46:0/2921194991 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3fc158fb0 con 0x7fa3fc06ea50 2023-12-02T00:29:30.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.321+0000 7fa401b33640 1 -- 172.21.15.46:0/2921194991 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3fc069400 con 0x7fa3fc06ea50 2023-12-02T00:29:30.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.321+0000 7fa3e3fff640 1 -- 172.21.15.46:0/2921194991 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa3ec003c80 con 0x7fa3fc06ea50 2023-12-02T00:29:30.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.322+0000 7fa3e3fff640 1 --2- 172.21.15.46:0/2921194991 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa3e806ae40 0x7fa3e806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:30.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.322+0000 7fa3e3fff640 1 -- 172.21.15.46:0/2921194991 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fa3ec088c30 con 0x7fa3fc06ea50 2023-12-02T00:29:30.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.322+0000 7fa3faffd640 1 --2- 172.21.15.46:0/2921194991 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa3e806ae40 0x7fa3e806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:30.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.323+0000 7fa3faffd640 1 --2- 172.21.15.46:0/2921194991 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa3e806ae40 0x7fa3e806d300 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0036000 tx=0x7fa3f0008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:30.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.326+0000 7fa3e3fff640 1 -- 172.21.15.46:0/2921194991 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa3ec053ce0 con 0x7fa3fc06ea50 2023-12-02T00:29:30.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.503+0000 7fa401b33640 1 -- 172.21.15.46:0/2921194991 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa3fc070d30 con 0x7fa3fc06ea50 2023-12-02T00:29:30.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.504+0000 7fa3e3fff640 1 -- 172.21.15.46:0/2921194991 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa3ec017020 con 0x7fa3fc06ea50 2023-12-02T00:29:30.506 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-12-02T00:29:30.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.505+0000 7fa401b33640 1 -- 172.21.15.46:0/2921194991 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa3e806ae40 msgr2=0x7fa3e806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:30.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.505+0000 7fa401b33640 1 --2- 172.21.15.46:0/2921194991 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa3e806ae40 0x7fa3e806d300 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0036000 tx=0x7fa3f0008030 comp rx=0 tx=0).stop 2023-12-02T00:29:30.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.505+0000 7fa401b33640 1 -- 172.21.15.46:0/2921194991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3fc06ea50 msgr2=0x7fa3fc0b4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:30.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.505+0000 7fa401b33640 1 --2- 172.21.15.46:0/2921194991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3fc06ea50 0x7fa3fc0b4170 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec00c410 tx=0x7fa3ec00c8d0 comp rx=0 tx=0).stop 2023-12-02T00:29:30.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.505+0000 7fa401b33640 1 -- 172.21.15.46:0/2921194991 shutdown_connections 2023-12-02T00:29:30.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.505+0000 7fa401b33640 1 --2- 172.21.15.46:0/2921194991 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa3e806ae40 0x7fa3e806d300 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:30.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.505+0000 7fa401b33640 1 --2- 172.21.15.46:0/2921194991 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3fc160fb0 0x7fa3fc1594d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:30.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.505+0000 7fa401b33640 1 --2- 172.21.15.46:0/2921194991 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa3fc0d0ed0 0x7fa3fc14c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:30.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.505+0000 7fa401b33640 1 --2- 172.21.15.46:0/2921194991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3fc06ea50 0x7fa3fc0b4170 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:30.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.505+0000 7fa401b33640 1 -- 172.21.15.46:0/2921194991 >> 172.21.15.46:0/2921194991 conn(0x7fa3fc06a6b0 msgr2=0x7fa3fc0b1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:30.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.506+0000 7fa401b33640 1 -- 172.21.15.46:0/2921194991 shutdown_connections 2023-12-02T00:29:30.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.506+0000 7fa401b33640 1 -- 172.21.15.46:0/2921194991 wait complete. 2023-12-02T00:29:30.519 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-02T00:29:30.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.625+0000 7f965c609640 1 Processor -- start 2023-12-02T00:29:30.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.625+0000 7f965c609640 1 -- start start 2023-12-02T00:29:30.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.626+0000 7f965c609640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f965406abe0 0x7f96541618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:30.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.626+0000 7f965c609640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96541406f0 con 0x7f9654140310 2023-12-02T00:29:30.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.626+0000 7f965c609640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9654143960 con 0x7f96541409d0 2023-12-02T00:29:30.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.626+0000 7f965c609640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9654161e20 con 0x7f965406abe0 2023-12-02T00:29:30.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.626+0000 7f965a3a5640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9654140310 0x7f965406a4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50472/0 (socket says 172.21.15.46:50472) 2023-12-02T00:29:30.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.626+0000 7f965a3a5640 1 -- 172.21.15.46:0/1786129178 learned_addr learned my addr 172.21.15.46:0/1786129178 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:30.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.627+0000 7f9659ba4640 1 --2- 172.21.15.46:0/1786129178 >> v2:172.21.15.162:3300/0 conn(0x7f965406abe0 0x7f96541618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:30.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.627+0000 7f96593a3640 1 -- 172.21.15.46:0/1786129178 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1951404248 0 0) 0x7f96541406f0 con 0x7f9654140310 2023-12-02T00:29:30.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.627+0000 7f96593a3640 1 -- 172.21.15.46:0/1786129178 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9640003520 con 0x7f9654140310 2023-12-02T00:29:30.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.627+0000 7f96593a3640 1 -- 172.21.15.46:0/1786129178 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1958469056 0 0) 0x7f9640003520 con 0x7f9654140310 2023-12-02T00:29:30.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.627+0000 7f96593a3640 1 -- 172.21.15.46:0/1786129178 >> v2:172.21.15.162:3300/0 conn(0x7f965406abe0 msgr2=0x7f96541618e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:30.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.627+0000 7f96593a3640 1 --2- 172.21.15.46:0/1786129178 >> v2:172.21.15.162:3300/0 conn(0x7f965406abe0 0x7f96541618e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:30.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.627+0000 7f96593a3640 1 -- 172.21.15.46:0/1786129178 >> v1:172.21.15.162:6789/0 conn(0x7f96541409d0 legacy=0x7f965406efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:30.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.627+0000 7f96593a3640 1 -- 172.21.15.46:0/1786129178 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9654142de0 con 0x7f9654140310 2023-12-02T00:29:30.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.627+0000 7f96593a3640 1 -- 172.21.15.46:0/1786129178 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7f964c001670 con 0x7f9654140310 2023-12-02T00:29:30.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.627+0000 7f96593a3640 1 -- 172.21.15.46:0/1786129178 >> v1:172.21.15.46:6789/0 conn(0x7f9654140310 legacy=0x7f965406a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:30.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.627+0000 7f96593a3640 1 --2- 172.21.15.46:0/1786129178 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9640003970 0x7f9640003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:30.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.627+0000 7f96593a3640 1 --2- 172.21.15.46:0/1786129178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9640004370 0x7f9640004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:30.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.627+0000 7f96593a3640 1 --2- 172.21.15.46:0/1786129178 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96400050e0 0x7f964000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:30.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.627+0000 7f96593a3640 1 -- 172.21.15.46:0/1786129178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9654161e20 con 0x7f9640004370 2023-12-02T00:29:30.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.627+0000 7f96593a3640 1 -- 172.21.15.46:0/1786129178 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964000c020 con 0x7f96400050e0 2023-12-02T00:29:30.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.627+0000 7f96593a3640 1 -- 172.21.15.46:0/1786129178 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f964000c1e0 con 0x7f9640003970 2023-12-02T00:29:30.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.627+0000 7f9659ba4640 1 --2- 172.21.15.46:0/1786129178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9640004370 0x7f9640004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:30.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.627+0000 7f965a3a5640 1 --2- 172.21.15.46:0/1786129178 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9640003970 0x7f9640003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:30.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.627+0000 7f965aba6640 1 --2- 172.21.15.46:0/1786129178 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96400050e0 0x7f964000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:30.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.627+0000 7f9659ba4640 1 -- 172.21.15.46:0/1786129178 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9640003970 msgr2=0x7f9640003d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:30.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.627+0000 7f9659ba4640 1 --2- 172.21.15.46:0/1786129178 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9640003970 0x7f9640003d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:30.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.627+0000 7f9659ba4640 1 -- 172.21.15.46:0/1786129178 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96400050e0 msgr2=0x7f964000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:30.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.627+0000 7f9659ba4640 1 --2- 172.21.15.46:0/1786129178 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96400050e0 0x7f964000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:30.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.627+0000 7f9659ba4640 1 -- 172.21.15.46:0/1786129178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9640003520 con 0x7f9640004370 2023-12-02T00:29:30.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.627+0000 7f965aba6640 1 --2- 172.21.15.46:0/1786129178 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96400050e0 0x7f964000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:30.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.627+0000 7f965aba6640 1 -- 172.21.15.46:0/1786129178 reap_dead start 2023-12-02T00:29:30.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.628+0000 7f9659ba4640 1 --2- 172.21.15.46:0/1786129178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9640004370 0x7f9640004840 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9644009ac0 tx=0x7f96400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f4d187a7fd3388 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:30.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.628+0000 7f96593a3640 1 -- 172.21.15.46:0/1786129178 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964402c6b0 con 0x7f9640004370 2023-12-02T00:29:30.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.628+0000 7f96593a3640 1 -- 172.21.15.46:0/1786129178 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f964402c810 con 0x7f9640004370 2023-12-02T00:29:30.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.628+0000 7f96593a3640 1 -- 172.21.15.46:0/1786129178 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964402cba0 con 0x7f9640004370 2023-12-02T00:29:30.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.629+0000 7f965c609640 1 -- 172.21.15.46:0/1786129178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9640004370 msgr2=0x7f9640004840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:30.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.629+0000 7f965c609640 1 --2- 172.21.15.46:0/1786129178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9640004370 0x7f9640004840 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9644009ac0 tx=0x7f96400057f0 comp rx=0 tx=0).stop 2023-12-02T00:29:30.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.629+0000 7f965c609640 1 -- 172.21.15.46:0/1786129178 shutdown_connections 2023-12-02T00:29:30.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.629+0000 7f965c609640 1 --2- 172.21.15.46:0/1786129178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9640004370 0x7f9640004840 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:30.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.629+0000 7f965c609640 1 -- 172.21.15.46:0/1786129178 >> 172.21.15.46:0/1786129178 conn(0x7f96540b1e50 msgr2=0x7f96540b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:30.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.629+0000 7f965c609640 1 -- 172.21.15.46:0/1786129178 shutdown_connections 2023-12-02T00:29:30.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.629+0000 7f965c609640 1 -- 172.21.15.46:0/1786129178 wait complete. 2023-12-02T00:29:30.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.630+0000 7f965c609640 1 Processor -- start 2023-12-02T00:29:30.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.630+0000 7f965c609640 1 -- start start 2023-12-02T00:29:30.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.631+0000 7f965c609640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9640004370 0x7f96540af790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:30.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.631+0000 7f965c609640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96540ad5a0 0x7f965413cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:30.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.631+0000 7f965c609640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f96540714a0 0x7f9654071920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:30.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.631+0000 7f965c609640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9654071e60 con 0x7f9640004370 2023-12-02T00:29:30.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.631+0000 7f965c609640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965413d0b0 con 0x7f96540ad5a0 2023-12-02T00:29:30.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.631+0000 7f965c609640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96540add10 con 0x7f96540714a0 2023-12-02T00:29:30.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.631+0000 7f965a3a5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9640004370 0x7f96540af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:30.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.631+0000 7f965a3a5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9640004370 0x7f96540af790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52976/0 (socket says 172.21.15.46:52976) 2023-12-02T00:29:30.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.631+0000 7f965a3a5640 1 -- 172.21.15.46:0/3678599357 learned_addr learned my addr 172.21.15.46:0/3678599357 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:30.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.631+0000 7f9659ba4640 1 --2- 172.21.15.46:0/3678599357 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96540ad5a0 0x7f965413cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:30.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.631+0000 7f965aba6640 1 --2- 172.21.15.46:0/3678599357 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f96540714a0 0x7f9654071920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:30.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.631+0000 7f965a3a5640 1 -- 172.21.15.46:0/3678599357 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f96540714a0 msgr2=0x7f9654071920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:30.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.631+0000 7f965a3a5640 1 --2- 172.21.15.46:0/3678599357 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f96540714a0 0x7f9654071920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:30.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.631+0000 7f965a3a5640 1 -- 172.21.15.46:0/3678599357 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96540ad5a0 msgr2=0x7f965413cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:30.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.631+0000 7f965a3a5640 1 --2- 172.21.15.46:0/3678599357 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96540ad5a0 0x7f965413cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:30.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.631+0000 7f965a3a5640 1 -- 172.21.15.46:0/3678599357 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96540adff0 con 0x7f9640004370 2023-12-02T00:29:30.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.631+0000 7f965aba6640 1 --2- 172.21.15.46:0/3678599357 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f96540714a0 0x7f9654071920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:30.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.631+0000 7f965a3a5640 1 --2- 172.21.15.46:0/3678599357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9640004370 0x7f96540af790 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f964c00e7b0 tx=0x7f964c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:30.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.631+0000 7f964affd640 1 -- 172.21.15.46:0/3678599357 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964c00eec0 con 0x7f9640004370 2023-12-02T00:29:30.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.632+0000 7f964affd640 1 -- 172.21.15.46:0/3678599357 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f964c00c8d0 con 0x7f9640004370 2023-12-02T00:29:30.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.632+0000 7f965c609640 1 -- 172.21.15.46:0/3678599357 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96540ae2d0 con 0x7f9640004370 2023-12-02T00:29:30.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.632+0000 7f964affd640 1 -- 172.21.15.46:0/3678599357 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964c00cc80 con 0x7f9640004370 2023-12-02T00:29:30.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.632+0000 7f965c609640 1 -- 172.21.15.46:0/3678599357 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96540ada20 con 0x7f9640004370 2023-12-02T00:29:30.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.632+0000 7f965c609640 1 -- 172.21.15.46:0/3678599357 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9654161030 con 0x7f9640004370 2023-12-02T00:29:30.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.633+0000 7f964affd640 1 -- 172.21.15.46:0/3678599357 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f964c01d460 con 0x7f9640004370 2023-12-02T00:29:30.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.633+0000 7f964affd640 1 --2- 172.21.15.46:0/3678599357 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f963806ae40 0x7f963806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:30.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.634+0000 7f964affd640 1 -- 172.21.15.46:0/3678599357 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f964c089630 con 0x7f9640004370 2023-12-02T00:29:30.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.634+0000 7f9659ba4640 1 --2- 172.21.15.46:0/3678599357 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f963806ae40 0x7f963806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:30.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.634+0000 7f9659ba4640 1 --2- 172.21.15.46:0/3678599357 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f963806ae40 0x7f963806d300 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9644036000 tx=0x7f9644030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:30.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.638+0000 7f964affd640 1 -- 172.21.15.46:0/3678599357 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f964c054650 con 0x7f9640004370 2023-12-02T00:29:30.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.827+0000 7f965c609640 1 -- 172.21.15.46:0/3678599357 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9654069400 con 0x7f9640004370 2023-12-02T00:29:30.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.827+0000 7f964affd640 1 -- 172.21.15.46:0/3678599357 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f964c057c70 con 0x7f9640004370 2023-12-02T00:29:30.829 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2023-12-02T00:29:30.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.829+0000 7f965c609640 1 -- 172.21.15.46:0/3678599357 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f963806ae40 msgr2=0x7f963806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:30.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.829+0000 7f965c609640 1 --2- 172.21.15.46:0/3678599357 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f963806ae40 0x7f963806d300 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9644036000 tx=0x7f9644030000 comp rx=0 tx=0).stop 2023-12-02T00:29:30.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.829+0000 7f965c609640 1 -- 172.21.15.46:0/3678599357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9640004370 msgr2=0x7f96540af790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:30.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.829+0000 7f965c609640 1 --2- 172.21.15.46:0/3678599357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9640004370 0x7f96540af790 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f964c00e7b0 tx=0x7f964c00ec70 comp rx=0 tx=0).stop 2023-12-02T00:29:30.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.829+0000 7f965c609640 1 -- 172.21.15.46:0/3678599357 shutdown_connections 2023-12-02T00:29:30.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.829+0000 7f965c609640 1 --2- 172.21.15.46:0/3678599357 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f963806ae40 0x7f963806d300 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:30.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.829+0000 7f965c609640 1 --2- 172.21.15.46:0/3678599357 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f96540714a0 0x7f9654071920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:30.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.829+0000 7f965c609640 1 --2- 172.21.15.46:0/3678599357 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96540ad5a0 0x7f965413cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:30.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.829+0000 7f965c609640 1 --2- 172.21.15.46:0/3678599357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9640004370 0x7f96540af790 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:30.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.829+0000 7f965c609640 1 -- 172.21.15.46:0/3678599357 >> 172.21.15.46:0/3678599357 conn(0x7f96540b1e50 msgr2=0x7f96540733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:30.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.829+0000 7f965c609640 1 -- 172.21.15.46:0/3678599357 shutdown_connections 2023-12-02T00:29:30.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:30.829+0000 7f965c609640 1 -- 172.21.15.46:0/3678599357 wait complete. 2023-12-02T00:29:30.842 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-02T00:29:30.842 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-02T00:29:30.858 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:29:31.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.002+0000 7f7837b6e640 1 Processor -- start 2023-12-02T00:29:31.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.003+0000 7f7837b6e640 1 -- start start 2023-12-02T00:29:31.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.003+0000 7f7837b6e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7830150560 0x7f78300b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:31.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.003+0000 7f7837b6e640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78300b49f0 con 0x7f7830150940 2023-12-02T00:29:31.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.003+0000 7f7837b6e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78300b3de0 con 0x7f78300b3390 2023-12-02T00:29:31.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.003+0000 7f7837b6e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78300b40c0 con 0x7f7830150560 2023-12-02T00:29:31.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.004+0000 7f783610b640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7830150940 0x7f78300ce0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50484/0 (socket says 172.21.15.46:50484) 2023-12-02T00:29:31.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.004+0000 7f783610b640 1 -- 172.21.15.46:0/844686161 learned_addr learned my addr 172.21.15.46:0/844686161 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:31.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.004+0000 7f783590a640 1 --2- 172.21.15.46:0/844686161 >> v2:172.21.15.162:3300/0 conn(0x7f7830150560 0x7f78300b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:31.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.004+0000 7f7834908640 1 -- 172.21.15.46:0/844686161 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 113338528 0 0) 0x7f78300b49f0 con 0x7f7830150940 2023-12-02T00:29:31.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.004+0000 7f7834908640 1 -- 172.21.15.46:0/844686161 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7818003520 con 0x7f7830150940 2023-12-02T00:29:31.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.004+0000 7f7834908640 1 -- 172.21.15.46:0/844686161 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3741944027 0 0) 0x7f7818003520 con 0x7f7830150940 2023-12-02T00:29:31.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.004+0000 7f7834908640 1 -- 172.21.15.46:0/844686161 >> v2:172.21.15.162:3300/0 conn(0x7f7830150560 msgr2=0x7f78300b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:31.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.004+0000 7f7834908640 1 --2- 172.21.15.46:0/844686161 >> v2:172.21.15.162:3300/0 conn(0x7f7830150560 0x7f78300b2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:31.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.004+0000 7f7834908640 1 -- 172.21.15.46:0/844686161 >> v1:172.21.15.162:6789/0 conn(0x7f78300b3390 legacy=0x7f783015d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:31.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.004+0000 7f7834908640 1 -- 172.21.15.46:0/844686161 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78300b43a0 con 0x7f7830150940 2023-12-02T00:29:31.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.004+0000 7f7834908640 1 -- 172.21.15.46:0/844686161 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7f78200019f0 con 0x7f7830150940 2023-12-02T00:29:31.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.004+0000 7f7834908640 1 -- 172.21.15.46:0/844686161 >> v1:172.21.15.46:6789/0 conn(0x7f7830150940 legacy=0x7f78300ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:31.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.004+0000 7f7834908640 1 --2- 172.21.15.46:0/844686161 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7818003970 0x7f7818003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:31.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.004+0000 7f7834908640 1 --2- 172.21.15.46:0/844686161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78180047c0 0x7f7818004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:31.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.004+0000 7f7834908640 1 --2- 172.21.15.46:0/844686161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7818005180 0x7f781800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:31.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.004+0000 7f7834908640 1 -- 172.21.15.46:0/844686161 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78300b40c0 con 0x7f78180047c0 2023-12-02T00:29:31.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.004+0000 7f7834908640 1 -- 172.21.15.46:0/844686161 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f781800c020 con 0x7f7818005180 2023-12-02T00:29:31.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.004+0000 7f7834908640 1 -- 172.21.15.46:0/844686161 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f781800c1e0 con 0x7f7818003970 2023-12-02T00:29:31.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.004+0000 7f7835109640 1 --2- 172.21.15.46:0/844686161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78180047c0 0x7f7818004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:31.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.005+0000 7f783590a640 1 --2- 172.21.15.46:0/844686161 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7818003970 0x7f7818003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:31.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.005+0000 7f783610b640 1 --2- 172.21.15.46:0/844686161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7818005180 0x7f781800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:31.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.005+0000 7f7835109640 1 -- 172.21.15.46:0/844686161 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7818003970 msgr2=0x7f7818003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:31.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.005+0000 7f7835109640 1 --2- 172.21.15.46:0/844686161 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7818003970 0x7f7818003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:31.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.005+0000 7f7835109640 1 -- 172.21.15.46:0/844686161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7818005180 msgr2=0x7f781800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:31.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.005+0000 7f7835109640 1 --2- 172.21.15.46:0/844686161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7818005180 0x7f781800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:31.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.005+0000 7f7835109640 1 -- 172.21.15.46:0/844686161 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7818003520 con 0x7f78180047c0 2023-12-02T00:29:31.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.005+0000 7f783610b640 1 --2- 172.21.15.46:0/844686161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7818005180 0x7f781800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:31.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.005+0000 7f783610b640 1 -- 172.21.15.46:0/844686161 reap_dead start 2023-12-02T00:29:31.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.006+0000 7f7835109640 1 --2- 172.21.15.46:0/844686161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78180047c0 0x7f7818004c40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7824001750 tx=0x7f78180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe45b1afaa1643d5 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:31.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.006+0000 7f7834908640 1 -- 172.21.15.46:0/844686161 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78240052d0 con 0x7f78180047c0 2023-12-02T00:29:31.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.006+0000 7f7834908640 1 -- 172.21.15.46:0/844686161 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f782402e050 con 0x7f78180047c0 2023-12-02T00:29:31.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.006+0000 7f7834908640 1 -- 172.21.15.46:0/844686161 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7824003830 con 0x7f78180047c0 2023-12-02T00:29:31.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.006+0000 7f7837b6e640 1 -- 172.21.15.46:0/844686161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78180047c0 msgr2=0x7f7818004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:31.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.006+0000 7f7837b6e640 1 --2- 172.21.15.46:0/844686161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78180047c0 0x7f7818004c40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7824001750 tx=0x7f78180057f0 comp rx=0 tx=0).stop 2023-12-02T00:29:31.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.006+0000 7f7837b6e640 1 -- 172.21.15.46:0/844686161 shutdown_connections 2023-12-02T00:29:31.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.006+0000 7f7837b6e640 1 --2- 172.21.15.46:0/844686161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78180047c0 0x7f7818004c40 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:31.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.006+0000 7f7837b6e640 1 -- 172.21.15.46:0/844686161 >> 172.21.15.46:0/844686161 conn(0x7f783006a4d0 msgr2=0x7f783006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:31.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.006+0000 7f7837b6e640 1 -- 172.21.15.46:0/844686161 shutdown_connections 2023-12-02T00:29:31.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.006+0000 7f7837b6e640 1 -- 172.21.15.46:0/844686161 wait complete. 2023-12-02T00:29:31.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.007+0000 7f7837b6e640 1 Processor -- start 2023-12-02T00:29:31.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.008+0000 7f7837b6e640 1 -- start start 2023-12-02T00:29:31.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.008+0000 7f7837b6e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f78180047c0 0x7f7830145b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:31.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.008+0000 7f7837b6e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f783014a720 0x7f783014aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:31.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.008+0000 7f7837b6e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7830147560 0x7f78301479e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:31.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.008+0000 7f7837b6e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f783014a300 con 0x7f7830147560 2023-12-02T00:29:31.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.008+0000 7f7837b6e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78301f50e0 con 0x7f783014a720 2023-12-02T00:29:31.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.008+0000 7f7837b6e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7830149960 con 0x7f78180047c0 2023-12-02T00:29:31.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.008+0000 7f783610b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7830147560 0x7f78301479e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:31.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.008+0000 7f783610b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7830147560 0x7f78301479e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52984/0 (socket says 172.21.15.46:52984) 2023-12-02T00:29:31.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.008+0000 7f783610b640 1 -- 172.21.15.46:0/2700710237 learned_addr learned my addr 172.21.15.46:0/2700710237 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:31.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.009+0000 7f783590a640 1 --2- 172.21.15.46:0/2700710237 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f78180047c0 0x7f7830145b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:31.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.009+0000 7f7835109640 1 --2- 172.21.15.46:0/2700710237 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f783014a720 0x7f783014aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:31.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.009+0000 7f783610b640 1 -- 172.21.15.46:0/2700710237 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f78180047c0 msgr2=0x7f7830145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:31.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.009+0000 7f783610b640 1 --2- 172.21.15.46:0/2700710237 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f78180047c0 0x7f7830145b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:31.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.009+0000 7f783610b640 1 -- 172.21.15.46:0/2700710237 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f783014a720 msgr2=0x7f783014aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:31.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.009+0000 7f783610b640 1 --2- 172.21.15.46:0/2700710237 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f783014a720 0x7f783014aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:31.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.009+0000 7f783610b640 1 -- 172.21.15.46:0/2700710237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7830149c40 con 0x7f7830147560 2023-12-02T00:29:31.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.009+0000 7f7835109640 1 --2- 172.21.15.46:0/2700710237 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f783014a720 0x7f783014aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:31.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.009+0000 7f783590a640 1 --2- 172.21.15.46:0/2700710237 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f78180047c0 0x7f7830145b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:31.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.009+0000 7f783610b640 1 --2- 172.21.15.46:0/2700710237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7830147560 0x7f78301479e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f78300cc860 tx=0x7f7820007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:31.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.009+0000 7f782e7fc640 1 -- 172.21.15.46:0/2700710237 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782000ea80 con 0x7f7830147560 2023-12-02T00:29:31.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.009+0000 7f782e7fc640 1 -- 172.21.15.46:0/2700710237 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f782000ec00 con 0x7f7830147560 2023-12-02T00:29:31.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.009+0000 7f782e7fc640 1 -- 172.21.15.46:0/2700710237 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7820013420 con 0x7f7830147560 2023-12-02T00:29:31.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.009+0000 7f7837b6e640 1 -- 172.21.15.46:0/2700710237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7830149f20 con 0x7f7830147560 2023-12-02T00:29:31.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.009+0000 7f7837b6e640 1 -- 172.21.15.46:0/2700710237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78300b2230 con 0x7f7830147560 2023-12-02T00:29:31.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.010+0000 7f7837b6e640 1 -- 172.21.15.46:0/2700710237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78300b3180 con 0x7f7830147560 2023-12-02T00:29:31.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.011+0000 7f782e7fc640 1 -- 172.21.15.46:0/2700710237 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7820013580 con 0x7f7830147560 2023-12-02T00:29:31.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.011+0000 7f782e7fc640 1 --2- 172.21.15.46:0/2700710237 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f781c06ae70 0x7f781c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:31.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.011+0000 7f782e7fc640 1 -- 172.21.15.46:0/2700710237 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f7820089a40 con 0x7f7830147560 2023-12-02T00:29:31.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.011+0000 7f783590a640 1 --2- 172.21.15.46:0/2700710237 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f781c06ae70 0x7f781c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:31.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.012+0000 7f783590a640 1 --2- 172.21.15.46:0/2700710237 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f781c06ae70 0x7f781c06d330 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f782800ad40 tx=0x7f7828008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:31.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.015+0000 7f782e7fc640 1 -- 172.21.15.46:0/2700710237 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f782008be60 con 0x7f7830147560 2023-12-02T00:29:31.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.193+0000 7f7837b6e640 1 -- 172.21.15.46:0/2700710237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7830069400 con 0x7f7830147560 2023-12-02T00:29:31.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.194+0000 7f782e7fc640 1 -- 172.21.15.46:0/2700710237 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16777 (secure 0 0 0) 0x7f7820054a60 con 0x7f7830147560 2023-12-02T00:29:31.196 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:29:31.196 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":23,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:29:30.868939+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-02T00:28:47.273337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-02T00:28:48.060115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.14","mappings":[{"from":4,"to":2}]},{"pgid":"3.1e","mappings":[{"from":3,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:29:31.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.196+0000 7f7837b6e640 1 -- 172.21.15.46:0/2700710237 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f781c06ae70 msgr2=0x7f781c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:31.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.196+0000 7f7837b6e640 1 --2- 172.21.15.46:0/2700710237 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f781c06ae70 0x7f781c06d330 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f782800ad40 tx=0x7f7828008040 comp rx=0 tx=0).stop 2023-12-02T00:29:31.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.196+0000 7f7837b6e640 1 -- 172.21.15.46:0/2700710237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7830147560 msgr2=0x7f78301479e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:31.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.196+0000 7f7837b6e640 1 --2- 172.21.15.46:0/2700710237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7830147560 0x7f78301479e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f78300cc860 tx=0x7f7820007bc0 comp rx=0 tx=0).stop 2023-12-02T00:29:31.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.196+0000 7f7837b6e640 1 -- 172.21.15.46:0/2700710237 shutdown_connections 2023-12-02T00:29:31.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.196+0000 7f7837b6e640 1 --2- 172.21.15.46:0/2700710237 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f781c06ae70 0x7f781c06d330 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:31.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.196+0000 7f7837b6e640 1 --2- 172.21.15.46:0/2700710237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7830147560 0x7f78301479e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:31.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.196+0000 7f7837b6e640 1 --2- 172.21.15.46:0/2700710237 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f783014a720 0x7f783014aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:31.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.196+0000 7f7837b6e640 1 --2- 172.21.15.46:0/2700710237 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f78180047c0 0x7f7830145b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:31.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.196+0000 7f7837b6e640 1 -- 172.21.15.46:0/2700710237 >> 172.21.15.46:0/2700710237 conn(0x7f783006a4d0 msgr2=0x7f783006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:31.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.196+0000 7f7837b6e640 1 -- 172.21.15.46:0/2700710237 shutdown_connections 2023-12-02T00:29:31.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.196+0000 7f7837b6e640 1 -- 172.21.15.46:0/2700710237 wait complete. 2023-12-02T00:29:31.209 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-02T00:28:37.213645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-02T00:28:47.273337+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-02T00:28:48.060115+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-02T00:29:31.210 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-02T00:29:31.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.317+0000 7f0b52316640 1 Processor -- start 2023-12-02T00:29:31.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.317+0000 7f0b52316640 1 -- start start 2023-12-02T00:29:31.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.318+0000 7f0b52316640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0b4c06f210 0x7f0b4c140800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:31.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.318+0000 7f0b52316640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0b4c140d40 0x7f0b4c13f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:31.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.318+0000 7f0b52316640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b4c15ba40 con 0x7f0b4c15c2f0 2023-12-02T00:29:31.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.318+0000 7f0b52316640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b4c15c6d0 con 0x7f0b4c06f210 2023-12-02T00:29:31.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.318+0000 7f0b52316640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b4c13e530 con 0x7f0b4c140d40 2023-12-02T00:29:31.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.318+0000 7f0b4bfff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0b4c15c2f0 0x7f0b4c06eb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50490/0 (socket says 172.21.15.46:50490) 2023-12-02T00:29:31.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.318+0000 7f0b4bfff640 1 -- 172.21.15.46:0/205429415 learned_addr learned my addr 172.21.15.46:0/205429415 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:31.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.318+0000 7f0b4b7fe640 1 --2- 172.21.15.46:0/205429415 >> v2:172.21.15.46:3300/0 conn(0x7f0b4c06f210 0x7f0b4c140800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:31.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.318+0000 7f0b508b3640 1 --2- 172.21.15.46:0/205429415 >> v2:172.21.15.162:3300/0 conn(0x7f0b4c140d40 0x7f0b4c13f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:31.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.319+0000 7f0b4affd640 1 -- 172.21.15.46:0/205429415 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3666324085 0 0) 0x7f0b4c15ba40 con 0x7f0b4c15c2f0 2023-12-02T00:29:31.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.319+0000 7f0b4b7fe640 1 -- 172.21.15.46:0/205429415 >> v2:172.21.15.162:3300/0 conn(0x7f0b4c140d40 msgr2=0x7f0b4c13f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:31.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.319+0000 7f0b4b7fe640 1 --2- 172.21.15.46:0/205429415 >> v2:172.21.15.162:3300/0 conn(0x7f0b4c140d40 0x7f0b4c13f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:31.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.319+0000 7f0b4b7fe640 1 -- 172.21.15.46:0/205429415 >> v1:172.21.15.46:6789/0 conn(0x7f0b4c15c2f0 legacy=0x7f0b4c06eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:31.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.319+0000 7f0b4b7fe640 1 -- 172.21.15.46:0/205429415 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b4c141190 con 0x7f0b4c06f210 2023-12-02T00:29:31.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.319+0000 7f0b4b7fe640 1 --2- 172.21.15.46:0/205429415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b4c06f210 0x7f0b4c140800 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c155c40 tx=0x7f0b3c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61cdbf0a0e3d0375 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:31.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.319+0000 7f0b4affd640 1 -- 172.21.15.46:0/205429415 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c031830 con 0x7f0b4c06f210 2023-12-02T00:29:31.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.319+0000 7f0b4affd640 1 -- 172.21.15.46:0/205429415 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b3c031990 con 0x7f0b4c06f210 2023-12-02T00:29:31.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.320+0000 7f0b4affd640 1 -- 172.21.15.46:0/205429415 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c031d20 con 0x7f0b4c06f210 2023-12-02T00:29:31.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.320+0000 7f0b52316640 1 -- 172.21.15.46:0/205429415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b4c06f210 msgr2=0x7f0b4c140800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:31.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.320+0000 7f0b52316640 1 --2- 172.21.15.46:0/205429415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b4c06f210 0x7f0b4c140800 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c155c40 tx=0x7f0b3c02b8d0 comp rx=0 tx=0).stop 2023-12-02T00:29:31.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.320+0000 7f0b52316640 1 -- 172.21.15.46:0/205429415 shutdown_connections 2023-12-02T00:29:31.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.320+0000 7f0b52316640 1 --2- 172.21.15.46:0/205429415 >> v2:172.21.15.162:3300/0 conn(0x7f0b4c140d40 0x7f0b4c13f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:31.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.320+0000 7f0b52316640 1 --2- 172.21.15.46:0/205429415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b4c06f210 0x7f0b4c140800 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:31.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.320+0000 7f0b52316640 1 -- 172.21.15.46:0/205429415 >> 172.21.15.46:0/205429415 conn(0x7f0b4c06a6b0 msgr2=0x7f0b4c06aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:31.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.320+0000 7f0b52316640 1 -- 172.21.15.46:0/205429415 shutdown_connections 2023-12-02T00:29:31.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.320+0000 7f0b52316640 1 -- 172.21.15.46:0/205429415 wait complete. 2023-12-02T00:29:31.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.321+0000 7f0b52316640 1 Processor -- start 2023-12-02T00:29:31.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.322+0000 7f0b52316640 1 -- start start 2023-12-02T00:29:31.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.322+0000 7f0b52316640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0b4c06f210 0x7f0b4c0b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:31.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.322+0000 7f0b52316640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b4c140d40 0x7f0b4c13c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:31.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.322+0000 7f0b52316640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b4c15c2f0 0x7f0b4c13cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:31.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.322+0000 7f0b52316640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4c0b4890 con 0x7f0b4c140d40 2023-12-02T00:29:31.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.322+0000 7f0b52316640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4c0b60c0 con 0x7f0b4c15c2f0 2023-12-02T00:29:31.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.322+0000 7f0b52316640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4c0b63a0 con 0x7f0b4c06f210 2023-12-02T00:29:31.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.322+0000 7f0b4b7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b4c140d40 0x7f0b4c13c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:31.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.322+0000 7f0b4b7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b4c140d40 0x7f0b4c13c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53006/0 (socket says 172.21.15.46:53006) 2023-12-02T00:29:31.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.322+0000 7f0b4b7fe640 1 -- 172.21.15.46:0/1141141847 learned_addr learned my addr 172.21.15.46:0/1141141847 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:31.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.323+0000 7f0b508b3640 1 --2- 172.21.15.46:0/1141141847 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b4c15c2f0 0x7f0b4c13cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:31.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.323+0000 7f0b4bfff640 1 --2- 172.21.15.46:0/1141141847 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0b4c06f210 0x7f0b4c0b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:31.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.323+0000 7f0b4b7fe640 1 -- 172.21.15.46:0/1141141847 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0b4c06f210 msgr2=0x7f0b4c0b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:31.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.323+0000 7f0b4b7fe640 1 --2- 172.21.15.46:0/1141141847 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0b4c06f210 0x7f0b4c0b4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:31.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.323+0000 7f0b4b7fe640 1 -- 172.21.15.46:0/1141141847 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b4c15c2f0 msgr2=0x7f0b4c13cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:31.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.323+0000 7f0b4b7fe640 1 --2- 172.21.15.46:0/1141141847 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b4c15c2f0 0x7f0b4c13cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:31.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.323+0000 7f0b4b7fe640 1 -- 172.21.15.46:0/1141141847 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b3c009400 con 0x7f0b4c140d40 2023-12-02T00:29:31.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.323+0000 7f0b508b3640 1 --2- 172.21.15.46:0/1141141847 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b4c15c2f0 0x7f0b4c13cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:31.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.323+0000 7f0b4b7fe640 1 --2- 172.21.15.46:0/1141141847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b4c140d40 0x7f0b4c13c6f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c009890 tx=0x7f0b3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:31.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.323+0000 7f0b48ff9640 1 -- 172.21.15.46:0/1141141847 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c03a6f0 con 0x7f0b4c140d40 2023-12-02T00:29:31.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.323+0000 7f0b48ff9640 1 -- 172.21.15.46:0/1141141847 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b3c03a870 con 0x7f0b4c140d40 2023-12-02T00:29:31.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.323+0000 7f0b52316640 1 -- 172.21.15.46:0/1141141847 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b4c0b4240 con 0x7f0b4c140d40 2023-12-02T00:29:31.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.323+0000 7f0b48ff9640 1 -- 172.21.15.46:0/1141141847 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c03ac20 con 0x7f0b4c140d40 2023-12-02T00:29:31.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.323+0000 7f0b52316640 1 -- 172.21.15.46:0/1141141847 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b4c0b5d00 con 0x7f0b4c140d40 2023-12-02T00:29:31.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.324+0000 7f0b52316640 1 -- 172.21.15.46:0/1141141847 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b4c0b5e90 con 0x7f0b4c140d40 2023-12-02T00:29:31.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.324+0000 7f0b48ff9640 1 -- 172.21.15.46:0/1141141847 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0b3c043410 con 0x7f0b4c140d40 2023-12-02T00:29:31.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.325+0000 7f0b48ff9640 1 --2- 172.21.15.46:0/1141141847 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f0b3806ae40 0x7f0b3806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:31.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.325+0000 7f0b48ff9640 1 -- 172.21.15.46:0/1141141847 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f0b3c0ab330 con 0x7f0b4c140d40 2023-12-02T00:29:31.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.325+0000 7f0b4bfff640 1 --2- 172.21.15.46:0/1141141847 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f0b3806ae40 0x7f0b3806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:31.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.325+0000 7f0b4bfff640 1 --2- 172.21.15.46:0/1141141847 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f0b3806ae40 0x7f0b3806d300 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0b40000f60 tx=0x7f0b40007c00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:31.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.329+0000 7f0b48ff9640 1 -- 172.21.15.46:0/1141141847 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0b3c076470 con 0x7f0b4c140d40 2023-12-02T00:29:31.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.505+0000 7f0b52316640 1 -- 172.21.15.46:0/1141141847 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0b4c069400 con 0x7f0b4c140d40 2023-12-02T00:29:31.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.506+0000 7f0b48ff9640 1 -- 172.21.15.46:0/1141141847 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0b3c079a90 con 0x7f0b4c140d40 2023-12-02T00:29:31.508 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-12-02T00:29:31.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.507+0000 7f0b52316640 1 -- 172.21.15.46:0/1141141847 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f0b3806ae40 msgr2=0x7f0b3806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:31.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.507+0000 7f0b52316640 1 --2- 172.21.15.46:0/1141141847 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f0b3806ae40 0x7f0b3806d300 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0b40000f60 tx=0x7f0b40007c00 comp rx=0 tx=0).stop 2023-12-02T00:29:31.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.507+0000 7f0b52316640 1 -- 172.21.15.46:0/1141141847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b4c140d40 msgr2=0x7f0b4c13c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:31.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.507+0000 7f0b52316640 1 --2- 172.21.15.46:0/1141141847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b4c140d40 0x7f0b4c13c6f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c009890 tx=0x7f0b3c0046c0 comp rx=0 tx=0).stop 2023-12-02T00:29:31.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.507+0000 7f0b52316640 1 -- 172.21.15.46:0/1141141847 shutdown_connections 2023-12-02T00:29:31.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.507+0000 7f0b52316640 1 --2- 172.21.15.46:0/1141141847 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f0b3806ae40 0x7f0b3806d300 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:31.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.507+0000 7f0b52316640 1 --2- 172.21.15.46:0/1141141847 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b4c15c2f0 0x7f0b4c13cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:31.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.507+0000 7f0b52316640 1 --2- 172.21.15.46:0/1141141847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b4c140d40 0x7f0b4c13c6f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:31.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.507+0000 7f0b52316640 1 --2- 172.21.15.46:0/1141141847 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0b4c06f210 0x7f0b4c0b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:31.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.507+0000 7f0b52316640 1 -- 172.21.15.46:0/1141141847 >> 172.21.15.46:0/1141141847 conn(0x7f0b4c06a6b0 msgr2=0x7f0b4c0700d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:31.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.507+0000 7f0b52316640 1 -- 172.21.15.46:0/1141141847 shutdown_connections 2023-12-02T00:29:31.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.508+0000 7f0b52316640 1 -- 172.21.15.46:0/1141141847 wait complete. 2023-12-02T00:29:31.520 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-02T00:29:31.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.627+0000 7f355850f640 1 Processor -- start 2023-12-02T00:29:31.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.628+0000 7f355850f640 1 -- start start 2023-12-02T00:29:31.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.628+0000 7f355850f640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f355015d330 0x7f35500b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:31.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.628+0000 7f355850f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35500d2910 0x7f35500d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:31.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.628+0000 7f355850f640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f35500d3730 0x7f35500b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:31.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.628+0000 7f355850f640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35500b2b10 con 0x7f355015d330 2023-12-02T00:29:31.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.628+0000 7f355850f640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35500d3b50 con 0x7f35500d3730 2023-12-02T00:29:31.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.628+0000 7f355850f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35500d1590 con 0x7f35500d2910 2023-12-02T00:29:31.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.629+0000 7f35562ab640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f355015d330 0x7f35500b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:31.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.629+0000 7f35562ab640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f355015d330 0x7f35500b8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53018/0 (socket says 172.21.15.46:53018) 2023-12-02T00:29:31.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.629+0000 7f35562ab640 1 -- 172.21.15.46:0/2059849056 learned_addr learned my addr 172.21.15.46:0/2059849056 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:31.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.629+0000 7f3555aaa640 1 --2- 172.21.15.46:0/2059849056 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35500d2910 0x7f35500d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:31.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.629+0000 7f3556aac640 1 --2- 172.21.15.46:0/2059849056 >> v2:172.21.15.162:3300/0 conn(0x7f35500d3730 0x7f35500b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:31.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.629+0000 7f35562ab640 1 -- 172.21.15.46:0/2059849056 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35500d2910 msgr2=0x7f35500d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:31.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.629+0000 7f35562ab640 1 --2- 172.21.15.46:0/2059849056 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35500d2910 0x7f35500d31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:31.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.629+0000 7f35562ab640 1 -- 172.21.15.46:0/2059849056 >> v2:172.21.15.162:3300/0 conn(0x7f35500d3730 msgr2=0x7f35500b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:31.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.629+0000 7f35562ab640 1 --2- 172.21.15.46:0/2059849056 >> v2:172.21.15.162:3300/0 conn(0x7f35500d3730 0x7f35500b6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:31.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.629+0000 7f35562ab640 1 -- 172.21.15.46:0/2059849056 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35500d1870 con 0x7f355015d330 2023-12-02T00:29:31.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.629+0000 7f3555aaa640 1 --2- 172.21.15.46:0/2059849056 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35500d2910 0x7f35500d31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:29:31.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.629+0000 7f3556aac640 1 --2- 172.21.15.46:0/2059849056 >> v2:172.21.15.162:3300/0 conn(0x7f35500d3730 0x7f35500b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:29:31.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.630+0000 7f35562ab640 1 --2- 172.21.15.46:0/2059849056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f355015d330 0x7f35500b8b80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f35500d1510 tx=0x7f354c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38126ed9a410e66 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:31.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.630+0000 7f35552a9640 1 -- 172.21.15.46:0/2059849056 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354c0317e0 con 0x7f355015d330 2023-12-02T00:29:31.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.630+0000 7f35552a9640 1 -- 172.21.15.46:0/2059849056 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f354c031940 con 0x7f355015d330 2023-12-02T00:29:31.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.630+0000 7f35552a9640 1 -- 172.21.15.46:0/2059849056 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354c031cd0 con 0x7f355015d330 2023-12-02T00:29:31.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.630+0000 7f355850f640 1 -- 172.21.15.46:0/2059849056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f355015d330 msgr2=0x7f35500b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:31.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.630+0000 7f355850f640 1 --2- 172.21.15.46:0/2059849056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f355015d330 0x7f35500b8b80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f35500d1510 tx=0x7f354c02b880 comp rx=0 tx=0).stop 2023-12-02T00:29:31.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.631+0000 7f355850f640 1 -- 172.21.15.46:0/2059849056 shutdown_connections 2023-12-02T00:29:31.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.631+0000 7f355850f640 1 --2- 172.21.15.46:0/2059849056 >> v2:172.21.15.162:3300/0 conn(0x7f35500d3730 0x7f35500b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:31.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.631+0000 7f355850f640 1 --2- 172.21.15.46:0/2059849056 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35500d2910 0x7f35500d31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:31.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.631+0000 7f355850f640 1 --2- 172.21.15.46:0/2059849056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f355015d330 0x7f35500b8b80 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:31.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.631+0000 7f355850f640 1 -- 172.21.15.46:0/2059849056 >> 172.21.15.46:0/2059849056 conn(0x7f3550072020 msgr2=0x7f3550072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:31.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.631+0000 7f355850f640 1 -- 172.21.15.46:0/2059849056 shutdown_connections 2023-12-02T00:29:31.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.631+0000 7f355850f640 1 -- 172.21.15.46:0/2059849056 wait complete. 2023-12-02T00:29:31.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.632+0000 7f355850f640 1 Processor -- start 2023-12-02T00:29:31.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.632+0000 7f355850f640 1 -- start start 2023-12-02T00:29:31.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.633+0000 7f355850f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35500d2910 0x7f35501f6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:31.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.633+0000 7f355850f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f35500d3730 0x7f3550155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:31.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.633+0000 7f355850f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f355015d330 0x7f3550155e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:31.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.633+0000 7f355850f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3550143210 con 0x7f35500d2910 2023-12-02T00:29:31.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.633+0000 7f355850f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35500c8050 con 0x7f35500d3730 2023-12-02T00:29:31.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.633+0000 7f355850f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3550156a90 con 0x7f355015d330 2023-12-02T00:29:31.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.633+0000 7f35562ab640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35500d2910 0x7f35501f6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:31.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.633+0000 7f35562ab640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35500d2910 0x7f35501f6bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53022/0 (socket says 172.21.15.46:53022) 2023-12-02T00:29:31.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.633+0000 7f35562ab640 1 -- 172.21.15.46:0/3120042369 learned_addr learned my addr 172.21.15.46:0/3120042369 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:31.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.633+0000 7f35562ab640 1 -- 172.21.15.46:0/3120042369 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f355015d330 msgr2=0x7f3550155e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:31.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.633+0000 7f35562ab640 1 --2- 172.21.15.46:0/3120042369 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f355015d330 0x7f3550155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:31.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.633+0000 7f35562ab640 1 -- 172.21.15.46:0/3120042369 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f35500d3730 msgr2=0x7f3550155910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:31.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.633+0000 7f35562ab640 1 --2- 172.21.15.46:0/3120042369 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f35500d3730 0x7f3550155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:31.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.633+0000 7f35562ab640 1 -- 172.21.15.46:0/3120042369 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f354c0094d0 con 0x7f35500d2910 2023-12-02T00:29:31.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.633+0000 7f35562ab640 1 --2- 172.21.15.46:0/3120042369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35500d2910 0x7f35501f6bc0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f354c000c00 tx=0x7f354c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:31.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.633+0000 7f3546ffd640 1 -- 172.21.15.46:0/3120042369 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354c032900 con 0x7f35500d2910 2023-12-02T00:29:31.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.633+0000 7f3546ffd640 1 -- 172.21.15.46:0/3120042369 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f354c032a80 con 0x7f35500d2910 2023-12-02T00:29:31.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.633+0000 7f355850f640 1 -- 172.21.15.46:0/3120042369 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35500c7d20 con 0x7f35500d2910 2023-12-02T00:29:31.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.633+0000 7f3546ffd640 1 -- 172.21.15.46:0/3120042369 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354c032e30 con 0x7f35500d2910 2023-12-02T00:29:31.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.633+0000 7f355850f640 1 -- 172.21.15.46:0/3120042369 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35501567b0 con 0x7f35500d2910 2023-12-02T00:29:31.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.634+0000 7f355850f640 1 -- 172.21.15.46:0/3120042369 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3550069400 con 0x7f35500d2910 2023-12-02T00:29:31.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.635+0000 7f3546ffd640 1 -- 172.21.15.46:0/3120042369 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f354c04e450 con 0x7f35500d2910 2023-12-02T00:29:31.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.635+0000 7f3546ffd640 1 --2- 172.21.15.46:0/3120042369 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f353406ae70 0x7f353406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:31.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.635+0000 7f3546ffd640 1 -- 172.21.15.46:0/3120042369 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f354c0ab8f0 con 0x7f35500d2910 2023-12-02T00:29:31.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.635+0000 7f3555aaa640 1 --2- 172.21.15.46:0/3120042369 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f353406ae70 0x7f353406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:31.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.636+0000 7f3555aaa640 1 --2- 172.21.15.46:0/3120042369 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f353406ae70 0x7f353406d330 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3540006b10 tx=0x7f3540008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:31.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.639+0000 7f3546ffd640 1 -- 172.21.15.46:0/3120042369 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f354c076a50 con 0x7f35500d2910 2023-12-02T00:29:31.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.817+0000 7f355850f640 1 -- 172.21.15.46:0/3120042369 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f35501f7100 con 0x7f35500d2910 2023-12-02T00:29:31.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.818+0000 7f3546ffd640 1 -- 172.21.15.46:0/3120042369 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f354c07a070 con 0x7f35500d2910 2023-12-02T00:29:31.820 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-12-02T00:29:31.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.820+0000 7f355850f640 1 -- 172.21.15.46:0/3120042369 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f353406ae70 msgr2=0x7f353406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:31.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.820+0000 7f355850f640 1 --2- 172.21.15.46:0/3120042369 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f353406ae70 0x7f353406d330 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3540006b10 tx=0x7f3540008040 comp rx=0 tx=0).stop 2023-12-02T00:29:31.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.820+0000 7f355850f640 1 -- 172.21.15.46:0/3120042369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35500d2910 msgr2=0x7f35501f6bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:31.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.820+0000 7f355850f640 1 --2- 172.21.15.46:0/3120042369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35500d2910 0x7f35501f6bc0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f354c000c00 tx=0x7f354c0046c0 comp rx=0 tx=0).stop 2023-12-02T00:29:31.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.820+0000 7f355850f640 1 -- 172.21.15.46:0/3120042369 shutdown_connections 2023-12-02T00:29:31.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.820+0000 7f355850f640 1 --2- 172.21.15.46:0/3120042369 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f353406ae70 0x7f353406d330 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:31.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.820+0000 7f355850f640 1 --2- 172.21.15.46:0/3120042369 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f355015d330 0x7f3550155e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:31.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.820+0000 7f355850f640 1 --2- 172.21.15.46:0/3120042369 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f35500d3730 0x7f3550155910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:31.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.820+0000 7f355850f640 1 --2- 172.21.15.46:0/3120042369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35500d2910 0x7f35501f6bc0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:31.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.820+0000 7f355850f640 1 -- 172.21.15.46:0/3120042369 >> 172.21.15.46:0/3120042369 conn(0x7f3550072020 msgr2=0x7f35500b6910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:31.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.820+0000 7f355850f640 1 -- 172.21.15.46:0/3120042369 shutdown_connections 2023-12-02T00:29:31.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.820+0000 7f355850f640 1 -- 172.21.15.46:0/3120042369 wait complete. 2023-12-02T00:29:31.833 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-02T00:29:31.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.941+0000 7fe05ab39640 1 Processor -- start 2023-12-02T00:29:31.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.941+0000 7fe05ab39640 1 -- start start 2023-12-02T00:29:31.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.942+0000 7fe05ab39640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe054160f70 0x7fe0540cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:31.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.942+0000 7fe05ab39640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0540ce480 0x7fe0540d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:31.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.942+0000 7fe05ab39640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0541614e0 con 0x7fe05415d2d0 2023-12-02T00:29:31.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.942+0000 7fe05ab39640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0541617c0 con 0x7fe054160f70 2023-12-02T00:29:31.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.942+0000 7fe05ab39640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe054161aa0 con 0x7fe0540ce480 2023-12-02T00:29:31.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.942+0000 7fe053fff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe054160f70 0x7fe0540cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:31.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.942+0000 7fe0588d5640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe05415d2d0 0x7fe0540b8b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50504/0 (socket says 172.21.15.46:50504) 2023-12-02T00:29:31.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.942+0000 7fe0588d5640 1 -- 172.21.15.46:0/4134588708 learned_addr learned my addr 172.21.15.46:0/4134588708 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:31.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.942+0000 7fe053fff640 1 -- 172.21.15.46:0/4134588708 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0540ce480 msgr2=0x7fe0540d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:31.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.942+0000 7fe0590d6640 1 --2- 172.21.15.46:0/4134588708 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0540ce480 0x7fe0540d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:31.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.943+0000 7fe053fff640 1 --2- 172.21.15.46:0/4134588708 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0540ce480 0x7fe0540d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:31.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.942+0000 7fe0537fe640 1 -- 172.21.15.46:0/4134588708 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2074937806 0 0) 0x7fe0541614e0 con 0x7fe05415d2d0 2023-12-02T00:29:31.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.943+0000 7fe053fff640 1 -- 172.21.15.46:0/4134588708 >> v1:172.21.15.46:6789/0 conn(0x7fe05415d2d0 legacy=0x7fe0540b8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:31.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.943+0000 7fe053fff640 1 -- 172.21.15.46:0/4134588708 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe054161d80 con 0x7fe054160f70 2023-12-02T00:29:31.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.943+0000 7fe0590d6640 1 --2- 172.21.15.46:0/4134588708 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0540ce480 0x7fe0540d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:29:31.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.943+0000 7fe053fff640 1 --2- 172.21.15.46:0/4134588708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe054160f70 0x7fe0540cdf40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe0480097a0 tx=0x7fe04802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd8301b771f6d78 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:31.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.943+0000 7fe0537fe640 1 -- 172.21.15.46:0/4134588708 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe048031830 con 0x7fe054160f70 2023-12-02T00:29:31.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.943+0000 7fe0537fe640 1 -- 172.21.15.46:0/4134588708 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe048031990 con 0x7fe054160f70 2023-12-02T00:29:31.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.943+0000 7fe0537fe640 1 -- 172.21.15.46:0/4134588708 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe048031d20 con 0x7fe054160f70 2023-12-02T00:29:31.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.944+0000 7fe05ab39640 1 -- 172.21.15.46:0/4134588708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe054160f70 msgr2=0x7fe0540cdf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:31.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.944+0000 7fe05ab39640 1 --2- 172.21.15.46:0/4134588708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe054160f70 0x7fe0540cdf40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe0480097a0 tx=0x7fe04802b8d0 comp rx=0 tx=0).stop 2023-12-02T00:29:31.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.944+0000 7fe05ab39640 1 -- 172.21.15.46:0/4134588708 shutdown_connections 2023-12-02T00:29:31.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.944+0000 7fe05ab39640 1 --2- 172.21.15.46:0/4134588708 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0540ce480 0x7fe0540d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:31.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.944+0000 7fe05ab39640 1 --2- 172.21.15.46:0/4134588708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe054160f70 0x7fe0540cdf40 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:31.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.944+0000 7fe05ab39640 1 -- 172.21.15.46:0/4134588708 >> 172.21.15.46:0/4134588708 conn(0x7fe054072230 msgr2=0x7fe0540cfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:31.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.944+0000 7fe05ab39640 1 -- 172.21.15.46:0/4134588708 shutdown_connections 2023-12-02T00:29:31.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.944+0000 7fe05ab39640 1 -- 172.21.15.46:0/4134588708 wait complete. 2023-12-02T00:29:31.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.945+0000 7fe05ab39640 1 Processor -- start 2023-12-02T00:29:31.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.945+0000 7fe05ab39640 1 -- start start 2023-12-02T00:29:31.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.946+0000 7fe05ab39640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0540ce480 0x7fe0540b2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:31.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.946+0000 7fe05ab39640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe05415d2d0 0x7fe05415f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:31.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.946+0000 7fe05ab39640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe054160f70 0x7fe0540c8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:31.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.946+0000 7fe05ab39640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0540b3ec0 con 0x7fe0540ce480 2023-12-02T00:29:31.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.946+0000 7fe05ab39640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05415edd0 con 0x7fe054160f70 2023-12-02T00:29:31.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.946+0000 7fe05ab39640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0540b3240 con 0x7fe05415d2d0 2023-12-02T00:29:31.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.946+0000 7fe0588d5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0540ce480 0x7fe0540b2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:31.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.946+0000 7fe0588d5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0540ce480 0x7fe0540b2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53042/0 (socket says 172.21.15.46:53042) 2023-12-02T00:29:31.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.946+0000 7fe0588d5640 1 -- 172.21.15.46:0/1733971880 learned_addr learned my addr 172.21.15.46:0/1733971880 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:31.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.946+0000 7fe053fff640 1 --2- 172.21.15.46:0/1733971880 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe05415d2d0 0x7fe05415f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:31.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.947+0000 7fe0590d6640 1 --2- 172.21.15.46:0/1733971880 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe054160f70 0x7fe0540c8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:31.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.947+0000 7fe0588d5640 1 -- 172.21.15.46:0/1733971880 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe05415d2d0 msgr2=0x7fe05415f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:31.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.947+0000 7fe0588d5640 1 --2- 172.21.15.46:0/1733971880 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe05415d2d0 0x7fe05415f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:31.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.947+0000 7fe0588d5640 1 -- 172.21.15.46:0/1733971880 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe054160f70 msgr2=0x7fe0540c8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:31.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.947+0000 7fe0588d5640 1 --2- 172.21.15.46:0/1733971880 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe054160f70 0x7fe0540c8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:31.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.947+0000 7fe0588d5640 1 -- 172.21.15.46:0/1733971880 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe048009400 con 0x7fe0540ce480 2023-12-02T00:29:31.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.947+0000 7fe0590d6640 1 --2- 172.21.15.46:0/1733971880 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe054160f70 0x7fe0540c8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:29:31.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.947+0000 7fe0588d5640 1 --2- 172.21.15.46:0/1733971880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0540ce480 0x7fe0540b2b60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe04c00c890 tx=0x7fe04c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:31.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.947+0000 7fe0517fa640 1 -- 172.21.15.46:0/1733971880 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04c00a640 con 0x7fe0540ce480 2023-12-02T00:29:31.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.947+0000 7fe0517fa640 1 -- 172.21.15.46:0/1733971880 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe04c00a7c0 con 0x7fe0540ce480 2023-12-02T00:29:31.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.947+0000 7fe05ab39640 1 -- 172.21.15.46:0/1733971880 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0540b3520 con 0x7fe0540ce480 2023-12-02T00:29:31.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.947+0000 7fe0517fa640 1 -- 172.21.15.46:0/1733971880 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04c00ab70 con 0x7fe0540ce480 2023-12-02T00:29:31.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.947+0000 7fe05ab39640 1 -- 172.21.15.46:0/1733971880 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0540b3ae0 con 0x7fe0540ce480 2023-12-02T00:29:31.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.948+0000 7fe05ab39640 1 -- 172.21.15.46:0/1733971880 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe054069400 con 0x7fe0540ce480 2023-12-02T00:29:31.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.948+0000 7fe0517fa640 1 -- 172.21.15.46:0/1733971880 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe04c00acd0 con 0x7fe0540ce480 2023-12-02T00:29:31.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.949+0000 7fe0517fa640 1 --2- 172.21.15.46:0/1733971880 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe04006ae40 0x7fe04006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:31.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.949+0000 7fe0517fa640 1 -- 172.21.15.46:0/1733971880 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fe04c088f00 con 0x7fe0540ce480 2023-12-02T00:29:31.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.949+0000 7fe053fff640 1 --2- 172.21.15.46:0/1733971880 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe04006ae40 0x7fe04006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:31.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.950+0000 7fe053fff640 1 --2- 172.21.15.46:0/1733971880 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe04006ae40 0x7fe04006d300 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe048036000 tx=0x7fe04803c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:31.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:31.953+0000 7fe0517fa640 1 -- 172.21.15.46:0/1733971880 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe04c053fb0 con 0x7fe0540ce480 2023-12-02T00:29:32.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.129+0000 7fe05ab39640 1 -- 172.21.15.46:0/1733971880 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe0540d0aa0 con 0x7fe0540ce480 2023-12-02T00:29:32.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.129+0000 7fe0517fa640 1 -- 172.21.15.46:0/1733971880 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe04c0180a0 con 0x7fe0540ce480 2023-12-02T00:29:32.132 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-12-02T00:29:32.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.131+0000 7fe05ab39640 1 -- 172.21.15.46:0/1733971880 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe04006ae40 msgr2=0x7fe04006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:32.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.131+0000 7fe05ab39640 1 --2- 172.21.15.46:0/1733971880 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe04006ae40 0x7fe04006d300 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe048036000 tx=0x7fe04803c040 comp rx=0 tx=0).stop 2023-12-02T00:29:32.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.131+0000 7fe05ab39640 1 -- 172.21.15.46:0/1733971880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0540ce480 msgr2=0x7fe0540b2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:32.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.131+0000 7fe05ab39640 1 --2- 172.21.15.46:0/1733971880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0540ce480 0x7fe0540b2b60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe04c00c890 tx=0x7fe04c00cd50 comp rx=0 tx=0).stop 2023-12-02T00:29:32.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.131+0000 7fe05ab39640 1 -- 172.21.15.46:0/1733971880 shutdown_connections 2023-12-02T00:29:32.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.131+0000 7fe05ab39640 1 --2- 172.21.15.46:0/1733971880 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe04006ae40 0x7fe04006d300 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:32.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.131+0000 7fe05ab39640 1 --2- 172.21.15.46:0/1733971880 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe054160f70 0x7fe0540c8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:32.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.131+0000 7fe05ab39640 1 --2- 172.21.15.46:0/1733971880 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe05415d2d0 0x7fe05415f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:32.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.131+0000 7fe05ab39640 1 --2- 172.21.15.46:0/1733971880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0540ce480 0x7fe0540b2b60 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:32.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.131+0000 7fe05ab39640 1 -- 172.21.15.46:0/1733971880 >> 172.21.15.46:0/1733971880 conn(0x7fe054072230 msgr2=0x7fe054157a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:32.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.131+0000 7fe05ab39640 1 -- 172.21.15.46:0/1733971880 shutdown_connections 2023-12-02T00:29:32.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.131+0000 7fe05ab39640 1 -- 172.21.15.46:0/1733971880 wait complete. 2023-12-02T00:29:32.144 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-02T00:29:32.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.250+0000 7f2d97ea6640 1 Processor -- start 2023-12-02T00:29:32.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.250+0000 7f2d97ea6640 1 -- start start 2023-12-02T00:29:32.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.251+0000 7f2d97ea6640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2d9006abe0 0x7f2d901618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:32.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.251+0000 7f2d97ea6640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d901406f0 con 0x7f2d90140310 2023-12-02T00:29:32.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.251+0000 7f2d97ea6640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d90143960 con 0x7f2d901409d0 2023-12-02T00:29:32.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.251+0000 7f2d97ea6640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d90161e20 con 0x7f2d9006abe0 2023-12-02T00:29:32.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.251+0000 7f2d95c42640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2d90140310 0x7f2d9006a4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50510/0 (socket says 172.21.15.46:50510) 2023-12-02T00:29:32.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.251+0000 7f2d95c42640 1 -- 172.21.15.46:0/3310524214 learned_addr learned my addr 172.21.15.46:0/3310524214 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:32.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.251+0000 7f2d95441640 1 --2- 172.21.15.46:0/3310524214 >> v2:172.21.15.162:3300/0 conn(0x7f2d9006abe0 0x7f2d901618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:32.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.252+0000 7f2d94c40640 1 -- 172.21.15.46:0/3310524214 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3960187595 0 0) 0x7f2d901406f0 con 0x7f2d90140310 2023-12-02T00:29:32.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.252+0000 7f2d94c40640 1 -- 172.21.15.46:0/3310524214 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d78003520 con 0x7f2d90140310 2023-12-02T00:29:32.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.252+0000 7f2d94c40640 1 -- 172.21.15.46:0/3310524214 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3164114654 0 0) 0x7f2d78003520 con 0x7f2d90140310 2023-12-02T00:29:32.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.252+0000 7f2d94c40640 1 -- 172.21.15.46:0/3310524214 >> v2:172.21.15.162:3300/0 conn(0x7f2d9006abe0 msgr2=0x7f2d901618e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:32.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.252+0000 7f2d94c40640 1 --2- 172.21.15.46:0/3310524214 >> v2:172.21.15.162:3300/0 conn(0x7f2d9006abe0 0x7f2d901618e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:32.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.252+0000 7f2d94c40640 1 -- 172.21.15.46:0/3310524214 >> v1:172.21.15.162:6789/0 conn(0x7f2d901409d0 legacy=0x7f2d9006efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:32.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.252+0000 7f2d94c40640 1 -- 172.21.15.46:0/3310524214 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d90142de0 con 0x7f2d90140310 2023-12-02T00:29:32.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.252+0000 7f2d94c40640 1 -- 172.21.15.46:0/3310524214 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7f2d88001670 con 0x7f2d90140310 2023-12-02T00:29:32.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.252+0000 7f2d94c40640 1 -- 172.21.15.46:0/3310524214 >> v1:172.21.15.46:6789/0 conn(0x7f2d90140310 legacy=0x7f2d9006a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:32.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.252+0000 7f2d94c40640 1 --2- 172.21.15.46:0/3310524214 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d78003970 0x7f2d78003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:32.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.252+0000 7f2d94c40640 1 --2- 172.21.15.46:0/3310524214 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d780047c0 0x7f2d78004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:32.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.252+0000 7f2d94c40640 1 --2- 172.21.15.46:0/3310524214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d78005180 0x7f2d7800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:32.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.252+0000 7f2d94c40640 1 -- 172.21.15.46:0/3310524214 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d90161e20 con 0x7f2d78005180 2023-12-02T00:29:32.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.252+0000 7f2d94c40640 1 -- 172.21.15.46:0/3310524214 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7800c120 con 0x7f2d780047c0 2023-12-02T00:29:32.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.252+0000 7f2d94c40640 1 -- 172.21.15.46:0/3310524214 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7800c2e0 con 0x7f2d78003970 2023-12-02T00:29:32.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.252+0000 7f2d96443640 1 --2- 172.21.15.46:0/3310524214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d78005180 0x7f2d7800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:32.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.252+0000 7f2d95441640 1 --2- 172.21.15.46:0/3310524214 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d780047c0 0x7f2d78004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:32.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.252+0000 7f2d95c42640 1 --2- 172.21.15.46:0/3310524214 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d78003970 0x7f2d78003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:32.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.252+0000 7f2d96443640 1 -- 172.21.15.46:0/3310524214 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d78003970 msgr2=0x7f2d78003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:32.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.252+0000 7f2d96443640 1 --2- 172.21.15.46:0/3310524214 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d78003970 0x7f2d78003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:32.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.252+0000 7f2d96443640 1 -- 172.21.15.46:0/3310524214 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d780047c0 msgr2=0x7f2d78004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:32.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.252+0000 7f2d96443640 1 --2- 172.21.15.46:0/3310524214 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d780047c0 0x7f2d78004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:32.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.252+0000 7f2d96443640 1 -- 172.21.15.46:0/3310524214 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d78003520 con 0x7f2d78005180 2023-12-02T00:29:32.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.253+0000 7f2d96443640 1 -- 172.21.15.46:0/3310524214 reap_dead start 2023-12-02T00:29:32.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.253+0000 7f2d96443640 1 --2- 172.21.15.46:0/3310524214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d78005180 0x7f2d7800b980 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2d80001e50 tx=0x7f2d780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7862aed8cc015975 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:32.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.253+0000 7f2d94c40640 1 -- 172.21.15.46:0/3310524214 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d80003b70 con 0x7f2d78005180 2023-12-02T00:29:32.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.253+0000 7f2d94c40640 1 -- 172.21.15.46:0/3310524214 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d80003cf0 con 0x7f2d78005180 2023-12-02T00:29:32.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.253+0000 7f2d94c40640 1 -- 172.21.15.46:0/3310524214 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d80005310 con 0x7f2d78005180 2023-12-02T00:29:32.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.254+0000 7f2d97ea6640 1 -- 172.21.15.46:0/3310524214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d78005180 msgr2=0x7f2d7800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:32.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.254+0000 7f2d97ea6640 1 --2- 172.21.15.46:0/3310524214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d78005180 0x7f2d7800b980 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2d80001e50 tx=0x7f2d780057f0 comp rx=0 tx=0).stop 2023-12-02T00:29:32.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.254+0000 7f2d97ea6640 1 -- 172.21.15.46:0/3310524214 shutdown_connections 2023-12-02T00:29:32.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.254+0000 7f2d97ea6640 1 --2- 172.21.15.46:0/3310524214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d78005180 0x7f2d7800b980 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:32.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.254+0000 7f2d97ea6640 1 -- 172.21.15.46:0/3310524214 >> 172.21.15.46:0/3310524214 conn(0x7f2d900b1e50 msgr2=0x7f2d900b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:32.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.254+0000 7f2d97ea6640 1 -- 172.21.15.46:0/3310524214 shutdown_connections 2023-12-02T00:29:32.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.254+0000 7f2d97ea6640 1 -- 172.21.15.46:0/3310524214 wait complete. 2023-12-02T00:29:32.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.255+0000 7f2d97ea6640 1 Processor -- start 2023-12-02T00:29:32.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.255+0000 7f2d97ea6640 1 -- start start 2023-12-02T00:29:32.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.256+0000 7f2d97ea6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d78005180 0x7f2d901f3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:32.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.256+0000 7f2d97ea6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d9015c700 0x7f2d90145900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:32.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.256+0000 7f2d97ea6640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d9013f2c0 0x7f2d9013f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:32.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.256+0000 7f2d97ea6640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d900b5020 con 0x7f2d78005180 2023-12-02T00:29:32.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.256+0000 7f2d97ea6640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9015e980 con 0x7f2d9013f2c0 2023-12-02T00:29:32.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.256+0000 7f2d97ea6640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d900af7a0 con 0x7f2d9015c700 2023-12-02T00:29:32.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.256+0000 7f2d95c42640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d78005180 0x7f2d901f3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:32.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.256+0000 7f2d95c42640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d78005180 0x7f2d901f3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53064/0 (socket says 172.21.15.46:53064) 2023-12-02T00:29:32.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.256+0000 7f2d95c42640 1 -- 172.21.15.46:0/711907675 learned_addr learned my addr 172.21.15.46:0/711907675 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:32.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.256+0000 7f2d96443640 1 --2- 172.21.15.46:0/711907675 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d9013f2c0 0x7f2d9013f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:32.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.256+0000 7f2d95441640 1 --2- 172.21.15.46:0/711907675 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d9015c700 0x7f2d90145900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:32.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.256+0000 7f2d95c42640 1 -- 172.21.15.46:0/711907675 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d9015c700 msgr2=0x7f2d90145900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:32.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.256+0000 7f2d95c42640 1 --2- 172.21.15.46:0/711907675 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d9015c700 0x7f2d90145900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:32.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.256+0000 7f2d95c42640 1 -- 172.21.15.46:0/711907675 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d9013f2c0 msgr2=0x7f2d9013f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:32.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.256+0000 7f2d95c42640 1 --2- 172.21.15.46:0/711907675 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d9013f2c0 0x7f2d9013f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:32.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.256+0000 7f2d95c42640 1 -- 172.21.15.46:0/711907675 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d900b47c0 con 0x7f2d78005180 2023-12-02T00:29:32.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.256+0000 7f2d95c42640 1 --2- 172.21.15.46:0/711907675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d78005180 0x7f2d901f3430 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2d880072f0 tx=0x7f2d880077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:32.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.257+0000 7f2d8e7fc640 1 -- 172.21.15.46:0/711907675 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d88004ea0 con 0x7f2d78005180 2023-12-02T00:29:32.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.257+0000 7f2d8e7fc640 1 -- 172.21.15.46:0/711907675 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d8800f990 con 0x7f2d78005180 2023-12-02T00:29:32.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.257+0000 7f2d97ea6640 1 -- 172.21.15.46:0/711907675 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d90143cf0 con 0x7f2d78005180 2023-12-02T00:29:32.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.257+0000 7f2d8e7fc640 1 -- 172.21.15.46:0/711907675 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8800fd20 con 0x7f2d78005180 2023-12-02T00:29:32.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.257+0000 7f2d97ea6640 1 -- 172.21.15.46:0/711907675 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d900b4d40 con 0x7f2d78005180 2023-12-02T00:29:32.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.258+0000 7f2d97ea6640 1 -- 172.21.15.46:0/711907675 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d90143b50 con 0x7f2d78005180 2023-12-02T00:29:32.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.258+0000 7f2d8e7fc640 1 -- 172.21.15.46:0/711907675 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2d8801e3b0 con 0x7f2d78005180 2023-12-02T00:29:32.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.258+0000 7f2d8e7fc640 1 --2- 172.21.15.46:0/711907675 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2d7c06ae40 0x7f2d7c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:32.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.259+0000 7f2d8e7fc640 1 -- 172.21.15.46:0/711907675 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f2d88089270 con 0x7f2d78005180 2023-12-02T00:29:32.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.259+0000 7f2d95441640 1 --2- 172.21.15.46:0/711907675 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2d7c06ae40 0x7f2d7c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:32.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.259+0000 7f2d95441640 1 --2- 172.21.15.46:0/711907675 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2d7c06ae40 0x7f2d7c06d300 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2d900b5210 tx=0x7f2d84008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:32.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.263+0000 7f2d8e7fc640 1 -- 172.21.15.46:0/711907675 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2d90143b50 con 0x7f2d78005180 2023-12-02T00:29:32.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.453+0000 7f2d97ea6640 1 -- 172.21.15.46:0/711907675 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2d90161030 con 0x7f2d78005180 2023-12-02T00:29:32.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.454+0000 7f2d8e7fc640 1 -- 172.21.15.46:0/711907675 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2d88057d20 con 0x7f2d78005180 2023-12-02T00:29:32.456 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2023-12-02T00:29:32.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.456+0000 7f2d97ea6640 1 -- 172.21.15.46:0/711907675 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2d7c06ae40 msgr2=0x7f2d7c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:32.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.456+0000 7f2d97ea6640 1 --2- 172.21.15.46:0/711907675 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2d7c06ae40 0x7f2d7c06d300 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2d900b5210 tx=0x7f2d84008040 comp rx=0 tx=0).stop 2023-12-02T00:29:32.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.456+0000 7f2d97ea6640 1 -- 172.21.15.46:0/711907675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d78005180 msgr2=0x7f2d901f3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:32.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.456+0000 7f2d97ea6640 1 --2- 172.21.15.46:0/711907675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d78005180 0x7f2d901f3430 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2d880072f0 tx=0x7f2d880077b0 comp rx=0 tx=0).stop 2023-12-02T00:29:32.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.456+0000 7f2d97ea6640 1 -- 172.21.15.46:0/711907675 shutdown_connections 2023-12-02T00:29:32.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.456+0000 7f2d97ea6640 1 --2- 172.21.15.46:0/711907675 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2d7c06ae40 0x7f2d7c06d300 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:32.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.456+0000 7f2d97ea6640 1 --2- 172.21.15.46:0/711907675 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d9013f2c0 0x7f2d9013f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:32.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.456+0000 7f2d97ea6640 1 --2- 172.21.15.46:0/711907675 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d9015c700 0x7f2d90145900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:32.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.456+0000 7f2d97ea6640 1 --2- 172.21.15.46:0/711907675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d78005180 0x7f2d901f3430 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:32.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.456+0000 7f2d97ea6640 1 -- 172.21.15.46:0/711907675 >> 172.21.15.46:0/711907675 conn(0x7f2d900b1e50 msgr2=0x7f2d900b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:32.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.456+0000 7f2d97ea6640 1 -- 172.21.15.46:0/711907675 shutdown_connections 2023-12-02T00:29:32.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.456+0000 7f2d97ea6640 1 -- 172.21.15.46:0/711907675 wait complete. 2023-12-02T00:29:32.468 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi046.front.sepia.ceph.com... 2023-12-02T00:29:32.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:32.469 DEBUG:teuthology.orchestra.run.smithi046:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2023-12-02T00:29:32.501 INFO:teuthology.orchestra.run:waiting for 300 2023-12-02T00:29:32.501 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2023-12-02T00:29:32.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:32.501 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-12-02T00:29:32.556 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-12-02T00:29:32.557 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-02T00:29:32.578 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-12-02T00:29:32.578 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-02T00:29:32.636 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:29:32.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.742+0000 7f1a999d0640 1 Processor -- start 2023-12-02T00:29:32.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.742+0000 7f1a999d0640 1 -- start start 2023-12-02T00:29:32.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.742+0000 7f1a999d0640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1a9406ea50 0x7f1a9406ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:32.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.742+0000 7f1a999d0640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1a940d0ed0 0x7f1a940d1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:32.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.742+0000 7f1a999d0640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a94161d20 con 0x7f1a94160fb0 2023-12-02T00:29:32.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.742+0000 7f1a999d0640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a9406f370 con 0x7f1a940d0ed0 2023-12-02T00:29:32.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.742+0000 7f1a999d0640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a9415e760 con 0x7f1a9406ea50 2023-12-02T00:29:32.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.742+0000 7f1a937fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1a940d0ed0 0x7f1a940d1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:32.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.742+0000 7f1a927fc640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1a94160fb0 0x7f1a94161440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59270/0 (socket says 172.21.15.46:59270) 2023-12-02T00:29:32.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.742+0000 7f1a927fc640 1 -- 172.21.15.46:0/1297266976 learned_addr learned my addr 172.21.15.46:0/1297266976 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:32.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.742+0000 7f1a92ffd640 1 --2- 172.21.15.46:0/1297266976 >> v2:172.21.15.162:3300/0 conn(0x7f1a9406ea50 0x7f1a9406ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:32.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.742+0000 7f1a937fe640 1 -- 172.21.15.46:0/1297266976 >> v2:172.21.15.162:3300/0 conn(0x7f1a9406ea50 msgr2=0x7f1a9406ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:32.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.742+0000 7f1a937fe640 1 --2- 172.21.15.46:0/1297266976 >> v2:172.21.15.162:3300/0 conn(0x7f1a9406ea50 0x7f1a9406ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:32.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.742+0000 7f1a937fe640 1 -- 172.21.15.46:0/1297266976 >> v1:172.21.15.46:6789/0 conn(0x7f1a94160fb0 legacy=0x7f1a94161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:32.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.742+0000 7f1a937fe640 1 -- 172.21.15.46:0/1297266976 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a9415d9a0 con 0x7f1a940d0ed0 2023-12-02T00:29:32.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.742+0000 7f1a91ffb640 1 -- 172.21.15.46:0/1297266976 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 382334655 0 0) 0x7f1a94161d20 con 0x7f1a94160fb0 2023-12-02T00:29:32.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.742+0000 7f1a92ffd640 1 --2- 172.21.15.46:0/1297266976 >> v2:172.21.15.162:3300/0 conn(0x7f1a9406ea50 0x7f1a9406ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:32.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.743+0000 7f1a937fe640 1 --2- 172.21.15.46:0/1297266976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a940d0ed0 0x7f1a940d1320 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1a9406f680 tx=0x7f1a8402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c887c4b57111d5 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:32.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.743+0000 7f1a91ffb640 1 -- 172.21.15.46:0/1297266976 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8402cbf0 con 0x7f1a940d0ed0 2023-12-02T00:29:32.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.743+0000 7f1a91ffb640 1 -- 172.21.15.46:0/1297266976 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a8402cd50 con 0x7f1a940d0ed0 2023-12-02T00:29:32.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.743+0000 7f1a91ffb640 1 -- 172.21.15.46:0/1297266976 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a84034660 con 0x7f1a940d0ed0 2023-12-02T00:29:32.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.744+0000 7f1a999d0640 1 -- 172.21.15.46:0/1297266976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a940d0ed0 msgr2=0x7f1a940d1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:32.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.744+0000 7f1a999d0640 1 --2- 172.21.15.46:0/1297266976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a940d0ed0 0x7f1a940d1320 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1a9406f680 tx=0x7f1a8402be40 comp rx=0 tx=0).stop 2023-12-02T00:29:32.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.744+0000 7f1a999d0640 1 -- 172.21.15.46:0/1297266976 shutdown_connections 2023-12-02T00:29:32.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.744+0000 7f1a999d0640 1 --2- 172.21.15.46:0/1297266976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a940d0ed0 0x7f1a940d1320 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:32.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.744+0000 7f1a999d0640 1 --2- 172.21.15.46:0/1297266976 >> v2:172.21.15.162:3300/0 conn(0x7f1a9406ea50 0x7f1a9406ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:32.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.744+0000 7f1a999d0640 1 -- 172.21.15.46:0/1297266976 >> 172.21.15.46:0/1297266976 conn(0x7f1a9406a6b0 msgr2=0x7f1a9406aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:32.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.744+0000 7f1a999d0640 1 -- 172.21.15.46:0/1297266976 shutdown_connections 2023-12-02T00:29:32.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.744+0000 7f1a999d0640 1 -- 172.21.15.46:0/1297266976 wait complete. 2023-12-02T00:29:32.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.745+0000 7f1a999d0640 1 Processor -- start 2023-12-02T00:29:32.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.745+0000 7f1a999d0640 1 -- start start 2023-12-02T00:29:32.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.746+0000 7f1a999d0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a9406ea50 0x7f1a9413f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:32.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.746+0000 7f1a999d0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a940d0ed0 0x7f1a9415fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:32.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.746+0000 7f1a999d0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a94160fb0 0x7f1a9413d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:32.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.746+0000 7f1a999d0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9413fc40 con 0x7f1a94160fb0 2023-12-02T00:29:32.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.746+0000 7f1a999d0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a941e6bf0 con 0x7f1a940d0ed0 2023-12-02T00:29:32.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.746+0000 7f1a999d0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9415ab50 con 0x7f1a9406ea50 2023-12-02T00:29:32.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.746+0000 7f1a937fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a94160fb0 0x7f1a9413d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:32.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.746+0000 7f1a937fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a94160fb0 0x7f1a9413d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57204/0 (socket says 172.21.15.46:57204) 2023-12-02T00:29:32.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.746+0000 7f1a937fe640 1 -- 172.21.15.46:0/4290780286 learned_addr learned my addr 172.21.15.46:0/4290780286 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:32.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.746+0000 7f1a927fc640 1 --2- 172.21.15.46:0/4290780286 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a940d0ed0 0x7f1a9415fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:32.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.746+0000 7f1a937fe640 1 -- 172.21.15.46:0/4290780286 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a9406ea50 msgr2=0x7f1a9413f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:32.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.746+0000 7f1a937fe640 1 --2- 172.21.15.46:0/4290780286 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a9406ea50 0x7f1a9413f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:32.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.746+0000 7f1a937fe640 1 -- 172.21.15.46:0/4290780286 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a940d0ed0 msgr2=0x7f1a9415fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:32.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.746+0000 7f1a937fe640 1 --2- 172.21.15.46:0/4290780286 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a940d0ed0 0x7f1a9415fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:32.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.746+0000 7f1a937fe640 1 -- 172.21.15.46:0/4290780286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a84009a70 con 0x7f1a94160fb0 2023-12-02T00:29:32.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.746+0000 7f1a927fc640 1 --2- 172.21.15.46:0/4290780286 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a940d0ed0 0x7f1a9415fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:29:32.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.747+0000 7f1a937fe640 1 --2- 172.21.15.46:0/4290780286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a94160fb0 0x7f1a9413d9b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1a84009e80 tx=0x7f1a84004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:32.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.747+0000 7f1a7bfff640 1 -- 172.21.15.46:0/4290780286 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a84040650 con 0x7f1a94160fb0 2023-12-02T00:29:32.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.747+0000 7f1a7bfff640 1 -- 172.21.15.46:0/4290780286 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a840407b0 con 0x7f1a94160fb0 2023-12-02T00:29:32.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.747+0000 7f1a7bfff640 1 -- 172.21.15.46:0/4290780286 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a84040b60 con 0x7f1a94160fb0 2023-12-02T00:29:32.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.747+0000 7f1a999d0640 1 -- 172.21.15.46:0/4290780286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a9415a2b0 con 0x7f1a94160fb0 2023-12-02T00:29:32.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.747+0000 7f1a999d0640 1 -- 172.21.15.46:0/4290780286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a9415a870 con 0x7f1a94160fb0 2023-12-02T00:29:32.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.748+0000 7f1a999d0640 1 -- 172.21.15.46:0/4290780286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a94069400 con 0x7f1a94160fb0 2023-12-02T00:29:32.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.748+0000 7f1a7bfff640 1 -- 172.21.15.46:0/4290780286 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1a84040cc0 con 0x7f1a94160fb0 2023-12-02T00:29:32.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.748+0000 7f1a7bfff640 1 --2- 172.21.15.46:0/4290780286 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f1a8006ae40 0x7f1a8006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:32.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.749+0000 7f1a7bfff640 1 -- 172.21.15.46:0/4290780286 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f1a840ac3c0 con 0x7f1a94160fb0 2023-12-02T00:29:32.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.749+0000 7f1a92ffd640 1 --2- 172.21.15.46:0/4290780286 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f1a8006ae40 0x7f1a8006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:32.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.749+0000 7f1a92ffd640 1 --2- 172.21.15.46:0/4290780286 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f1a8006ae40 0x7f1a8006d300 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c006e80 tx=0x7f1a8c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:32.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.754+0000 7f1a7bfff640 1 -- 172.21.15.46:0/4290780286 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1a84077470 con 0x7f1a94160fb0 2023-12-02T00:29:32.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.931+0000 7f1a999d0640 1 -- 172.21.15.46:0/4290780286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1a941e6d60 con 0x7f1a94160fb0 2023-12-02T00:29:32.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.932+0000 7f1a7bfff640 1 -- 172.21.15.46:0/4290780286 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16777 (secure 0 0 0) 0x7f1a84009450 con 0x7f1a94160fb0 2023-12-02T00:29:32.934 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:29:32.934 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":23,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:29:30.868939+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-02T00:28:47.273337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-02T00:28:48.060115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.14","mappings":[{"from":4,"to":2}]},{"pgid":"3.1e","mappings":[{"from":3,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:29:32.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.934+0000 7f1a999d0640 1 -- 172.21.15.46:0/4290780286 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f1a8006ae40 msgr2=0x7f1a8006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:32.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.934+0000 7f1a999d0640 1 --2- 172.21.15.46:0/4290780286 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f1a8006ae40 0x7f1a8006d300 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c006e80 tx=0x7f1a8c008040 comp rx=0 tx=0).stop 2023-12-02T00:29:32.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.934+0000 7f1a999d0640 1 -- 172.21.15.46:0/4290780286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a94160fb0 msgr2=0x7f1a9413d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:32.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.934+0000 7f1a999d0640 1 --2- 172.21.15.46:0/4290780286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a94160fb0 0x7f1a9413d9b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1a84009e80 tx=0x7f1a84004a40 comp rx=0 tx=0).stop 2023-12-02T00:29:32.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.934+0000 7f1a999d0640 1 -- 172.21.15.46:0/4290780286 shutdown_connections 2023-12-02T00:29:32.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.934+0000 7f1a999d0640 1 --2- 172.21.15.46:0/4290780286 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f1a8006ae40 0x7f1a8006d300 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:32.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.934+0000 7f1a999d0640 1 --2- 172.21.15.46:0/4290780286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a94160fb0 0x7f1a9413d9b0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:32.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.934+0000 7f1a999d0640 1 --2- 172.21.15.46:0/4290780286 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a940d0ed0 0x7f1a9415fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:32.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.934+0000 7f1a999d0640 1 --2- 172.21.15.46:0/4290780286 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a9406ea50 0x7f1a9413f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:32.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.934+0000 7f1a999d0640 1 -- 172.21.15.46:0/4290780286 >> 172.21.15.46:0/4290780286 conn(0x7f1a9406a6b0 msgr2=0x7f1a941416d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:32.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.934+0000 7f1a999d0640 1 -- 172.21.15.46:0/4290780286 shutdown_connections 2023-12-02T00:29:32.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:32.934+0000 7f1a999d0640 1 -- 172.21.15.46:0/4290780286 wait complete. 2023-12-02T00:29:32.947 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-02T00:28:37.213645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-02T00:28:47.273337+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-02T00:28:48.060115+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-02T00:29:32.948 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-02T00:29:33.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.055+0000 7f95e6116640 1 Processor -- start 2023-12-02T00:29:33.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.055+0000 7f95e6116640 1 -- start start 2023-12-02T00:29:33.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.056+0000 7f95e6116640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f95e0142270 0x7f95e01426c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:33.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.056+0000 7f95e6116640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95e0161d20 con 0x7f95e0160fb0 2023-12-02T00:29:33.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.056+0000 7f95e6116640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95e015e760 con 0x7f95e006ea50 2023-12-02T00:29:33.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.056+0000 7f95e6116640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95e00d1ee0 con 0x7f95e0142270 2023-12-02T00:29:33.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.056+0000 7f95deffd640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f95e0160fb0 0x7f95e0161440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59284/0 (socket says 172.21.15.46:59284) 2023-12-02T00:29:33.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.056+0000 7f95deffd640 1 -- 172.21.15.46:0/1773253941 learned_addr learned my addr 172.21.15.46:0/1773253941 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:33.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.056+0000 7f95de7fc640 1 -- 172.21.15.46:0/1773253941 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 828624464 0 0) 0x7f95e0161d20 con 0x7f95e0160fb0 2023-12-02T00:29:33.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.056+0000 7f95dffff640 1 --2- 172.21.15.46:0/1773253941 >> v2:172.21.15.162:3300/0 conn(0x7f95e0142270 0x7f95e01426c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:33.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.056+0000 7f95de7fc640 1 -- 172.21.15.46:0/1773253941 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95c8003520 con 0x7f95e0160fb0 2023-12-02T00:29:33.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.057+0000 7f95de7fc640 1 -- 172.21.15.46:0/1773253941 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3635696202 0 0) 0x7f95c8003520 con 0x7f95e0160fb0 2023-12-02T00:29:33.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.057+0000 7f95de7fc640 1 -- 172.21.15.46:0/1773253941 >> v2:172.21.15.162:3300/0 conn(0x7f95e0142270 msgr2=0x7f95e01426c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:33.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.057+0000 7f95de7fc640 1 --2- 172.21.15.46:0/1773253941 >> v2:172.21.15.162:3300/0 conn(0x7f95e0142270 0x7f95e01426c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:33.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.057+0000 7f95de7fc640 1 -- 172.21.15.46:0/1773253941 >> v1:172.21.15.162:6789/0 conn(0x7f95e006ea50 legacy=0x7f95e006ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:33.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.057+0000 7f95de7fc640 1 -- 172.21.15.46:0/1773253941 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e015d9a0 con 0x7f95e0160fb0 2023-12-02T00:29:33.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.057+0000 7f95de7fc640 1 -- 172.21.15.46:0/1773253941 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7f95d4001670 con 0x7f95e0160fb0 2023-12-02T00:29:33.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.057+0000 7f95de7fc640 1 -- 172.21.15.46:0/1773253941 >> v1:172.21.15.46:6789/0 conn(0x7f95e0160fb0 legacy=0x7f95e0161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:33.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.057+0000 7f95de7fc640 1 --2- 172.21.15.46:0/1773253941 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95c8003970 0x7f95c8003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:33.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.057+0000 7f95de7fc640 1 --2- 172.21.15.46:0/1773253941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95c8004370 0x7f95c8004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:33.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.057+0000 7f95de7fc640 1 --2- 172.21.15.46:0/1773253941 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95c80050e0 0x7f95c800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:33.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.057+0000 7f95de7fc640 1 -- 172.21.15.46:0/1773253941 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e00d1ee0 con 0x7f95c8004370 2023-12-02T00:29:33.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.057+0000 7f95de7fc640 1 -- 172.21.15.46:0/1773253941 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c800c020 con 0x7f95c80050e0 2023-12-02T00:29:33.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.057+0000 7f95de7fc640 1 -- 172.21.15.46:0/1773253941 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95c800c1e0 con 0x7f95c8003970 2023-12-02T00:29:33.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.057+0000 7f95deffd640 1 --2- 172.21.15.46:0/1773253941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95c8004370 0x7f95c8004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:33.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.057+0000 7f95dffff640 1 --2- 172.21.15.46:0/1773253941 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95c80050e0 0x7f95c800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:33.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.057+0000 7f95df7fe640 1 --2- 172.21.15.46:0/1773253941 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95c8003970 0x7f95c8003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:33.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.057+0000 7f95deffd640 1 -- 172.21.15.46:0/1773253941 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95c8003970 msgr2=0x7f95c8003d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:33.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.057+0000 7f95deffd640 1 --2- 172.21.15.46:0/1773253941 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95c8003970 0x7f95c8003d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:33.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.057+0000 7f95deffd640 1 -- 172.21.15.46:0/1773253941 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95c80050e0 msgr2=0x7f95c800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:33.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.057+0000 7f95deffd640 1 --2- 172.21.15.46:0/1773253941 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95c80050e0 0x7f95c800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:33.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.057+0000 7f95deffd640 1 -- 172.21.15.46:0/1773253941 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c8003520 con 0x7f95c8004370 2023-12-02T00:29:33.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.057+0000 7f95dffff640 1 --2- 172.21.15.46:0/1773253941 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95c80050e0 0x7f95c800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:33.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.057+0000 7f95dffff640 1 -- 172.21.15.46:0/1773253941 reap_dead start 2023-12-02T00:29:33.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.058+0000 7f95deffd640 1 --2- 172.21.15.46:0/1773253941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95c8004370 0x7f95c8004840 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f95d400c210 tx=0x7f95c80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46306e81928d8487 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:33.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.058+0000 7f95de7fc640 1 -- 172.21.15.46:0/1773253941 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d4005ad0 con 0x7f95c8004370 2023-12-02T00:29:33.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.058+0000 7f95de7fc640 1 -- 172.21.15.46:0/1773253941 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95d4005c50 con 0x7f95c8004370 2023-12-02T00:29:33.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.058+0000 7f95de7fc640 1 -- 172.21.15.46:0/1773253941 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d402c890 con 0x7f95c8004370 2023-12-02T00:29:33.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.059+0000 7f95e6116640 1 -- 172.21.15.46:0/1773253941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95c8004370 msgr2=0x7f95c8004840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:33.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.059+0000 7f95e6116640 1 --2- 172.21.15.46:0/1773253941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95c8004370 0x7f95c8004840 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f95d400c210 tx=0x7f95c80057f0 comp rx=0 tx=0).stop 2023-12-02T00:29:33.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.059+0000 7f95e6116640 1 -- 172.21.15.46:0/1773253941 shutdown_connections 2023-12-02T00:29:33.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.059+0000 7f95e6116640 1 --2- 172.21.15.46:0/1773253941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95c8004370 0x7f95c8004840 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:33.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.059+0000 7f95e6116640 1 -- 172.21.15.46:0/1773253941 >> 172.21.15.46:0/1773253941 conn(0x7f95e006a6b0 msgr2=0x7f95e006aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:33.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.059+0000 7f95e6116640 1 -- 172.21.15.46:0/1773253941 shutdown_connections 2023-12-02T00:29:33.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.059+0000 7f95e6116640 1 -- 172.21.15.46:0/1773253941 wait complete. 2023-12-02T00:29:33.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.060+0000 7f95e6116640 1 Processor -- start 2023-12-02T00:29:33.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.060+0000 7f95e6116640 1 -- start start 2023-12-02T00:29:33.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.061+0000 7f95e6116640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95c8004370 0x7f95e00b0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:33.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.061+0000 7f95e6116640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95e00b2720 0x7f95e01539c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:33.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.061+0000 7f95e6116640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95e006c630 0x7f95e006cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:33.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.061+0000 7f95e6116640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e006cff0 con 0x7f95e00b2720 2023-12-02T00:29:33.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.061+0000 7f95e6116640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e0154cf0 con 0x7f95c8004370 2023-12-02T00:29:33.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.061+0000 7f95e6116640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95e0156430 con 0x7f95e006c630 2023-12-02T00:29:33.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.061+0000 7f95deffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95e00b2720 0x7f95e01539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:33.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.061+0000 7f95deffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95e00b2720 0x7f95e01539c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57208/0 (socket says 172.21.15.46:57208) 2023-12-02T00:29:33.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.061+0000 7f95deffd640 1 -- 172.21.15.46:0/2434559674 learned_addr learned my addr 172.21.15.46:0/2434559674 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:33.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.061+0000 7f95df7fe640 1 --2- 172.21.15.46:0/2434559674 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95c8004370 0x7f95e00b0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:33.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.061+0000 7f95dffff640 1 --2- 172.21.15.46:0/2434559674 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95e006c630 0x7f95e006cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:33.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.061+0000 7f95deffd640 1 -- 172.21.15.46:0/2434559674 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95e006c630 msgr2=0x7f95e006cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:33.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.061+0000 7f95deffd640 1 --2- 172.21.15.46:0/2434559674 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95e006c630 0x7f95e006cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:33.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.061+0000 7f95deffd640 1 -- 172.21.15.46:0/2434559674 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95c8004370 msgr2=0x7f95e00b0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:33.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.061+0000 7f95deffd640 1 --2- 172.21.15.46:0/2434559674 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95c8004370 0x7f95e00b0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:33.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.061+0000 7f95deffd640 1 -- 172.21.15.46:0/2434559674 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e00b1140 con 0x7f95e00b2720 2023-12-02T00:29:33.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.061+0000 7f95df7fe640 1 --2- 172.21.15.46:0/2434559674 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95c8004370 0x7f95e00b0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:33.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.061+0000 7f95deffd640 1 --2- 172.21.15.46:0/2434559674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95e00b2720 0x7f95e01539c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f95d402c430 tx=0x7f95d402e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:33.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.061+0000 7f95c7fff640 1 -- 172.21.15.46:0/2434559674 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d403c040 con 0x7f95e00b2720 2023-12-02T00:29:33.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.061+0000 7f95c7fff640 1 -- 172.21.15.46:0/2434559674 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95d402cca0 con 0x7f95e00b2720 2023-12-02T00:29:33.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.062+0000 7f95e6116640 1 -- 172.21.15.46:0/2434559674 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95e00b1420 con 0x7f95e00b2720 2023-12-02T00:29:33.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.062+0000 7f95c7fff640 1 -- 172.21.15.46:0/2434559674 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d4001670 con 0x7f95e00b2720 2023-12-02T00:29:33.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.062+0000 7f95e6116640 1 -- 172.21.15.46:0/2434559674 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95e00b19e0 con 0x7f95e00b2720 2023-12-02T00:29:33.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.063+0000 7f95e6116640 1 -- 172.21.15.46:0/2434559674 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95e00b1b70 con 0x7f95e00b2720 2023-12-02T00:29:33.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.063+0000 7f95c7fff640 1 -- 172.21.15.46:0/2434559674 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f95d4032070 con 0x7f95e00b2720 2023-12-02T00:29:33.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.063+0000 7f95c7fff640 1 --2- 172.21.15.46:0/2434559674 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f95cc06ae70 0x7f95cc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:33.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.063+0000 7f95c7fff640 1 -- 172.21.15.46:0/2434559674 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f95d40aa930 con 0x7f95e00b2720 2023-12-02T00:29:33.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.064+0000 7f95df7fe640 1 --2- 172.21.15.46:0/2434559674 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f95cc06ae70 0x7f95cc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:33.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.064+0000 7f95df7fe640 1 --2- 172.21.15.46:0/2434559674 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f95cc06ae70 0x7f95cc06d330 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f95d00035f0 tx=0x7f95d0007c50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:33.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.069+0000 7f95c7fff640 1 -- 172.21.15.46:0/2434559674 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f95d4075950 con 0x7f95e00b2720 2023-12-02T00:29:33.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.241+0000 7f95e6116640 1 -- 172.21.15.46:0/2434559674 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f95e0069400 con 0x7f95e00b2720 2023-12-02T00:29:33.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.241+0000 7f95c7fff640 1 -- 172.21.15.46:0/2434559674 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f95d4078f70 con 0x7f95e00b2720 2023-12-02T00:29:33.243 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-12-02T00:29:33.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.243+0000 7f95e6116640 1 -- 172.21.15.46:0/2434559674 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f95cc06ae70 msgr2=0x7f95cc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:35.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.243+0000 7f95e6116640 1 --2- 172.21.15.46:0/2434559674 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f95cc06ae70 0x7f95cc06d330 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f95d00035f0 tx=0x7f95d0007c50 comp rx=0 tx=0).stop 2023-12-02T00:29:35.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.243+0000 7f95e6116640 1 -- 172.21.15.46:0/2434559674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95e00b2720 msgr2=0x7f95e01539c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:35.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.243+0000 7f95e6116640 1 --2- 172.21.15.46:0/2434559674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95e00b2720 0x7f95e01539c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f95d402c430 tx=0x7f95d402e070 comp rx=0 tx=0).stop 2023-12-02T00:29:35.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.243+0000 7f95e6116640 1 -- 172.21.15.46:0/2434559674 shutdown_connections 2023-12-02T00:29:35.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.243+0000 7f95e6116640 1 --2- 172.21.15.46:0/2434559674 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f95cc06ae70 0x7f95cc06d330 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:35.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.243+0000 7f95e6116640 1 --2- 172.21.15.46:0/2434559674 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95e006c630 0x7f95e006cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:35.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.243+0000 7f95e6116640 1 --2- 172.21.15.46:0/2434559674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95e00b2720 0x7f95e01539c0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:35.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.243+0000 7f95e6116640 1 --2- 172.21.15.46:0/2434559674 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95c8004370 0x7f95e00b0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:35.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.243+0000 7f95e6116640 1 -- 172.21.15.46:0/2434559674 >> 172.21.15.46:0/2434559674 conn(0x7f95e006a6b0 msgr2=0x7f95e006aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:35.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.243+0000 7f95e6116640 1 -- 172.21.15.46:0/2434559674 shutdown_connections 2023-12-02T00:29:35.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:33.243+0000 7f95e6116640 1 -- 172.21.15.46:0/2434559674 wait complete. 2023-12-02T00:29:35.183 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-02T00:29:35.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.304+0000 7f6926925640 1 Processor -- start 2023-12-02T00:29:35.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.304+0000 7f6926925640 1 -- start start 2023-12-02T00:29:35.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.304+0000 7f6926925640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f69200d2910 0x7f69200d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:35.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.304+0000 7f6926925640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f69200d3730 0x7f69200b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:35.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.304+0000 7f6926925640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69200b2b10 con 0x7f692015d330 2023-12-02T00:29:35.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.304+0000 7f6926925640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69200d3b50 con 0x7f69200d3730 2023-12-02T00:29:35.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.304+0000 7f6926925640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69200d1590 con 0x7f69200d2910 2023-12-02T00:29:35.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.305+0000 7f6924ec2640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f69200d3730 0x7f69200b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:35.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.305+0000 7f6924ec2640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f69200d3730 0x7f69200b6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57218/0 (socket says 172.21.15.46:57218) 2023-12-02T00:29:35.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.305+0000 7f6924ec2640 1 -- 172.21.15.46:0/4209370196 learned_addr learned my addr 172.21.15.46:0/4209370196 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:35.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.305+0000 7f691f7fe640 1 --2- 172.21.15.46:0/4209370196 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f69200d2910 0x7f69200d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:35.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.305+0000 7f6924ec2640 1 -- 172.21.15.46:0/4209370196 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f69200d2910 msgr2=0x7f69200d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:35.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.305+0000 7f6924ec2640 1 --2- 172.21.15.46:0/4209370196 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f69200d2910 0x7f69200d31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:35.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.305+0000 7f6924ec2640 1 -- 172.21.15.46:0/4209370196 >> v1:172.21.15.162:6789/0 conn(0x7f692015d330 legacy=0x7f69200b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:35.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.305+0000 7f6924ec2640 1 -- 172.21.15.46:0/4209370196 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69200d1870 con 0x7f69200d3730 2023-12-02T00:29:35.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.305+0000 7f691f7fe640 1 --2- 172.21.15.46:0/4209370196 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f69200d2910 0x7f69200d31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:35.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.306+0000 7f6924ec2640 1 --2- 172.21.15.46:0/4209370196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69200d3730 0x7f69200b6390 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6918009e50 tx=0x7f691802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86c0431b561e8f server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:35.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.306+0000 7f691effd640 1 -- 172.21.15.46:0/4209370196 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691802ca70 con 0x7f69200d3730 2023-12-02T00:29:35.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.306+0000 7f691effd640 1 -- 172.21.15.46:0/4209370196 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f691802cbd0 con 0x7f69200d3730 2023-12-02T00:29:35.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.306+0000 7f691effd640 1 -- 172.21.15.46:0/4209370196 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69180343f0 con 0x7f69200d3730 2023-12-02T00:29:35.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.306+0000 7f6926925640 1 -- 172.21.15.46:0/4209370196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69200d3730 msgr2=0x7f69200b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:35.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.306+0000 7f6926925640 1 --2- 172.21.15.46:0/4209370196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69200d3730 0x7f69200b6390 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6918009e50 tx=0x7f691802bed0 comp rx=0 tx=0).stop 2023-12-02T00:29:35.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.307+0000 7f6926925640 1 -- 172.21.15.46:0/4209370196 shutdown_connections 2023-12-02T00:29:35.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.307+0000 7f6926925640 1 --2- 172.21.15.46:0/4209370196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69200d3730 0x7f69200b6390 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:35.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.307+0000 7f6926925640 1 --2- 172.21.15.46:0/4209370196 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f69200d2910 0x7f69200d31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:35.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.307+0000 7f6926925640 1 -- 172.21.15.46:0/4209370196 >> 172.21.15.46:0/4209370196 conn(0x7f6920072020 msgr2=0x7f6920072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:35.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.307+0000 7f6926925640 1 -- 172.21.15.46:0/4209370196 shutdown_connections 2023-12-02T00:29:35.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.307+0000 7f6926925640 1 -- 172.21.15.46:0/4209370196 wait complete. 2023-12-02T00:29:35.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.308+0000 7f6926925640 1 Processor -- start 2023-12-02T00:29:35.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.308+0000 7f6926925640 1 -- start start 2023-12-02T00:29:35.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.309+0000 7f6926925640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69200d2910 0x7f69200cc860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:35.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.309+0000 7f6926925640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f69200d3730 0x7f6920152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:35.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.309+0000 7f6926925640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f692015d330 0x7f69200cad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:35.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.309+0000 7f6926925640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69200cb590 con 0x7f692015d330 2023-12-02T00:29:35.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.309+0000 7f6926925640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69200c6c40 con 0x7f69200d2910 2023-12-02T00:29:35.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.309+0000 7f6926925640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69200c6f20 con 0x7f69200d3730 2023-12-02T00:29:35.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.309+0000 7f6924ec2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f692015d330 0x7f69200cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:35.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.309+0000 7f6924ec2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f692015d330 0x7f69200cad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57228/0 (socket says 172.21.15.46:57228) 2023-12-02T00:29:35.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.309+0000 7f6924ec2640 1 -- 172.21.15.46:0/3667046676 learned_addr learned my addr 172.21.15.46:0/3667046676 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:35.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.309+0000 7f691ffff640 1 --2- 172.21.15.46:0/3667046676 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69200d2910 0x7f69200cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:35.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.309+0000 7f691f7fe640 1 --2- 172.21.15.46:0/3667046676 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f69200d3730 0x7f6920152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:35.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.309+0000 7f6924ec2640 1 -- 172.21.15.46:0/3667046676 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f69200d3730 msgr2=0x7f6920152ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:35.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.309+0000 7f6924ec2640 1 --2- 172.21.15.46:0/3667046676 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f69200d3730 0x7f6920152ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:35.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.309+0000 7f6924ec2640 1 -- 172.21.15.46:0/3667046676 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69200d2910 msgr2=0x7f69200cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:35.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.309+0000 7f6924ec2640 1 --2- 172.21.15.46:0/3667046676 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69200d2910 0x7f69200cc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:35.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.309+0000 7f6924ec2640 1 -- 172.21.15.46:0/3667046676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6918009b30 con 0x7f692015d330 2023-12-02T00:29:35.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.310+0000 7f6924ec2640 1 --2- 172.21.15.46:0/3667046676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f692015d330 0x7f69200cad40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6918009e50 tx=0x7f6918004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:35.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.310+0000 7f691cff9640 1 -- 172.21.15.46:0/3667046676 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6918009510 con 0x7f692015d330 2023-12-02T00:29:35.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.310+0000 7f691cff9640 1 -- 172.21.15.46:0/3667046676 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f691803e450 con 0x7f692015d330 2023-12-02T00:29:35.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.310+0000 7f6926925640 1 -- 172.21.15.46:0/3667046676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69200c7200 con 0x7f692015d330 2023-12-02T00:29:35.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.310+0000 7f691cff9640 1 -- 172.21.15.46:0/3667046676 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691803e800 con 0x7f692015d330 2023-12-02T00:29:35.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.310+0000 7f6926925640 1 -- 172.21.15.46:0/3667046676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69200cb280 con 0x7f692015d330 2023-12-02T00:29:35.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.311+0000 7f6926925640 1 -- 172.21.15.46:0/3667046676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69200c7720 con 0x7f692015d330 2023-12-02T00:29:35.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.311+0000 7f691cff9640 1 -- 172.21.15.46:0/3667046676 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f691803e960 con 0x7f692015d330 2023-12-02T00:29:35.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.312+0000 7f691cff9640 1 --2- 172.21.15.46:0/3667046676 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f690c06ae40 0x7f690c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:35.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.312+0000 7f691cff9640 1 -- 172.21.15.46:0/3667046676 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f69180ac260 con 0x7f692015d330 2023-12-02T00:29:35.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.312+0000 7f691ffff640 1 --2- 172.21.15.46:0/3667046676 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f690c06ae40 0x7f690c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:35.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.313+0000 7f691ffff640 1 --2- 172.21.15.46:0/3667046676 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f690c06ae40 0x7f690c06d300 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6910000f60 tx=0x7f6910009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:35.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.316+0000 7f691cff9640 1 -- 172.21.15.46:0/3667046676 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f69200c7720 con 0x7f692015d330 2023-12-02T00:29:35.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.491+0000 7f6926925640 1 -- 172.21.15.46:0/3667046676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f69200c7720 con 0x7f692015d330 2023-12-02T00:29:35.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.491+0000 7f691cff9640 1 -- 172.21.15.46:0/3667046676 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f69200c7720 con 0x7f692015d330 2023-12-02T00:29:35.493 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-12-02T00:29:35.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.493+0000 7f6926925640 1 -- 172.21.15.46:0/3667046676 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f690c06ae40 msgr2=0x7f690c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:35.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.493+0000 7f6926925640 1 --2- 172.21.15.46:0/3667046676 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f690c06ae40 0x7f690c06d300 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6910000f60 tx=0x7f6910009040 comp rx=0 tx=0).stop 2023-12-02T00:29:35.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.493+0000 7f6926925640 1 -- 172.21.15.46:0/3667046676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f692015d330 msgr2=0x7f69200cad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:35.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.493+0000 7f6926925640 1 --2- 172.21.15.46:0/3667046676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f692015d330 0x7f69200cad40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6918009e50 tx=0x7f6918004a40 comp rx=0 tx=0).stop 2023-12-02T00:29:35.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.493+0000 7f6926925640 1 -- 172.21.15.46:0/3667046676 shutdown_connections 2023-12-02T00:29:35.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.493+0000 7f6926925640 1 --2- 172.21.15.46:0/3667046676 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f690c06ae40 0x7f690c06d300 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:35.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.493+0000 7f6926925640 1 --2- 172.21.15.46:0/3667046676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f692015d330 0x7f69200cad40 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:35.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.493+0000 7f6926925640 1 --2- 172.21.15.46:0/3667046676 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f69200d3730 0x7f6920152ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:35.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.493+0000 7f6926925640 1 --2- 172.21.15.46:0/3667046676 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69200d2910 0x7f69200cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:35.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.493+0000 7f6926925640 1 -- 172.21.15.46:0/3667046676 >> 172.21.15.46:0/3667046676 conn(0x7f6920072020 msgr2=0x7f69200b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:35.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.493+0000 7f6926925640 1 -- 172.21.15.46:0/3667046676 shutdown_connections 2023-12-02T00:29:35.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.493+0000 7f6926925640 1 -- 172.21.15.46:0/3667046676 wait complete. 2023-12-02T00:29:35.506 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-02T00:29:35.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.659+0000 7f581b9d5640 1 Processor -- start 2023-12-02T00:29:35.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.659+0000 7f581b9d5640 1 -- start start 2023-12-02T00:29:35.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.660+0000 7f581b9d5640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f581415d1d0 0x7f581406b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:35.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.660+0000 7f581b9d5640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f58140aeb80 0x7f581406ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:35.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.660+0000 7f581b9d5640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58140adae0 con 0x7f58140ad640 2023-12-02T00:29:35.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.660+0000 7f581b9d5640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58140addc0 con 0x7f58140aeb80 2023-12-02T00:29:35.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.660+0000 7f581b9d5640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58140ae0a0 con 0x7f581415d1d0 2023-12-02T00:29:35.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.660+0000 7f5819f72640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f58140aeb80 0x7f581406ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:35.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.660+0000 7f5818f70640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f58140ad640 0x7f5814160ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59300/0 (socket says 172.21.15.46:59300) 2023-12-02T00:29:35.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.660+0000 7f5818f70640 1 -- 172.21.15.46:0/1473612762 learned_addr learned my addr 172.21.15.46:0/1473612762 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:35.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.660+0000 7f5819771640 1 --2- 172.21.15.46:0/1473612762 >> v2:172.21.15.162:3300/0 conn(0x7f581415d1d0 0x7f581406b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:35.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.660+0000 7f5819f72640 1 -- 172.21.15.46:0/1473612762 >> v2:172.21.15.162:3300/0 conn(0x7f581415d1d0 msgr2=0x7f581406b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:35.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.660+0000 7f5819f72640 1 --2- 172.21.15.46:0/1473612762 >> v2:172.21.15.162:3300/0 conn(0x7f581415d1d0 0x7f581406b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:35.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.660+0000 7f5819f72640 1 -- 172.21.15.46:0/1473612762 >> v1:172.21.15.46:6789/0 conn(0x7f58140ad640 legacy=0x7f5814160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:35.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.660+0000 7f5819f72640 1 -- 172.21.15.46:0/1473612762 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58140ae380 con 0x7f58140aeb80 2023-12-02T00:29:35.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.660+0000 7f5813fff640 1 -- 172.21.15.46:0/1473612762 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2855285012 0 0) 0x7f58140adae0 con 0x7f58140ad640 2023-12-02T00:29:35.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.661+0000 7f5819f72640 1 --2- 172.21.15.46:0/1473612762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58140aeb80 0x7f581406ba20 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5814072310 tx=0x7f580402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5946a82b767c413f server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:35.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.661+0000 7f5813fff640 1 -- 172.21.15.46:0/1473612762 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580402cc80 con 0x7f58140aeb80 2023-12-02T00:29:35.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.661+0000 7f5813fff640 1 -- 172.21.15.46:0/1473612762 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f580402cde0 con 0x7f58140aeb80 2023-12-02T00:29:35.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.661+0000 7f5813fff640 1 -- 172.21.15.46:0/1473612762 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5804034660 con 0x7f58140aeb80 2023-12-02T00:29:35.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.662+0000 7f581b9d5640 1 -- 172.21.15.46:0/1473612762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58140aeb80 msgr2=0x7f581406ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:35.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.662+0000 7f581b9d5640 1 --2- 172.21.15.46:0/1473612762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58140aeb80 0x7f581406ba20 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5814072310 tx=0x7f580402bed0 comp rx=0 tx=0).stop 2023-12-02T00:29:35.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.662+0000 7f581b9d5640 1 -- 172.21.15.46:0/1473612762 shutdown_connections 2023-12-02T00:29:35.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.662+0000 7f581b9d5640 1 --2- 172.21.15.46:0/1473612762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58140aeb80 0x7f581406ba20 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:35.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.662+0000 7f581b9d5640 1 --2- 172.21.15.46:0/1473612762 >> v2:172.21.15.162:3300/0 conn(0x7f581415d1d0 0x7f581406b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:35.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.662+0000 7f581b9d5640 1 -- 172.21.15.46:0/1473612762 >> 172.21.15.46:0/1473612762 conn(0x7f5814070e70 msgr2=0x7f5814071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:35.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.662+0000 7f581b9d5640 1 -- 172.21.15.46:0/1473612762 shutdown_connections 2023-12-02T00:29:35.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.662+0000 7f581b9d5640 1 -- 172.21.15.46:0/1473612762 wait complete. 2023-12-02T00:29:35.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.663+0000 7f581b9d5640 1 Processor -- start 2023-12-02T00:29:35.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.663+0000 7f581b9d5640 1 -- start start 2023-12-02T00:29:35.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.664+0000 7f581b9d5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f58140ad640 0x7f5814143ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:35.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.664+0000 7f581b9d5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58140aeb80 0x7f58141f2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:35.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.664+0000 7f581b9d5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f581415d1d0 0x7f58140b07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:35.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.664+0000 7f581b9d5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581415f000 con 0x7f581415d1d0 2023-12-02T00:29:35.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.664+0000 7f581b9d5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581415f190 con 0x7f58140aeb80 2023-12-02T00:29:35.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.664+0000 7f581b9d5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5814140da0 con 0x7f58140ad640 2023-12-02T00:29:35.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.664+0000 7f5819f72640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f581415d1d0 0x7f58140b07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:35.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.664+0000 7f5819f72640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f581415d1d0 0x7f58140b07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57236/0 (socket says 172.21.15.46:57236) 2023-12-02T00:29:35.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.664+0000 7f5819f72640 1 -- 172.21.15.46:0/2814303469 learned_addr learned my addr 172.21.15.46:0/2814303469 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:35.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.664+0000 7f5819771640 1 --2- 172.21.15.46:0/2814303469 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f58140ad640 0x7f5814143ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:35.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.664+0000 7f5818f70640 1 --2- 172.21.15.46:0/2814303469 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58140aeb80 0x7f58141f2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:35.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.664+0000 7f5819f72640 1 -- 172.21.15.46:0/2814303469 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f58140ad640 msgr2=0x7f5814143ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:35.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.664+0000 7f5819f72640 1 --2- 172.21.15.46:0/2814303469 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f58140ad640 0x7f5814143ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:35.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.664+0000 7f5819f72640 1 -- 172.21.15.46:0/2814303469 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58140aeb80 msgr2=0x7f58141f2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:35.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.664+0000 7f5819f72640 1 --2- 172.21.15.46:0/2814303469 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58140aeb80 0x7f58141f2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:35.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.664+0000 7f5819f72640 1 -- 172.21.15.46:0/2814303469 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5804009a70 con 0x7f581415d1d0 2023-12-02T00:29:35.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.665+0000 7f5819f72640 1 --2- 172.21.15.46:0/2814303469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f581415d1d0 0x7f58140b07d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5804009e80 tx=0x7f5804004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:35.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.665+0000 7f5811ffb640 1 -- 172.21.15.46:0/2814303469 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5804040650 con 0x7f581415d1d0 2023-12-02T00:29:35.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.665+0000 7f5811ffb640 1 -- 172.21.15.46:0/2814303469 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58040407b0 con 0x7f581415d1d0 2023-12-02T00:29:35.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.665+0000 7f5811ffb640 1 -- 172.21.15.46:0/2814303469 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5804040b60 con 0x7f581415d1d0 2023-12-02T00:29:35.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.665+0000 7f581b9d5640 1 -- 172.21.15.46:0/2814303469 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58141f2a50 con 0x7f581415d1d0 2023-12-02T00:29:35.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.665+0000 7f581b9d5640 1 -- 172.21.15.46:0/2814303469 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58140cef10 con 0x7f581415d1d0 2023-12-02T00:29:35.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.666+0000 7f581b9d5640 1 -- 172.21.15.46:0/2814303469 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f581415f300 con 0x7f581415d1d0 2023-12-02T00:29:35.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.666+0000 7f5811ffb640 1 -- 172.21.15.46:0/2814303469 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5804040cc0 con 0x7f581415d1d0 2023-12-02T00:29:35.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.667+0000 7f5811ffb640 1 --2- 172.21.15.46:0/2814303469 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f580006ae40 0x7f580006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:35.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.667+0000 7f5811ffb640 1 -- 172.21.15.46:0/2814303469 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f58040ac4f0 con 0x7f581415d1d0 2023-12-02T00:29:35.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.667+0000 7f5819771640 1 --2- 172.21.15.46:0/2814303469 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f580006ae40 0x7f580006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:35.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.667+0000 7f5819771640 1 --2- 172.21.15.46:0/2814303469 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f580006ae40 0x7f580006d300 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f580c006fd0 tx=0x7f580c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:35.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.671+0000 7f5811ffb640 1 -- 172.21.15.46:0/2814303469 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5804077470 con 0x7f581415d1d0 2023-12-02T00:29:35.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.847+0000 7f581b9d5640 1 -- 172.21.15.46:0/2814303469 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5814069400 con 0x7f581415d1d0 2023-12-02T00:29:35.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.847+0000 7f5811ffb640 1 -- 172.21.15.46:0/2814303469 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5804009450 con 0x7f581415d1d0 2023-12-02T00:29:35.849 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-12-02T00:29:35.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.849+0000 7f581b9d5640 1 -- 172.21.15.46:0/2814303469 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f580006ae40 msgr2=0x7f580006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:35.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.849+0000 7f581b9d5640 1 --2- 172.21.15.46:0/2814303469 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f580006ae40 0x7f580006d300 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f580c006fd0 tx=0x7f580c008040 comp rx=0 tx=0).stop 2023-12-02T00:29:35.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.849+0000 7f581b9d5640 1 -- 172.21.15.46:0/2814303469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f581415d1d0 msgr2=0x7f58140b07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:35.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.849+0000 7f581b9d5640 1 --2- 172.21.15.46:0/2814303469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f581415d1d0 0x7f58140b07d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5804009e80 tx=0x7f5804004a40 comp rx=0 tx=0).stop 2023-12-02T00:29:35.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.850+0000 7f581b9d5640 1 -- 172.21.15.46:0/2814303469 shutdown_connections 2023-12-02T00:29:35.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.850+0000 7f581b9d5640 1 --2- 172.21.15.46:0/2814303469 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f580006ae40 0x7f580006d300 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:35.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.850+0000 7f581b9d5640 1 --2- 172.21.15.46:0/2814303469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f581415d1d0 0x7f58140b07d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:35.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.850+0000 7f581b9d5640 1 --2- 172.21.15.46:0/2814303469 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58140aeb80 0x7f58141f2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:35.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.850+0000 7f581b9d5640 1 --2- 172.21.15.46:0/2814303469 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f58140ad640 0x7f5814143ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:35.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.850+0000 7f581b9d5640 1 -- 172.21.15.46:0/2814303469 >> 172.21.15.46:0/2814303469 conn(0x7f5814070e70 msgr2=0x7f58140b45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:35.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.850+0000 7f581b9d5640 1 -- 172.21.15.46:0/2814303469 shutdown_connections 2023-12-02T00:29:35.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.850+0000 7f581b9d5640 1 -- 172.21.15.46:0/2814303469 wait complete. 2023-12-02T00:29:35.862 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-02T00:29:35.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.969+0000 7f88b38ee640 1 Processor -- start 2023-12-02T00:29:35.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.969+0000 7f88b38ee640 1 -- start start 2023-12-02T00:29:35.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.970+0000 7f88b38ee640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f88ac0ad640 0x7f88ac06d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:35.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.970+0000 7f88b38ee640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88ac0adae0 con 0x7f88ac15d1d0 2023-12-02T00:29:35.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.970+0000 7f88b38ee640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88ac0addc0 con 0x7f88ac06bbf0 2023-12-02T00:29:35.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.970+0000 7f88b38ee640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88ac0ae0a0 con 0x7f88ac0ad640 2023-12-02T00:29:35.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.970+0000 7f88b168a640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f88ac15d1d0 0x7f88ac06b4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59310/0 (socket says 172.21.15.46:59310) 2023-12-02T00:29:35.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.970+0000 7f88b168a640 1 -- 172.21.15.46:0/3214480840 learned_addr learned my addr 172.21.15.46:0/3214480840 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:35.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.970+0000 7f88a3fff640 1 -- 172.21.15.46:0/3214480840 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1663716702 0 0) 0x7f88ac0adae0 con 0x7f88ac15d1d0 2023-12-02T00:29:35.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.970+0000 7f88a3fff640 1 -- 172.21.15.46:0/3214480840 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8898003520 con 0x7f88ac15d1d0 2023-12-02T00:29:35.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.970+0000 7f88b0e89640 1 --2- 172.21.15.46:0/3214480840 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f88ac0ad640 0x7f88ac06d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:35.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.970+0000 7f88a3fff640 1 -- 172.21.15.46:0/3214480840 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2468222198 0 0) 0x7f8898003520 con 0x7f88ac15d1d0 2023-12-02T00:29:35.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.970+0000 7f88a3fff640 1 -- 172.21.15.46:0/3214480840 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f88ac0ad640 msgr2=0x7f88ac06d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:35.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.970+0000 7f88a3fff640 1 --2- 172.21.15.46:0/3214480840 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f88ac0ad640 0x7f88ac06d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:35.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.970+0000 7f88a3fff640 1 -- 172.21.15.46:0/3214480840 >> v1:172.21.15.162:6789/0 conn(0x7f88ac06bbf0 legacy=0x7f88ac06d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:35.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.970+0000 7f88a3fff640 1 -- 172.21.15.46:0/3214480840 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88ac0ae380 con 0x7f88ac15d1d0 2023-12-02T00:29:35.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.970+0000 7f88a3fff640 1 -- 172.21.15.46:0/3214480840 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7f88a8001670 con 0x7f88ac15d1d0 2023-12-02T00:29:35.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.970+0000 7f88a3fff640 1 -- 172.21.15.46:0/3214480840 >> v1:172.21.15.46:6789/0 conn(0x7f88ac15d1d0 legacy=0x7f88ac06b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:35.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.970+0000 7f88a3fff640 1 --2- 172.21.15.46:0/3214480840 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f88980039a0 0x7f8898003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:35.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.970+0000 7f88a3fff640 1 --2- 172.21.15.46:0/3214480840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88980047c0 0x7f8898004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:35.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.970+0000 7f88a3fff640 1 --2- 172.21.15.46:0/3214480840 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f88ac0ad640 0x7f8898005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:35.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.970+0000 7f88a3fff640 1 -- 172.21.15.46:0/3214480840 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ac0ae0a0 con 0x7f88980047c0 2023-12-02T00:29:35.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.971+0000 7f88a3fff640 1 -- 172.21.15.46:0/3214480840 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889800ba30 con 0x7f88980039a0 2023-12-02T00:29:35.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.971+0000 7f88a3fff640 1 -- 172.21.15.46:0/3214480840 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f889800bbf0 con 0x7f88ac0ad640 2023-12-02T00:29:35.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.971+0000 7f88b0e89640 1 --2- 172.21.15.46:0/3214480840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88980047c0 0x7f8898004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:35.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.971+0000 7f88b168a640 1 --2- 172.21.15.46:0/3214480840 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f88980039a0 0x7f8898003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:35.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.971+0000 7f88b1e8b640 1 --2- 172.21.15.46:0/3214480840 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f88ac0ad640 0x7f8898005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:35.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.971+0000 7f88b0e89640 1 -- 172.21.15.46:0/3214480840 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f88ac0ad640 msgr2=0x7f8898005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:35.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.971+0000 7f88b0e89640 1 --2- 172.21.15.46:0/3214480840 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f88ac0ad640 0x7f8898005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:35.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.971+0000 7f88b0e89640 1 -- 172.21.15.46:0/3214480840 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f88980039a0 msgr2=0x7f8898003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:35.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.971+0000 7f88b0e89640 1 --2- 172.21.15.46:0/3214480840 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f88980039a0 0x7f8898003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:35.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.971+0000 7f88b0e89640 1 -- 172.21.15.46:0/3214480840 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8898003520 con 0x7f88980047c0 2023-12-02T00:29:35.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.972+0000 7f88b0e89640 1 --2- 172.21.15.46:0/3214480840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88980047c0 0x7f8898004c40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f889c009a90 tx=0x7f88980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=635c9ca69cf029cb server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:35.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.972+0000 7f88a3fff640 1 -- 172.21.15.46:0/3214480840 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889c02c680 con 0x7f88980047c0 2023-12-02T00:29:35.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.972+0000 7f88a3fff640 1 -- 172.21.15.46:0/3214480840 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f889c02c7e0 con 0x7f88980047c0 2023-12-02T00:29:35.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.972+0000 7f88a3fff640 1 -- 172.21.15.46:0/3214480840 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889c02cb70 con 0x7f88980047c0 2023-12-02T00:29:35.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.972+0000 7f88b38ee640 1 -- 172.21.15.46:0/3214480840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88980047c0 msgr2=0x7f8898004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:35.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.972+0000 7f88b38ee640 1 --2- 172.21.15.46:0/3214480840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88980047c0 0x7f8898004c40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f889c009a90 tx=0x7f88980057f0 comp rx=0 tx=0).stop 2023-12-02T00:29:35.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.972+0000 7f88b1e8b640 1 -- 172.21.15.46:0/3214480840 reap_dead start 2023-12-02T00:29:35.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.972+0000 7f88b38ee640 1 -- 172.21.15.46:0/3214480840 shutdown_connections 2023-12-02T00:29:35.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.972+0000 7f88b38ee640 1 -- 172.21.15.46:0/3214480840 >> 172.21.15.46:0/3214480840 conn(0x7f88ac070e70 msgr2=0x7f88ac071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:35.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.972+0000 7f88b38ee640 1 -- 172.21.15.46:0/3214480840 shutdown_connections 2023-12-02T00:29:35.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.972+0000 7f88b38ee640 1 -- 172.21.15.46:0/3214480840 wait complete. 2023-12-02T00:29:35.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.974+0000 7f88b38ee640 1 Processor -- start 2023-12-02T00:29:35.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.974+0000 7f88b38ee640 1 -- start start 2023-12-02T00:29:35.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.974+0000 7f88b38ee640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f88980047c0 0x7f88ac154110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:35.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.974+0000 7f88b38ee640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88ac15c6f0 0x7f88ac15ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:35.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.974+0000 7f88b38ee640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f88ac15fd80 0x7f88ac159390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:35.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.974+0000 7f88b38ee640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ac15b3b0 con 0x7f88ac15c6f0 2023-12-02T00:29:35.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.974+0000 7f88b38ee640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ac15b650 con 0x7f88ac15fd80 2023-12-02T00:29:35.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.974+0000 7f88b38ee640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88ac15b930 con 0x7f88980047c0 2023-12-02T00:29:35.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.974+0000 7f88b0e89640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88ac15c6f0 0x7f88ac15ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:35.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.974+0000 7f88b0e89640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88ac15c6f0 0x7f88ac15ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57244/0 (socket says 172.21.15.46:57244) 2023-12-02T00:29:35.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.974+0000 7f88b0e89640 1 -- 172.21.15.46:0/3143660990 learned_addr learned my addr 172.21.15.46:0/3143660990 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:35.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.975+0000 7f88b168a640 1 --2- 172.21.15.46:0/3143660990 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f88980047c0 0x7f88ac154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:35.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.975+0000 7f88b1e8b640 1 --2- 172.21.15.46:0/3143660990 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f88ac15fd80 0x7f88ac159390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:35.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.975+0000 7f88b0e89640 1 -- 172.21.15.46:0/3143660990 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f88980047c0 msgr2=0x7f88ac154110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:35.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.975+0000 7f88b0e89640 1 --2- 172.21.15.46:0/3143660990 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f88980047c0 0x7f88ac154110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:35.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.975+0000 7f88b0e89640 1 -- 172.21.15.46:0/3143660990 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f88ac15fd80 msgr2=0x7f88ac159390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:35.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.975+0000 7f88b0e89640 1 --2- 172.21.15.46:0/3143660990 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f88ac15fd80 0x7f88ac159390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:35.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.975+0000 7f88b0e89640 1 -- 172.21.15.46:0/3143660990 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88ac15bc10 con 0x7f88ac15c6f0 2023-12-02T00:29:35.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.975+0000 7f88b0e89640 1 --2- 172.21.15.46:0/3143660990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88ac15c6f0 0x7f88ac15ae70 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f889c02c220 tx=0x7f889c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:35.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.975+0000 7f88a1ffb640 1 -- 172.21.15.46:0/3143660990 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889c003810 con 0x7f88ac15c6f0 2023-12-02T00:29:35.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.975+0000 7f88a1ffb640 1 -- 172.21.15.46:0/3143660990 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f889c003970 con 0x7f88ac15c6f0 2023-12-02T00:29:35.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.975+0000 7f88b38ee640 1 -- 172.21.15.46:0/3143660990 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88ac15bef0 con 0x7f88ac15c6f0 2023-12-02T00:29:35.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.975+0000 7f88a1ffb640 1 -- 172.21.15.46:0/3143660990 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889c003ce0 con 0x7f88ac15c6f0 2023-12-02T00:29:35.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.975+0000 7f88b38ee640 1 -- 172.21.15.46:0/3143660990 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88ac160200 con 0x7f88ac15c6f0 2023-12-02T00:29:35.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.976+0000 7f88b38ee640 1 -- 172.21.15.46:0/3143660990 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88ac069400 con 0x7f88ac15c6f0 2023-12-02T00:29:35.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.977+0000 7f88a1ffb640 1 -- 172.21.15.46:0/3143660990 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f889c034440 con 0x7f88ac15c6f0 2023-12-02T00:29:35.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.977+0000 7f88a1ffb640 1 --2- 172.21.15.46:0/3143660990 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f889006ae70 0x7f889006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:35.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.977+0000 7f88a1ffb640 1 -- 172.21.15.46:0/3143660990 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f889c0ab6a0 con 0x7f88ac15c6f0 2023-12-02T00:29:35.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.978+0000 7f88b168a640 1 --2- 172.21.15.46:0/3143660990 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f889006ae70 0x7f889006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:35.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.978+0000 7f88b168a640 1 --2- 172.21.15.46:0/3143660990 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f889006ae70 0x7f889006d330 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f88a8007da0 tx=0x7f88a8001bd0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:35.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:35.982+0000 7f88a1ffb640 1 -- 172.21.15.46:0/3143660990 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f889c076760 con 0x7f88ac15c6f0 2023-12-02T00:29:36.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.171+0000 7f88b38ee640 1 -- 172.21.15.46:0/3143660990 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f88ac153760 con 0x7f88ac15c6f0 2023-12-02T00:29:36.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.171+0000 7f88a1ffb640 1 -- 172.21.15.46:0/3143660990 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7f889c0044e0 con 0x7f88ac15c6f0 2023-12-02T00:29:36.173 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2023-12-02T00:29:36.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.173+0000 7f88b38ee640 1 -- 172.21.15.46:0/3143660990 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f889006ae70 msgr2=0x7f889006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:36.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.173+0000 7f88b38ee640 1 --2- 172.21.15.46:0/3143660990 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f889006ae70 0x7f889006d330 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f88a8007da0 tx=0x7f88a8001bd0 comp rx=0 tx=0).stop 2023-12-02T00:29:36.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.173+0000 7f88b38ee640 1 -- 172.21.15.46:0/3143660990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88ac15c6f0 msgr2=0x7f88ac15ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:36.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.173+0000 7f88b38ee640 1 --2- 172.21.15.46:0/3143660990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88ac15c6f0 0x7f88ac15ae70 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f889c02c220 tx=0x7f889c004730 comp rx=0 tx=0).stop 2023-12-02T00:29:36.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.173+0000 7f88b38ee640 1 -- 172.21.15.46:0/3143660990 shutdown_connections 2023-12-02T00:29:36.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.173+0000 7f88b38ee640 1 --2- 172.21.15.46:0/3143660990 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f889006ae70 0x7f889006d330 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:36.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.173+0000 7f88b38ee640 1 --2- 172.21.15.46:0/3143660990 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f88ac15fd80 0x7f88ac159390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:36.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.173+0000 7f88b38ee640 1 --2- 172.21.15.46:0/3143660990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88ac15c6f0 0x7f88ac15ae70 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:36.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.173+0000 7f88b38ee640 1 --2- 172.21.15.46:0/3143660990 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f88980047c0 0x7f88ac154110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:36.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.173+0000 7f88b38ee640 1 -- 172.21.15.46:0/3143660990 >> 172.21.15.46:0/3143660990 conn(0x7f88ac070e70 msgr2=0x7f88ac0b0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:36.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.173+0000 7f88b38ee640 1 -- 172.21.15.46:0/3143660990 shutdown_connections 2023-12-02T00:29:36.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.173+0000 7f88b38ee640 1 -- 172.21.15.46:0/3143660990 wait complete. 2023-12-02T00:29:36.185 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-12-02T00:29:36.186 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-02T00:29:36.208 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:29:36.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.315+0000 7f031ba9b640 1 Processor -- start 2023-12-02T00:29:36.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.315+0000 7f031ba9b640 1 -- start start 2023-12-02T00:29:36.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.316+0000 7f031ba9b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0314144980 0x7f031415c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:36.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.316+0000 7f031ba9b640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f03140ceb70 0x7f03140cc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:36.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.316+0000 7f031ba9b640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0314071770 con 0x7f03140b4370 2023-12-02T00:29:36.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.316+0000 7f031ba9b640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03140ccdc0 con 0x7f03140ceb70 2023-12-02T00:29:36.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.316+0000 7f031ba9b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03140cd0a0 con 0x7f0314144980 2023-12-02T00:29:36.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.316+0000 7f031a038640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f03140b4370 0x7f0314071c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59326/0 (socket says 172.21.15.46:59326) 2023-12-02T00:29:36.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.316+0000 7f031a038640 1 -- 172.21.15.46:0/3838340201 learned_addr learned my addr 172.21.15.46:0/3838340201 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:36.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.316+0000 7f0319036640 1 --2- 172.21.15.46:0/3838340201 >> v2:172.21.15.162:3300/0 conn(0x7f03140ceb70 0x7f03140cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:36.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.316+0000 7f0319837640 1 --2- 172.21.15.46:0/3838340201 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0314144980 0x7f031415c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:36.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.316+0000 7f0318835640 1 -- 172.21.15.46:0/3838340201 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2226919400 0 0) 0x7f0314071770 con 0x7f03140b4370 2023-12-02T00:29:36.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.316+0000 7f0318835640 1 -- 172.21.15.46:0/3838340201 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0300003520 con 0x7f03140b4370 2023-12-02T00:29:36.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.316+0000 7f0318835640 1 -- 172.21.15.46:0/3838340201 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4200071647 0 0) 0x7f0300003520 con 0x7f03140b4370 2023-12-02T00:29:36.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.316+0000 7f0318835640 1 -- 172.21.15.46:0/3838340201 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0314144980 msgr2=0x7f031415c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:36.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.316+0000 7f0318835640 1 --2- 172.21.15.46:0/3838340201 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0314144980 0x7f031415c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:36.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.316+0000 7f0318835640 1 -- 172.21.15.46:0/3838340201 >> v2:172.21.15.162:3300/0 conn(0x7f03140ceb70 msgr2=0x7f03140cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:36.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.316+0000 7f0318835640 1 --2- 172.21.15.46:0/3838340201 >> v2:172.21.15.162:3300/0 conn(0x7f03140ceb70 0x7f03140cc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:36.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.316+0000 7f0318835640 1 -- 172.21.15.46:0/3838340201 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03140cd380 con 0x7f03140b4370 2023-12-02T00:29:36.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.316+0000 7f0318835640 1 -- 172.21.15.46:0/3838340201 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7f030c0019f0 con 0x7f03140b4370 2023-12-02T00:29:36.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.316+0000 7f0318835640 1 -- 172.21.15.46:0/3838340201 >> v1:172.21.15.46:6789/0 conn(0x7f03140b4370 legacy=0x7f0314071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:36.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.316+0000 7f0318835640 1 --2- 172.21.15.46:0/3838340201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0300003970 0x7f0300003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:36.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.316+0000 7f0318835640 1 --2- 172.21.15.46:0/3838340201 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03000047c0 0x7f0300004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:36.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.316+0000 7f0318835640 1 --2- 172.21.15.46:0/3838340201 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0314144980 0x7f0300005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:36.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.316+0000 7f0318835640 1 -- 172.21.15.46:0/3838340201 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03140ccdc0 con 0x7f0300003970 2023-12-02T00:29:36.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.316+0000 7f0318835640 1 -- 172.21.15.46:0/3838340201 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03140cd0a0 con 0x7f03000047c0 2023-12-02T00:29:36.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.317+0000 7f0318835640 1 -- 172.21.15.46:0/3838340201 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f030000b980 con 0x7f0314144980 2023-12-02T00:29:36.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.317+0000 7f0319837640 1 --2- 172.21.15.46:0/3838340201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0300003970 0x7f0300003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:36.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.317+0000 7f0319036640 1 --2- 172.21.15.46:0/3838340201 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03000047c0 0x7f0300004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:36.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.317+0000 7f031a038640 1 --2- 172.21.15.46:0/3838340201 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0314144980 0x7f0300005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:36.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.317+0000 7f0319837640 1 -- 172.21.15.46:0/3838340201 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0314144980 msgr2=0x7f0300005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:36.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.317+0000 7f0319837640 1 --2- 172.21.15.46:0/3838340201 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0314144980 0x7f0300005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:36.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.317+0000 7f0319837640 1 -- 172.21.15.46:0/3838340201 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03000047c0 msgr2=0x7f0300004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:36.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.317+0000 7f0319837640 1 --2- 172.21.15.46:0/3838340201 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03000047c0 0x7f0300004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:36.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.317+0000 7f0319837640 1 -- 172.21.15.46:0/3838340201 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0300003520 con 0x7f0300003970 2023-12-02T00:29:36.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.318+0000 7f0319837640 1 --2- 172.21.15.46:0/3838340201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0300003970 0x7f0300003d90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0310009ac0 tx=0x7f03000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43653f824ac52d4e server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:36.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.318+0000 7f0318835640 1 -- 172.21.15.46:0/3838340201 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031002c6b0 con 0x7f0300003970 2023-12-02T00:29:36.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.318+0000 7f0318835640 1 -- 172.21.15.46:0/3838340201 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f031002c810 con 0x7f0300003970 2023-12-02T00:29:36.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.318+0000 7f0318835640 1 -- 172.21.15.46:0/3838340201 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031002cba0 con 0x7f0300003970 2023-12-02T00:29:36.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.318+0000 7f031ba9b640 1 -- 172.21.15.46:0/3838340201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0300003970 msgr2=0x7f0300003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:36.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.318+0000 7f031ba9b640 1 --2- 172.21.15.46:0/3838340201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0300003970 0x7f0300003d90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0310009ac0 tx=0x7f03000057f0 comp rx=0 tx=0).stop 2023-12-02T00:29:36.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.318+0000 7f031a038640 1 -- 172.21.15.46:0/3838340201 reap_dead start 2023-12-02T00:29:36.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.318+0000 7f031ba9b640 1 -- 172.21.15.46:0/3838340201 shutdown_connections 2023-12-02T00:29:36.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.318+0000 7f031ba9b640 1 -- 172.21.15.46:0/3838340201 >> 172.21.15.46:0/3838340201 conn(0x7f031406a4d0 msgr2=0x7f031406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:36.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.318+0000 7f031ba9b640 1 -- 172.21.15.46:0/3838340201 shutdown_connections 2023-12-02T00:29:36.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.319+0000 7f031ba9b640 1 -- 172.21.15.46:0/3838340201 wait complete. 2023-12-02T00:29:36.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.320+0000 7f031ba9b640 1 Processor -- start 2023-12-02T00:29:36.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.320+0000 7f031ba9b640 1 -- start start 2023-12-02T00:29:36.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.320+0000 7f031ba9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0300003970 0x7f03140c7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:36.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.320+0000 7f031ba9b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0314155510 0x7f03140c6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:36.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.320+0000 7f031ba9b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0314132ff0 0x7f0314133470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:36.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.320+0000 7f031ba9b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03140c86a0 con 0x7f0300003970 2023-12-02T00:29:36.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.320+0000 7f031ba9b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03140c7040 con 0x7f0314155510 2023-12-02T00:29:36.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.320+0000 7f031ba9b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03141339b0 con 0x7f0314132ff0 2023-12-02T00:29:36.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.320+0000 7f0319837640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0300003970 0x7f03140c7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:36.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.321+0000 7f0319837640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0300003970 0x7f03140c7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57260/0 (socket says 172.21.15.46:57260) 2023-12-02T00:29:36.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.321+0000 7f0319837640 1 -- 172.21.15.46:0/388841535 learned_addr learned my addr 172.21.15.46:0/388841535 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:36.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.321+0000 7f031a038640 1 --2- 172.21.15.46:0/388841535 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0314132ff0 0x7f0314133470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:36.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.321+0000 7f0319036640 1 --2- 172.21.15.46:0/388841535 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0314155510 0x7f03140c6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:36.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.321+0000 7f0319837640 1 -- 172.21.15.46:0/388841535 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0314132ff0 msgr2=0x7f0314133470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:36.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.321+0000 7f0319837640 1 --2- 172.21.15.46:0/388841535 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0314132ff0 0x7f0314133470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:36.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.321+0000 7f0319837640 1 -- 172.21.15.46:0/388841535 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0314155510 msgr2=0x7f03140c6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:36.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.321+0000 7f0319837640 1 --2- 172.21.15.46:0/388841535 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0314155510 0x7f03140c6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:36.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.321+0000 7f0319837640 1 -- 172.21.15.46:0/388841535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0314155990 con 0x7f0300003970 2023-12-02T00:29:36.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.321+0000 7f0319837640 1 --2- 172.21.15.46:0/388841535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0300003970 0x7f03140c7420 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0310009ac0 tx=0x7f0310004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:36.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.321+0000 7f030a7fc640 1 -- 172.21.15.46:0/388841535 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0310003810 con 0x7f0300003970 2023-12-02T00:29:36.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.321+0000 7f030a7fc640 1 -- 172.21.15.46:0/388841535 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0310003970 con 0x7f0300003970 2023-12-02T00:29:36.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.321+0000 7f031ba9b640 1 -- 172.21.15.46:0/388841535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03140c7a90 con 0x7f0300003970 2023-12-02T00:29:36.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.321+0000 7f030a7fc640 1 -- 172.21.15.46:0/388841535 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0310003d00 con 0x7f0300003970 2023-12-02T00:29:36.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.321+0000 7f031ba9b640 1 -- 172.21.15.46:0/388841535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03140c8050 con 0x7f0300003970 2023-12-02T00:29:36.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.322+0000 7f031ba9b640 1 -- 172.21.15.46:0/388841535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0314069400 con 0x7f0300003970 2023-12-02T00:29:36.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.323+0000 7f030a7fc640 1 -- 172.21.15.46:0/388841535 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0310040450 con 0x7f0300003970 2023-12-02T00:29:36.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.323+0000 7f030a7fc640 1 --2- 172.21.15.46:0/388841535 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f02f806ae40 0x7f02f806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:36.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.323+0000 7f030a7fc640 1 -- 172.21.15.46:0/388841535 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f03100ab480 con 0x7f0300003970 2023-12-02T00:29:36.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.323+0000 7f0319036640 1 --2- 172.21.15.46:0/388841535 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f02f806ae40 0x7f02f806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:36.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.324+0000 7f0319036640 1 --2- 172.21.15.46:0/388841535 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f02f806ae40 0x7f02f806d300 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f03140a86e0 tx=0x7f0304009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:36.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.327+0000 7f030a7fc640 1 -- 172.21.15.46:0/388841535 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f03100765e0 con 0x7f0300003970 2023-12-02T00:29:36.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.505+0000 7f031ba9b640 1 -- 172.21.15.46:0/388841535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03140c7200 con 0x7f0300003970 2023-12-02T00:29:36.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.507+0000 7f030a7fc640 1 -- 172.21.15.46:0/388841535 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16777 (secure 0 0 0) 0x7f03100044e0 con 0x7f0300003970 2023-12-02T00:29:36.508 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:29:36.508 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":23,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:29:30.868939+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-02T00:28:47.273337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-02T00:28:48.060115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.14","mappings":[{"from":4,"to":2}]},{"pgid":"3.1e","mappings":[{"from":3,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:29:36.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.508+0000 7f031ba9b640 1 -- 172.21.15.46:0/388841535 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f02f806ae40 msgr2=0x7f02f806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:36.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.508+0000 7f031ba9b640 1 --2- 172.21.15.46:0/388841535 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f02f806ae40 0x7f02f806d300 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f03140a86e0 tx=0x7f0304009190 comp rx=0 tx=0).stop 2023-12-02T00:29:36.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.508+0000 7f031ba9b640 1 -- 172.21.15.46:0/388841535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0300003970 msgr2=0x7f03140c7420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:36.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.508+0000 7f031ba9b640 1 --2- 172.21.15.46:0/388841535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0300003970 0x7f03140c7420 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0310009ac0 tx=0x7f0310004730 comp rx=0 tx=0).stop 2023-12-02T00:29:36.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.508+0000 7f031ba9b640 1 -- 172.21.15.46:0/388841535 shutdown_connections 2023-12-02T00:29:36.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.508+0000 7f031ba9b640 1 --2- 172.21.15.46:0/388841535 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f02f806ae40 0x7f02f806d300 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:36.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.508+0000 7f031ba9b640 1 --2- 172.21.15.46:0/388841535 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0314132ff0 0x7f0314133470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:36.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.508+0000 7f031ba9b640 1 --2- 172.21.15.46:0/388841535 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0314155510 0x7f03140c6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:36.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.508+0000 7f031ba9b640 1 --2- 172.21.15.46:0/388841535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0300003970 0x7f03140c7420 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:36.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.508+0000 7f031ba9b640 1 -- 172.21.15.46:0/388841535 >> 172.21.15.46:0/388841535 conn(0x7f031406a4d0 msgr2=0x7f0314157720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:36.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.509+0000 7f031ba9b640 1 -- 172.21.15.46:0/388841535 shutdown_connections 2023-12-02T00:29:36.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.509+0000 7f031ba9b640 1 -- 172.21.15.46:0/388841535 wait complete. 2023-12-02T00:29:36.521 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-02T00:28:37.213645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-02T00:28:47.273337+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-02T00:28:48.060115+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-02T00:29:36.521 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-02T00:29:36.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.630+0000 7faa9a610640 1 Processor -- start 2023-12-02T00:29:36.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.630+0000 7faa9a610640 1 -- start start 2023-12-02T00:29:36.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.631+0000 7faa9a610640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faa9406abe0 0x7faa941618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:36.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.631+0000 7faa9a610640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7faa941409d0 0x7faa9406efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:36.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.631+0000 7faa9a610640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa941406f0 con 0x7faa94140310 2023-12-02T00:29:36.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.631+0000 7faa9a610640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa94161e20 con 0x7faa941409d0 2023-12-02T00:29:36.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.631+0000 7faa9a610640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa94143960 con 0x7faa9406abe0 2023-12-02T00:29:36.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.631+0000 7faa93fff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7faa94140310 0x7faa9406a4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59336/0 (socket says 172.21.15.46:59336) 2023-12-02T00:29:36.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.631+0000 7faa93fff640 1 -- 172.21.15.46:0/976086307 learned_addr learned my addr 172.21.15.46:0/976086307 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:36.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.631+0000 7faa98bad640 1 --2- 172.21.15.46:0/976086307 >> v2:172.21.15.162:3300/0 conn(0x7faa941409d0 0x7faa9406efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:36.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.631+0000 7faa937fe640 1 --2- 172.21.15.46:0/976086307 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faa9406abe0 0x7faa941618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:36.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.631+0000 7faa92ffd640 1 -- 172.21.15.46:0/976086307 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1852364184 0 0) 0x7faa941406f0 con 0x7faa94140310 2023-12-02T00:29:36.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.632+0000 7faa92ffd640 1 -- 172.21.15.46:0/976086307 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa7c003520 con 0x7faa94140310 2023-12-02T00:29:36.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.632+0000 7faa92ffd640 1 -- 172.21.15.46:0/976086307 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 298515786 0 0) 0x7faa7c003520 con 0x7faa94140310 2023-12-02T00:29:36.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.632+0000 7faa92ffd640 1 -- 172.21.15.46:0/976086307 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faa9406abe0 msgr2=0x7faa941618e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:36.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.632+0000 7faa92ffd640 1 --2- 172.21.15.46:0/976086307 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faa9406abe0 0x7faa941618e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:36.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.632+0000 7faa92ffd640 1 -- 172.21.15.46:0/976086307 >> v2:172.21.15.162:3300/0 conn(0x7faa941409d0 msgr2=0x7faa9406efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:36.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.632+0000 7faa92ffd640 1 --2- 172.21.15.46:0/976086307 >> v2:172.21.15.162:3300/0 conn(0x7faa941409d0 0x7faa9406efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:36.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.632+0000 7faa92ffd640 1 -- 172.21.15.46:0/976086307 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa94142de0 con 0x7faa94140310 2023-12-02T00:29:36.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.632+0000 7faa92ffd640 1 -- 172.21.15.46:0/976086307 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7faa88001670 con 0x7faa94140310 2023-12-02T00:29:36.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.632+0000 7faa98bad640 1 --2- 172.21.15.46:0/976086307 >> v2:172.21.15.162:3300/0 conn(0x7faa941409d0 0x7faa9406efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:29:36.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.632+0000 7faa92ffd640 1 -- 172.21.15.46:0/976086307 >> v1:172.21.15.46:6789/0 conn(0x7faa94140310 legacy=0x7faa9406a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:36.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.632+0000 7faa92ffd640 1 --2- 172.21.15.46:0/976086307 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faa9406abe0 0x7faa7c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:36.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.632+0000 7faa92ffd640 1 --2- 172.21.15.46:0/976086307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa7c003ef0 0x7faa7c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:36.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.632+0000 7faa92ffd640 1 --2- 172.21.15.46:0/976086307 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa7c004d00 0x7faa7c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:36.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.632+0000 7faa92ffd640 1 -- 172.21.15.46:0/976086307 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa94161e20 con 0x7faa7c003ef0 2023-12-02T00:29:36.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.632+0000 7faa92ffd640 1 -- 172.21.15.46:0/976086307 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa94143960 con 0x7faa7c004d00 2023-12-02T00:29:36.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.632+0000 7faa92ffd640 1 -- 172.21.15.46:0/976086307 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa7c00b9d0 con 0x7faa9406abe0 2023-12-02T00:29:36.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.632+0000 7faa937fe640 1 --2- 172.21.15.46:0/976086307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa7c003ef0 0x7faa7c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:36.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.632+0000 7faa93fff640 1 --2- 172.21.15.46:0/976086307 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faa9406abe0 0x7faa7c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:36.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.632+0000 7faa98bad640 1 --2- 172.21.15.46:0/976086307 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa7c004d00 0x7faa7c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:36.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.632+0000 7faa937fe640 1 -- 172.21.15.46:0/976086307 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faa9406abe0 msgr2=0x7faa7c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:36.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.632+0000 7faa937fe640 1 --2- 172.21.15.46:0/976086307 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faa9406abe0 0x7faa7c003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:36.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.632+0000 7faa937fe640 1 -- 172.21.15.46:0/976086307 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa7c004d00 msgr2=0x7faa7c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:36.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.632+0000 7faa937fe640 1 --2- 172.21.15.46:0/976086307 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa7c004d00 0x7faa7c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:36.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.632+0000 7faa937fe640 1 -- 172.21.15.46:0/976086307 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa7c003520 con 0x7faa7c003ef0 2023-12-02T00:29:36.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.633+0000 7faa937fe640 1 --2- 172.21.15.46:0/976086307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa7c003ef0 0x7faa7c0047c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7faa84009af0 tx=0x7faa7c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60b95a6fcedcde30 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:36.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.633+0000 7faa92ffd640 1 -- 172.21.15.46:0/976086307 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8402c6b0 con 0x7faa7c003ef0 2023-12-02T00:29:36.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.633+0000 7faa92ffd640 1 -- 172.21.15.46:0/976086307 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa8402c810 con 0x7faa7c003ef0 2023-12-02T00:29:36.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.633+0000 7faa92ffd640 1 -- 172.21.15.46:0/976086307 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8402cba0 con 0x7faa7c003ef0 2023-12-02T00:29:36.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.634+0000 7faa9a610640 1 -- 172.21.15.46:0/976086307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa7c003ef0 msgr2=0x7faa7c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:36.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.634+0000 7faa9a610640 1 --2- 172.21.15.46:0/976086307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa7c003ef0 0x7faa7c0047c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7faa84009af0 tx=0x7faa7c0057f0 comp rx=0 tx=0).stop 2023-12-02T00:29:36.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.634+0000 7faa98bad640 1 -- 172.21.15.46:0/976086307 reap_dead start 2023-12-02T00:29:36.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.634+0000 7faa9a610640 1 -- 172.21.15.46:0/976086307 shutdown_connections 2023-12-02T00:29:36.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.634+0000 7faa9a610640 1 -- 172.21.15.46:0/976086307 >> 172.21.15.46:0/976086307 conn(0x7faa940b1e50 msgr2=0x7faa940b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:36.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.634+0000 7faa9a610640 1 -- 172.21.15.46:0/976086307 shutdown_connections 2023-12-02T00:29:36.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.634+0000 7faa9a610640 1 -- 172.21.15.46:0/976086307 wait complete. 2023-12-02T00:29:36.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.635+0000 7faa9a610640 1 Processor -- start 2023-12-02T00:29:36.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.635+0000 7faa9a610640 1 -- start start 2023-12-02T00:29:36.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.636+0000 7faa9a610640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa7c003ef0 0x7faa9413cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:36.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.636+0000 7faa9a610640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faa9415e860 0x7faa94146340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:36.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.636+0000 7faa9a610640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa94143d50 0x7faa9413e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:36.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.636+0000 7faa9a610640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa94146b20 con 0x7faa94143d50 2023-12-02T00:29:36.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.636+0000 7faa9a610640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa94160840 con 0x7faa7c003ef0 2023-12-02T00:29:36.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.636+0000 7faa9a610640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa94146880 con 0x7faa9415e860 2023-12-02T00:29:36.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.636+0000 7faa98bad640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa94143d50 0x7faa9413e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:36.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.636+0000 7faa98bad640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa94143d50 0x7faa9413e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57276/0 (socket says 172.21.15.46:57276) 2023-12-02T00:29:36.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.636+0000 7faa98bad640 1 -- 172.21.15.46:0/4203190690 learned_addr learned my addr 172.21.15.46:0/4203190690 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:36.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.636+0000 7faa93fff640 1 --2- 172.21.15.46:0/4203190690 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa7c003ef0 0x7faa9413cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:36.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.636+0000 7faa937fe640 1 --2- 172.21.15.46:0/4203190690 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faa9415e860 0x7faa94146340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:36.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.636+0000 7faa98bad640 1 -- 172.21.15.46:0/4203190690 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faa9415e860 msgr2=0x7faa94146340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:36.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.636+0000 7faa98bad640 1 --2- 172.21.15.46:0/4203190690 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faa9415e860 0x7faa94146340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:36.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.636+0000 7faa98bad640 1 -- 172.21.15.46:0/4203190690 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa7c003ef0 msgr2=0x7faa9413cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:36.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.636+0000 7faa98bad640 1 --2- 172.21.15.46:0/4203190690 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa7c003ef0 0x7faa9413cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:36.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.636+0000 7faa98bad640 1 -- 172.21.15.46:0/4203190690 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa9415daa0 con 0x7faa94143d50 2023-12-02T00:29:36.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.636+0000 7faa98bad640 1 --2- 172.21.15.46:0/4203190690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa94143d50 0x7faa9413e350 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7faa94147060 tx=0x7faa8c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:36.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.637+0000 7faa90ff9640 1 -- 172.21.15.46:0/4203190690 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8c00e040 con 0x7faa94143d50 2023-12-02T00:29:36.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.637+0000 7faa90ff9640 1 -- 172.21.15.46:0/4203190690 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa8c00f040 con 0x7faa94143d50 2023-12-02T00:29:36.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.637+0000 7faa9a610640 1 -- 172.21.15.46:0/4203190690 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa9415dd80 con 0x7faa94143d50 2023-12-02T00:29:36.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.637+0000 7faa90ff9640 1 -- 172.21.15.46:0/4203190690 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8c015670 con 0x7faa94143d50 2023-12-02T00:29:36.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.637+0000 7faa9a610640 1 -- 172.21.15.46:0/4203190690 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa9415e340 con 0x7faa94143d50 2023-12-02T00:29:36.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.638+0000 7faa9a610640 1 -- 172.21.15.46:0/4203190690 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa9415e530 con 0x7faa94143d50 2023-12-02T00:29:36.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.638+0000 7faa90ff9640 1 -- 172.21.15.46:0/4203190690 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faa8c0157d0 con 0x7faa94143d50 2023-12-02T00:29:36.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.639+0000 7faa90ff9640 1 --2- 172.21.15.46:0/4203190690 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7faa8006ae40 0x7faa8006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:36.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.639+0000 7faa90ff9640 1 -- 172.21.15.46:0/4203190690 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7faa8c089c70 con 0x7faa94143d50 2023-12-02T00:29:36.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.639+0000 7faa93fff640 1 --2- 172.21.15.46:0/4203190690 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7faa8006ae40 0x7faa8006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:36.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.639+0000 7faa93fff640 1 --2- 172.21.15.46:0/4203190690 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7faa8006ae40 0x7faa8006d300 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7faa88001eb0 tx=0x7faa88009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:36.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.643+0000 7faa90ff9640 1 -- 172.21.15.46:0/4203190690 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faa8c054d20 con 0x7faa94143d50 2023-12-02T00:29:36.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.819+0000 7faa9a610640 1 -- 172.21.15.46:0/4203190690 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faa940afd90 con 0x7faa94143d50 2023-12-02T00:29:36.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.819+0000 7faa90ff9640 1 -- 172.21.15.46:0/4203190690 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7faa940afd90 con 0x7faa94143d50 2023-12-02T00:29:36.820 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-12-02T00:29:36.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.821+0000 7faa9a610640 1 -- 172.21.15.46:0/4203190690 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7faa8006ae40 msgr2=0x7faa8006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:36.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.821+0000 7faa9a610640 1 --2- 172.21.15.46:0/4203190690 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7faa8006ae40 0x7faa8006d300 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7faa88001eb0 tx=0x7faa88009040 comp rx=0 tx=0).stop 2023-12-02T00:29:36.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.821+0000 7faa9a610640 1 -- 172.21.15.46:0/4203190690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa94143d50 msgr2=0x7faa9413e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:36.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.821+0000 7faa9a610640 1 --2- 172.21.15.46:0/4203190690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa94143d50 0x7faa9413e350 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7faa94147060 tx=0x7faa8c00f580 comp rx=0 tx=0).stop 2023-12-02T00:29:36.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.821+0000 7faa9a610640 1 -- 172.21.15.46:0/4203190690 shutdown_connections 2023-12-02T00:29:36.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.821+0000 7faa9a610640 1 --2- 172.21.15.46:0/4203190690 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7faa8006ae40 0x7faa8006d300 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:36.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.821+0000 7faa9a610640 1 --2- 172.21.15.46:0/4203190690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa94143d50 0x7faa9413e350 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:36.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.821+0000 7faa9a610640 1 --2- 172.21.15.46:0/4203190690 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faa9415e860 0x7faa94146340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:36.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.821+0000 7faa9a610640 1 --2- 172.21.15.46:0/4203190690 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa7c003ef0 0x7faa9413cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:36.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.821+0000 7faa9a610640 1 -- 172.21.15.46:0/4203190690 >> 172.21.15.46:0/4203190690 conn(0x7faa940b1e50 msgr2=0x7faa940734b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:36.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.822+0000 7faa9a610640 1 -- 172.21.15.46:0/4203190690 shutdown_connections 2023-12-02T00:29:36.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.822+0000 7faa9a610640 1 -- 172.21.15.46:0/4203190690 wait complete. 2023-12-02T00:29:36.834 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-02T00:29:36.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.942+0000 7f5f326da640 1 Processor -- start 2023-12-02T00:29:36.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.942+0000 7f5f326da640 1 -- start start 2023-12-02T00:29:36.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.943+0000 7f5f326da640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5f2c15d2d0 0x7f5f2c0b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:36.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.943+0000 7f5f326da640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5f2c0ce150 0x7f5f2c0b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:36.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.943+0000 7f5f326da640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f2c0d1bb0 con 0x7f5f2c0d2040 2023-12-02T00:29:36.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.943+0000 7f5f326da640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f2c15d6b0 con 0x7f5f2c0ce150 2023-12-02T00:29:36.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.943+0000 7f5f326da640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f2c0ce700 con 0x7f5f2c15d2d0 2023-12-02T00:29:36.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.943+0000 7f5f30c77640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5f2c0ce150 0x7f5f2c0b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:36.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.943+0000 7f5f2b7fe640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5f2c0d2040 0x7f5f2c0cda40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59342/0 (socket says 172.21.15.46:59342) 2023-12-02T00:29:36.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.943+0000 7f5f2b7fe640 1 -- 172.21.15.46:0/5154949 learned_addr learned my addr 172.21.15.46:0/5154949 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:36.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.943+0000 7f5f2bfff640 1 --2- 172.21.15.46:0/5154949 >> v2:172.21.15.162:3300/0 conn(0x7f5f2c15d2d0 0x7f5f2c0b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:36.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.943+0000 7f5f30c77640 1 -- 172.21.15.46:0/5154949 >> v2:172.21.15.162:3300/0 conn(0x7f5f2c15d2d0 msgr2=0x7f5f2c0b4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:36.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.943+0000 7f5f30c77640 1 --2- 172.21.15.46:0/5154949 >> v2:172.21.15.162:3300/0 conn(0x7f5f2c15d2d0 0x7f5f2c0b4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:36.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.943+0000 7f5f30c77640 1 -- 172.21.15.46:0/5154949 >> v1:172.21.15.46:6789/0 conn(0x7f5f2c0d2040 legacy=0x7f5f2c0cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:36.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.943+0000 7f5f2affd640 1 -- 172.21.15.46:0/5154949 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2870809389 0 0) 0x7f5f2c0d1bb0 con 0x7f5f2c0d2040 2023-12-02T00:29:36.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.943+0000 7f5f30c77640 1 -- 172.21.15.46:0/5154949 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f2c0d1280 con 0x7f5f2c0ce150 2023-12-02T00:29:36.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.943+0000 7f5f2bfff640 1 --2- 172.21.15.46:0/5154949 >> v2:172.21.15.162:3300/0 conn(0x7f5f2c15d2d0 0x7f5f2c0b4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:29:36.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.944+0000 7f5f30c77640 1 --2- 172.21.15.46:0/5154949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f2c0ce150 0x7f5f2c0b51a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c06c5d0 tx=0x7f5f2402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec7227a5c34ceb02 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:36.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.944+0000 7f5f2affd640 1 -- 172.21.15.46:0/5154949 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2402cc80 con 0x7f5f2c0ce150 2023-12-02T00:29:36.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.944+0000 7f5f2affd640 1 -- 172.21.15.46:0/5154949 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f2402cde0 con 0x7f5f2c0ce150 2023-12-02T00:29:36.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.944+0000 7f5f2affd640 1 -- 172.21.15.46:0/5154949 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f24034660 con 0x7f5f2c0ce150 2023-12-02T00:29:36.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.945+0000 7f5f326da640 1 -- 172.21.15.46:0/5154949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f2c0ce150 msgr2=0x7f5f2c0b51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:36.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.945+0000 7f5f326da640 1 --2- 172.21.15.46:0/5154949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f2c0ce150 0x7f5f2c0b51a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c06c5d0 tx=0x7f5f2402bed0 comp rx=0 tx=0).stop 2023-12-02T00:29:36.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.945+0000 7f5f326da640 1 -- 172.21.15.46:0/5154949 shutdown_connections 2023-12-02T00:29:36.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.945+0000 7f5f326da640 1 --2- 172.21.15.46:0/5154949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f2c0ce150 0x7f5f2c0b51a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:36.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.945+0000 7f5f326da640 1 --2- 172.21.15.46:0/5154949 >> v2:172.21.15.162:3300/0 conn(0x7f5f2c15d2d0 0x7f5f2c0b4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:36.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.945+0000 7f5f326da640 1 -- 172.21.15.46:0/5154949 >> 172.21.15.46:0/5154949 conn(0x7f5f2c072050 msgr2=0x7f5f2c072460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:36.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.945+0000 7f5f326da640 1 -- 172.21.15.46:0/5154949 shutdown_connections 2023-12-02T00:29:36.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.945+0000 7f5f326da640 1 -- 172.21.15.46:0/5154949 wait complete. 2023-12-02T00:29:36.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.946+0000 7f5f326da640 1 Processor -- start 2023-12-02T00:29:36.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.946+0000 7f5f326da640 1 -- start start 2023-12-02T00:29:36.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.947+0000 7f5f326da640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f2c0ce150 0x7f5f2c0cc980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:36.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.947+0000 7f5f326da640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f2c0d2040 0x7f5f2c1f1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:36.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.947+0000 7f5f326da640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f2c15d2d0 0x7f5f2c0ca2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:36.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.947+0000 7f5f326da640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c0ccec0 con 0x7f5f2c15d2d0 2023-12-02T00:29:36.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.947+0000 7f5f326da640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c0c8620 con 0x7f5f2c0d2040 2023-12-02T00:29:36.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.947+0000 7f5f326da640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c0cd0b0 con 0x7f5f2c0ce150 2023-12-02T00:29:36.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.947+0000 7f5f30c77640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f2c15d2d0 0x7f5f2c0ca2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:36.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.947+0000 7f5f30c77640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f2c15d2d0 0x7f5f2c0ca2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57296/0 (socket says 172.21.15.46:57296) 2023-12-02T00:29:36.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.947+0000 7f5f30c77640 1 -- 172.21.15.46:0/2838048792 learned_addr learned my addr 172.21.15.46:0/2838048792 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:36.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.947+0000 7f5f2bfff640 1 --2- 172.21.15.46:0/2838048792 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f2c0ce150 0x7f5f2c0cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:36.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.947+0000 7f5f2b7fe640 1 --2- 172.21.15.46:0/2838048792 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f2c0d2040 0x7f5f2c1f1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:36.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.947+0000 7f5f30c77640 1 -- 172.21.15.46:0/2838048792 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f2c0ce150 msgr2=0x7f5f2c0cc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:36.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.947+0000 7f5f30c77640 1 --2- 172.21.15.46:0/2838048792 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f2c0ce150 0x7f5f2c0cc980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:36.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.947+0000 7f5f30c77640 1 -- 172.21.15.46:0/2838048792 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f2c0d2040 msgr2=0x7f5f2c1f1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:36.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.947+0000 7f5f30c77640 1 --2- 172.21.15.46:0/2838048792 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f2c0d2040 0x7f5f2c1f1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:36.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.947+0000 7f5f30c77640 1 -- 172.21.15.46:0/2838048792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f24009a70 con 0x7f5f2c15d2d0 2023-12-02T00:29:36.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.948+0000 7f5f30c77640 1 --2- 172.21.15.46:0/2838048792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f2c15d2d0 0x7f5f2c0ca2b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5f2402c4e0 tx=0x7f5f24004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:36.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.948+0000 7f5f28ff9640 1 -- 172.21.15.46:0/2838048792 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f24040650 con 0x7f5f2c15d2d0 2023-12-02T00:29:36.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.948+0000 7f5f28ff9640 1 -- 172.21.15.46:0/2838048792 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f240407b0 con 0x7f5f2c15d2d0 2023-12-02T00:29:36.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.948+0000 7f5f326da640 1 -- 172.21.15.46:0/2838048792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f2c0c7cf0 con 0x7f5f2c15d2d0 2023-12-02T00:29:36.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.948+0000 7f5f28ff9640 1 -- 172.21.15.46:0/2838048792 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f24040b60 con 0x7f5f2c15d2d0 2023-12-02T00:29:36.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.948+0000 7f5f326da640 1 -- 172.21.15.46:0/2838048792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f2c0c82b0 con 0x7f5f2c15d2d0 2023-12-02T00:29:36.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.949+0000 7f5f326da640 1 -- 172.21.15.46:0/2838048792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f2c0c87b0 con 0x7f5f2c15d2d0 2023-12-02T00:29:36.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.949+0000 7f5f28ff9640 1 -- 172.21.15.46:0/2838048792 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5f24040cc0 con 0x7f5f2c15d2d0 2023-12-02T00:29:36.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.950+0000 7f5f28ff9640 1 --2- 172.21.15.46:0/2838048792 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5f1806ae40 0x7f5f1806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:36.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.950+0000 7f5f28ff9640 1 -- 172.21.15.46:0/2838048792 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f5f240ac8b0 con 0x7f5f2c15d2d0 2023-12-02T00:29:36.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.950+0000 7f5f2bfff640 1 --2- 172.21.15.46:0/2838048792 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5f1806ae40 0x7f5f1806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:36.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.951+0000 7f5f2bfff640 1 --2- 172.21.15.46:0/2838048792 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5f1806ae40 0x7f5f1806d300 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5f2000b840 tx=0x7f5f20008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:36.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:36.954+0000 7f5f28ff9640 1 -- 172.21.15.46:0/2838048792 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5f24077a10 con 0x7f5f2c15d2d0 2023-12-02T00:29:37.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.128+0000 7f5f326da640 1 -- 172.21.15.46:0/2838048792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5f2c069400 con 0x7f5f2c15d2d0 2023-12-02T00:29:37.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.128+0000 7f5f28ff9640 1 -- 172.21.15.46:0/2838048792 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5f2407b030 con 0x7f5f2c15d2d0 2023-12-02T00:29:37.130 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-12-02T00:29:37.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.130+0000 7f5f326da640 1 -- 172.21.15.46:0/2838048792 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5f1806ae40 msgr2=0x7f5f1806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:37.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.130+0000 7f5f326da640 1 --2- 172.21.15.46:0/2838048792 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5f1806ae40 0x7f5f1806d300 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5f2000b840 tx=0x7f5f20008040 comp rx=0 tx=0).stop 2023-12-02T00:29:37.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.130+0000 7f5f326da640 1 -- 172.21.15.46:0/2838048792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f2c15d2d0 msgr2=0x7f5f2c0ca2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:37.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.130+0000 7f5f326da640 1 --2- 172.21.15.46:0/2838048792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f2c15d2d0 0x7f5f2c0ca2b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5f2402c4e0 tx=0x7f5f24004a40 comp rx=0 tx=0).stop 2023-12-02T00:29:37.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.130+0000 7f5f326da640 1 -- 172.21.15.46:0/2838048792 shutdown_connections 2023-12-02T00:29:37.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.130+0000 7f5f326da640 1 --2- 172.21.15.46:0/2838048792 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5f1806ae40 0x7f5f1806d300 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:37.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.130+0000 7f5f326da640 1 --2- 172.21.15.46:0/2838048792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f2c15d2d0 0x7f5f2c0ca2b0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:37.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.130+0000 7f5f326da640 1 --2- 172.21.15.46:0/2838048792 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f2c0d2040 0x7f5f2c1f1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:37.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.130+0000 7f5f326da640 1 --2- 172.21.15.46:0/2838048792 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f2c0ce150 0x7f5f2c0cc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:37.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.130+0000 7f5f326da640 1 -- 172.21.15.46:0/2838048792 >> 172.21.15.46:0/2838048792 conn(0x7f5f2c072050 msgr2=0x7f5f2c0b8530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:37.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.130+0000 7f5f326da640 1 -- 172.21.15.46:0/2838048792 shutdown_connections 2023-12-02T00:29:37.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.130+0000 7f5f326da640 1 -- 172.21.15.46:0/2838048792 wait complete. 2023-12-02T00:29:37.142 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-02T00:29:37.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.251+0000 7f141bdb7640 1 Processor -- start 2023-12-02T00:29:37.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.251+0000 7f141bdb7640 1 -- start start 2023-12-02T00:29:37.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.252+0000 7f141bdb7640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1414160fb0 0x7f1414161440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:37.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.252+0000 7f141bdb7640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f14140d0ed0 0x7f14140d1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:37.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.252+0000 7f141bdb7640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1414161d20 con 0x7f141406ea50 2023-12-02T00:29:37.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.252+0000 7f141bdb7640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f141415e760 con 0x7f1414160fb0 2023-12-02T00:29:37.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.252+0000 7f141bdb7640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f141415d9a0 con 0x7f14140d0ed0 2023-12-02T00:29:37.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.252+0000 7f1419352640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1414160fb0 0x7f1414161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:37.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.252+0000 7f1419352640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1414160fb0 0x7f1414161440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57312/0 (socket says 172.21.15.46:57312) 2023-12-02T00:29:37.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.252+0000 7f1419352640 1 -- 172.21.15.46:0/1009247444 learned_addr learned my addr 172.21.15.46:0/1009247444 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:37.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.252+0000 7f1419352640 1 -- 172.21.15.46:0/1009247444 >> v2:172.21.15.162:3300/0 conn(0x7f14140d0ed0 msgr2=0x7f14140d1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:37.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.252+0000 7f1419352640 1 --2- 172.21.15.46:0/1009247444 >> v2:172.21.15.162:3300/0 conn(0x7f14140d0ed0 0x7f14140d1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:37.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.252+0000 7f1419352640 1 -- 172.21.15.46:0/1009247444 >> v1:172.21.15.162:6789/0 conn(0x7f141406ea50 legacy=0x7f141406ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:37.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.252+0000 7f1419352640 1 -- 172.21.15.46:0/1009247444 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f141415dc80 con 0x7f1414160fb0 2023-12-02T00:29:37.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.253+0000 7f1419352640 1 --2- 172.21.15.46:0/1009247444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1414160fb0 0x7f1414161440 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f14140b55a0 tx=0x7f141002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5b67cede8b1b4f9 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:37.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.253+0000 7f1418b51640 1 -- 172.21.15.46:0/1009247444 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14100317e0 con 0x7f1414160fb0 2023-12-02T00:29:37.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.253+0000 7f1418b51640 1 -- 172.21.15.46:0/1009247444 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1410031940 con 0x7f1414160fb0 2023-12-02T00:29:37.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.253+0000 7f1418b51640 1 -- 172.21.15.46:0/1009247444 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1410031cd0 con 0x7f1414160fb0 2023-12-02T00:29:37.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.253+0000 7f141bdb7640 1 -- 172.21.15.46:0/1009247444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1414160fb0 msgr2=0x7f1414161440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:37.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.253+0000 7f141bdb7640 1 --2- 172.21.15.46:0/1009247444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1414160fb0 0x7f1414161440 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f14140b55a0 tx=0x7f141002b880 comp rx=0 tx=0).stop 2023-12-02T00:29:37.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.253+0000 7f141bdb7640 1 -- 172.21.15.46:0/1009247444 shutdown_connections 2023-12-02T00:29:37.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.253+0000 7f141bdb7640 1 --2- 172.21.15.46:0/1009247444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1414160fb0 0x7f1414161440 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:37.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.253+0000 7f141bdb7640 1 --2- 172.21.15.46:0/1009247444 >> v2:172.21.15.162:3300/0 conn(0x7f14140d0ed0 0x7f14140d1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:37.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.253+0000 7f141bdb7640 1 -- 172.21.15.46:0/1009247444 >> 172.21.15.46:0/1009247444 conn(0x7f141406a6b0 msgr2=0x7f141406aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:37.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.254+0000 7f141bdb7640 1 -- 172.21.15.46:0/1009247444 shutdown_connections 2023-12-02T00:29:37.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.254+0000 7f141bdb7640 1 -- 172.21.15.46:0/1009247444 wait complete. 2023-12-02T00:29:37.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.255+0000 7f141bdb7640 1 Processor -- start 2023-12-02T00:29:37.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.255+0000 7f141bdb7640 1 -- start start 2023-12-02T00:29:37.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.255+0000 7f141bdb7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f141406ea50 0x7f14141e8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:37.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.255+0000 7f141bdb7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14140d0ed0 0x7f14140b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:37.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.255+0000 7f141bdb7640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1414160fb0 0x7f14140b4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:37.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.255+0000 7f141bdb7640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1414158db0 con 0x7f141406ea50 2023-12-02T00:29:37.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.255+0000 7f141bdb7640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1414145560 con 0x7f1414160fb0 2023-12-02T00:29:37.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.255+0000 7f141bdb7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1414145840 con 0x7f14140d0ed0 2023-12-02T00:29:37.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.256+0000 7f1419b53640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f141406ea50 0x7f14141e8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:37.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.256+0000 7f1419b53640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f141406ea50 0x7f14141e8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57322/0 (socket says 172.21.15.46:57322) 2023-12-02T00:29:37.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.256+0000 7f1419b53640 1 -- 172.21.15.46:0/3746786346 learned_addr learned my addr 172.21.15.46:0/3746786346 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:37.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.256+0000 7f141a354640 1 --2- 172.21.15.46:0/3746786346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1414160fb0 0x7f14140b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:37.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.256+0000 7f1419352640 1 --2- 172.21.15.46:0/3746786346 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14140d0ed0 0x7f14140b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:37.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.256+0000 7f1419b53640 1 -- 172.21.15.46:0/3746786346 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14140d0ed0 msgr2=0x7f14140b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:37.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.256+0000 7f1419b53640 1 --2- 172.21.15.46:0/3746786346 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14140d0ed0 0x7f14140b3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:37.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.256+0000 7f1419b53640 1 -- 172.21.15.46:0/3746786346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1414160fb0 msgr2=0x7f14140b4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:37.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.256+0000 7f1419b53640 1 --2- 172.21.15.46:0/3746786346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1414160fb0 0x7f14140b4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:37.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.256+0000 7f1419b53640 1 -- 172.21.15.46:0/3746786346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14100094d0 con 0x7f141406ea50 2023-12-02T00:29:37.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.256+0000 7f1419352640 1 --2- 172.21.15.46:0/3746786346 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14140d0ed0 0x7f14140b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:29:37.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.256+0000 7f1419b53640 1 --2- 172.21.15.46:0/3746786346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f141406ea50 0x7f14141e8110 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f140800b3e0 tx=0x7f140800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:37.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.256+0000 7f14067fc640 1 -- 172.21.15.46:0/3746786346 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1408011830 con 0x7f141406ea50 2023-12-02T00:29:37.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.256+0000 7f14067fc640 1 -- 172.21.15.46:0/3746786346 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14080119b0 con 0x7f141406ea50 2023-12-02T00:29:37.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.256+0000 7f141bdb7640 1 -- 172.21.15.46:0/3746786346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1414145b20 con 0x7f141406ea50 2023-12-02T00:29:37.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.256+0000 7f14067fc640 1 -- 172.21.15.46:0/3746786346 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1408011d60 con 0x7f141406ea50 2023-12-02T00:29:37.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.256+0000 7f141bdb7640 1 -- 172.21.15.46:0/3746786346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1414146320 con 0x7f141406ea50 2023-12-02T00:29:37.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.257+0000 7f141bdb7640 1 -- 172.21.15.46:0/3746786346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1414146040 con 0x7f141406ea50 2023-12-02T00:29:37.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.258+0000 7f14067fc640 1 -- 172.21.15.46:0/3746786346 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f140801a400 con 0x7f141406ea50 2023-12-02T00:29:37.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.258+0000 7f14067fc640 1 --2- 172.21.15.46:0/3746786346 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f13f806ae40 0x7f13f806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:37.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.259+0000 7f14067fc640 1 -- 172.21.15.46:0/3746786346 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f1408089850 con 0x7f141406ea50 2023-12-02T00:29:37.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.259+0000 7f1419352640 1 --2- 172.21.15.46:0/3746786346 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f13f806ae40 0x7f13f806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:37.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.259+0000 7f1419352640 1 --2- 172.21.15.46:0/3746786346 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f13f806ae40 0x7f13f806d300 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1410036000 tx=0x7f141003d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:37.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.263+0000 7f14067fc640 1 -- 172.21.15.46:0/3746786346 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1414146040 con 0x7f141406ea50 2023-12-02T00:29:37.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.440+0000 7f141bdb7640 1 -- 172.21.15.46:0/3746786346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1414146040 con 0x7f141406ea50 2023-12-02T00:29:37.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.440+0000 7f14067fc640 1 -- 172.21.15.46:0/3746786346 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1414146040 con 0x7f141406ea50 2023-12-02T00:29:37.441 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-12-02T00:29:37.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.442+0000 7f141bdb7640 1 -- 172.21.15.46:0/3746786346 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f13f806ae40 msgr2=0x7f13f806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:37.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.442+0000 7f141bdb7640 1 --2- 172.21.15.46:0/3746786346 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f13f806ae40 0x7f13f806d300 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1410036000 tx=0x7f141003d040 comp rx=0 tx=0).stop 2023-12-02T00:29:37.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.442+0000 7f141bdb7640 1 -- 172.21.15.46:0/3746786346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f141406ea50 msgr2=0x7f14141e8110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:37.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.442+0000 7f141bdb7640 1 --2- 172.21.15.46:0/3746786346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f141406ea50 0x7f14141e8110 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f140800b3e0 tx=0x7f140800b8a0 comp rx=0 tx=0).stop 2023-12-02T00:29:37.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.442+0000 7f141bdb7640 1 -- 172.21.15.46:0/3746786346 shutdown_connections 2023-12-02T00:29:37.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.442+0000 7f141bdb7640 1 --2- 172.21.15.46:0/3746786346 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f13f806ae40 0x7f13f806d300 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:37.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.442+0000 7f141bdb7640 1 --2- 172.21.15.46:0/3746786346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1414160fb0 0x7f14140b4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:37.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.442+0000 7f141bdb7640 1 --2- 172.21.15.46:0/3746786346 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14140d0ed0 0x7f14140b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:37.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.442+0000 7f141bdb7640 1 --2- 172.21.15.46:0/3746786346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f141406ea50 0x7f14141e8110 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:37.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.442+0000 7f141bdb7640 1 -- 172.21.15.46:0/3746786346 >> 172.21.15.46:0/3746786346 conn(0x7f141406a6b0 msgr2=0x7f14140d1320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:37.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.442+0000 7f141bdb7640 1 -- 172.21.15.46:0/3746786346 shutdown_connections 2023-12-02T00:29:37.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.442+0000 7f141bdb7640 1 -- 172.21.15.46:0/3746786346 wait complete. 2023-12-02T00:29:37.454 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-02T00:29:37.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.561+0000 7fa9ed226640 1 Processor -- start 2023-12-02T00:29:37.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.561+0000 7fa9ed226640 1 -- start start 2023-12-02T00:29:37.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.562+0000 7fa9ed226640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa9e80d2910 0x7fa9e80d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:37.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.562+0000 7fa9ed226640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9e80b2b10 con 0x7fa9e815d330 2023-12-02T00:29:37.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.562+0000 7fa9ed226640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9e80d1590 con 0x7fa9e80d3730 2023-12-02T00:29:37.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.562+0000 7fa9ed226640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9e80d3bc0 con 0x7fa9e80d2910 2023-12-02T00:29:37.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.562+0000 7fa9e6d9d640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa9e815d330 0x7fa9e80b8b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59350/0 (socket says 172.21.15.46:59350) 2023-12-02T00:29:37.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.562+0000 7fa9e6d9d640 1 -- 172.21.15.46:0/2085043084 learned_addr learned my addr 172.21.15.46:0/2085043084 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:37.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.562+0000 7fa9dffff640 1 --2- 172.21.15.46:0/2085043084 >> v2:172.21.15.46:3300/0 conn(0x7fa9e80d2910 0x7fa9e80d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:37.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.562+0000 7fa9dffff640 1 -- 172.21.15.46:0/2085043084 >> v1:172.21.15.162:6789/0 conn(0x7fa9e80d3730 legacy=0x7fa9e80b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:37.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.562+0000 7fa9dffff640 1 -- 172.21.15.46:0/2085043084 >> v1:172.21.15.46:6789/0 conn(0x7fa9e815d330 legacy=0x7fa9e80b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:37.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.562+0000 7fa9e659c640 1 -- 172.21.15.46:0/2085043084 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4105170082 0 0) 0x7fa9e80b2b10 con 0x7fa9e815d330 2023-12-02T00:29:37.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.562+0000 7fa9dffff640 1 -- 172.21.15.46:0/2085043084 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e80d1870 con 0x7fa9e80d2910 2023-12-02T00:29:37.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.563+0000 7fa9dffff640 1 --2- 172.21.15.46:0/2085043084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9e80d2910 0x7fa9e80d31f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa9d00097a0 tx=0x7fa9d002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9a3dbff62f51826 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:37.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.563+0000 7fa9e659c640 1 -- 172.21.15.46:0/2085043084 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d0031830 con 0x7fa9e80d2910 2023-12-02T00:29:37.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.563+0000 7fa9e659c640 1 -- 172.21.15.46:0/2085043084 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d0031990 con 0x7fa9e80d2910 2023-12-02T00:29:37.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.563+0000 7fa9e659c640 1 -- 172.21.15.46:0/2085043084 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d0031d20 con 0x7fa9e80d2910 2023-12-02T00:29:37.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.564+0000 7fa9ed226640 1 -- 172.21.15.46:0/2085043084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9e80d2910 msgr2=0x7fa9e80d31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:37.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.564+0000 7fa9ed226640 1 --2- 172.21.15.46:0/2085043084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9e80d2910 0x7fa9e80d31f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa9d00097a0 tx=0x7fa9d002b8d0 comp rx=0 tx=0).stop 2023-12-02T00:29:37.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.564+0000 7fa9ed226640 1 -- 172.21.15.46:0/2085043084 shutdown_connections 2023-12-02T00:29:37.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.564+0000 7fa9ed226640 1 --2- 172.21.15.46:0/2085043084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9e80d2910 0x7fa9e80d31f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:37.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.564+0000 7fa9ed226640 1 -- 172.21.15.46:0/2085043084 >> 172.21.15.46:0/2085043084 conn(0x7fa9e8072020 msgr2=0x7fa9e8072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:37.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.564+0000 7fa9ed226640 1 -- 172.21.15.46:0/2085043084 shutdown_connections 2023-12-02T00:29:37.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.564+0000 7fa9ed226640 1 -- 172.21.15.46:0/2085043084 wait complete. 2023-12-02T00:29:37.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.565+0000 7fa9ed226640 1 Processor -- start 2023-12-02T00:29:37.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.565+0000 7fa9ed226640 1 -- start start 2023-12-02T00:29:37.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.566+0000 7fa9ed226640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9e80d2910 0x7fa9e81570a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:37.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.566+0000 7fa9ed226640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9e80d3730 0x7fa9e80cce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:37.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.566+0000 7fa9ed226640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9e815d330 0x7fa9e815d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:37.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.566+0000 7fa9ed226640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e8158e70 con 0x7fa9e80d2910 2023-12-02T00:29:37.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.566+0000 7fa9ed226640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e8158260 con 0x7fa9e815d330 2023-12-02T00:29:37.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.566+0000 7fa9ed226640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e8158540 con 0x7fa9e80d3730 2023-12-02T00:29:37.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.566+0000 7fa9e6d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9e80d2910 0x7fa9e81570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:37.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.566+0000 7fa9e6d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9e80d2910 0x7fa9e81570a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57342/0 (socket says 172.21.15.46:57342) 2023-12-02T00:29:37.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.566+0000 7fa9e6d9d640 1 -- 172.21.15.46:0/1624616339 learned_addr learned my addr 172.21.15.46:0/1624616339 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:37.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.566+0000 7fa9e759e640 1 --2- 172.21.15.46:0/1624616339 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9e815d330 0x7fa9e815d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:37.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.566+0000 7fa9dffff640 1 --2- 172.21.15.46:0/1624616339 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9e80d3730 0x7fa9e80cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:37.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.566+0000 7fa9e6d9d640 1 -- 172.21.15.46:0/1624616339 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9e80d3730 msgr2=0x7fa9e80cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:37.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.566+0000 7fa9e6d9d640 1 --2- 172.21.15.46:0/1624616339 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9e80d3730 0x7fa9e80cce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:37.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.566+0000 7fa9e6d9d640 1 -- 172.21.15.46:0/1624616339 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9e815d330 msgr2=0x7fa9e815d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:37.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.566+0000 7fa9e6d9d640 1 --2- 172.21.15.46:0/1624616339 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9e815d330 0x7fa9e815d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:37.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.566+0000 7fa9e6d9d640 1 -- 172.21.15.46:0/1624616339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d0009400 con 0x7fa9e80d2910 2023-12-02T00:29:37.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.566+0000 7fa9e759e640 1 --2- 172.21.15.46:0/1624616339 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9e815d330 0x7fa9e815d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:37.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.567+0000 7fa9e6d9d640 1 --2- 172.21.15.46:0/1624616339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9e80d2910 0x7fa9e81570a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa9d800c3f0 tx=0x7fa9d800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:37.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.567+0000 7fa9df7fe640 1 -- 172.21.15.46:0/1624616339 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d80074e0 con 0x7fa9e80d2910 2023-12-02T00:29:37.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.567+0000 7fa9df7fe640 1 -- 172.21.15.46:0/1624616339 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d8007640 con 0x7fa9e80d2910 2023-12-02T00:29:37.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.567+0000 7fa9ed226640 1 -- 172.21.15.46:0/1624616339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9e8158820 con 0x7fa9e80d2910 2023-12-02T00:29:37.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.567+0000 7fa9df7fe640 1 -- 172.21.15.46:0/1624616339 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d8004a00 con 0x7fa9e80d2910 2023-12-02T00:29:37.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.567+0000 7fa9ed226640 1 -- 172.21.15.46:0/1624616339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9e81575e0 con 0x7fa9e80d2910 2023-12-02T00:29:37.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.568+0000 7fa9ed226640 1 -- 172.21.15.46:0/1624616339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9e8069400 con 0x7fa9e80d2910 2023-12-02T00:29:37.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.568+0000 7fa9df7fe640 1 -- 172.21.15.46:0/1624616339 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa9d8004b60 con 0x7fa9e80d2910 2023-12-02T00:29:37.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.568+0000 7fa9df7fe640 1 --2- 172.21.15.46:0/1624616339 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa9c806ae40 0x7fa9c806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:37.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.569+0000 7fa9df7fe640 1 -- 172.21.15.46:0/1624616339 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fa9d8089520 con 0x7fa9e80d2910 2023-12-02T00:29:37.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.569+0000 7fa9dffff640 1 --2- 172.21.15.46:0/1624616339 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa9c806ae40 0x7fa9c806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:37.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.569+0000 7fa9dffff640 1 --2- 172.21.15.46:0/1624616339 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa9c806ae40 0x7fa9c806d300 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0036000 tx=0x7fa9d0008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:37.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.573+0000 7fa9df7fe640 1 -- 172.21.15.46:0/1624616339 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa9d8054540 con 0x7fa9e80d2910 2023-12-02T00:29:37.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.763+0000 7fa9ed226640 1 -- 172.21.15.46:0/1624616339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa9e80b48e0 con 0x7fa9e80d2910 2023-12-02T00:29:37.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.763+0000 7fa9df7fe640 1 -- 172.21.15.46:0/1624616339 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa9d8057b60 con 0x7fa9e80d2910 2023-12-02T00:29:37.765 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2023-12-02T00:29:37.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.766+0000 7fa9ed226640 1 -- 172.21.15.46:0/1624616339 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa9c806ae40 msgr2=0x7fa9c806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:37.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.766+0000 7fa9ed226640 1 --2- 172.21.15.46:0/1624616339 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa9c806ae40 0x7fa9c806d300 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0036000 tx=0x7fa9d0008030 comp rx=0 tx=0).stop 2023-12-02T00:29:37.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.766+0000 7fa9ed226640 1 -- 172.21.15.46:0/1624616339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9e80d2910 msgr2=0x7fa9e81570a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:37.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.766+0000 7fa9ed226640 1 --2- 172.21.15.46:0/1624616339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9e80d2910 0x7fa9e81570a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa9d800c3f0 tx=0x7fa9d800c8b0 comp rx=0 tx=0).stop 2023-12-02T00:29:37.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.766+0000 7fa9ed226640 1 -- 172.21.15.46:0/1624616339 shutdown_connections 2023-12-02T00:29:37.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.766+0000 7fa9ed226640 1 --2- 172.21.15.46:0/1624616339 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa9c806ae40 0x7fa9c806d300 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:37.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.766+0000 7fa9ed226640 1 --2- 172.21.15.46:0/1624616339 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9e815d330 0x7fa9e815d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:37.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.766+0000 7fa9ed226640 1 --2- 172.21.15.46:0/1624616339 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9e80d3730 0x7fa9e80cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:37.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.766+0000 7fa9ed226640 1 --2- 172.21.15.46:0/1624616339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9e80d2910 0x7fa9e81570a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:37.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.766+0000 7fa9ed226640 1 -- 172.21.15.46:0/1624616339 >> 172.21.15.46:0/1624616339 conn(0x7fa9e8072020 msgr2=0x7fa9e8072400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:37.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.766+0000 7fa9ed226640 1 -- 172.21.15.46:0/1624616339 shutdown_connections 2023-12-02T00:29:37.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.766+0000 7fa9ed226640 1 -- 172.21.15.46:0/1624616339 wait complete. 2023-12-02T00:29:37.778 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi162.front.sepia.ceph.com... 2023-12-02T00:29:37.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:37.778 DEBUG:teuthology.orchestra.run.smithi162:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2023-12-02T00:29:37.811 INFO:teuthology.orchestra.run:waiting for 300 2023-12-02T00:29:37.811 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi162.front.sepia.ceph.com 2023-12-02T00:29:37.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:29:37.811 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-12-02T00:29:37.868 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-02T00:29:37.868 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-02T00:29:37.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.976+0000 7fa20f1a0640 1 Processor -- start 2023-12-02T00:29:37.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.976+0000 7fa20f1a0640 1 -- start start 2023-12-02T00:29:37.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.977+0000 7fa20f1a0640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa20815e8f0 0x7fa2080b3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:37.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.977+0000 7fa20f1a0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa2080ccbf0 0x7fa2080ceca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:37.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.977+0000 7fa20f1a0640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2080cc050 con 0x7fa2080b23a0 2023-12-02T00:29:37.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.977+0000 7fa20f1a0640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2080ce610 con 0x7fa20815e8f0 2023-12-02T00:29:37.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.977+0000 7fa20f1a0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa20815ecd0 con 0x7fa2080ccbf0 2023-12-02T00:29:37.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.977+0000 7fa20e19e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa20815e8f0 0x7fa2080b3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:37.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.977+0000 7fa20d99d640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa2080b23a0 0x7fa2080cc4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59356/0 (socket says 172.21.15.46:59356) 2023-12-02T00:29:37.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.977+0000 7fa20d99d640 1 -- 172.21.15.46:0/3949078056 learned_addr learned my addr 172.21.15.46:0/3949078056 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:37.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.977+0000 7fa20e19e640 1 -- 172.21.15.46:0/3949078056 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa2080ccbf0 msgr2=0x7fa2080ceca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:37.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.977+0000 7fa20e19e640 1 --2- 172.21.15.46:0/3949078056 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa2080ccbf0 0x7fa2080ceca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:37.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.977+0000 7fa20e19e640 1 -- 172.21.15.46:0/3949078056 >> v1:172.21.15.46:6789/0 conn(0x7fa2080b23a0 legacy=0x7fa2080cc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:37.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.977+0000 7fa20d19c640 1 -- 172.21.15.46:0/3949078056 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3830248996 0 0) 0x7fa2080cc050 con 0x7fa2080b23a0 2023-12-02T00:29:37.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.977+0000 7fa20e19e640 1 -- 172.21.15.46:0/3949078056 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2080cba60 con 0x7fa20815e8f0 2023-12-02T00:29:37.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.978+0000 7fa20e19e640 1 --2- 172.21.15.46:0/3949078056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa20815e8f0 0x7fa2080b3bf0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa2040097a0 tx=0x7fa20402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efcc5a0f632bbf17 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:37.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.978+0000 7fa20d19c640 1 -- 172.21.15.46:0/3949078056 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa204031830 con 0x7fa20815e8f0 2023-12-02T00:29:37.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.978+0000 7fa20d19c640 1 -- 172.21.15.46:0/3949078056 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa204031990 con 0x7fa20815e8f0 2023-12-02T00:29:37.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.978+0000 7fa20d19c640 1 -- 172.21.15.46:0/3949078056 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa204031d20 con 0x7fa20815e8f0 2023-12-02T00:29:37.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.978+0000 7fa20f1a0640 1 -- 172.21.15.46:0/3949078056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa20815e8f0 msgr2=0x7fa2080b3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:37.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.978+0000 7fa20f1a0640 1 --2- 172.21.15.46:0/3949078056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa20815e8f0 0x7fa2080b3bf0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa2040097a0 tx=0x7fa20402b8d0 comp rx=0 tx=0).stop 2023-12-02T00:29:37.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.979+0000 7fa20f1a0640 1 -- 172.21.15.46:0/3949078056 shutdown_connections 2023-12-02T00:29:37.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.979+0000 7fa20f1a0640 1 --2- 172.21.15.46:0/3949078056 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa2080ccbf0 0x7fa2080ceca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:37.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.979+0000 7fa20f1a0640 1 --2- 172.21.15.46:0/3949078056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa20815e8f0 0x7fa2080b3bf0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:37.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.979+0000 7fa20f1a0640 1 -- 172.21.15.46:0/3949078056 >> 172.21.15.46:0/3949078056 conn(0x7fa2080b28f0 msgr2=0x7fa2080b2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:37.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.979+0000 7fa20f1a0640 1 -- 172.21.15.46:0/3949078056 shutdown_connections 2023-12-02T00:29:37.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.979+0000 7fa20f1a0640 1 -- 172.21.15.46:0/3949078056 wait complete. 2023-12-02T00:29:37.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.980+0000 7fa20f1a0640 1 Processor -- start 2023-12-02T00:29:37.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.980+0000 7fa20f1a0640 1 -- start start 2023-12-02T00:29:37.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.980+0000 7fa20f1a0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2080b23a0 0x7fa20815a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:37.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.980+0000 7fa20f1a0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2080ccbf0 0x7fa20814de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:37.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.980+0000 7fa20f1a0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa20815e8f0 0x7fa20814e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:37.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.980+0000 7fa20f1a0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa20806b5e0 con 0x7fa2080b23a0 2023-12-02T00:29:37.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.980+0000 7fa20f1a0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2080b58b0 con 0x7fa2080ccbf0 2023-12-02T00:29:37.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.980+0000 7fa20f1a0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2080b5b90 con 0x7fa20815e8f0 2023-12-02T00:29:37.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.981+0000 7fa20e19e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2080b23a0 0x7fa20815a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:37.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.981+0000 7fa20e19e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2080b23a0 0x7fa20815a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57350/0 (socket says 172.21.15.46:57350) 2023-12-02T00:29:37.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.981+0000 7fa20e19e640 1 -- 172.21.15.46:0/3313858557 learned_addr learned my addr 172.21.15.46:0/3313858557 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:37.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.981+0000 7fa20d99d640 1 --2- 172.21.15.46:0/3313858557 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2080ccbf0 0x7fa20814de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:37.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.981+0000 7fa20e99f640 1 --2- 172.21.15.46:0/3313858557 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa20815e8f0 0x7fa20814e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:37.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.981+0000 7fa20e19e640 1 -- 172.21.15.46:0/3313858557 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa20815e8f0 msgr2=0x7fa20814e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:37.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.981+0000 7fa20e19e640 1 --2- 172.21.15.46:0/3313858557 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa20815e8f0 0x7fa20814e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:37.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.981+0000 7fa20e19e640 1 -- 172.21.15.46:0/3313858557 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2080ccbf0 msgr2=0x7fa20814de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:37.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.981+0000 7fa20e19e640 1 --2- 172.21.15.46:0/3313858557 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2080ccbf0 0x7fa20814de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:37.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.981+0000 7fa20e19e640 1 -- 172.21.15.46:0/3313858557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa204009400 con 0x7fa2080b23a0 2023-12-02T00:29:37.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.981+0000 7fa20e19e640 1 --2- 172.21.15.46:0/3313858557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2080b23a0 0x7fa20815a910 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa204000c00 tx=0x7fa2040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:37.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.981+0000 7fa1feffd640 1 -- 172.21.15.46:0/3313858557 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa204004230 con 0x7fa2080b23a0 2023-12-02T00:29:37.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.981+0000 7fa1feffd640 1 -- 172.21.15.46:0/3313858557 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa204004390 con 0x7fa2080b23a0 2023-12-02T00:29:37.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.981+0000 7fa1feffd640 1 -- 172.21.15.46:0/3313858557 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20403a940 con 0x7fa2080b23a0 2023-12-02T00:29:37.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.981+0000 7fa20f1a0640 1 -- 172.21.15.46:0/3313858557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa20806b830 con 0x7fa2080b23a0 2023-12-02T00:29:37.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.981+0000 7fa20f1a0640 1 -- 172.21.15.46:0/3313858557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa20815da40 con 0x7fa2080b23a0 2023-12-02T00:29:37.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.982+0000 7fa20f1a0640 1 -- 172.21.15.46:0/3313858557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa20806bd50 con 0x7fa2080b23a0 2023-12-02T00:29:37.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.983+0000 7fa1feffd640 1 -- 172.21.15.46:0/3313858557 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa20403aaa0 con 0x7fa2080b23a0 2023-12-02T00:29:37.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.983+0000 7fa1feffd640 1 --2- 172.21.15.46:0/3313858557 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa1ec06ae40 0x7fa1ec06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:37.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.983+0000 7fa1feffd640 1 -- 172.21.15.46:0/3313858557 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fa2040abbe0 con 0x7fa2080b23a0 2023-12-02T00:29:37.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.983+0000 7fa20d99d640 1 --2- 172.21.15.46:0/3313858557 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa1ec06ae40 0x7fa1ec06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:37.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.984+0000 7fa20d99d640 1 --2- 172.21.15.46:0/3313858557 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa1ec06ae40 0x7fa1ec06d300 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa2080c5800 tx=0x7fa1f8007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:37.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:37.987+0000 7fa1feffd640 1 -- 172.21.15.46:0/3313858557 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa20806bd50 con 0x7fa2080b23a0 2023-12-02T00:29:38.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.198+0000 7fa20f1a0640 1 -- 172.21.15.46:0/3313858557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa20806bd50 con 0x7fa2080b23a0 2023-12-02T00:29:38.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.198+0000 7fa1feffd640 1 -- 172.21.15.46:0/3313858557 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4907 (secure 0 0 0) 0x7fa20806bd50 con 0x7fa2080b23a0 2023-12-02T00:29:38.200 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:29:38.200 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/2059113401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2059113401},{"type":"v1","addr":"172.21.15.46:6834","nonce":2059113401}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/987532381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":987532381},{"type":"v1","addr":"172.21.15.46:6835","nonce":987532381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/449974862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":449974862},{"type":"v1","addr":"172.21.15.162:6836","nonce":449974862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-02T00:28:49.086548+0000","modified":"2023-12-02T00:28:51.082469+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4253":{"gid":4253,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/1231844478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":1231844478},{"type":"v1","addr":"172.21.15.162:6837","nonce":1231844478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-02T00:29:38.200 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 6 2023-12-02T00:29:38.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.200+0000 7fa20f1a0640 1 -- 172.21.15.46:0/3313858557 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa1ec06ae40 msgr2=0x7fa1ec06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:38.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.200+0000 7fa20f1a0640 1 --2- 172.21.15.46:0/3313858557 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa1ec06ae40 0x7fa1ec06d300 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa2080c5800 tx=0x7fa1f8007de0 comp rx=0 tx=0).stop 2023-12-02T00:29:38.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.200+0000 7fa20f1a0640 1 -- 172.21.15.46:0/3313858557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2080b23a0 msgr2=0x7fa20815a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:38.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.200+0000 7fa20f1a0640 1 --2- 172.21.15.46:0/3313858557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2080b23a0 0x7fa20815a910 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa204000c00 tx=0x7fa2040046c0 comp rx=0 tx=0).stop 2023-12-02T00:29:38.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.200+0000 7fa20f1a0640 1 -- 172.21.15.46:0/3313858557 shutdown_connections 2023-12-02T00:29:38.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.200+0000 7fa20f1a0640 1 --2- 172.21.15.46:0/3313858557 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa1ec06ae40 0x7fa1ec06d300 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:38.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.200+0000 7fa20f1a0640 1 --2- 172.21.15.46:0/3313858557 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa20815e8f0 0x7fa20814e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:38.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.200+0000 7fa20f1a0640 1 --2- 172.21.15.46:0/3313858557 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2080ccbf0 0x7fa20814de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:38.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.200+0000 7fa20f1a0640 1 --2- 172.21.15.46:0/3313858557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2080b23a0 0x7fa20815a910 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:38.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.200+0000 7fa20f1a0640 1 -- 172.21.15.46:0/3313858557 >> 172.21.15.46:0/3313858557 conn(0x7fa2080b28f0 msgr2=0x7fa2080c4990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:38.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.200+0000 7fa20f1a0640 1 -- 172.21.15.46:0/3313858557 shutdown_connections 2023-12-02T00:29:38.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.200+0000 7fa20f1a0640 1 -- 172.21.15.46:0/3313858557 wait complete. 2023-12-02T00:29:38.212 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:29:38.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.318+0000 7f79f609b640 1 Processor -- start 2023-12-02T00:29:38.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.318+0000 7f79f609b640 1 -- start start 2023-12-02T00:29:38.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.319+0000 7f79f609b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f79f0144980 0x7f79f015c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:38.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.319+0000 7f79f609b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79f00b4370 0x7f79f00b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:38.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.319+0000 7f79f609b640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79f0071770 con 0x7f79f00ceb70 2023-12-02T00:29:38.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.319+0000 7f79f609b640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79f00ccb60 con 0x7f79f0144980 2023-12-02T00:29:38.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.319+0000 7f79f609b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f00b4ce0 con 0x7f79f00b4370 2023-12-02T00:29:38.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.319+0000 7f79ef7fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f79f0144980 0x7f79f015c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:38.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.319+0000 7f79ef7fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f79f0144980 0x7f79f015c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57356/0 (socket says 172.21.15.46:57356) 2023-12-02T00:29:38.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.319+0000 7f79ef7fe640 1 -- 172.21.15.46:0/3678392388 learned_addr learned my addr 172.21.15.46:0/3678392388 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:38.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.319+0000 7f79effff640 1 --2- 172.21.15.46:0/3678392388 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79f00b4370 0x7f79f00b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:38.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.319+0000 7f79ef7fe640 1 -- 172.21.15.46:0/3678392388 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79f00b4370 msgr2=0x7f79f00b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:38.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.319+0000 7f79ef7fe640 1 --2- 172.21.15.46:0/3678392388 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79f00b4370 0x7f79f00b47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:38.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.319+0000 7f79ef7fe640 1 -- 172.21.15.46:0/3678392388 >> v1:172.21.15.162:6789/0 conn(0x7f79f00ceb70 legacy=0x7f79f00cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:38.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.319+0000 7f79ef7fe640 1 -- 172.21.15.46:0/3678392388 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f00ccdc0 con 0x7f79f0144980 2023-12-02T00:29:38.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.320+0000 7f79ef7fe640 1 --2- 172.21.15.46:0/3678392388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79f0144980 0x7f79f015c7c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f79e0009870 tx=0x7f79e002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=580e46c74259ed17 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:38.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.320+0000 7f79ee7fc640 1 -- 172.21.15.46:0/3678392388 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e0030840 con 0x7f79f0144980 2023-12-02T00:29:38.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.320+0000 7f79ee7fc640 1 -- 172.21.15.46:0/3678392388 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e00309a0 con 0x7f79f0144980 2023-12-02T00:29:38.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.320+0000 7f79ee7fc640 1 -- 172.21.15.46:0/3678392388 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e0030d30 con 0x7f79f0144980 2023-12-02T00:29:38.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.321+0000 7f79f609b640 1 -- 172.21.15.46:0/3678392388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79f0144980 msgr2=0x7f79f015c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:38.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.321+0000 7f79f609b640 1 --2- 172.21.15.46:0/3678392388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79f0144980 0x7f79f015c7c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f79e0009870 tx=0x7f79e002b980 comp rx=0 tx=0).stop 2023-12-02T00:29:38.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.321+0000 7f79f609b640 1 -- 172.21.15.46:0/3678392388 shutdown_connections 2023-12-02T00:29:38.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.321+0000 7f79f609b640 1 --2- 172.21.15.46:0/3678392388 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79f00b4370 0x7f79f00b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:38.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.321+0000 7f79f609b640 1 --2- 172.21.15.46:0/3678392388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79f0144980 0x7f79f015c7c0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:38.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.321+0000 7f79f609b640 1 -- 172.21.15.46:0/3678392388 >> 172.21.15.46:0/3678392388 conn(0x7f79f006a4d0 msgr2=0x7f79f006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:38.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.321+0000 7f79f609b640 1 -- 172.21.15.46:0/3678392388 shutdown_connections 2023-12-02T00:29:38.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.321+0000 7f79f609b640 1 -- 172.21.15.46:0/3678392388 wait complete. 2023-12-02T00:29:38.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.322+0000 7f79f609b640 1 Processor -- start 2023-12-02T00:29:38.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.322+0000 7f79f609b640 1 -- start start 2023-12-02T00:29:38.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.323+0000 7f79f609b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79f00b4370 0x7f79f0155510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:38.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.323+0000 7f79f609b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79f00ceb70 0x7f79f00b0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:38.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.323+0000 7f79f609b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79f0144980 0x7f79f00ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:38.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.323+0000 7f79f609b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f0155a50 con 0x7f79f00ceb70 2023-12-02T00:29:38.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.323+0000 7f79f609b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f00b1010 con 0x7f79f0144980 2023-12-02T00:29:38.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.323+0000 7f79f609b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f00b12f0 con 0x7f79f00b4370 2023-12-02T00:29:38.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.323+0000 7f79eeffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79f00ceb70 0x7f79f00b0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:38.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.323+0000 7f79eeffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79f00ceb70 0x7f79f00b0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57366/0 (socket says 172.21.15.46:57366) 2023-12-02T00:29:38.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.323+0000 7f79eeffd640 1 -- 172.21.15.46:0/2111592819 learned_addr learned my addr 172.21.15.46:0/2111592819 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:38.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.323+0000 7f79ef7fe640 1 --2- 172.21.15.46:0/2111592819 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79f00b4370 0x7f79f0155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:38.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.323+0000 7f79effff640 1 --2- 172.21.15.46:0/2111592819 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79f0144980 0x7f79f00ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:38.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.323+0000 7f79eeffd640 1 -- 172.21.15.46:0/2111592819 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79f00b4370 msgr2=0x7f79f0155510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:38.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.323+0000 7f79eeffd640 1 --2- 172.21.15.46:0/2111592819 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79f00b4370 0x7f79f0155510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:38.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.323+0000 7f79eeffd640 1 -- 172.21.15.46:0/2111592819 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79f0144980 msgr2=0x7f79f00ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:38.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.323+0000 7f79eeffd640 1 --2- 172.21.15.46:0/2111592819 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79f0144980 0x7f79f00ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:38.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.323+0000 7f79eeffd640 1 -- 172.21.15.46:0/2111592819 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e00094d0 con 0x7f79f00ceb70 2023-12-02T00:29:38.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.324+0000 7f79eeffd640 1 --2- 172.21.15.46:0/2111592819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79f00ceb70 0x7f79f00b0730 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f79e400c450 tx=0x7f79e400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:38.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.324+0000 7f79d7fff640 1 -- 172.21.15.46:0/2111592819 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e40052d0 con 0x7f79f00ceb70 2023-12-02T00:29:38.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.324+0000 7f79d7fff640 1 -- 172.21.15.46:0/2111592819 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e4005430 con 0x7f79f00ceb70 2023-12-02T00:29:38.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.324+0000 7f79f609b640 1 -- 172.21.15.46:0/2111592819 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79f0133670 con 0x7f79f00ceb70 2023-12-02T00:29:38.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.324+0000 7f79d7fff640 1 -- 172.21.15.46:0/2111592819 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e4012870 con 0x7f79f00ceb70 2023-12-02T00:29:38.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.324+0000 7f79f609b640 1 -- 172.21.15.46:0/2111592819 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79f00b15d0 con 0x7f79f00ceb70 2023-12-02T00:29:38.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.325+0000 7f79f609b640 1 -- 172.21.15.46:0/2111592819 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79f0133b90 con 0x7f79f00ceb70 2023-12-02T00:29:38.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.325+0000 7f79d7fff640 1 -- 172.21.15.46:0/2111592819 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f79e40129d0 con 0x7f79f00ceb70 2023-12-02T00:29:38.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.326+0000 7f79d7fff640 1 --2- 172.21.15.46:0/2111592819 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f79dc06ae70 0x7f79dc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:38.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.326+0000 7f79d7fff640 1 -- 172.21.15.46:0/2111592819 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f79e4088f40 con 0x7f79f00ceb70 2023-12-02T00:29:38.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.326+0000 7f79ef7fe640 1 --2- 172.21.15.46:0/2111592819 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f79dc06ae70 0x7f79dc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:38.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.326+0000 7f79ef7fe640 1 --2- 172.21.15.46:0/2111592819 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f79dc06ae70 0x7f79dc06d330 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f79e0036000 tx=0x7f79e0008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:38.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.330+0000 7f79d7fff640 1 -- 172.21.15.46:0/2111592819 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f79f0133b90 con 0x7f79f00ceb70 2023-12-02T00:29:38.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.507+0000 7f79f609b640 1 -- 172.21.15.46:0/2111592819 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79f0133b90 con 0x7f79f00ceb70 2023-12-02T00:29:38.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.508+0000 7f79d7fff640 1 -- 172.21.15.46:0/2111592819 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16777 (secure 0 0 0) 0x7f79f0133b90 con 0x7f79f00ceb70 2023-12-02T00:29:38.510 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:29:38.510 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":23,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:29:30.868939+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-02T00:28:47.273337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-02T00:28:48.060115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.14","mappings":[{"from":4,"to":2}]},{"pgid":"3.1e","mappings":[{"from":3,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:29:38.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.510+0000 7f79f609b640 1 -- 172.21.15.46:0/2111592819 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f79dc06ae70 msgr2=0x7f79dc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:38.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.510+0000 7f79f609b640 1 --2- 172.21.15.46:0/2111592819 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f79dc06ae70 0x7f79dc06d330 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f79e0036000 tx=0x7f79e0008030 comp rx=0 tx=0).stop 2023-12-02T00:29:38.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.510+0000 7f79f609b640 1 -- 172.21.15.46:0/2111592819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79f00ceb70 msgr2=0x7f79f00b0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:38.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.510+0000 7f79f609b640 1 --2- 172.21.15.46:0/2111592819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79f00ceb70 0x7f79f00b0730 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f79e400c450 tx=0x7f79e400c910 comp rx=0 tx=0).stop 2023-12-02T00:29:38.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.510+0000 7f79f609b640 1 -- 172.21.15.46:0/2111592819 shutdown_connections 2023-12-02T00:29:38.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.510+0000 7f79f609b640 1 --2- 172.21.15.46:0/2111592819 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f79dc06ae70 0x7f79dc06d330 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:38.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.510+0000 7f79f609b640 1 --2- 172.21.15.46:0/2111592819 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79f0144980 0x7f79f00ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:38.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.510+0000 7f79f609b640 1 --2- 172.21.15.46:0/2111592819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79f00ceb70 0x7f79f00b0730 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:38.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.510+0000 7f79f609b640 1 --2- 172.21.15.46:0/2111592819 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79f00b4370 0x7f79f0155510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:38.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.510+0000 7f79f609b640 1 -- 172.21.15.46:0/2111592819 >> 172.21.15.46:0/2111592819 conn(0x7f79f006a4d0 msgr2=0x7f79f0157cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:38.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.510+0000 7f79f609b640 1 -- 172.21.15.46:0/2111592819 shutdown_connections 2023-12-02T00:29:38.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.510+0000 7f79f609b640 1 -- 172.21.15.46:0/2111592819 wait complete. 2023-12-02T00:29:38.522 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-02T00:28:37.213645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-02T00:28:47.273337+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-02T00:28:48.060115+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-02T00:29:38.523 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-02T00:29:38.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.631+0000 7f4bf7b68640 1 Processor -- start 2023-12-02T00:29:38.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.631+0000 7f4bf7b68640 1 -- start start 2023-12-02T00:29:38.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.632+0000 7f4bf7b68640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4bf006ea50 0x7f4bf006ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:38.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.632+0000 7f4bf7b68640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4bf0160fb0 0x7f4bf0161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:38.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.632+0000 7f4bf7b68640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4bf00b3ff0 0x7f4bf00b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:38.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.632+0000 7f4bf7b68640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bf0161d20 con 0x7f4bf0160fb0 2023-12-02T00:29:38.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.632+0000 7f4bf7b68640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bf006f370 con 0x7f4bf00b3ff0 2023-12-02T00:29:38.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.632+0000 7f4bf7b68640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf015e760 con 0x7f4bf006ea50 2023-12-02T00:29:38.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.632+0000 7f4bf5103640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4bf0160fb0 0x7f4bf0161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:38.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.632+0000 7f4bf5103640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4bf0160fb0 0x7f4bf0161410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57380/0 (socket says 172.21.15.46:57380) 2023-12-02T00:29:38.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.632+0000 7f4bf5103640 1 -- 172.21.15.46:0/642592425 learned_addr learned my addr 172.21.15.46:0/642592425 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:38.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.632+0000 7f4bf6105640 1 --2- 172.21.15.46:0/642592425 >> v2:172.21.15.162:3300/0 conn(0x7f4bf00b3ff0 0x7f4bf00b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:38.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.632+0000 7f4bf5904640 1 --2- 172.21.15.46:0/642592425 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4bf006ea50 0x7f4bf006ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:38.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.632+0000 7f4bf5103640 1 -- 172.21.15.46:0/642592425 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4bf006ea50 msgr2=0x7f4bf006ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:38.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.632+0000 7f4bf5103640 1 --2- 172.21.15.46:0/642592425 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4bf006ea50 0x7f4bf006ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:38.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.632+0000 7f4bf5103640 1 -- 172.21.15.46:0/642592425 >> v2:172.21.15.162:3300/0 conn(0x7f4bf00b3ff0 msgr2=0x7f4bf00b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:38.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.632+0000 7f4bf5103640 1 --2- 172.21.15.46:0/642592425 >> v2:172.21.15.162:3300/0 conn(0x7f4bf00b3ff0 0x7f4bf00b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:38.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.632+0000 7f4bf5103640 1 -- 172.21.15.46:0/642592425 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf015d9a0 con 0x7f4bf0160fb0 2023-12-02T00:29:38.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.632+0000 7f4bf5904640 1 --2- 172.21.15.46:0/642592425 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4bf006ea50 0x7f4bf006ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:29:38.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.633+0000 7f4bf5103640 1 --2- 172.21.15.46:0/642592425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bf0160fb0 0x7f4bf0161410 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4bf00b66c0 tx=0x7f4be402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce5fea3b3c994a3b server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:38.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.633+0000 7f4bf4902640 1 -- 172.21.15.46:0/642592425 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be40317e0 con 0x7f4bf0160fb0 2023-12-02T00:29:38.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.633+0000 7f4bf4902640 1 -- 172.21.15.46:0/642592425 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be4031940 con 0x7f4bf0160fb0 2023-12-02T00:29:38.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.633+0000 7f4bf4902640 1 -- 172.21.15.46:0/642592425 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be4031cd0 con 0x7f4bf0160fb0 2023-12-02T00:29:38.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.633+0000 7f4bf7b68640 1 -- 172.21.15.46:0/642592425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bf0160fb0 msgr2=0x7f4bf0161410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:38.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.633+0000 7f4bf7b68640 1 --2- 172.21.15.46:0/642592425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bf0160fb0 0x7f4bf0161410 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4bf00b66c0 tx=0x7f4be402b880 comp rx=0 tx=0).stop 2023-12-02T00:29:38.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.633+0000 7f4bf7b68640 1 -- 172.21.15.46:0/642592425 shutdown_connections 2023-12-02T00:29:38.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.633+0000 7f4bf7b68640 1 --2- 172.21.15.46:0/642592425 >> v2:172.21.15.162:3300/0 conn(0x7f4bf00b3ff0 0x7f4bf00b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:38.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.633+0000 7f4bf7b68640 1 --2- 172.21.15.46:0/642592425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bf0160fb0 0x7f4bf0161410 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:38.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.633+0000 7f4bf7b68640 1 --2- 172.21.15.46:0/642592425 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4bf006ea50 0x7f4bf006ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:38.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.633+0000 7f4bf7b68640 1 -- 172.21.15.46:0/642592425 >> 172.21.15.46:0/642592425 conn(0x7f4bf006a6b0 msgr2=0x7f4bf006aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:38.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.634+0000 7f4bf7b68640 1 -- 172.21.15.46:0/642592425 shutdown_connections 2023-12-02T00:29:38.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.634+0000 7f4bf7b68640 1 -- 172.21.15.46:0/642592425 wait complete. 2023-12-02T00:29:38.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.635+0000 7f4bf7b68640 1 Processor -- start 2023-12-02T00:29:38.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.635+0000 7f4bf7b68640 1 -- start start 2023-12-02T00:29:38.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.635+0000 7f4bf7b68640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4bf006ea50 0x7f4bf015aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:38.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.635+0000 7f4bf7b68640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bf00b3ff0 0x7f4bf013f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:38.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.635+0000 7f4bf7b68640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4bf0160fb0 0x7f4bf015a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:38.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.635+0000 7f4bf7b68640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf01464c0 con 0x7f4bf00b3ff0 2023-12-02T00:29:38.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.635+0000 7f4bf7b68640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf0157f50 con 0x7f4bf0160fb0 2023-12-02T00:29:38.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.635+0000 7f4bf7b68640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf013f5f0 con 0x7f4bf006ea50 2023-12-02T00:29:38.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.636+0000 7f4bf5103640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bf00b3ff0 0x7f4bf013f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:38.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.636+0000 7f4bf5103640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bf00b3ff0 0x7f4bf013f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57384/0 (socket says 172.21.15.46:57384) 2023-12-02T00:29:38.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.636+0000 7f4bf5103640 1 -- 172.21.15.46:0/4290957720 learned_addr learned my addr 172.21.15.46:0/4290957720 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:38.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.636+0000 7f4bf6105640 1 --2- 172.21.15.46:0/4290957720 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4bf0160fb0 0x7f4bf015a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:38.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.636+0000 7f4bf5904640 1 --2- 172.21.15.46:0/4290957720 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4bf006ea50 0x7f4bf015aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:38.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.636+0000 7f4bf5103640 1 -- 172.21.15.46:0/4290957720 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4bf006ea50 msgr2=0x7f4bf015aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:38.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.636+0000 7f4bf5103640 1 --2- 172.21.15.46:0/4290957720 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4bf006ea50 0x7f4bf015aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:38.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.636+0000 7f4bf5103640 1 -- 172.21.15.46:0/4290957720 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4bf0160fb0 msgr2=0x7f4bf015a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:38.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.636+0000 7f4bf5103640 1 --2- 172.21.15.46:0/4290957720 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4bf0160fb0 0x7f4bf015a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:38.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.636+0000 7f4bf5103640 1 -- 172.21.15.46:0/4290957720 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be40094d0 con 0x7f4bf00b3ff0 2023-12-02T00:29:38.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.636+0000 7f4bf5904640 1 --2- 172.21.15.46:0/4290957720 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4bf006ea50 0x7f4bf015aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:38.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.636+0000 7f4bf5103640 1 --2- 172.21.15.46:0/4290957720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bf00b3ff0 0x7f4bf013f0b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4be40090b0 tx=0x7f4be40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:38.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.636+0000 7f4bee7fc640 1 -- 172.21.15.46:0/4290957720 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be4032880 con 0x7f4bf00b3ff0 2023-12-02T00:29:38.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.636+0000 7f4bee7fc640 1 -- 172.21.15.46:0/4290957720 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be40329e0 con 0x7f4bf00b3ff0 2023-12-02T00:29:38.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.636+0000 7f4bf7b68640 1 -- 172.21.15.46:0/4290957720 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bf015a5a0 con 0x7f4bf00b3ff0 2023-12-02T00:29:38.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.636+0000 7f4bee7fc640 1 -- 172.21.15.46:0/4290957720 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be4032d90 con 0x7f4bf00b3ff0 2023-12-02T00:29:38.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.636+0000 7f4bf7b68640 1 -- 172.21.15.46:0/4290957720 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bf01469f0 con 0x7f4bf00b3ff0 2023-12-02T00:29:38.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.637+0000 7f4bf7b68640 1 -- 172.21.15.46:0/4290957720 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bf00b6290 con 0x7f4bf00b3ff0 2023-12-02T00:29:38.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.638+0000 7f4bee7fc640 1 -- 172.21.15.46:0/4290957720 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4be404e450 con 0x7f4bf00b3ff0 2023-12-02T00:29:38.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.638+0000 7f4bee7fc640 1 --2- 172.21.15.46:0/4290957720 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f4bdc06ae70 0x7f4bdc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:38.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.638+0000 7f4bee7fc640 1 -- 172.21.15.46:0/4290957720 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f4be40ab750 con 0x7f4bf00b3ff0 2023-12-02T00:29:38.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.639+0000 7f4bf5904640 1 --2- 172.21.15.46:0/4290957720 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f4bdc06ae70 0x7f4bdc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:38.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.639+0000 7f4bf5904640 1 --2- 172.21.15.46:0/4290957720 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f4bdc06ae70 0x7f4bdc06d330 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4be800a990 tx=0x7f4be8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:38.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.642+0000 7f4bee7fc640 1 -- 172.21.15.46:0/4290957720 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4be40768b0 con 0x7f4bf00b3ff0 2023-12-02T00:29:38.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.817+0000 7f4bf7b68640 1 -- 172.21.15.46:0/4290957720 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4bf00d1d60 con 0x7f4bf00b3ff0 2023-12-02T00:29:38.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.817+0000 7f4bee7fc640 1 -- 172.21.15.46:0/4290957720 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4be4079ed0 con 0x7f4bf00b3ff0 2023-12-02T00:29:38.819 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-12-02T00:29:38.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.819+0000 7f4bf7b68640 1 -- 172.21.15.46:0/4290957720 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f4bdc06ae70 msgr2=0x7f4bdc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:38.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.819+0000 7f4bf7b68640 1 --2- 172.21.15.46:0/4290957720 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f4bdc06ae70 0x7f4bdc06d330 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4be800a990 tx=0x7f4be8008040 comp rx=0 tx=0).stop 2023-12-02T00:29:38.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.819+0000 7f4bf7b68640 1 -- 172.21.15.46:0/4290957720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bf00b3ff0 msgr2=0x7f4bf013f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:38.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.819+0000 7f4bf7b68640 1 --2- 172.21.15.46:0/4290957720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bf00b3ff0 0x7f4bf013f0b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4be40090b0 tx=0x7f4be40046c0 comp rx=0 tx=0).stop 2023-12-02T00:29:38.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.819+0000 7f4bf7b68640 1 -- 172.21.15.46:0/4290957720 shutdown_connections 2023-12-02T00:29:38.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.819+0000 7f4bf7b68640 1 --2- 172.21.15.46:0/4290957720 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f4bdc06ae70 0x7f4bdc06d330 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:38.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.819+0000 7f4bf7b68640 1 --2- 172.21.15.46:0/4290957720 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4bf0160fb0 0x7f4bf015a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:38.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.819+0000 7f4bf7b68640 1 --2- 172.21.15.46:0/4290957720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bf00b3ff0 0x7f4bf013f0b0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:38.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.819+0000 7f4bf7b68640 1 --2- 172.21.15.46:0/4290957720 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4bf006ea50 0x7f4bf015aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:38.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.819+0000 7f4bf7b68640 1 -- 172.21.15.46:0/4290957720 >> 172.21.15.46:0/4290957720 conn(0x7f4bf006a6b0 msgr2=0x7f4bf0140d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:38.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.819+0000 7f4bf7b68640 1 -- 172.21.15.46:0/4290957720 shutdown_connections 2023-12-02T00:29:38.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.819+0000 7f4bf7b68640 1 -- 172.21.15.46:0/4290957720 wait complete. 2023-12-02T00:29:38.831 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-02T00:29:38.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.940+0000 7f95ead83640 1 Processor -- start 2023-12-02T00:29:38.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.940+0000 7f95ead83640 1 -- start start 2023-12-02T00:29:38.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.941+0000 7f95ead83640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f95e4160f70 0x7f95e40cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:38.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.941+0000 7f95ead83640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95e41614e0 con 0x7f95e415d2d0 2023-12-02T00:29:38.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.941+0000 7f95ead83640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95e41617c0 con 0x7f95e40ce480 2023-12-02T00:29:38.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.941+0000 7f95ead83640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95e4161aa0 con 0x7f95e4160f70 2023-12-02T00:29:38.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.941+0000 7f95e8b1f640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f95e415d2d0 0x7f95e40b8b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59362/0 (socket says 172.21.15.46:59362) 2023-12-02T00:29:38.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.941+0000 7f95e8b1f640 1 -- 172.21.15.46:0/3886343894 learned_addr learned my addr 172.21.15.46:0/3886343894 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:38.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.941+0000 7f95e3fff640 1 --2- 172.21.15.46:0/3886343894 >> v2:172.21.15.162:3300/0 conn(0x7f95e4160f70 0x7f95e40cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:38.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.941+0000 7f95e37fe640 1 -- 172.21.15.46:0/3886343894 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3893255162 0 0) 0x7f95e41614e0 con 0x7f95e415d2d0 2023-12-02T00:29:38.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.942+0000 7f95e37fe640 1 -- 172.21.15.46:0/3886343894 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95d0003520 con 0x7f95e415d2d0 2023-12-02T00:29:38.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.942+0000 7f95e37fe640 1 -- 172.21.15.46:0/3886343894 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3828872019 0 0) 0x7f95e41617c0 con 0x7f95e40ce480 2023-12-02T00:29:38.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.942+0000 7f95e37fe640 1 -- 172.21.15.46:0/3886343894 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95e41614e0 con 0x7f95e40ce480 2023-12-02T00:29:38.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.942+0000 7f95e37fe640 1 -- 172.21.15.46:0/3886343894 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3800426595 0 0) 0x7f95d0003520 con 0x7f95e415d2d0 2023-12-02T00:29:38.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.942+0000 7f95e37fe640 1 -- 172.21.15.46:0/3886343894 >> v2:172.21.15.162:3300/0 conn(0x7f95e4160f70 msgr2=0x7f95e40cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:38.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.942+0000 7f95e37fe640 1 --2- 172.21.15.46:0/3886343894 >> v2:172.21.15.162:3300/0 conn(0x7f95e4160f70 0x7f95e40cdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:38.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.942+0000 7f95e37fe640 1 -- 172.21.15.46:0/3886343894 >> v1:172.21.15.162:6789/0 conn(0x7f95e40ce480 legacy=0x7f95e40d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:38.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.942+0000 7f95e37fe640 1 -- 172.21.15.46:0/3886343894 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e4161d80 con 0x7f95e415d2d0 2023-12-02T00:29:38.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.942+0000 7f95e37fe640 1 -- 172.21.15.46:0/3886343894 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7f95dc0019f0 con 0x7f95e415d2d0 2023-12-02T00:29:38.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.942+0000 7f95e3fff640 1 --2- 172.21.15.46:0/3886343894 >> v2:172.21.15.162:3300/0 conn(0x7f95e4160f70 0x7f95e40cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:29:38.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.942+0000 7f95e37fe640 1 -- 172.21.15.46:0/3886343894 >> v1:172.21.15.46:6789/0 conn(0x7f95e415d2d0 legacy=0x7f95e40b8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:38.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.942+0000 7f95e37fe640 1 --2- 172.21.15.46:0/3886343894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95d0003b50 0x7f95d0003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:38.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.942+0000 7f95e37fe640 1 --2- 172.21.15.46:0/3886343894 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95d00047c0 0x7f95d0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:38.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.942+0000 7f95e37fe640 1 --2- 172.21.15.46:0/3886343894 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95d0005180 0x7f95d000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:38.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.942+0000 7f95e37fe640 1 -- 172.21.15.46:0/3886343894 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e4161aa0 con 0x7f95d0003b50 2023-12-02T00:29:38.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.942+0000 7f95e37fe640 1 -- 172.21.15.46:0/3886343894 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d000c050 con 0x7f95d00047c0 2023-12-02T00:29:38.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.942+0000 7f95e37fe640 1 -- 172.21.15.46:0/3886343894 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95d000c210 con 0x7f95d0005180 2023-12-02T00:29:38.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.942+0000 7f95e8b1f640 1 --2- 172.21.15.46:0/3886343894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95d0003b50 0x7f95d0003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:38.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.942+0000 7f95e3fff640 1 --2- 172.21.15.46:0/3886343894 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95d00047c0 0x7f95d0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:38.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.942+0000 7f95e9320640 1 --2- 172.21.15.46:0/3886343894 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95d0005180 0x7f95d000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:38.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.942+0000 7f95e8b1f640 1 -- 172.21.15.46:0/3886343894 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95d0005180 msgr2=0x7f95d000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:38.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.942+0000 7f95e8b1f640 1 --2- 172.21.15.46:0/3886343894 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95d0005180 0x7f95d000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:38.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.942+0000 7f95e8b1f640 1 -- 172.21.15.46:0/3886343894 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95d00047c0 msgr2=0x7f95d0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:38.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.942+0000 7f95e8b1f640 1 --2- 172.21.15.46:0/3886343894 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95d00047c0 0x7f95d0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.942+0000 7f95e8b1f640 1 -- 172.21.15.46:0/3886343894 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d0003520 con 0x7f95d0003b50 2023-12-02T00:29:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.942+0000 7f95e9320640 1 --2- 172.21.15.46:0/3886343894 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95d0005180 0x7f95d000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.942+0000 7f95e9320640 1 -- 172.21.15.46:0/3886343894 reap_dead start 2023-12-02T00:29:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.943+0000 7f95e8b1f640 1 --2- 172.21.15.46:0/3886343894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95d0003b50 0x7f95d0003f50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f95dc00a710 tx=0x7f95d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3538f49687a070e5 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.943+0000 7f95e37fe640 1 -- 172.21.15.46:0/3886343894 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95dc034630 con 0x7f95d0003b50 2023-12-02T00:29:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.943+0000 7f95e37fe640 1 -- 172.21.15.46:0/3886343894 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95dc02f070 con 0x7f95d0003b50 2023-12-02T00:29:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.943+0000 7f95e37fe640 1 -- 172.21.15.46:0/3886343894 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95dc034a30 con 0x7f95d0003b50 2023-12-02T00:29:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.944+0000 7f95ead83640 1 -- 172.21.15.46:0/3886343894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95d0003b50 msgr2=0x7f95d0003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.944+0000 7f95ead83640 1 --2- 172.21.15.46:0/3886343894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95d0003b50 0x7f95d0003f50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f95dc00a710 tx=0x7f95d00057f0 comp rx=0 tx=0).stop 2023-12-02T00:29:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.944+0000 7f95ead83640 1 -- 172.21.15.46:0/3886343894 shutdown_connections 2023-12-02T00:29:38.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.944+0000 7f95ead83640 1 --2- 172.21.15.46:0/3886343894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95d0003b50 0x7f95d0003f50 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:38.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.944+0000 7f95ead83640 1 -- 172.21.15.46:0/3886343894 >> 172.21.15.46:0/3886343894 conn(0x7f95e4072230 msgr2=0x7f95e40cfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:38.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.944+0000 7f95ead83640 1 -- 172.21.15.46:0/3886343894 shutdown_connections 2023-12-02T00:29:38.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.944+0000 7f95ead83640 1 -- 172.21.15.46:0/3886343894 wait complete. 2023-12-02T00:29:38.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.945+0000 7f95ead83640 1 Processor -- start 2023-12-02T00:29:38.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.945+0000 7f95ead83640 1 -- start start 2023-12-02T00:29:38.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.946+0000 7f95ead83640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95d0003b50 0x7f95e4150e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:38.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.946+0000 7f95ead83640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95e41535d0 0x7f95e4153a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:38.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.946+0000 7f95ead83640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95e40b46a0 0x7f95e40b4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:38.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.946+0000 7f95ead83640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e40b5060 con 0x7f95d0003b50 2023-12-02T00:29:38.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.946+0000 7f95ead83640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e41513b0 con 0x7f95e40b46a0 2023-12-02T00:29:38.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.946+0000 7f95ead83640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95e4152530 con 0x7f95e41535d0 2023-12-02T00:29:38.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.946+0000 7f95e8b1f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95d0003b50 0x7f95e4150e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:38.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.946+0000 7f95e8b1f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95d0003b50 0x7f95e4150e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57406/0 (socket says 172.21.15.46:57406) 2023-12-02T00:29:38.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.946+0000 7f95e8b1f640 1 -- 172.21.15.46:0/2403691761 learned_addr learned my addr 172.21.15.46:0/2403691761 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:38.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.946+0000 7f95e9320640 1 --2- 172.21.15.46:0/2403691761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95e40b46a0 0x7f95e40b4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:38.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.946+0000 7f95e3fff640 1 --2- 172.21.15.46:0/2403691761 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95e41535d0 0x7f95e4153a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:38.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.946+0000 7f95e8b1f640 1 -- 172.21.15.46:0/2403691761 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95e41535d0 msgr2=0x7f95e4153a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:38.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.946+0000 7f95e8b1f640 1 --2- 172.21.15.46:0/2403691761 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95e41535d0 0x7f95e4153a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:38.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.946+0000 7f95e8b1f640 1 -- 172.21.15.46:0/2403691761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95e40b46a0 msgr2=0x7f95e40b4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:38.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.946+0000 7f95e8b1f640 1 --2- 172.21.15.46:0/2403691761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95e40b46a0 0x7f95e40b4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:38.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.946+0000 7f95e8b1f640 1 -- 172.21.15.46:0/2403691761 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e4152810 con 0x7f95d0003b50 2023-12-02T00:29:38.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.946+0000 7f95e9320640 1 --2- 172.21.15.46:0/2403691761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95e40b46a0 0x7f95e40b4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:38.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.947+0000 7f95e8b1f640 1 --2- 172.21.15.46:0/2403691761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95d0003b50 0x7f95e4150e70 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f95dc02ca60 tx=0x7f95e41616b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:38.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.947+0000 7f95e17fa640 1 -- 172.21.15.46:0/2403691761 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95dc034900 con 0x7f95d0003b50 2023-12-02T00:29:38.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.947+0000 7f95e17fa640 1 -- 172.21.15.46:0/2403691761 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95dc034a60 con 0x7f95d0003b50 2023-12-02T00:29:38.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.947+0000 7f95e17fa640 1 -- 172.21.15.46:0/2403691761 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95dc034dd0 con 0x7f95d0003b50 2023-12-02T00:29:38.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.947+0000 7f95ead83640 1 -- 172.21.15.46:0/2403691761 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95e4152af0 con 0x7f95d0003b50 2023-12-02T00:29:38.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.947+0000 7f95ead83640 1 -- 172.21.15.46:0/2403691761 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95e41530b0 con 0x7f95d0003b50 2023-12-02T00:29:38.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.948+0000 7f95ead83640 1 -- 172.21.15.46:0/2403691761 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95e4153240 con 0x7f95d0003b50 2023-12-02T00:29:38.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.948+0000 7f95e17fa640 1 -- 172.21.15.46:0/2403691761 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f95dc031030 con 0x7f95d0003b50 2023-12-02T00:29:38.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.949+0000 7f95e17fa640 1 --2- 172.21.15.46:0/2403691761 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f95c406ae40 0x7f95c406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:38.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.949+0000 7f95e17fa640 1 -- 172.21.15.46:0/2403691761 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f95dc0abce0 con 0x7f95d0003b50 2023-12-02T00:29:38.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.949+0000 7f95e3fff640 1 --2- 172.21.15.46:0/2403691761 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f95c406ae40 0x7f95c406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:38.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.950+0000 7f95e3fff640 1 --2- 172.21.15.46:0/2403691761 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f95c406ae40 0x7f95c406d300 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f95e41f6940 tx=0x7f95d4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:38.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:38.953+0000 7f95e17fa640 1 -- 172.21.15.46:0/2403691761 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f95dc076d90 con 0x7f95d0003b50 2023-12-02T00:29:39.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.127+0000 7f95ead83640 1 -- 172.21.15.46:0/2403691761 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f95e4153240 con 0x7f95d0003b50 2023-12-02T00:29:39.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.128+0000 7f95e17fa640 1 -- 172.21.15.46:0/2403691761 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f95dc032e60 con 0x7f95d0003b50 2023-12-02T00:29:39.129 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-12-02T00:29:39.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.129+0000 7f95ead83640 1 -- 172.21.15.46:0/2403691761 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f95c406ae40 msgr2=0x7f95c406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:39.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.129+0000 7f95ead83640 1 --2- 172.21.15.46:0/2403691761 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f95c406ae40 0x7f95c406d300 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f95e41f6940 tx=0x7f95d4008040 comp rx=0 tx=0).stop 2023-12-02T00:29:39.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.129+0000 7f95ead83640 1 -- 172.21.15.46:0/2403691761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95d0003b50 msgr2=0x7f95e4150e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:39.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.129+0000 7f95ead83640 1 --2- 172.21.15.46:0/2403691761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95d0003b50 0x7f95e4150e70 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f95dc02ca60 tx=0x7f95e41616b0 comp rx=0 tx=0).stop 2023-12-02T00:29:39.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.130+0000 7f95ead83640 1 -- 172.21.15.46:0/2403691761 shutdown_connections 2023-12-02T00:29:39.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.130+0000 7f95ead83640 1 --2- 172.21.15.46:0/2403691761 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f95c406ae40 0x7f95c406d300 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:39.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.130+0000 7f95ead83640 1 --2- 172.21.15.46:0/2403691761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95e40b46a0 0x7f95e40b4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:39.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.130+0000 7f95ead83640 1 --2- 172.21.15.46:0/2403691761 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95e41535d0 0x7f95e4153a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:39.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.130+0000 7f95ead83640 1 --2- 172.21.15.46:0/2403691761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95d0003b50 0x7f95e4150e70 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:39.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.130+0000 7f95ead83640 1 -- 172.21.15.46:0/2403691761 >> 172.21.15.46:0/2403691761 conn(0x7f95e4072230 msgr2=0x7f95e415da10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:39.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.130+0000 7f95ead83640 1 -- 172.21.15.46:0/2403691761 shutdown_connections 2023-12-02T00:29:39.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.130+0000 7f95ead83640 1 -- 172.21.15.46:0/2403691761 wait complete. 2023-12-02T00:29:39.142 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-02T00:29:39.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.251+0000 7f8e67ed7640 1 Processor -- start 2023-12-02T00:29:39.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.251+0000 7f8e67ed7640 1 -- start start 2023-12-02T00:29:39.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.252+0000 7f8e67ed7640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8e600d2910 0x7f8e600d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:39.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.252+0000 7f8e67ed7640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e600b2b10 con 0x7f8e600d3730 2023-12-02T00:29:39.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.252+0000 7f8e67ed7640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e600d1590 con 0x7f8e6015d330 2023-12-02T00:29:39.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.252+0000 7f8e67ed7640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e600d3bc0 con 0x7f8e600d2910 2023-12-02T00:29:39.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.252+0000 7f8e66474640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8e600d3730 0x7f8e600b6390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59374/0 (socket says 172.21.15.46:59374) 2023-12-02T00:29:39.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.252+0000 7f8e66474640 1 -- 172.21.15.46:0/1001284419 learned_addr learned my addr 172.21.15.46:0/1001284419 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:39.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.252+0000 7f8e65472640 1 --2- 172.21.15.46:0/1001284419 >> v2:172.21.15.162:3300/0 conn(0x7f8e600d2910 0x7f8e600d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:39.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.252+0000 7f8e64c71640 1 -- 172.21.15.46:0/1001284419 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3615929171 0 0) 0x7f8e600b2b10 con 0x7f8e600d3730 2023-12-02T00:29:39.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.252+0000 7f8e64c71640 1 -- 172.21.15.46:0/1001284419 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e4c003520 con 0x7f8e600d3730 2023-12-02T00:29:39.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.252+0000 7f8e64c71640 1 -- 172.21.15.46:0/1001284419 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 340466119 0 0) 0x7f8e4c003520 con 0x7f8e600d3730 2023-12-02T00:29:39.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.252+0000 7f8e64c71640 1 -- 172.21.15.46:0/1001284419 >> v2:172.21.15.162:3300/0 conn(0x7f8e600d2910 msgr2=0x7f8e600d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:39.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.252+0000 7f8e64c71640 1 --2- 172.21.15.46:0/1001284419 >> v2:172.21.15.162:3300/0 conn(0x7f8e600d2910 0x7f8e600d31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:39.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.252+0000 7f8e64c71640 1 -- 172.21.15.46:0/1001284419 >> v1:172.21.15.162:6789/0 conn(0x7f8e6015d330 legacy=0x7f8e600b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:39.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.252+0000 7f8e64c71640 1 -- 172.21.15.46:0/1001284419 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e600d1870 con 0x7f8e600d3730 2023-12-02T00:29:39.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.252+0000 7f8e64c71640 1 -- 172.21.15.46:0/1001284419 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7f8e580019f0 con 0x7f8e600d3730 2023-12-02T00:29:39.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.252+0000 7f8e64c71640 1 -- 172.21.15.46:0/1001284419 >> v1:172.21.15.46:6789/0 conn(0x7f8e600d3730 legacy=0x7f8e600b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:39.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.253+0000 7f8e64c71640 1 --2- 172.21.15.46:0/1001284419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e4c003970 0x7f8e4c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:39.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.253+0000 7f8e64c71640 1 --2- 172.21.15.46:0/1001284419 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e4c0047c0 0x7f8e4c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:39.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.253+0000 7f8e64c71640 1 --2- 172.21.15.46:0/1001284419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e4c005180 0x7f8e4c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:39.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.253+0000 7f8e64c71640 1 -- 172.21.15.46:0/1001284419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e600d3bc0 con 0x7f8e4c003970 2023-12-02T00:29:39.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.253+0000 7f8e64c71640 1 -- 172.21.15.46:0/1001284419 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c00c120 con 0x7f8e4c005180 2023-12-02T00:29:39.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.253+0000 7f8e64c71640 1 -- 172.21.15.46:0/1001284419 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c00c2e0 con 0x7f8e4c0047c0 2023-12-02T00:29:39.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.253+0000 7f8e65c73640 1 --2- 172.21.15.46:0/1001284419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e4c003970 0x7f8e4c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:39.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.253+0000 7f8e65472640 1 --2- 172.21.15.46:0/1001284419 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e4c0047c0 0x7f8e4c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:39.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.253+0000 7f8e66474640 1 --2- 172.21.15.46:0/1001284419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e4c005180 0x7f8e4c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:39.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.253+0000 7f8e65c73640 1 -- 172.21.15.46:0/1001284419 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e4c0047c0 msgr2=0x7f8e4c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:39.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.253+0000 7f8e65c73640 1 --2- 172.21.15.46:0/1001284419 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e4c0047c0 0x7f8e4c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:39.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.253+0000 7f8e65c73640 1 -- 172.21.15.46:0/1001284419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e4c005180 msgr2=0x7f8e4c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:39.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.253+0000 7f8e65c73640 1 --2- 172.21.15.46:0/1001284419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e4c005180 0x7f8e4c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:39.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.253+0000 7f8e65c73640 1 -- 172.21.15.46:0/1001284419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e4c003520 con 0x7f8e4c003970 2023-12-02T00:29:39.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.253+0000 7f8e66474640 1 -- 172.21.15.46:0/1001284419 reap_dead start 2023-12-02T00:29:39.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.254+0000 7f8e65c73640 1 --2- 172.21.15.46:0/1001284419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e4c003970 0x7f8e4c003d90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c001750 tx=0x7f8e4c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf78674515a98e3f server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:39.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.254+0000 7f8e64c71640 1 -- 172.21.15.46:0/1001284419 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c0052d0 con 0x7f8e4c003970 2023-12-02T00:29:39.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.254+0000 7f8e64c71640 1 -- 172.21.15.46:0/1001284419 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e5c02e050 con 0x7f8e4c003970 2023-12-02T00:29:39.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.254+0000 7f8e64c71640 1 -- 172.21.15.46:0/1001284419 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c003830 con 0x7f8e4c003970 2023-12-02T00:29:39.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.254+0000 7f8e67ed7640 1 -- 172.21.15.46:0/1001284419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e4c003970 msgr2=0x7f8e4c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:39.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.254+0000 7f8e67ed7640 1 --2- 172.21.15.46:0/1001284419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e4c003970 0x7f8e4c003d90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c001750 tx=0x7f8e4c0057f0 comp rx=0 tx=0).stop 2023-12-02T00:29:39.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.254+0000 7f8e67ed7640 1 -- 172.21.15.46:0/1001284419 shutdown_connections 2023-12-02T00:29:39.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.254+0000 7f8e67ed7640 1 --2- 172.21.15.46:0/1001284419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e4c003970 0x7f8e4c003d90 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:39.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.254+0000 7f8e67ed7640 1 -- 172.21.15.46:0/1001284419 >> 172.21.15.46:0/1001284419 conn(0x7f8e60072020 msgr2=0x7f8e60072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:39.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.254+0000 7f8e67ed7640 1 -- 172.21.15.46:0/1001284419 shutdown_connections 2023-12-02T00:29:39.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.255+0000 7f8e67ed7640 1 -- 172.21.15.46:0/1001284419 wait complete. 2023-12-02T00:29:39.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.256+0000 7f8e67ed7640 1 Processor -- start 2023-12-02T00:29:39.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.256+0000 7f8e67ed7640 1 -- start start 2023-12-02T00:29:39.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.256+0000 7f8e67ed7640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e4c003970 0x7f8e60153480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:39.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.256+0000 7f8e67ed7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e60146100 0x7f8e601456f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:39.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.256+0000 7f8e67ed7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e600c58d0 0x7f8e600c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:39.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.256+0000 7f8e67ed7640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e601539c0 con 0x7f8e600c58d0 2023-12-02T00:29:39.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.256+0000 7f8e67ed7640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e600c6290 con 0x7f8e4c003970 2023-12-02T00:29:39.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.256+0000 7f8e67ed7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e601558f0 con 0x7f8e60146100 2023-12-02T00:29:39.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.257+0000 7f8e66474640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e600c58d0 0x7f8e600c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:39.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.257+0000 7f8e66474640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e600c58d0 0x7f8e600c5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57434/0 (socket says 172.21.15.46:57434) 2023-12-02T00:29:39.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.257+0000 7f8e66474640 1 -- 172.21.15.46:0/3255909497 learned_addr learned my addr 172.21.15.46:0/3255909497 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:39.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.257+0000 7f8e65c73640 1 --2- 172.21.15.46:0/3255909497 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e4c003970 0x7f8e60153480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:39.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.257+0000 7f8e65472640 1 --2- 172.21.15.46:0/3255909497 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e60146100 0x7f8e601456f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:39.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.257+0000 7f8e66474640 1 -- 172.21.15.46:0/3255909497 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e60146100 msgr2=0x7f8e601456f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:39.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.257+0000 7f8e66474640 1 --2- 172.21.15.46:0/3255909497 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e60146100 0x7f8e601456f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:39.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.257+0000 7f8e66474640 1 -- 172.21.15.46:0/3255909497 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e4c003970 msgr2=0x7f8e60153480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:39.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.257+0000 7f8e66474640 1 --2- 172.21.15.46:0/3255909497 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e4c003970 0x7f8e60153480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:39.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.257+0000 7f8e66474640 1 -- 172.21.15.46:0/3255909497 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e60155bd0 con 0x7f8e600c58d0 2023-12-02T00:29:39.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.257+0000 7f8e65472640 1 --2- 172.21.15.46:0/3255909497 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e60146100 0x7f8e601456f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:39.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.257+0000 7f8e65c73640 1 --2- 172.21.15.46:0/3255909497 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e4c003970 0x7f8e60153480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:39.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.257+0000 7f8e66474640 1 --2- 172.21.15.46:0/3255909497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e600c58d0 0x7f8e600c5d50 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8e6008c510 tx=0x7f8e58007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:39.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.257+0000 7f8e567fc640 1 -- 172.21.15.46:0/3255909497 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e5800aaf0 con 0x7f8e600c58d0 2023-12-02T00:29:39.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.257+0000 7f8e567fc640 1 -- 172.21.15.46:0/3255909497 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e5800ac50 con 0x7f8e600c58d0 2023-12-02T00:29:39.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.257+0000 7f8e67ed7640 1 -- 172.21.15.46:0/3255909497 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e60155eb0 con 0x7f8e600c58d0 2023-12-02T00:29:39.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.257+0000 7f8e567fc640 1 -- 172.21.15.46:0/3255909497 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e580133f0 con 0x7f8e600c58d0 2023-12-02T00:29:39.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.257+0000 7f8e67ed7640 1 -- 172.21.15.46:0/3255909497 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e60156470 con 0x7f8e600c58d0 2023-12-02T00:29:39.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.258+0000 7f8e67ed7640 1 -- 172.21.15.46:0/3255909497 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e60069400 con 0x7f8e600c58d0 2023-12-02T00:29:39.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.259+0000 7f8e567fc640 1 -- 172.21.15.46:0/3255909497 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8e58013550 con 0x7f8e600c58d0 2023-12-02T00:29:39.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.259+0000 7f8e567fc640 1 --2- 172.21.15.46:0/3255909497 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8e4406ae70 0x7f8e4406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:39.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.259+0000 7f8e567fc640 1 -- 172.21.15.46:0/3255909497 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f8e58089cd0 con 0x7f8e600c58d0 2023-12-02T00:29:39.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.259+0000 7f8e65c73640 1 --2- 172.21.15.46:0/3255909497 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8e4406ae70 0x7f8e4406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:39.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.260+0000 7f8e65c73640 1 --2- 172.21.15.46:0/3255909497 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8e4406ae70 0x7f8e4406d330 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c033000 tx=0x7f8e5c009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:39.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.263+0000 7f8e567fc640 1 -- 172.21.15.46:0/3255909497 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8e58054cf0 con 0x7f8e600c58d0 2023-12-02T00:29:39.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.439+0000 7f8e67ed7640 1 -- 172.21.15.46:0/3255909497 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8e600c9230 con 0x7f8e600c58d0 2023-12-02T00:29:39.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.439+0000 7f8e567fc640 1 -- 172.21.15.46:0/3255909497 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8e58058310 con 0x7f8e600c58d0 2023-12-02T00:29:39.443 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-12-02T00:29:39.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.441+0000 7f8e67ed7640 1 -- 172.21.15.46:0/3255909497 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8e4406ae70 msgr2=0x7f8e4406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:39.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.441+0000 7f8e67ed7640 1 --2- 172.21.15.46:0/3255909497 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8e4406ae70 0x7f8e4406d330 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c033000 tx=0x7f8e5c009030 comp rx=0 tx=0).stop 2023-12-02T00:29:39.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.441+0000 7f8e67ed7640 1 -- 172.21.15.46:0/3255909497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e600c58d0 msgr2=0x7f8e600c5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:39.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.441+0000 7f8e67ed7640 1 --2- 172.21.15.46:0/3255909497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e600c58d0 0x7f8e600c5d50 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8e6008c510 tx=0x7f8e58007bd0 comp rx=0 tx=0).stop 2023-12-02T00:29:39.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.441+0000 7f8e67ed7640 1 -- 172.21.15.46:0/3255909497 shutdown_connections 2023-12-02T00:29:39.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.441+0000 7f8e67ed7640 1 --2- 172.21.15.46:0/3255909497 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8e4406ae70 0x7f8e4406d330 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:39.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.441+0000 7f8e67ed7640 1 --2- 172.21.15.46:0/3255909497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e600c58d0 0x7f8e600c5d50 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:39.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.441+0000 7f8e67ed7640 1 --2- 172.21.15.46:0/3255909497 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e60146100 0x7f8e601456f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:39.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.441+0000 7f8e67ed7640 1 --2- 172.21.15.46:0/3255909497 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e4c003970 0x7f8e60153480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:39.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.441+0000 7f8e67ed7640 1 -- 172.21.15.46:0/3255909497 >> 172.21.15.46:0/3255909497 conn(0x7f8e60072020 msgr2=0x7f8e600c85d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:39.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.441+0000 7f8e67ed7640 1 -- 172.21.15.46:0/3255909497 shutdown_connections 2023-12-02T00:29:39.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.441+0000 7f8e67ed7640 1 -- 172.21.15.46:0/3255909497 wait complete. 2023-12-02T00:29:39.455 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-02T00:29:39.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.561+0000 7f669e048640 1 Processor -- start 2023-12-02T00:29:39.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.561+0000 7f669e048640 1 -- start start 2023-12-02T00:29:39.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.562+0000 7f669e048640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f669815d270 0x7f669806ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:39.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.562+0000 7f669e048640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66980cdee0 0x7f66980cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:39.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.562+0000 7f669e048640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66980d0c40 con 0x7f66980709c0 2023-12-02T00:29:39.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.562+0000 7f669e048640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66980ced90 con 0x7f669815d270 2023-12-02T00:29:39.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.562+0000 7f669e048640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66980cf070 con 0x7f66980cdee0 2023-12-02T00:29:39.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.562+0000 7f66977fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f669815d270 0x7f669806ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:39.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.562+0000 7f66977fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f669815d270 0x7f669806ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57448/0 (socket says 172.21.15.46:57448) 2023-12-02T00:29:39.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.562+0000 7f66977fe640 1 -- 172.21.15.46:0/1894029974 learned_addr learned my addr 172.21.15.46:0/1894029974 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:39.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.563+0000 7f6697fff640 1 --2- 172.21.15.46:0/1894029974 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66980cdee0 0x7f66980cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:39.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.563+0000 7f66977fe640 1 -- 172.21.15.46:0/1894029974 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66980cdee0 msgr2=0x7f66980cc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:39.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.563+0000 7f66977fe640 1 --2- 172.21.15.46:0/1894029974 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66980cdee0 0x7f66980cc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:39.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.563+0000 7f66977fe640 1 -- 172.21.15.46:0/1894029974 >> v1:172.21.15.162:6789/0 conn(0x7f66980709c0 legacy=0x7f66980cd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:39.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.563+0000 7f66977fe640 1 -- 172.21.15.46:0/1894029974 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66980d08f0 con 0x7f669815d270 2023-12-02T00:29:39.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.563+0000 7f6697fff640 1 --2- 172.21.15.46:0/1894029974 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66980cdee0 0x7f66980cc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:29:39.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.563+0000 7f66977fe640 1 --2- 172.21.15.46:0/1894029974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f669815d270 0x7f669806ce70 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6688009870 tx=0x7f668802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ef715ac0383fb31 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:39.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.564+0000 7f66967fc640 1 -- 172.21.15.46:0/1894029974 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66880317c0 con 0x7f669815d270 2023-12-02T00:29:39.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.564+0000 7f66967fc640 1 -- 172.21.15.46:0/1894029974 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6688031920 con 0x7f669815d270 2023-12-02T00:29:39.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.564+0000 7f66967fc640 1 -- 172.21.15.46:0/1894029974 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6688031cb0 con 0x7f669815d270 2023-12-02T00:29:39.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.564+0000 7f669e048640 1 -- 172.21.15.46:0/1894029974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f669815d270 msgr2=0x7f669806ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:39.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.564+0000 7f669e048640 1 --2- 172.21.15.46:0/1894029974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f669815d270 0x7f669806ce70 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6688009870 tx=0x7f668802b860 comp rx=0 tx=0).stop 2023-12-02T00:29:39.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.564+0000 7f669e048640 1 -- 172.21.15.46:0/1894029974 shutdown_connections 2023-12-02T00:29:39.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.564+0000 7f669e048640 1 --2- 172.21.15.46:0/1894029974 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66980cdee0 0x7f66980cc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:39.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.564+0000 7f669e048640 1 --2- 172.21.15.46:0/1894029974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f669815d270 0x7f669806ce70 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:39.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.564+0000 7f669e048640 1 -- 172.21.15.46:0/1894029974 >> 172.21.15.46:0/1894029974 conn(0x7f66980b2c10 msgr2=0x7f66980b2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:39.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.564+0000 7f669e048640 1 -- 172.21.15.46:0/1894029974 shutdown_connections 2023-12-02T00:29:39.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.564+0000 7f669e048640 1 -- 172.21.15.46:0/1894029974 wait complete. 2023-12-02T00:29:39.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.565+0000 7f669e048640 1 Processor -- start 2023-12-02T00:29:39.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.565+0000 7f669e048640 1 -- start start 2023-12-02T00:29:39.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.566+0000 7f669e048640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66980709c0 0x7f66980c46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:39.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.566+0000 7f669e048640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66980cdee0 0x7f66980c4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:39.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.566+0000 7f669e048640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f669815d270 0x7f66980c6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:39.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.566+0000 7f669e048640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66980c7d70 con 0x7f66980709c0 2023-12-02T00:29:39.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.566+0000 7f669e048640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66980c74d0 con 0x7f66980cdee0 2023-12-02T00:29:39.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.566+0000 7f669e048640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66980c77b0 con 0x7f669815d270 2023-12-02T00:29:39.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.566+0000 7f66977fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66980709c0 0x7f66980c46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:39.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.566+0000 7f66977fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66980709c0 0x7f66980c46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57462/0 (socket says 172.21.15.46:57462) 2023-12-02T00:29:39.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.566+0000 7f66977fe640 1 -- 172.21.15.46:0/1787381009 learned_addr learned my addr 172.21.15.46:0/1787381009 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:39.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.567+0000 7f6696ffd640 1 --2- 172.21.15.46:0/1787381009 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66980cdee0 0x7f66980c4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:39.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.567+0000 7f6697fff640 1 --2- 172.21.15.46:0/1787381009 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f669815d270 0x7f66980c6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:39.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.567+0000 7f66977fe640 1 -- 172.21.15.46:0/1787381009 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f669815d270 msgr2=0x7f66980c6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:39.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.567+0000 7f66977fe640 1 --2- 172.21.15.46:0/1787381009 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f669815d270 0x7f66980c6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:39.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.567+0000 7f66977fe640 1 -- 172.21.15.46:0/1787381009 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66980cdee0 msgr2=0x7f66980c4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:39.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.567+0000 7f66977fe640 1 --2- 172.21.15.46:0/1787381009 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66980cdee0 0x7f66980c4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:39.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.567+0000 7f66977fe640 1 -- 172.21.15.46:0/1787381009 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66880094d0 con 0x7f66980709c0 2023-12-02T00:29:39.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.567+0000 7f6697fff640 1 --2- 172.21.15.46:0/1787381009 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f669815d270 0x7f66980c6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:29:39.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.567+0000 7f6696ffd640 1 --2- 172.21.15.46:0/1787381009 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66980cdee0 0x7f66980c4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:29:39.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.567+0000 7f66977fe640 1 --2- 172.21.15.46:0/1787381009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66980709c0 0x7f66980c46f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f668802b8a0 tx=0x7f66880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:39.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.567+0000 7f667ffff640 1 -- 172.21.15.46:0/1787381009 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66880329a0 con 0x7f66980709c0 2023-12-02T00:29:39.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.567+0000 7f667ffff640 1 -- 172.21.15.46:0/1787381009 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6688032b40 con 0x7f66980709c0 2023-12-02T00:29:39.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.567+0000 7f669e048640 1 -- 172.21.15.46:0/1787381009 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66980c7a90 con 0x7f66980709c0 2023-12-02T00:29:39.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.567+0000 7f667ffff640 1 -- 172.21.15.46:0/1787381009 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66880453f0 con 0x7f66980709c0 2023-12-02T00:29:39.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.567+0000 7f669e048640 1 -- 172.21.15.46:0/1787381009 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f669814f730 con 0x7f66980709c0 2023-12-02T00:29:39.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.568+0000 7f669e048640 1 -- 172.21.15.46:0/1787381009 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6698067fc0 con 0x7f66980709c0 2023-12-02T00:29:39.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.569+0000 7f667ffff640 1 -- 172.21.15.46:0/1787381009 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f668804e450 con 0x7f66980709c0 2023-12-02T00:29:39.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.569+0000 7f667ffff640 1 --2- 172.21.15.46:0/1787381009 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f668406ae40 0x7f668406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:39.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.569+0000 7f667ffff640 1 -- 172.21.15.46:0/1787381009 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f66880abac0 con 0x7f66980709c0 2023-12-02T00:29:39.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.569+0000 7f6696ffd640 1 --2- 172.21.15.46:0/1787381009 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f668406ae40 0x7f668406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:39.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.570+0000 7f6696ffd640 1 --2- 172.21.15.46:0/1787381009 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f668406ae40 0x7f668406d300 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f669806f6a0 tx=0x7f668c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:39.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.573+0000 7f667ffff640 1 -- 172.21.15.46:0/1787381009 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6688076c00 con 0x7f66980709c0 2023-12-02T00:29:39.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.784+0000 7f669e048640 1 -- 172.21.15.46:0/1787381009 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66980c9190 con 0x7f66980709c0 2023-12-02T00:29:39.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.784+0000 7f667ffff640 1 -- 172.21.15.46:0/1787381009 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4907 (secure 0 0 0) 0x7f66980c9190 con 0x7f66980709c0 2023-12-02T00:29:39.786 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:29:39.786 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/2059113401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2059113401},{"type":"v1","addr":"172.21.15.46:6834","nonce":2059113401}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/987532381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":987532381},{"type":"v1","addr":"172.21.15.46:6835","nonce":987532381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/449974862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":449974862},{"type":"v1","addr":"172.21.15.162:6836","nonce":449974862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-02T00:28:49.086548+0000","modified":"2023-12-02T00:28:51.082469+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4253":{"gid":4253,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/1231844478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":1231844478},{"type":"v1","addr":"172.21.15.162:6837","nonce":1231844478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-02T00:29:39.786 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 6 2023-12-02T00:29:39.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.786+0000 7f669e048640 1 -- 172.21.15.46:0/1787381009 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f668406ae40 msgr2=0x7f668406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:39.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.786+0000 7f669e048640 1 --2- 172.21.15.46:0/1787381009 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f668406ae40 0x7f668406d300 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f669806f6a0 tx=0x7f668c009040 comp rx=0 tx=0).stop 2023-12-02T00:29:39.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.786+0000 7f669e048640 1 -- 172.21.15.46:0/1787381009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66980709c0 msgr2=0x7f66980c46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:39.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.786+0000 7f669e048640 1 --2- 172.21.15.46:0/1787381009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66980709c0 0x7f66980c46f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f668802b8a0 tx=0x7f66880046c0 comp rx=0 tx=0).stop 2023-12-02T00:29:39.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.786+0000 7f669e048640 1 -- 172.21.15.46:0/1787381009 shutdown_connections 2023-12-02T00:29:39.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.786+0000 7f669e048640 1 --2- 172.21.15.46:0/1787381009 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f668406ae40 0x7f668406d300 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:39.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.786+0000 7f669e048640 1 --2- 172.21.15.46:0/1787381009 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f669815d270 0x7f66980c6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:39.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.786+0000 7f669e048640 1 --2- 172.21.15.46:0/1787381009 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66980cdee0 0x7f66980c4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:39.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.786+0000 7f669e048640 1 --2- 172.21.15.46:0/1787381009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66980709c0 0x7f66980c46f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:39.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.786+0000 7f669e048640 1 -- 172.21.15.46:0/1787381009 >> 172.21.15.46:0/1787381009 conn(0x7f66980b2c10 msgr2=0x7f66980b40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:39.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.786+0000 7f669e048640 1 -- 172.21.15.46:0/1787381009 shutdown_connections 2023-12-02T00:29:39.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.786+0000 7f669e048640 1 -- 172.21.15.46:0/1787381009 wait complete. 2023-12-02T00:29:39.799 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:29:39.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.904+0000 7f3e0d210640 1 Processor -- start 2023-12-02T00:29:39.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.904+0000 7f3e0d210640 1 -- start start 2023-12-02T00:29:39.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.905+0000 7f3e0d210640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e0815b130 0x7f3e080d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:39.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.905+0000 7f3e0d210640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3e0815f760 0x7f3e080cec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:39.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.905+0000 7f3e0d210640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3e080cf150 0x7f3e0806ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:39.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.905+0000 7f3e0d210640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e0815fd20 con 0x7f3e0815f760 2023-12-02T00:29:39.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.905+0000 7f3e0d210640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e080d0730 con 0x7f3e080cf150 2023-12-02T00:29:39.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.905+0000 7f3e0d210640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e080cf5c0 con 0x7f3e0815b130 2023-12-02T00:29:39.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.905+0000 7f3e0659c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3e0815f760 0x7f3e080cec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:39.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.905+0000 7f3e0659c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3e0815f760 0x7f3e080cec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57466/0 (socket says 172.21.15.46:57466) 2023-12-02T00:29:39.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.905+0000 7f3e0659c640 1 -- 172.21.15.46:0/2411461888 learned_addr learned my addr 172.21.15.46:0/2411461888 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:39.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.905+0000 7f3e0759e640 1 --2- 172.21.15.46:0/2411461888 >> v2:172.21.15.162:3300/0 conn(0x7f3e080cf150 0x7f3e0806ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:39.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.905+0000 7f3e06d9d640 1 --2- 172.21.15.46:0/2411461888 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e0815b130 0x7f3e080d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:39.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.906+0000 7f3e0659c640 1 -- 172.21.15.46:0/2411461888 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e0815b130 msgr2=0x7f3e080d01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:39.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.906+0000 7f3e0659c640 1 --2- 172.21.15.46:0/2411461888 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e0815b130 0x7f3e080d01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:39.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.906+0000 7f3e0659c640 1 -- 172.21.15.46:0/2411461888 >> v2:172.21.15.162:3300/0 conn(0x7f3e080cf150 msgr2=0x7f3e0806ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:39.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.906+0000 7f3e0659c640 1 --2- 172.21.15.46:0/2411461888 >> v2:172.21.15.162:3300/0 conn(0x7f3e080cf150 0x7f3e0806ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:39.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.906+0000 7f3e0659c640 1 -- 172.21.15.46:0/2411461888 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e080b3f10 con 0x7f3e0815f760 2023-12-02T00:29:39.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.906+0000 7f3e0659c640 1 --2- 172.21.15.46:0/2411461888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e0815f760 0x7f3e080cec10 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3e080cce80 tx=0x7f3dfc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1b6560a0cbfb68d server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:39.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.907+0000 7f3e05d9b640 1 -- 172.21.15.46:0/2411461888 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc031640 con 0x7f3e0815f760 2023-12-02T00:29:39.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.907+0000 7f3e05d9b640 1 -- 172.21.15.46:0/2411461888 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dfc0317a0 con 0x7f3e0815f760 2023-12-02T00:29:39.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.907+0000 7f3e05d9b640 1 -- 172.21.15.46:0/2411461888 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc031b30 con 0x7f3e0815f760 2023-12-02T00:29:39.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.907+0000 7f3e0d210640 1 -- 172.21.15.46:0/2411461888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e0815f760 msgr2=0x7f3e080cec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:39.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.907+0000 7f3e0d210640 1 --2- 172.21.15.46:0/2411461888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e0815f760 0x7f3e080cec10 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3e080cce80 tx=0x7f3dfc02b8f0 comp rx=0 tx=0).stop 2023-12-02T00:29:39.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.907+0000 7f3e0d210640 1 -- 172.21.15.46:0/2411461888 shutdown_connections 2023-12-02T00:29:39.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.907+0000 7f3e0d210640 1 --2- 172.21.15.46:0/2411461888 >> v2:172.21.15.162:3300/0 conn(0x7f3e080cf150 0x7f3e0806ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:39.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.907+0000 7f3e0d210640 1 --2- 172.21.15.46:0/2411461888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e0815f760 0x7f3e080cec10 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:39.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.907+0000 7f3e0d210640 1 --2- 172.21.15.46:0/2411461888 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e0815b130 0x7f3e080d01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:39.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.907+0000 7f3e0d210640 1 -- 172.21.15.46:0/2411461888 >> 172.21.15.46:0/2411461888 conn(0x7f3e080b2c10 msgr2=0x7f3e080b2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:39.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.907+0000 7f3e0d210640 1 -- 172.21.15.46:0/2411461888 shutdown_connections 2023-12-02T00:29:39.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.907+0000 7f3e0d210640 1 -- 172.21.15.46:0/2411461888 wait complete. 2023-12-02T00:29:39.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.909+0000 7f3e0d210640 1 Processor -- start 2023-12-02T00:29:39.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.909+0000 7f3e0d210640 1 -- start start 2023-12-02T00:29:39.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.909+0000 7f3e0d210640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e080cf150 0x7f3e081556f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:39.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.909+0000 7f3e0d210640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e0815b130 0x7f3e08155c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:39.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.909+0000 7f3e0d210640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e0815f760 0x7f3e080c5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:39.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.909+0000 7f3e0d210640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e080c7750 con 0x7f3e0815f760 2023-12-02T00:29:39.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.909+0000 7f3e0d210640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e080c66b0 con 0x7f3e0815b130 2023-12-02T00:29:39.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.909+0000 7f3e0d210640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e080c6990 con 0x7f3e080cf150 2023-12-02T00:29:39.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.909+0000 7f3e0759e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e0815f760 0x7f3e080c5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:39.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.909+0000 7f3e0759e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e0815f760 0x7f3e080c5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57480/0 (socket says 172.21.15.46:57480) 2023-12-02T00:29:39.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.909+0000 7f3e0759e640 1 -- 172.21.15.46:0/616821541 learned_addr learned my addr 172.21.15.46:0/616821541 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:39.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.910+0000 7f3e0659c640 1 --2- 172.21.15.46:0/616821541 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e0815b130 0x7f3e08155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:39.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.910+0000 7f3e06d9d640 1 --2- 172.21.15.46:0/616821541 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e080cf150 0x7f3e081556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:39.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.910+0000 7f3e0759e640 1 -- 172.21.15.46:0/616821541 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e080cf150 msgr2=0x7f3e081556f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:39.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.910+0000 7f3e0759e640 1 --2- 172.21.15.46:0/616821541 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e080cf150 0x7f3e081556f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:39.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.910+0000 7f3e0759e640 1 -- 172.21.15.46:0/616821541 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e0815b130 msgr2=0x7f3e08155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:39.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.910+0000 7f3e0759e640 1 --2- 172.21.15.46:0/616821541 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e0815b130 0x7f3e08155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:39.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.910+0000 7f3e0759e640 1 -- 172.21.15.46:0/616821541 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dfc0094d0 con 0x7f3e0815f760 2023-12-02T00:29:39.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.910+0000 7f3e0659c640 1 --2- 172.21.15.46:0/616821541 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e0815b130 0x7f3e08155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:39.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.910+0000 7f3e0759e640 1 --2- 172.21.15.46:0/616821541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e0815f760 0x7f3e080c5060 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3df800eed0 tx=0x7f3df800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:39.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.910+0000 7f3def7fe640 1 -- 172.21.15.46:0/616821541 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df8010040 con 0x7f3e0815f760 2023-12-02T00:29:39.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.910+0000 7f3def7fe640 1 -- 172.21.15.46:0/616821541 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3df8014400 con 0x7f3e0815f760 2023-12-02T00:29:39.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.910+0000 7f3e0d210640 1 -- 172.21.15.46:0/616821541 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e080c6c70 con 0x7f3e0815f760 2023-12-02T00:29:39.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.910+0000 7f3def7fe640 1 -- 172.21.15.46:0/616821541 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df8009ea0 con 0x7f3e0815f760 2023-12-02T00:29:39.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.910+0000 7f3e0d210640 1 -- 172.21.15.46:0/616821541 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e080c7230 con 0x7f3e0815f760 2023-12-02T00:29:39.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.911+0000 7f3e0d210640 1 -- 172.21.15.46:0/616821541 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e08067fc0 con 0x7f3e0815f760 2023-12-02T00:29:39.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.912+0000 7f3def7fe640 1 -- 172.21.15.46:0/616821541 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3df80147b0 con 0x7f3e0815f760 2023-12-02T00:29:39.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.912+0000 7f3def7fe640 1 --2- 172.21.15.46:0/616821541 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3df406ae40 0x7f3df406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:39.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.912+0000 7f3def7fe640 1 -- 172.21.15.46:0/616821541 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f3df8089db0 con 0x7f3e0815f760 2023-12-02T00:29:39.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.912+0000 7f3e06d9d640 1 --2- 172.21.15.46:0/616821541 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3df406ae40 0x7f3df406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:39.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.913+0000 7f3e06d9d640 1 --2- 172.21.15.46:0/616821541 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3df406ae40 0x7f3df406d300 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3e00009ad0 tx=0x7f3e00008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:39.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:39.917+0000 7f3def7fe640 1 -- 172.21.15.46:0/616821541 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3df8054e60 con 0x7f3e0815f760 2023-12-02T00:29:40.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.093+0000 7f3e0d210640 1 -- 172.21.15.46:0/616821541 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3e080c7420 con 0x7f3e0815f760 2023-12-02T00:29:40.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.095+0000 7f3def7fe640 1 -- 172.21.15.46:0/616821541 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16777 (secure 0 0 0) 0x7f3df8018020 con 0x7f3e0815f760 2023-12-02T00:29:40.097 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:29:40.097 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":23,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:29:30.868939+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-02T00:28:47.273337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-02T00:28:48.060115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.14","mappings":[{"from":4,"to":2}]},{"pgid":"3.1e","mappings":[{"from":3,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:29:40.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.097+0000 7f3e0d210640 1 -- 172.21.15.46:0/616821541 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3df406ae40 msgr2=0x7f3df406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:40.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.097+0000 7f3e0d210640 1 --2- 172.21.15.46:0/616821541 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3df406ae40 0x7f3df406d300 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3e00009ad0 tx=0x7f3e00008040 comp rx=0 tx=0).stop 2023-12-02T00:29:40.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.097+0000 7f3e0d210640 1 -- 172.21.15.46:0/616821541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e0815f760 msgr2=0x7f3e080c5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:40.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.097+0000 7f3e0d210640 1 --2- 172.21.15.46:0/616821541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e0815f760 0x7f3e080c5060 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3df800eed0 tx=0x7f3df800b480 comp rx=0 tx=0).stop 2023-12-02T00:29:40.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.097+0000 7f3e0d210640 1 -- 172.21.15.46:0/616821541 shutdown_connections 2023-12-02T00:29:40.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.097+0000 7f3e0d210640 1 --2- 172.21.15.46:0/616821541 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3df406ae40 0x7f3df406d300 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:40.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.097+0000 7f3e0d210640 1 --2- 172.21.15.46:0/616821541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e0815f760 0x7f3e080c5060 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:40.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.097+0000 7f3e0d210640 1 --2- 172.21.15.46:0/616821541 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e0815b130 0x7f3e08155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:40.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.097+0000 7f3e0d210640 1 --2- 172.21.15.46:0/616821541 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e080cf150 0x7f3e081556f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:40.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.097+0000 7f3e0d210640 1 -- 172.21.15.46:0/616821541 >> 172.21.15.46:0/616821541 conn(0x7f3e080b2c10 msgr2=0x7f3e080b4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:40.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.097+0000 7f3e0d210640 1 -- 172.21.15.46:0/616821541 shutdown_connections 2023-12-02T00:29:40.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.097+0000 7f3e0d210640 1 -- 172.21.15.46:0/616821541 wait complete. 2023-12-02T00:29:40.113 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-02T00:29:40.113 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-02T00:29:40.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.219+0000 7fb7a7fb0640 1 Processor -- start 2023-12-02T00:29:40.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.219+0000 7fb7a7fb0640 1 -- start start 2023-12-02T00:29:40.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.220+0000 7fb7a7fb0640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb7a0143490 0x7fb7a00b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:40.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.220+0000 7fb7a7fb0640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb7a015abf0 0x7fb7a00b1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:40.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.220+0000 7fb7a7fb0640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7a015e2a0 con 0x7fb7a015e730 2023-12-02T00:29:40.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.220+0000 7fb7a7fb0640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7a015b520 con 0x7fb7a0143490 2023-12-02T00:29:40.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.220+0000 7fb7a7fb0640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7a0143870 con 0x7fb7a015abf0 2023-12-02T00:29:40.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.220+0000 7fb7a5d4c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb7a0143490 0x7fb7a00b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:40.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.220+0000 7fb7a5d4c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb7a0143490 0x7fb7a00b0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57494/0 (socket says 172.21.15.46:57494) 2023-12-02T00:29:40.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.220+0000 7fb7a5d4c640 1 -- 172.21.15.46:0/539534869 learned_addr learned my addr 172.21.15.46:0/539534869 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:40.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.220+0000 7fb7a654d640 1 --2- 172.21.15.46:0/539534869 >> v2:172.21.15.162:3300/0 conn(0x7fb7a015abf0 0x7fb7a00b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:40.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.220+0000 7fb7a5d4c640 1 -- 172.21.15.46:0/539534869 >> v2:172.21.15.162:3300/0 conn(0x7fb7a015abf0 msgr2=0x7fb7a00b1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:40.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.220+0000 7fb7a5d4c640 1 --2- 172.21.15.46:0/539534869 >> v2:172.21.15.162:3300/0 conn(0x7fb7a015abf0 0x7fb7a00b1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:40.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.220+0000 7fb7a5d4c640 1 -- 172.21.15.46:0/539534869 >> v1:172.21.15.162:6789/0 conn(0x7fb7a015e730 legacy=0x7fb7a015a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:40.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.220+0000 7fb7a5d4c640 1 -- 172.21.15.46:0/539534869 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a015b240 con 0x7fb7a0143490 2023-12-02T00:29:40.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.220+0000 7fb7a654d640 1 --2- 172.21.15.46:0/539534869 >> v2:172.21.15.162:3300/0 conn(0x7fb7a015abf0 0x7fb7a00b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:29:40.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.221+0000 7fb7a5d4c640 1 --2- 172.21.15.46:0/539534869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7a0143490 0x7fb7a00b0cf0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb7a00b4ca0 tx=0x7fb79c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e76d107fb5430e server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:40.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.221+0000 7fb7a4d4a640 1 -- 172.21.15.46:0/539534869 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79c031870 con 0x7fb7a0143490 2023-12-02T00:29:40.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.221+0000 7fb7a4d4a640 1 -- 172.21.15.46:0/539534869 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb79c0319d0 con 0x7fb7a0143490 2023-12-02T00:29:40.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.221+0000 7fb7a4d4a640 1 -- 172.21.15.46:0/539534869 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79c031d60 con 0x7fb7a0143490 2023-12-02T00:29:40.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.221+0000 7fb7a7fb0640 1 -- 172.21.15.46:0/539534869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7a0143490 msgr2=0x7fb7a00b0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:40.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.221+0000 7fb7a7fb0640 1 --2- 172.21.15.46:0/539534869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7a0143490 0x7fb7a00b0cf0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb7a00b4ca0 tx=0x7fb79c02b910 comp rx=0 tx=0).stop 2023-12-02T00:29:40.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.222+0000 7fb7a7fb0640 1 -- 172.21.15.46:0/539534869 shutdown_connections 2023-12-02T00:29:40.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.222+0000 7fb7a7fb0640 1 --2- 172.21.15.46:0/539534869 >> v2:172.21.15.162:3300/0 conn(0x7fb7a015abf0 0x7fb7a00b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:40.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.222+0000 7fb7a7fb0640 1 --2- 172.21.15.46:0/539534869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7a0143490 0x7fb7a00b0cf0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:40.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.222+0000 7fb7a7fb0640 1 -- 172.21.15.46:0/539534869 >> 172.21.15.46:0/539534869 conn(0x7fb7a006a4d0 msgr2=0x7fb7a006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:40.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.222+0000 7fb7a7fb0640 1 -- 172.21.15.46:0/539534869 shutdown_connections 2023-12-02T00:29:40.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.222+0000 7fb7a7fb0640 1 -- 172.21.15.46:0/539534869 wait complete. 2023-12-02T00:29:40.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.223+0000 7fb7a7fb0640 1 Processor -- start 2023-12-02T00:29:40.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.223+0000 7fb7a7fb0640 1 -- start start 2023-12-02T00:29:40.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.223+0000 7fb7a7fb0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7a0143490 0x7fb7a01f6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:40.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.224+0000 7fb7a7fb0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7a015abf0 0x7fb7a01595e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:40.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.224+0000 7fb7a7fb0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7a015e730 0x7fb7a0159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:40.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.224+0000 7fb7a7fb0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a015a060 con 0x7fb7a015e730 2023-12-02T00:29:40.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.224+0000 7fb7a7fb0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a0164df0 con 0x7fb7a015abf0 2023-12-02T00:29:40.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.224+0000 7fb7a7fb0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a01650d0 con 0x7fb7a0143490 2023-12-02T00:29:40.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.224+0000 7fb7a654d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7a015e730 0x7fb7a0159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:40.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.224+0000 7fb7a654d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7a015e730 0x7fb7a0159b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57504/0 (socket says 172.21.15.46:57504) 2023-12-02T00:29:40.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.224+0000 7fb7a654d640 1 -- 172.21.15.46:0/2247051533 learned_addr learned my addr 172.21.15.46:0/2247051533 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:40.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.224+0000 7fb7a554b640 1 --2- 172.21.15.46:0/2247051533 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7a015abf0 0x7fb7a01595e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:40.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.224+0000 7fb7a5d4c640 1 --2- 172.21.15.46:0/2247051533 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7a0143490 0x7fb7a01f6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:40.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.224+0000 7fb7a654d640 1 -- 172.21.15.46:0/2247051533 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7a0143490 msgr2=0x7fb7a01f6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:40.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.224+0000 7fb7a654d640 1 --2- 172.21.15.46:0/2247051533 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7a0143490 0x7fb7a01f6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:40.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.224+0000 7fb7a654d640 1 -- 172.21.15.46:0/2247051533 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7a015abf0 msgr2=0x7fb7a01595e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:40.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.224+0000 7fb7a654d640 1 --2- 172.21.15.46:0/2247051533 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7a015abf0 0x7fb7a01595e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:40.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.224+0000 7fb7a654d640 1 -- 172.21.15.46:0/2247051533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb79c0094d0 con 0x7fb7a015e730 2023-12-02T00:29:40.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.224+0000 7fb7a654d640 1 --2- 172.21.15.46:0/2247051533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7a015e730 0x7fb7a0159b20 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb79800cef0 tx=0x7fb79800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:40.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.224+0000 7fb7967fc640 1 -- 172.21.15.46:0/2247051533 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79800d840 con 0x7fb7a015e730 2023-12-02T00:29:40.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.224+0000 7fb7967fc640 1 -- 172.21.15.46:0/2247051533 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb79800d9c0 con 0x7fb7a015e730 2023-12-02T00:29:40.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.224+0000 7fb7967fc640 1 -- 172.21.15.46:0/2247051533 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79800dd70 con 0x7fb7a015e730 2023-12-02T00:29:40.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.224+0000 7fb7a7fb0640 1 -- 172.21.15.46:0/2247051533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7a01653b0 con 0x7fb7a015e730 2023-12-02T00:29:40.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.224+0000 7fb7a7fb0640 1 -- 172.21.15.46:0/2247051533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7a0165970 con 0x7fb7a015e730 2023-12-02T00:29:40.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.225+0000 7fb7a7fb0640 1 -- 172.21.15.46:0/2247051533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7a00b4350 con 0x7fb7a015e730 2023-12-02T00:29:40.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.226+0000 7fb7967fc640 1 -- 172.21.15.46:0/2247051533 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb79801e3b0 con 0x7fb7a015e730 2023-12-02T00:29:40.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.226+0000 7fb7967fc640 1 --2- 172.21.15.46:0/2247051533 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb78406ae70 0x7fb78406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:40.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.226+0000 7fb7967fc640 1 -- 172.21.15.46:0/2247051533 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fb79808a150 con 0x7fb7a015e730 2023-12-02T00:29:40.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.227+0000 7fb7a5d4c640 1 --2- 172.21.15.46:0/2247051533 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb78406ae70 0x7fb78406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:40.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.227+0000 7fb7a5d4c640 1 --2- 172.21.15.46:0/2247051533 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb78406ae70 0x7fb78406d330 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb79c036000 tx=0x7fb79c03c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:40.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.231+0000 7fb7967fc640 1 -- 172.21.15.46:0/2247051533 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb798055220 con 0x7fb7a015e730 2023-12-02T00:29:40.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.443+0000 7fb7a7fb0640 1 -- 172.21.15.46:0/2247051533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb7a0165b90 con 0x7fb7a015e730 2023-12-02T00:29:40.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.444+0000 7fb7967fc640 1 -- 172.21.15.46:0/2247051533 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4907 (secure 0 0 0) 0x7fb798058840 con 0x7fb7a015e730 2023-12-02T00:29:40.445 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:29:40.445 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/2059113401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2059113401},{"type":"v1","addr":"172.21.15.46:6834","nonce":2059113401}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/987532381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":987532381},{"type":"v1","addr":"172.21.15.46:6835","nonce":987532381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/449974862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":449974862},{"type":"v1","addr":"172.21.15.162:6836","nonce":449974862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-02T00:28:49.086548+0000","modified":"2023-12-02T00:28:51.082469+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4253":{"gid":4253,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/1231844478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":1231844478},{"type":"v1","addr":"172.21.15.162:6837","nonce":1231844478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-02T00:29:40.446 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 6 2023-12-02T00:29:40.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.446+0000 7fb7a7fb0640 1 -- 172.21.15.46:0/2247051533 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb78406ae70 msgr2=0x7fb78406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:40.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.446+0000 7fb7a7fb0640 1 --2- 172.21.15.46:0/2247051533 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb78406ae70 0x7fb78406d330 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb79c036000 tx=0x7fb79c03c040 comp rx=0 tx=0).stop 2023-12-02T00:29:40.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.446+0000 7fb7a7fb0640 1 -- 172.21.15.46:0/2247051533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7a015e730 msgr2=0x7fb7a0159b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:40.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.446+0000 7fb7a7fb0640 1 --2- 172.21.15.46:0/2247051533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7a015e730 0x7fb7a0159b20 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb79800cef0 tx=0x7fb79800a710 comp rx=0 tx=0).stop 2023-12-02T00:29:40.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.446+0000 7fb7a7fb0640 1 -- 172.21.15.46:0/2247051533 shutdown_connections 2023-12-02T00:29:40.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.446+0000 7fb7a7fb0640 1 --2- 172.21.15.46:0/2247051533 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb78406ae70 0x7fb78406d330 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:40.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.446+0000 7fb7a7fb0640 1 --2- 172.21.15.46:0/2247051533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7a015e730 0x7fb7a0159b20 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:40.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.446+0000 7fb7a7fb0640 1 --2- 172.21.15.46:0/2247051533 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7a015abf0 0x7fb7a01595e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:40.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.446+0000 7fb7a7fb0640 1 --2- 172.21.15.46:0/2247051533 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7a0143490 0x7fb7a01f6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:40.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.446+0000 7fb7a7fb0640 1 -- 172.21.15.46:0/2247051533 >> 172.21.15.46:0/2247051533 conn(0x7fb7a006a4d0 msgr2=0x7fb7a015b660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:40.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.446+0000 7fb7a7fb0640 1 -- 172.21.15.46:0/2247051533 shutdown_connections 2023-12-02T00:29:40.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.446+0000 7fb7a7fb0640 1 -- 172.21.15.46:0/2247051533 wait complete. 2023-12-02T00:29:40.458 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-02T00:29:40.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.563+0000 7f6bb61de640 1 Processor -- start 2023-12-02T00:29:40.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.563+0000 7f6bb61de640 1 -- start start 2023-12-02T00:29:40.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.564+0000 7f6bb61de640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6bb01409d0 0x7f6bb01618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:40.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.564+0000 7f6bb61de640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bb01406f0 con 0x7f6bb0140310 2023-12-02T00:29:40.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.564+0000 7f6bb61de640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bb0143960 con 0x7f6bb006abe0 2023-12-02T00:29:40.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.564+0000 7f6bb61de640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb0161e20 con 0x7f6bb01409d0 2023-12-02T00:29:40.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.564+0000 7f6baf7fe640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6bb0140310 0x7f6bb006a4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59386/0 (socket says 172.21.15.46:59386) 2023-12-02T00:29:40.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.564+0000 7f6baf7fe640 1 -- 172.21.15.46:0/2952671258 learned_addr learned my addr 172.21.15.46:0/2952671258 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:40.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.564+0000 7f6baffff640 1 --2- 172.21.15.46:0/2952671258 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6bb01409d0 0x7f6bb01618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:40.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.564+0000 7f6bae7fc640 1 -- 172.21.15.46:0/2952671258 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 234077509 0 0) 0x7f6bb01406f0 con 0x7f6bb0140310 2023-12-02T00:29:40.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.564+0000 7f6bae7fc640 1 -- 172.21.15.46:0/2952671258 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b98003520 con 0x7f6bb0140310 2023-12-02T00:29:40.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.564+0000 7f6bae7fc640 1 -- 172.21.15.46:0/2952671258 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 38265194 0 0) 0x7f6bb0143960 con 0x7f6bb006abe0 2023-12-02T00:29:40.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.564+0000 7f6bae7fc640 1 -- 172.21.15.46:0/2952671258 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bb01406f0 con 0x7f6bb006abe0 2023-12-02T00:29:40.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.564+0000 7f6baffff640 1 -- 172.21.15.46:0/2952671258 >> v1:172.21.15.162:6789/0 conn(0x7f6bb006abe0 legacy=0x7f6bb006efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:40.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.564+0000 7f6bae7fc640 1 -- 172.21.15.46:0/2952671258 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1396517883 0 0) 0x7f6b98003520 con 0x7f6bb0140310 2023-12-02T00:29:40.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.564+0000 7f6baffff640 1 -- 172.21.15.46:0/2952671258 >> v1:172.21.15.46:6789/0 conn(0x7f6bb0140310 legacy=0x7f6bb006a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:40.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.564+0000 7f6baffff640 1 -- 172.21.15.46:0/2952671258 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb0142de0 con 0x7f6bb01409d0 2023-12-02T00:29:40.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.565+0000 7f6baffff640 1 --2- 172.21.15.46:0/2952671258 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6bb01409d0 0x7f6bb01618e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8009dc0 tx=0x7f6ba802bd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=835ae8f8fd811542 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:40.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.565+0000 7f6bae7fc640 1 -- 172.21.15.46:0/2952671258 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0001670 con 0x7f6bb01409d0 2023-12-02T00:29:40.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.565+0000 7f6bae7fc640 1 -- 172.21.15.46:0/2952671258 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba802c880 con 0x7f6bb01409d0 2023-12-02T00:29:40.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.565+0000 7f6bae7fc640 1 -- 172.21.15.46:0/2952671258 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba802cbf0 con 0x7f6bb01409d0 2023-12-02T00:29:40.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.566+0000 7f6bb61de640 1 -- 172.21.15.46:0/2952671258 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6bb01409d0 msgr2=0x7f6bb01618e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:40.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.566+0000 7f6bb61de640 1 --2- 172.21.15.46:0/2952671258 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6bb01409d0 0x7f6bb01618e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8009dc0 tx=0x7f6ba802bd50 comp rx=0 tx=0).stop 2023-12-02T00:29:40.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.566+0000 7f6bb61de640 1 -- 172.21.15.46:0/2952671258 shutdown_connections 2023-12-02T00:29:40.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.566+0000 7f6bb61de640 1 --2- 172.21.15.46:0/2952671258 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6bb01409d0 0x7f6bb01618e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:40.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.566+0000 7f6bb61de640 1 -- 172.21.15.46:0/2952671258 >> 172.21.15.46:0/2952671258 conn(0x7f6bb00b1e50 msgr2=0x7f6bb00b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:40.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.566+0000 7f6bb61de640 1 -- 172.21.15.46:0/2952671258 shutdown_connections 2023-12-02T00:29:40.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.566+0000 7f6bb61de640 1 -- 172.21.15.46:0/2952671258 wait complete. 2023-12-02T00:29:40.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.567+0000 7f6bb61de640 1 Processor -- start 2023-12-02T00:29:40.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.567+0000 7f6bb61de640 1 -- start start 2023-12-02T00:29:40.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.568+0000 7f6bb61de640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bb006abe0 0x7f6bb013cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:40.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.568+0000 7f6bb61de640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6bb0140310 0x7f6bb00af0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:40.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.568+0000 7f6bb61de640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6bb01409d0 0x7f6bb015da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:40.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.568+0000 7f6bb61de640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb0145380 con 0x7f6bb006abe0 2023-12-02T00:29:40.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.568+0000 7f6bb61de640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb013d0b0 con 0x7f6bb0140310 2023-12-02T00:29:40.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.568+0000 7f6bb61de640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb01f2d20 con 0x7f6bb01409d0 2023-12-02T00:29:40.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.568+0000 7f6baf7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bb006abe0 0x7f6bb013cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:40.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.568+0000 7f6baf7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bb006abe0 0x7f6bb013cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57520/0 (socket says 172.21.15.46:57520) 2023-12-02T00:29:40.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.568+0000 7f6baf7fe640 1 -- 172.21.15.46:0/1414396235 learned_addr learned my addr 172.21.15.46:0/1414396235 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:40.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.568+0000 7f6baeffd640 1 --2- 172.21.15.46:0/1414396235 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6bb0140310 0x7f6bb00af0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:40.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.568+0000 7f6baffff640 1 --2- 172.21.15.46:0/1414396235 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6bb01409d0 0x7f6bb015da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:40.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.568+0000 7f6baf7fe640 1 -- 172.21.15.46:0/1414396235 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6bb01409d0 msgr2=0x7f6bb015da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:40.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.568+0000 7f6baf7fe640 1 --2- 172.21.15.46:0/1414396235 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6bb01409d0 0x7f6bb015da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:40.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.568+0000 7f6baf7fe640 1 -- 172.21.15.46:0/1414396235 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6bb0140310 msgr2=0x7f6bb00af0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:40.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.568+0000 7f6baf7fe640 1 --2- 172.21.15.46:0/1414396235 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6bb0140310 0x7f6bb00af0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:40.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.568+0000 7f6baf7fe640 1 -- 172.21.15.46:0/1414396235 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba8009a70 con 0x7f6bb006abe0 2023-12-02T00:29:40.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.569+0000 7f6baffff640 1 --2- 172.21.15.46:0/1414396235 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6bb01409d0 0x7f6bb015da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:40.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.569+0000 7f6baeffd640 1 --2- 172.21.15.46:0/1414396235 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6bb0140310 0x7f6bb00af0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:40.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.569+0000 7f6baf7fe640 1 --2- 172.21.15.46:0/1414396235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bb006abe0 0x7f6bb013cb70 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6ba000c780 tx=0x7f6ba000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:40.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.569+0000 7f6b97fff640 1 -- 172.21.15.46:0/1414396235 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0005ad0 con 0x7f6bb006abe0 2023-12-02T00:29:40.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.569+0000 7f6b97fff640 1 -- 172.21.15.46:0/1414396235 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba0005c30 con 0x7f6bb006abe0 2023-12-02T00:29:40.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.569+0000 7f6bb61de640 1 -- 172.21.15.46:0/1414396235 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bb013e730 con 0x7f6bb006abe0 2023-12-02T00:29:40.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.569+0000 7f6b97fff640 1 -- 172.21.15.46:0/1414396235 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0007070 con 0x7f6bb006abe0 2023-12-02T00:29:40.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.569+0000 7f6bb61de640 1 -- 172.21.15.46:0/1414396235 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bb013ecf0 con 0x7f6bb006abe0 2023-12-02T00:29:40.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.570+0000 7f6bb61de640 1 -- 172.21.15.46:0/1414396235 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bb0161030 con 0x7f6bb006abe0 2023-12-02T00:29:40.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.570+0000 7f6b97fff640 1 -- 172.21.15.46:0/1414396235 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6ba00071d0 con 0x7f6bb006abe0 2023-12-02T00:29:40.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.571+0000 7f6b97fff640 1 --2- 172.21.15.46:0/1414396235 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f6b9c06ae40 0x7f6b9c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:40.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.571+0000 7f6b97fff640 1 -- 172.21.15.46:0/1414396235 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f6ba0088be0 con 0x7f6bb006abe0 2023-12-02T00:29:40.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.571+0000 7f6baeffd640 1 --2- 172.21.15.46:0/1414396235 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f6b9c06ae40 0x7f6b9c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:40.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.572+0000 7f6baeffd640 1 --2- 172.21.15.46:0/1414396235 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f6b9c06ae40 0x7f6b9c06d300 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0160430 tx=0x7f6ba4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:40.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.575+0000 7f6b97fff640 1 -- 172.21.15.46:0/1414396235 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6ba0053d40 con 0x7f6bb006abe0 2023-12-02T00:29:40.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.786+0000 7f6bb61de640 1 -- 172.21.15.46:0/1414396235 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6bb0069400 con 0x7f6bb006abe0 2023-12-02T00:29:40.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.786+0000 7f6b97fff640 1 -- 172.21.15.46:0/1414396235 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4907 (secure 0 0 0) 0x7f6ba0017020 con 0x7f6bb006abe0 2023-12-02T00:29:40.787 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:29:40.788 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/2059113401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2059113401},{"type":"v1","addr":"172.21.15.46:6834","nonce":2059113401}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/987532381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":987532381},{"type":"v1","addr":"172.21.15.46:6835","nonce":987532381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/449974862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":449974862},{"type":"v1","addr":"172.21.15.162:6836","nonce":449974862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-02T00:28:49.086548+0000","modified":"2023-12-02T00:28:51.082469+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4253":{"gid":4253,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/1231844478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":1231844478},{"type":"v1","addr":"172.21.15.162:6837","nonce":1231844478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-02T00:29:40.788 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 6 2023-12-02T00:29:40.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.788+0000 7f6bb61de640 1 -- 172.21.15.46:0/1414396235 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f6b9c06ae40 msgr2=0x7f6b9c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:40.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.788+0000 7f6bb61de640 1 --2- 172.21.15.46:0/1414396235 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f6b9c06ae40 0x7f6b9c06d300 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0160430 tx=0x7f6ba4009040 comp rx=0 tx=0).stop 2023-12-02T00:29:40.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.788+0000 7f6bb61de640 1 -- 172.21.15.46:0/1414396235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bb006abe0 msgr2=0x7f6bb013cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:40.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.788+0000 7f6bb61de640 1 --2- 172.21.15.46:0/1414396235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bb006abe0 0x7f6bb013cb70 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6ba000c780 tx=0x7f6ba000cc40 comp rx=0 tx=0).stop 2023-12-02T00:29:40.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.788+0000 7f6bb61de640 1 -- 172.21.15.46:0/1414396235 shutdown_connections 2023-12-02T00:29:40.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.788+0000 7f6bb61de640 1 --2- 172.21.15.46:0/1414396235 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f6b9c06ae40 0x7f6b9c06d300 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:40.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.788+0000 7f6bb61de640 1 --2- 172.21.15.46:0/1414396235 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6bb01409d0 0x7f6bb015da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:40.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.788+0000 7f6bb61de640 1 --2- 172.21.15.46:0/1414396235 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6bb0140310 0x7f6bb00af0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:40.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.788+0000 7f6bb61de640 1 --2- 172.21.15.46:0/1414396235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bb006abe0 0x7f6bb013cb70 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:40.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.788+0000 7f6bb61de640 1 -- 172.21.15.46:0/1414396235 >> 172.21.15.46:0/1414396235 conn(0x7f6bb00b1e50 msgr2=0x7f6bb006a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:40.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.789+0000 7f6bb61de640 1 -- 172.21.15.46:0/1414396235 shutdown_connections 2023-12-02T00:29:40.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.789+0000 7f6bb61de640 1 -- 172.21.15.46:0/1414396235 wait complete. 2023-12-02T00:29:40.801 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:29:40.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.906+0000 7f38ed864640 1 Processor -- start 2023-12-02T00:29:40.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.906+0000 7f38ed864640 1 -- start start 2023-12-02T00:29:40.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.907+0000 7f38ed864640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f38e815c370 0x7f38e815f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:40.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.907+0000 7f38ed864640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38e815b640 con 0x7f38e8070540 2023-12-02T00:29:40.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.907+0000 7f38ed864640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38e815f980 con 0x7f38e8155960 2023-12-02T00:29:40.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.907+0000 7f38ed864640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38e815fc60 con 0x7f38e815c370 2023-12-02T00:29:40.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.907+0000 7f38e77fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f38e815c370 0x7f38e815f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:40.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.907+0000 7f38e67fc640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f38e8070540 0x7f38e815ec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59390/0 (socket says 172.21.15.46:59390) 2023-12-02T00:29:40.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.907+0000 7f38e67fc640 1 -- 172.21.15.46:0/1292968963 learned_addr learned my addr 172.21.15.46:0/1292968963 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:40.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.907+0000 7f38e77fe640 1 -- 172.21.15.46:0/1292968963 >> v1:172.21.15.162:6789/0 conn(0x7f38e8155960 legacy=0x7f38e806fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:40.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.907+0000 7f38e77fe640 1 -- 172.21.15.46:0/1292968963 >> v1:172.21.15.46:6789/0 conn(0x7f38e8070540 legacy=0x7f38e815ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:40.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.907+0000 7f38e5ffb640 1 -- 172.21.15.46:0/1292968963 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3984362305 0 0) 0x7f38e815b640 con 0x7f38e8070540 2023-12-02T00:29:40.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.907+0000 7f38e77fe640 1 -- 172.21.15.46:0/1292968963 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e815b890 con 0x7f38e815c370 2023-12-02T00:29:40.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.908+0000 7f38e77fe640 1 --2- 172.21.15.46:0/1292968963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38e815c370 0x7f38e815f3a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f38d8009d90 tx=0x7f38d802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e799dc06ae14ad14 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:40.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.908+0000 7f38e5ffb640 1 -- 172.21.15.46:0/1292968963 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d802cd00 con 0x7f38e815c370 2023-12-02T00:29:40.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.908+0000 7f38e5ffb640 1 -- 172.21.15.46:0/1292968963 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d802ce60 con 0x7f38e815c370 2023-12-02T00:29:40.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.908+0000 7f38e5ffb640 1 -- 172.21.15.46:0/1292968963 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d8034660 con 0x7f38e815c370 2023-12-02T00:29:40.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.909+0000 7f38ed864640 1 -- 172.21.15.46:0/1292968963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38e815c370 msgr2=0x7f38e815f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:40.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.909+0000 7f38ed864640 1 --2- 172.21.15.46:0/1292968963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38e815c370 0x7f38e815f3a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f38d8009d90 tx=0x7f38d802bf50 comp rx=0 tx=0).stop 2023-12-02T00:29:40.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.909+0000 7f38ed864640 1 -- 172.21.15.46:0/1292968963 shutdown_connections 2023-12-02T00:29:40.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.909+0000 7f38ed864640 1 --2- 172.21.15.46:0/1292968963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38e815c370 0x7f38e815f3a0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:40.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.909+0000 7f38ed864640 1 -- 172.21.15.46:0/1292968963 >> 172.21.15.46:0/1292968963 conn(0x7f38e806e7d0 msgr2=0x7f38e806ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:40.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.909+0000 7f38ed864640 1 -- 172.21.15.46:0/1292968963 shutdown_connections 2023-12-02T00:29:40.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.909+0000 7f38ed864640 1 -- 172.21.15.46:0/1292968963 wait complete. 2023-12-02T00:29:40.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.910+0000 7f38ed864640 1 Processor -- start 2023-12-02T00:29:40.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.910+0000 7f38ed864640 1 -- start start 2023-12-02T00:29:40.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.911+0000 7f38ed864640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f38e8070540 0x7f38e80caae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:40.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.911+0000 7f38ed864640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38e8155960 0x7f38e815ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:40.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.911+0000 7f38ed864640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38e815c370 0x7f38e8155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:40.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.911+0000 7f38ed864640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e80cb020 con 0x7f38e815c370 2023-12-02T00:29:40.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.911+0000 7f38ed864640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e80b1140 con 0x7f38e8155960 2023-12-02T00:29:40.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.911+0000 7f38ed864640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38e80b1420 con 0x7f38e8070540 2023-12-02T00:29:40.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.911+0000 7f38e77fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38e815c370 0x7f38e8155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:40.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.911+0000 7f38e77fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38e815c370 0x7f38e8155d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57544/0 (socket says 172.21.15.46:57544) 2023-12-02T00:29:40.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.911+0000 7f38e77fe640 1 -- 172.21.15.46:0/2882268939 learned_addr learned my addr 172.21.15.46:0/2882268939 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:40.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.911+0000 7f38e67fc640 1 --2- 172.21.15.46:0/2882268939 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38e8155960 0x7f38e815ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:40.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.911+0000 7f38e6ffd640 1 --2- 172.21.15.46:0/2882268939 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f38e8070540 0x7f38e80caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:40.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.911+0000 7f38e77fe640 1 -- 172.21.15.46:0/2882268939 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f38e8070540 msgr2=0x7f38e80caae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:40.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.911+0000 7f38e77fe640 1 --2- 172.21.15.46:0/2882268939 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f38e8070540 0x7f38e80caae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:40.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.911+0000 7f38e77fe640 1 -- 172.21.15.46:0/2882268939 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38e8155960 msgr2=0x7f38e815ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:40.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.911+0000 7f38e77fe640 1 --2- 172.21.15.46:0/2882268939 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38e8155960 0x7f38e815ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:40.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.911+0000 7f38e77fe640 1 -- 172.21.15.46:0/2882268939 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d8009a70 con 0x7f38e815c370 2023-12-02T00:29:40.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.911+0000 7f38e67fc640 1 --2- 172.21.15.46:0/2882268939 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38e8155960 0x7f38e815ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:40.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.911+0000 7f38e77fe640 1 --2- 172.21.15.46:0/2882268939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38e815c370 0x7f38e8155d60 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f38d8009d90 tx=0x7f38d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:40.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.912+0000 7f38dffff640 1 -- 172.21.15.46:0/2882268939 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d8003b90 con 0x7f38e815c370 2023-12-02T00:29:40.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.912+0000 7f38dffff640 1 -- 172.21.15.46:0/2882268939 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d8003cf0 con 0x7f38e815c370 2023-12-02T00:29:40.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.912+0000 7f38ed864640 1 -- 172.21.15.46:0/2882268939 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38e80b1700 con 0x7f38e815c370 2023-12-02T00:29:40.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.912+0000 7f38dffff640 1 -- 172.21.15.46:0/2882268939 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d8004230 con 0x7f38e815c370 2023-12-02T00:29:40.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.912+0000 7f38ed864640 1 -- 172.21.15.46:0/2882268939 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38e80b1cc0 con 0x7f38e815c370 2023-12-02T00:29:40.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.913+0000 7f38ed864640 1 -- 172.21.15.46:0/2882268939 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38e80b1eb0 con 0x7f38e815c370 2023-12-02T00:29:40.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.913+0000 7f38dffff640 1 -- 172.21.15.46:0/2882268939 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f38d803b050 con 0x7f38e815c370 2023-12-02T00:29:40.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.913+0000 7f38dffff640 1 --2- 172.21.15.46:0/2882268939 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f38c806ae70 0x7f38c806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:40.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.913+0000 7f38dffff640 1 -- 172.21.15.46:0/2882268939 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f38d80ac0a0 con 0x7f38e815c370 2023-12-02T00:29:40.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.914+0000 7f38e6ffd640 1 --2- 172.21.15.46:0/2882268939 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f38c806ae70 0x7f38c806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:40.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.914+0000 7f38e6ffd640 1 --2- 172.21.15.46:0/2882268939 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f38c806ae70 0x7f38c806d330 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f38e0000f60 tx=0x7f38e0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:40.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:40.917+0000 7f38dffff640 1 -- 172.21.15.46:0/2882268939 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f38d8077030 con 0x7f38e815c370 2023-12-02T00:29:41.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.093+0000 7f38ed864640 1 -- 172.21.15.46:0/2882268939 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38e8067420 con 0x7f38e815c370 2023-12-02T00:29:41.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.094+0000 7f38dffff640 1 -- 172.21.15.46:0/2882268939 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16777 (secure 0 0 0) 0x7f38d8039090 con 0x7f38e815c370 2023-12-02T00:29:41.095 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:29:41.096 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":23,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:29:30.868939+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-02T00:28:47.273337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-02T00:28:48.060115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.14","mappings":[{"from":4,"to":2}]},{"pgid":"3.1e","mappings":[{"from":3,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:29:41.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.096+0000 7f38ed864640 1 -- 172.21.15.46:0/2882268939 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f38c806ae70 msgr2=0x7f38c806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:41.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.096+0000 7f38ed864640 1 --2- 172.21.15.46:0/2882268939 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f38c806ae70 0x7f38c806d330 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f38e0000f60 tx=0x7f38e0009040 comp rx=0 tx=0).stop 2023-12-02T00:29:41.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.096+0000 7f38ed864640 1 -- 172.21.15.46:0/2882268939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38e815c370 msgr2=0x7f38e8155d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:41.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.096+0000 7f38ed864640 1 --2- 172.21.15.46:0/2882268939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38e815c370 0x7f38e8155d60 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f38d8009d90 tx=0x7f38d8004a40 comp rx=0 tx=0).stop 2023-12-02T00:29:41.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.096+0000 7f38ed864640 1 -- 172.21.15.46:0/2882268939 shutdown_connections 2023-12-02T00:29:41.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.096+0000 7f38ed864640 1 --2- 172.21.15.46:0/2882268939 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f38c806ae70 0x7f38c806d330 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:41.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.096+0000 7f38ed864640 1 --2- 172.21.15.46:0/2882268939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38e815c370 0x7f38e8155d60 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:41.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.096+0000 7f38ed864640 1 --2- 172.21.15.46:0/2882268939 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38e8155960 0x7f38e815ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:41.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.096+0000 7f38ed864640 1 --2- 172.21.15.46:0/2882268939 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f38e8070540 0x7f38e80caae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:41.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.096+0000 7f38ed864640 1 -- 172.21.15.46:0/2882268939 >> 172.21.15.46:0/2882268939 conn(0x7f38e806e7d0 msgr2=0x7f38e806ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:41.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.096+0000 7f38ed864640 1 -- 172.21.15.46:0/2882268939 shutdown_connections 2023-12-02T00:29:41.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.096+0000 7f38ed864640 1 -- 172.21.15.46:0/2882268939 wait complete. 2023-12-02T00:29:41.108 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-02T00:29:41.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.216+0000 7f3cc0cc2640 1 Processor -- start 2023-12-02T00:29:41.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.216+0000 7f3cc0cc2640 1 -- start start 2023-12-02T00:29:41.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.217+0000 7f3cc0cc2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3cbc15d2d0 0x7f3cbc0b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:41.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.217+0000 7f3cc0cc2640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3cbc0ce150 0x7f3cbc0b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:41.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.217+0000 7f3cc0cc2640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cbc0d1bb0 con 0x7f3cbc0d2040 2023-12-02T00:29:41.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.217+0000 7f3cc0cc2640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cbc15d6b0 con 0x7f3cbc0ce150 2023-12-02T00:29:41.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.217+0000 7f3cc0cc2640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cbc0ce700 con 0x7f3cbc15d2d0 2023-12-02T00:29:41.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.218+0000 7f3cba59c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3cbc15d2d0 0x7f3cbc0b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:41.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.218+0000 7f3cba59c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3cbc15d2d0 0x7f3cbc0b4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.46:48392/0 (socket says 172.21.15.46:48392) 2023-12-02T00:29:41.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.218+0000 7f3cbad9d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3cbc0ce150 0x7f3cbc0b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:41.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.218+0000 7f3cba59c640 1 -- 172.21.15.46:0/3354896613 learned_addr learned my addr 172.21.15.46:0/3354896613 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:41.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.218+0000 7f3cba59c640 1 -- 172.21.15.46:0/3354896613 >> v2:172.21.15.162:3300/0 conn(0x7f3cbc0ce150 msgr2=0x7f3cbc0b51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:41.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.218+0000 7f3cba59c640 1 --2- 172.21.15.46:0/3354896613 >> v2:172.21.15.162:3300/0 conn(0x7f3cbc0ce150 0x7f3cbc0b51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:41.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.218+0000 7f3cba59c640 1 -- 172.21.15.46:0/3354896613 >> v1:172.21.15.162:6789/0 conn(0x7f3cbc0d2040 legacy=0x7f3cbc0cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:41.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.218+0000 7f3cba59c640 1 -- 172.21.15.46:0/3354896613 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cbc0d1280 con 0x7f3cbc15d2d0 2023-12-02T00:29:41.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.219+0000 7f3cba59c640 1 --2- 172.21.15.46:0/3354896613 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3cbc15d2d0 0x7f3cbc0b4c60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3cac009e50 tx=0x7f3cac02bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be385165cdf189cb server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:41.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.219+0000 7f3cb959a640 1 -- 172.21.15.46:0/3354896613 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cac02cb00 con 0x7f3cbc15d2d0 2023-12-02T00:29:41.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.219+0000 7f3cb959a640 1 -- 172.21.15.46:0/3354896613 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cac02cc60 con 0x7f3cbc15d2d0 2023-12-02T00:29:41.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.219+0000 7f3cb959a640 1 -- 172.21.15.46:0/3354896613 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cac0343f0 con 0x7f3cbc15d2d0 2023-12-02T00:29:41.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.220+0000 7f3cc0cc2640 1 -- 172.21.15.46:0/3354896613 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3cbc15d2d0 msgr2=0x7f3cbc0b4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:41.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.220+0000 7f3cc0cc2640 1 --2- 172.21.15.46:0/3354896613 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3cbc15d2d0 0x7f3cbc0b4c60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3cac009e50 tx=0x7f3cac02bf60 comp rx=0 tx=0).stop 2023-12-02T00:29:41.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.220+0000 7f3cc0cc2640 1 -- 172.21.15.46:0/3354896613 shutdown_connections 2023-12-02T00:29:41.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.220+0000 7f3cc0cc2640 1 --2- 172.21.15.46:0/3354896613 >> v2:172.21.15.162:3300/0 conn(0x7f3cbc0ce150 0x7f3cbc0b51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:41.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.220+0000 7f3cc0cc2640 1 --2- 172.21.15.46:0/3354896613 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3cbc15d2d0 0x7f3cbc0b4c60 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:41.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.220+0000 7f3cc0cc2640 1 -- 172.21.15.46:0/3354896613 >> 172.21.15.46:0/3354896613 conn(0x7f3cbc072050 msgr2=0x7f3cbc072460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:41.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.220+0000 7f3cc0cc2640 1 -- 172.21.15.46:0/3354896613 shutdown_connections 2023-12-02T00:29:41.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.220+0000 7f3cc0cc2640 1 -- 172.21.15.46:0/3354896613 wait complete. 2023-12-02T00:29:41.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.221+0000 7f3cc0cc2640 1 Processor -- start 2023-12-02T00:29:41.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.221+0000 7f3cc0cc2640 1 -- start start 2023-12-02T00:29:41.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.222+0000 7f3cc0cc2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cbc0ce150 0x7f3cbc1524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:41.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.222+0000 7f3cc0cc2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3cbc0d2040 0x7f3cbc15ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:41.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.222+0000 7f3cc0cc2640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3cbc15d2d0 0x7f3cbc0b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:41.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.222+0000 7f3cc0cc2640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cbc0ccd20 con 0x7f3cbc0ce150 2023-12-02T00:29:41.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.222+0000 7f3cc0cc2640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cbc0cbeb0 con 0x7f3cbc15d2d0 2023-12-02T00:29:41.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.222+0000 7f3cc0cc2640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cbc0cc150 con 0x7f3cbc0d2040 2023-12-02T00:29:41.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.222+0000 7f3cba59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cbc0ce150 0x7f3cbc1524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:41.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.222+0000 7f3cba59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cbc0ce150 0x7f3cbc1524d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57550/0 (socket says 172.21.15.46:57550) 2023-12-02T00:29:41.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.222+0000 7f3cba59c640 1 -- 172.21.15.46:0/2112653675 learned_addr learned my addr 172.21.15.46:0/2112653675 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:41.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.222+0000 7f3cbad9d640 1 --2- 172.21.15.46:0/2112653675 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3cbc15d2d0 0x7f3cbc0b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:41.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.222+0000 7f3cb9d9b640 1 --2- 172.21.15.46:0/2112653675 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3cbc0d2040 0x7f3cbc15ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:41.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.222+0000 7f3cba59c640 1 -- 172.21.15.46:0/2112653675 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3cbc0d2040 msgr2=0x7f3cbc15ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:41.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.222+0000 7f3cba59c640 1 --2- 172.21.15.46:0/2112653675 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3cbc0d2040 0x7f3cbc15ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:41.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.222+0000 7f3cba59c640 1 -- 172.21.15.46:0/2112653675 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3cbc15d2d0 msgr2=0x7f3cbc0b4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:41.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.222+0000 7f3cba59c640 1 --2- 172.21.15.46:0/2112653675 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3cbc15d2d0 0x7f3cbc0b4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:41.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.222+0000 7f3cba59c640 1 -- 172.21.15.46:0/2112653675 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cac009b30 con 0x7f3cbc0ce150 2023-12-02T00:29:41.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.222+0000 7f3cba59c640 1 --2- 172.21.15.46:0/2112653675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cbc0ce150 0x7f3cbc1524d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3cac009e50 tx=0x7f3cac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:41.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.223+0000 7f3cb2ffd640 1 -- 172.21.15.46:0/2112653675 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cac009510 con 0x7f3cbc0ce150 2023-12-02T00:29:41.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.223+0000 7f3cb2ffd640 1 -- 172.21.15.46:0/2112653675 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cac03e450 con 0x7f3cbc0ce150 2023-12-02T00:29:41.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.223+0000 7f3cc0cc2640 1 -- 172.21.15.46:0/2112653675 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cbc0ca550 con 0x7f3cbc0ce150 2023-12-02T00:29:41.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.223+0000 7f3cb2ffd640 1 -- 172.21.15.46:0/2112653675 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cac03e800 con 0x7f3cbc0ce150 2023-12-02T00:29:41.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.223+0000 7f3cc0cc2640 1 -- 172.21.15.46:0/2112653675 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cbc0cc6d0 con 0x7f3cbc0ce150 2023-12-02T00:29:41.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.224+0000 7f3cc0cc2640 1 -- 172.21.15.46:0/2112653675 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cbc069400 con 0x7f3cbc0ce150 2023-12-02T00:29:41.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.224+0000 7f3cb2ffd640 1 -- 172.21.15.46:0/2112653675 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3cac03e960 con 0x7f3cbc0ce150 2023-12-02T00:29:41.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.224+0000 7f3cb2ffd640 1 --2- 172.21.15.46:0/2112653675 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3c9c06ae40 0x7f3c9c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:41.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.225+0000 7f3cb2ffd640 1 -- 172.21.15.46:0/2112653675 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f3cac0ac3d0 con 0x7f3cbc0ce150 2023-12-02T00:29:41.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.225+0000 7f3cb9d9b640 1 --2- 172.21.15.46:0/2112653675 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3c9c06ae40 0x7f3c9c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:41.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.225+0000 7f3cb9d9b640 1 --2- 172.21.15.46:0/2112653675 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3c9c06ae40 0x7f3c9c06d300 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc0b2100 tx=0x7f3ca4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:41.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.229+0000 7f3cb2ffd640 1 -- 172.21.15.46:0/2112653675 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3cac0773f0 con 0x7f3cbc0ce150 2023-12-02T00:29:41.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.443+0000 7f3cc0cc2640 1 -- 172.21.15.46:0/2112653675 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f3cbc072840 con 0x7f3cbc0ce150 2023-12-02T00:29:41.444 INFO:tasks.ceph.mon.a.smithi046.stderr:2023-12-02T00:29:41.443+0000 7f59c5f43640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-02T00:29:41.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.445+0000 7f3cb2ffd640 1 -- 172.21.15.46:0/2112653675 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v7) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3cac07aa10 con 0x7f3cbc0ce150 2023-12-02T00:29:41.445 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-02T00:29:41.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.447+0000 7f3cc0cc2640 1 -- 172.21.15.46:0/2112653675 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3c9c06ae40 msgr2=0x7f3c9c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:41.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.447+0000 7f3cc0cc2640 1 --2- 172.21.15.46:0/2112653675 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3c9c06ae40 0x7f3c9c06d300 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc0b2100 tx=0x7f3ca4009040 comp rx=0 tx=0).stop 2023-12-02T00:29:41.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.447+0000 7f3cc0cc2640 1 -- 172.21.15.46:0/2112653675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cbc0ce150 msgr2=0x7f3cbc1524d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:41.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.447+0000 7f3cc0cc2640 1 --2- 172.21.15.46:0/2112653675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cbc0ce150 0x7f3cbc1524d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3cac009e50 tx=0x7f3cac004a40 comp rx=0 tx=0).stop 2023-12-02T00:29:41.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.447+0000 7f3cc0cc2640 1 -- 172.21.15.46:0/2112653675 shutdown_connections 2023-12-02T00:29:41.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.447+0000 7f3cc0cc2640 1 --2- 172.21.15.46:0/2112653675 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3c9c06ae40 0x7f3c9c06d300 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:41.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.447+0000 7f3cc0cc2640 1 --2- 172.21.15.46:0/2112653675 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3cbc15d2d0 0x7f3cbc0b4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:41.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.447+0000 7f3cc0cc2640 1 --2- 172.21.15.46:0/2112653675 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3cbc0d2040 0x7f3cbc15ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:41.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.447+0000 7f3cc0cc2640 1 --2- 172.21.15.46:0/2112653675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cbc0ce150 0x7f3cbc1524d0 secure :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3cac009e50 tx=0x7f3cac004a40 comp rx=0 tx=0).stop 2023-12-02T00:29:41.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.447+0000 7f3cc0cc2640 1 -- 172.21.15.46:0/2112653675 >> 172.21.15.46:0/2112653675 conn(0x7f3cbc072050 msgr2=0x7f3cbc0d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:41.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.447+0000 7f3cc0cc2640 1 -- 172.21.15.46:0/2112653675 shutdown_connections 2023-12-02T00:29:41.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.447+0000 7f3cc0cc2640 1 -- 172.21.15.46:0/2112653675 wait complete. 2023-12-02T00:29:41.449 INFO:tasks.ceph.mds.d.smithi162.stderr: -4106> 2023-12-02T00:28:42.763+0000 7f7391f97640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-12-02T00:29:41.451 INFO:tasks.ceph.mds.d.smithi162.stderr: -2890> 2023-12-02T00:28:53.782+0000 7f738bf8b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-02T00:29:41.459 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-02T00:29:41.481 INFO:tasks.ceph.mds.d.smithi162.stdout:starting mds.d at 2023-12-02T00:29:41.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.568+0000 7f840bed0640 1 Processor -- start 2023-12-02T00:29:41.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.568+0000 7f840bed0640 1 -- start start 2023-12-02T00:29:41.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.569+0000 7f840bed0640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f840406ea50 0x7f840406ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:41.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.569+0000 7f840bed0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f84040d0ed0 0x7f84040d1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:41.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.569+0000 7f840bed0640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8404161d20 con 0x7f8404160fb0 2023-12-02T00:29:41.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.569+0000 7f840bed0640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f840406f370 con 0x7f840406ea50 2023-12-02T00:29:41.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.569+0000 7f840bed0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f840415e760 con 0x7f84040d0ed0 2023-12-02T00:29:41.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.569+0000 7f8409c6c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f840406ea50 0x7f840406ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:41.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.569+0000 7f8409c6c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f840406ea50 0x7f840406ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57560/0 (socket says 172.21.15.46:57560) 2023-12-02T00:29:41.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.569+0000 7f8409c6c640 1 -- 172.21.15.46:0/3191470895 learned_addr learned my addr 172.21.15.46:0/3191470895 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:41.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.570+0000 7f840a46d640 1 --2- 172.21.15.46:0/3191470895 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f84040d0ed0 0x7f84040d1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:41.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.570+0000 7f8409c6c640 1 -- 172.21.15.46:0/3191470895 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f84040d0ed0 msgr2=0x7f84040d1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:41.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.570+0000 7f8409c6c640 1 --2- 172.21.15.46:0/3191470895 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f84040d0ed0 0x7f84040d1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:41.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.570+0000 7f8409c6c640 1 -- 172.21.15.46:0/3191470895 >> v1:172.21.15.162:6789/0 conn(0x7f8404160fb0 legacy=0x7f8404161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:41.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.570+0000 7f8409c6c640 1 -- 172.21.15.46:0/3191470895 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f840415d9a0 con 0x7f840406ea50 2023-12-02T00:29:41.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.570+0000 7f8409c6c640 1 --2- 172.21.15.46:0/3191470895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f840406ea50 0x7f840406ee30 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f83fc009870 tx=0x7f83fc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fa85434a9c19393 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:41.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.570+0000 7f8408c6a640 1 -- 172.21.15.46:0/3191470895 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83fc031640 con 0x7f840406ea50 2023-12-02T00:29:41.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.571+0000 7f8408c6a640 1 -- 172.21.15.46:0/3191470895 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83fc0317a0 con 0x7f840406ea50 2023-12-02T00:29:41.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.571+0000 7f8408c6a640 1 -- 172.21.15.46:0/3191470895 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83fc031b30 con 0x7f840406ea50 2023-12-02T00:29:41.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.571+0000 7f840bed0640 1 -- 172.21.15.46:0/3191470895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f840406ea50 msgr2=0x7f840406ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:41.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.571+0000 7f840bed0640 1 --2- 172.21.15.46:0/3191470895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f840406ea50 0x7f840406ee30 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f83fc009870 tx=0x7f83fc02b8f0 comp rx=0 tx=0).stop 2023-12-02T00:29:41.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.571+0000 7f840bed0640 1 -- 172.21.15.46:0/3191470895 shutdown_connections 2023-12-02T00:29:41.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.571+0000 7f840bed0640 1 --2- 172.21.15.46:0/3191470895 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f84040d0ed0 0x7f84040d1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:41.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.571+0000 7f840bed0640 1 --2- 172.21.15.46:0/3191470895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f840406ea50 0x7f840406ee30 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:41.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.571+0000 7f840bed0640 1 -- 172.21.15.46:0/3191470895 >> 172.21.15.46:0/3191470895 conn(0x7f840406a6b0 msgr2=0x7f840406aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:41.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.571+0000 7f840bed0640 1 -- 172.21.15.46:0/3191470895 shutdown_connections 2023-12-02T00:29:41.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.571+0000 7f840bed0640 1 -- 172.21.15.46:0/3191470895 wait complete. 2023-12-02T00:29:41.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.572+0000 7f840bed0640 1 Processor -- start 2023-12-02T00:29:41.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.572+0000 7f840bed0640 1 -- start start 2023-12-02T00:29:41.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.573+0000 7f840bed0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f840406ea50 0x7f8404155fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:41.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.573+0000 7f840bed0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84040d0ed0 0x7f84041e9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:41.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.573+0000 7f840bed0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8404160fb0 0x7f840413f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:41.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.573+0000 7f840bed0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84041e7f60 con 0x7f8404160fb0 2023-12-02T00:29:41.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.573+0000 7f840bed0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84041e8240 con 0x7f84040d0ed0 2023-12-02T00:29:41.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.573+0000 7f840bed0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84041e8520 con 0x7f840406ea50 2023-12-02T00:29:41.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.573+0000 7f840a46d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8404160fb0 0x7f840413f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:41.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.573+0000 7f840a46d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8404160fb0 0x7f840413f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57574/0 (socket says 172.21.15.46:57574) 2023-12-02T00:29:41.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.573+0000 7f840a46d640 1 -- 172.21.15.46:0/1820429364 learned_addr learned my addr 172.21.15.46:0/1820429364 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:41.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.573+0000 7f8409c6c640 1 --2- 172.21.15.46:0/1820429364 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f840406ea50 0x7f8404155fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:41.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.573+0000 7f840946b640 1 --2- 172.21.15.46:0/1820429364 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84040d0ed0 0x7f84041e9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:41.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.574+0000 7f840a46d640 1 -- 172.21.15.46:0/1820429364 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f840406ea50 msgr2=0x7f8404155fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.574+0000 7f840a46d640 1 --2- 172.21.15.46:0/1820429364 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f840406ea50 0x7f8404155fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.574+0000 7f840a46d640 1 -- 172.21.15.46:0/1820429364 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84040d0ed0 msgr2=0x7f84041e9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.574+0000 7f840a46d640 1 --2- 172.21.15.46:0/1820429364 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84040d0ed0 0x7f84041e9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.574+0000 7f840a46d640 1 -- 172.21.15.46:0/1820429364 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83fc0094d0 con 0x7f8404160fb0 2023-12-02T00:29:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.574+0000 7f8409c6c640 1 --2- 172.21.15.46:0/1820429364 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f840406ea50 0x7f8404155fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.574+0000 7f840946b640 1 --2- 172.21.15.46:0/1820429364 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84040d0ed0 0x7f84041e9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.574+0000 7f840a46d640 1 --2- 172.21.15.46:0/1820429364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8404160fb0 0x7f840413f350 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f83f4007d90 tx=0x7f83f400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.574+0000 7f84027fc640 1 -- 172.21.15.46:0/1820429364 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f4014420 con 0x7f8404160fb0 2023-12-02T00:29:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.574+0000 7f84027fc640 1 -- 172.21.15.46:0/1820429364 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83f40145a0 con 0x7f8404160fb0 2023-12-02T00:29:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.574+0000 7f840bed0640 1 -- 172.21.15.46:0/1820429364 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84041e8800 con 0x7f8404160fb0 2023-12-02T00:29:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.574+0000 7f84027fc640 1 -- 172.21.15.46:0/1820429364 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f4014950 con 0x7f8404160fb0 2023-12-02T00:29:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.574+0000 7f840bed0640 1 -- 172.21.15.46:0/1820429364 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8404156500 con 0x7f8404160fb0 2023-12-02T00:29:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.575+0000 7f840bed0640 1 -- 172.21.15.46:0/1820429364 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84041e8d20 con 0x7f8404160fb0 2023-12-02T00:29:41.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.575+0000 7f84027fc640 1 -- 172.21.15.46:0/1820429364 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f83f4014ab0 con 0x7f8404160fb0 2023-12-02T00:29:41.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.576+0000 7f84027fc640 1 --2- 172.21.15.46:0/1820429364 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f83f006ae40 0x7f83f006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:41.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.576+0000 7f84027fc640 1 -- 172.21.15.46:0/1820429364 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5940+0+0 (secure 0 0 0) 0x7f83f408a060 con 0x7f8404160fb0 2023-12-02T00:29:41.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.576+0000 7f8409c6c640 1 --2- 172.21.15.46:0/1820429364 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f83f006ae40 0x7f83f006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:41.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.577+0000 7f8409c6c640 1 --2- 172.21.15.46:0/1820429364 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f83f006ae40 0x7f83f006d300 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f83fc036000 tx=0x7f83fc03c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:41.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.580+0000 7f84027fc640 1 -- 172.21.15.46:0/1820429364 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f84041e8d20 con 0x7f8404160fb0 2023-12-02T00:29:41.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:41.796+0000 7f840bed0640 1 -- 172.21.15.46:0/1820429364 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f84041e8d20 con 0x7f8404160fb0 2023-12-02T00:29:42.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.446+0000 7f84027fc640 1 -- 172.21.15.46:0/1820429364 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v8) v1 ==== 104+0+0 (secure 0 0 0) 0x7f84041e8d20 con 0x7f8404160fb0 2023-12-02T00:29:42.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.449+0000 7f840bed0640 1 -- 172.21.15.46:0/1820429364 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f83f006ae40 msgr2=0x7f83f006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:42.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.449+0000 7f840bed0640 1 --2- 172.21.15.46:0/1820429364 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f83f006ae40 0x7f83f006d300 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f83fc036000 tx=0x7f83fc03c040 comp rx=0 tx=0).stop 2023-12-02T00:29:42.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.449+0000 7f840bed0640 1 -- 172.21.15.46:0/1820429364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8404160fb0 msgr2=0x7f840413f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:42.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.449+0000 7f840bed0640 1 --2- 172.21.15.46:0/1820429364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8404160fb0 0x7f840413f350 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f83f4007d90 tx=0x7f83f400e4b0 comp rx=0 tx=0).stop 2023-12-02T00:29:42.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.449+0000 7f840bed0640 1 -- 172.21.15.46:0/1820429364 shutdown_connections 2023-12-02T00:29:42.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.449+0000 7f840bed0640 1 --2- 172.21.15.46:0/1820429364 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f83f006ae40 0x7f83f006d300 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:42.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.449+0000 7f840bed0640 1 --2- 172.21.15.46:0/1820429364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8404160fb0 0x7f840413f350 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:42.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.449+0000 7f840bed0640 1 --2- 172.21.15.46:0/1820429364 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84040d0ed0 0x7f84041e9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:42.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.449+0000 7f840bed0640 1 --2- 172.21.15.46:0/1820429364 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f840406ea50 0x7f8404155fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:42.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.449+0000 7f840bed0640 1 -- 172.21.15.46:0/1820429364 >> 172.21.15.46:0/1820429364 conn(0x7f840406a6b0 msgr2=0x7f840406aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:42.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.449+0000 7f840bed0640 1 -- 172.21.15.46:0/1820429364 shutdown_connections 2023-12-02T00:29:42.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.449+0000 7f840bed0640 1 -- 172.21.15.46:0/1820429364 wait complete. 2023-12-02T00:29:42.464 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-02T00:29:42.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.580+0000 7f66c12d4640 1 Processor -- start 2023-12-02T00:29:42.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.580+0000 7f66c12d4640 1 -- start start 2023-12-02T00:29:42.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.581+0000 7f66c12d4640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f66bc144980 0x7f66bc15c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:42.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.581+0000 7f66c12d4640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f66bc0b4370 0x7f66bc0b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:42.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.581+0000 7f66c12d4640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66bc071770 con 0x7f66bc0ceb70 2023-12-02T00:29:42.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.581+0000 7f66c12d4640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66bc0ccb60 con 0x7f66bc144980 2023-12-02T00:29:42.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.581+0000 7f66c12d4640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66bc0b4ce0 con 0x7f66bc0b4370 2023-12-02T00:29:42.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.581+0000 7f66baffd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f66bc144980 0x7f66bc15c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:42.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.581+0000 7f66baffd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f66bc144980 0x7f66bc15c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57584/0 (socket says 172.21.15.46:57584) 2023-12-02T00:29:42.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.581+0000 7f66baffd640 1 -- 172.21.15.46:0/2169990106 learned_addr learned my addr 172.21.15.46:0/2169990106 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:42.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.581+0000 7f66bb7fe640 1 --2- 172.21.15.46:0/2169990106 >> v2:172.21.15.162:3300/0 conn(0x7f66bc0b4370 0x7f66bc0b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:42.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.581+0000 7f66baffd640 1 -- 172.21.15.46:0/2169990106 >> v2:172.21.15.162:3300/0 conn(0x7f66bc0b4370 msgr2=0x7f66bc0b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:42.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.581+0000 7f66baffd640 1 --2- 172.21.15.46:0/2169990106 >> v2:172.21.15.162:3300/0 conn(0x7f66bc0b4370 0x7f66bc0b47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:42.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.581+0000 7f66baffd640 1 -- 172.21.15.46:0/2169990106 >> v1:172.21.15.162:6789/0 conn(0x7f66bc0ceb70 legacy=0x7f66bc0cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:42.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.581+0000 7f66baffd640 1 -- 172.21.15.46:0/2169990106 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66bc0ccdc0 con 0x7f66bc144980 2023-12-02T00:29:42.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.582+0000 7f66baffd640 1 --2- 172.21.15.46:0/2169990106 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66bc144980 0x7f66bc15c7c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f66bc0716e0 tx=0x7f66ac02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8da0db78410aef0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:42.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.582+0000 7f66ba7fc640 1 -- 172.21.15.46:0/2169990106 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ac031870 con 0x7f66bc144980 2023-12-02T00:29:42.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.582+0000 7f66ba7fc640 1 -- 172.21.15.46:0/2169990106 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66ac0319d0 con 0x7f66bc144980 2023-12-02T00:29:42.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.582+0000 7f66ba7fc640 1 -- 172.21.15.46:0/2169990106 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ac031d60 con 0x7f66bc144980 2023-12-02T00:29:42.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.583+0000 7f66c12d4640 1 -- 172.21.15.46:0/2169990106 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66bc144980 msgr2=0x7f66bc15c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:42.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.583+0000 7f66c12d4640 1 --2- 172.21.15.46:0/2169990106 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66bc144980 0x7f66bc15c7c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f66bc0716e0 tx=0x7f66ac02b910 comp rx=0 tx=0).stop 2023-12-02T00:29:42.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.583+0000 7f66c12d4640 1 -- 172.21.15.46:0/2169990106 shutdown_connections 2023-12-02T00:29:42.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.583+0000 7f66c12d4640 1 --2- 172.21.15.46:0/2169990106 >> v2:172.21.15.162:3300/0 conn(0x7f66bc0b4370 0x7f66bc0b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:42.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.583+0000 7f66c12d4640 1 --2- 172.21.15.46:0/2169990106 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66bc144980 0x7f66bc15c7c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:42.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.583+0000 7f66c12d4640 1 -- 172.21.15.46:0/2169990106 >> 172.21.15.46:0/2169990106 conn(0x7f66bc06a4d0 msgr2=0x7f66bc06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:42.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.583+0000 7f66c12d4640 1 -- 172.21.15.46:0/2169990106 shutdown_connections 2023-12-02T00:29:42.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.583+0000 7f66c12d4640 1 -- 172.21.15.46:0/2169990106 wait complete. 2023-12-02T00:29:42.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.584+0000 7f66c12d4640 1 Processor -- start 2023-12-02T00:29:42.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.584+0000 7f66c12d4640 1 -- start start 2023-12-02T00:29:42.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.585+0000 7f66c12d4640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66bc0b4370 0x7f66bc0b2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:42.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.585+0000 7f66c12d4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66bc0ceb70 0x7f66bc0ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:42.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.585+0000 7f66c12d4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66bc144980 0x7f66bc156d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:42.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.585+0000 7f66c12d4640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66bc157250 con 0x7f66bc0ceb70 2023-12-02T00:29:42.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.585+0000 7f66c12d4640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66bc0acf70 con 0x7f66bc0b4370 2023-12-02T00:29:42.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.585+0000 7f66c12d4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66bc0ad250 con 0x7f66bc144980 2023-12-02T00:29:42.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.585+0000 7f66b3fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66bc0ceb70 0x7f66bc0ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:42.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.585+0000 7f66b3fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66bc0ceb70 0x7f66bc0ac8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38414/0 (socket says 172.21.15.46:38414) 2023-12-02T00:29:42.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.585+0000 7f66b3fff640 1 -- 172.21.15.46:0/2012940235 learned_addr learned my addr 172.21.15.46:0/2012940235 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:42.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.585+0000 7f66bb7fe640 1 --2- 172.21.15.46:0/2012940235 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66bc144980 0x7f66bc156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:42.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.585+0000 7f66b3fff640 1 -- 172.21.15.46:0/2012940235 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66bc144980 msgr2=0x7f66bc156d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:42.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.585+0000 7f66baffd640 1 --2- 172.21.15.46:0/2012940235 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66bc0b4370 0x7f66bc0b2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:42.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.585+0000 7f66b3fff640 1 --2- 172.21.15.46:0/2012940235 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66bc144980 0x7f66bc156d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:42.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.585+0000 7f66b3fff640 1 -- 172.21.15.46:0/2012940235 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66bc0b4370 msgr2=0x7f66bc0b2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:42.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.585+0000 7f66b3fff640 1 --2- 172.21.15.46:0/2012940235 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66bc0b4370 0x7f66bc0b2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:42.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.585+0000 7f66b3fff640 1 -- 172.21.15.46:0/2012940235 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66ac0094d0 con 0x7f66bc0ceb70 2023-12-02T00:29:42.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.585+0000 7f66b3fff640 1 --2- 172.21.15.46:0/2012940235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66bc0ceb70 0x7f66bc0ac8e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f66a400c430 tx=0x7f66a400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:42.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.586+0000 7f66b37fe640 1 -- 172.21.15.46:0/2012940235 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a4011790 con 0x7f66bc0ceb70 2023-12-02T00:29:42.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.586+0000 7f66b37fe640 1 -- 172.21.15.46:0/2012940235 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66a40118f0 con 0x7f66bc0ceb70 2023-12-02T00:29:42.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.586+0000 7f66c12d4640 1 -- 172.21.15.46:0/2012940235 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66bc0ad530 con 0x7f66bc0ceb70 2023-12-02T00:29:42.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.586+0000 7f66b37fe640 1 -- 172.21.15.46:0/2012940235 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a4011ce0 con 0x7f66bc0ceb70 2023-12-02T00:29:42.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.586+0000 7f66c12d4640 1 -- 172.21.15.46:0/2012940235 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66bc1351c0 con 0x7f66bc0ceb70 2023-12-02T00:29:42.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.587+0000 7f66c12d4640 1 -- 172.21.15.46:0/2012940235 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66bc069400 con 0x7f66bc0ceb70 2023-12-02T00:29:42.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.587+0000 7f66b37fe640 1 -- 172.21.15.46:0/2012940235 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f66a401c460 con 0x7f66bc0ceb70 2023-12-02T00:29:42.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.587+0000 7f66b37fe640 1 --2- 172.21.15.46:0/2012940235 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f669c06ae40 0x7f669c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:42.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.588+0000 7f66b37fe640 1 -- 172.21.15.46:0/2012940235 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5940+0+0 (secure 0 0 0) 0x7f66a40896c0 con 0x7f66bc0ceb70 2023-12-02T00:29:42.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.588+0000 7f66baffd640 1 --2- 172.21.15.46:0/2012940235 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f669c06ae40 0x7f669c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:42.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.588+0000 7f66baffd640 1 --2- 172.21.15.46:0/2012940235 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f669c06ae40 0x7f669c06d300 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f66ac036000 tx=0x7f66ac03c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:42.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.592+0000 7f66b37fe640 1 -- 172.21.15.46:0/2012940235 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f66a4054680 con 0x7f66bc0ceb70 2023-12-02T00:29:42.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:42.770+0000 7f66c12d4640 1 -- 172.21.15.46:0/2012940235 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f66bc135380 con 0x7f66bc0ceb70 2023-12-02T00:29:43.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.448+0000 7f66b37fe640 1 -- 172.21.15.46:0/2012940235 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v25) v1 ==== 181+0+0 (secure 0 0 0) 0x7f66a4057ca0 con 0x7f66bc0ceb70 2023-12-02T00:29:43.449 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2023-12-02T00:29:43.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.451+0000 7f66c12d4640 1 -- 172.21.15.46:0/2012940235 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f669c06ae40 msgr2=0x7f669c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:43.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.451+0000 7f66c12d4640 1 --2- 172.21.15.46:0/2012940235 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f669c06ae40 0x7f669c06d300 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f66ac036000 tx=0x7f66ac03c040 comp rx=0 tx=0).stop 2023-12-02T00:29:43.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.451+0000 7f66c12d4640 1 -- 172.21.15.46:0/2012940235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66bc0ceb70 msgr2=0x7f66bc0ac8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:43.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.451+0000 7f66c12d4640 1 --2- 172.21.15.46:0/2012940235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66bc0ceb70 0x7f66bc0ac8e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f66a400c430 tx=0x7f66a400c8f0 comp rx=0 tx=0).stop 2023-12-02T00:29:43.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.452+0000 7f66c12d4640 1 -- 172.21.15.46:0/2012940235 shutdown_connections 2023-12-02T00:29:43.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.452+0000 7f66c12d4640 1 --2- 172.21.15.46:0/2012940235 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f669c06ae40 0x7f669c06d300 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:43.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.452+0000 7f66c12d4640 1 --2- 172.21.15.46:0/2012940235 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66bc144980 0x7f66bc156d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:43.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.452+0000 7f66c12d4640 1 --2- 172.21.15.46:0/2012940235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66bc0ceb70 0x7f66bc0ac8e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:43.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.452+0000 7f66c12d4640 1 --2- 172.21.15.46:0/2012940235 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66bc0b4370 0x7f66bc0b2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:43.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.452+0000 7f66c12d4640 1 -- 172.21.15.46:0/2012940235 >> 172.21.15.46:0/2012940235 conn(0x7f66bc06a4d0 msgr2=0x7f66bc143c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:43.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.452+0000 7f66c12d4640 1 -- 172.21.15.46:0/2012940235 shutdown_connections 2023-12-02T00:29:43.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.452+0000 7f66c12d4640 1 -- 172.21.15.46:0/2012940235 wait complete. 2023-12-02T00:29:43.468 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-02T00:29:43.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.583+0000 7f2fb3f39640 1 Processor -- start 2023-12-02T00:29:43.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.584+0000 7f2fb3f39640 1 -- start start 2023-12-02T00:29:43.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.584+0000 7f2fb3f39640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2fac15d330 0x7f2fac0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:43.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.584+0000 7f2fb3f39640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2fac0d3900 0x7f2fac0b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:43.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.584+0000 7f2fb3f39640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fac0b2b10 con 0x7f2fac0d2910 2023-12-02T00:29:43.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.584+0000 7f2fb3f39640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fac0d1590 con 0x7f2fac0d3900 2023-12-02T00:29:43.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.584+0000 7f2fb3f39640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fac0d1870 con 0x7f2fac15d330 2023-12-02T00:29:43.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.585+0000 7f2fb24d6640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2fac0d3900 0x7f2fac0b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:43.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.585+0000 7f2fb24d6640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2fac0d3900 0x7f2fac0b6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38418/0 (socket says 172.21.15.46:38418) 2023-12-02T00:29:43.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.585+0000 7f2fb24d6640 1 -- 172.21.15.46:0/2524224419 learned_addr learned my addr 172.21.15.46:0/2524224419 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:43.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.585+0000 7f2fb24d6640 1 -- 172.21.15.46:0/2524224419 >> v2:172.21.15.162:3300/0 conn(0x7f2fac15d330 msgr2=0x7f2fac0b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:43.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.585+0000 7f2fb1cd5640 1 --2- 172.21.15.46:0/2524224419 >> v2:172.21.15.162:3300/0 conn(0x7f2fac15d330 0x7f2fac0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:43.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.585+0000 7f2fb24d6640 1 --2- 172.21.15.46:0/2524224419 >> v2:172.21.15.162:3300/0 conn(0x7f2fac15d330 0x7f2fac0b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:43.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.585+0000 7f2fb24d6640 1 -- 172.21.15.46:0/2524224419 >> v1:172.21.15.162:6789/0 conn(0x7f2fac0d2910 legacy=0x7f2fac0d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:43.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.585+0000 7f2fb24d6640 1 -- 172.21.15.46:0/2524224419 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fac0d1b50 con 0x7f2fac0d3900 2023-12-02T00:29:43.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.585+0000 7f2fb1cd5640 1 --2- 172.21.15.46:0/2524224419 >> v2:172.21.15.162:3300/0 conn(0x7f2fac15d330 0x7f2fac0b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:29:43.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.586+0000 7f2fb24d6640 1 --2- 172.21.15.46:0/2524224419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fac0d3900 0x7f2fac0b6390 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2fac0d0d10 tx=0x7f2fa402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8770167101e4a622 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:43.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.586+0000 7f2fb0cd3640 1 -- 172.21.15.46:0/2524224419 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa402c9f0 con 0x7f2fac0d3900 2023-12-02T00:29:43.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.586+0000 7f2fb0cd3640 1 -- 172.21.15.46:0/2524224419 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa402cb50 con 0x7f2fac0d3900 2023-12-02T00:29:43.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.586+0000 7f2fb0cd3640 1 -- 172.21.15.46:0/2524224419 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa40343f0 con 0x7f2fac0d3900 2023-12-02T00:29:43.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.586+0000 7f2fb3f39640 1 -- 172.21.15.46:0/2524224419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fac0d3900 msgr2=0x7f2fac0b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:43.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.586+0000 7f2fb3f39640 1 --2- 172.21.15.46:0/2524224419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fac0d3900 0x7f2fac0b6390 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2fac0d0d10 tx=0x7f2fa402be50 comp rx=0 tx=0).stop 2023-12-02T00:29:43.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.586+0000 7f2fb3f39640 1 -- 172.21.15.46:0/2524224419 shutdown_connections 2023-12-02T00:29:43.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.586+0000 7f2fb3f39640 1 --2- 172.21.15.46:0/2524224419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fac0d3900 0x7f2fac0b6390 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:43.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.586+0000 7f2fb3f39640 1 --2- 172.21.15.46:0/2524224419 >> v2:172.21.15.162:3300/0 conn(0x7f2fac15d330 0x7f2fac0b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:43.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.586+0000 7f2fb3f39640 1 -- 172.21.15.46:0/2524224419 >> 172.21.15.46:0/2524224419 conn(0x7f2fac072020 msgr2=0x7f2fac072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:43.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.587+0000 7f2fb3f39640 1 -- 172.21.15.46:0/2524224419 shutdown_connections 2023-12-02T00:29:43.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.587+0000 7f2fb3f39640 1 -- 172.21.15.46:0/2524224419 wait complete. 2023-12-02T00:29:43.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.588+0000 7f2fb3f39640 1 Processor -- start 2023-12-02T00:29:43.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.588+0000 7f2fb3f39640 1 -- start start 2023-12-02T00:29:43.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.588+0000 7f2fb3f39640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2fac0d2910 0x7f2fac0b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:43.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.588+0000 7f2fb3f39640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fac0d3900 0x7f2fac0b4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:43.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.588+0000 7f2fb3f39640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2fac15d330 0x7f2fac0c77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:43.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.588+0000 7f2fb3f39640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fac0c7390 con 0x7f2fac0d3900 2023-12-02T00:29:43.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.588+0000 7f2fb3f39640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fac0ca330 con 0x7f2fac15d330 2023-12-02T00:29:43.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.588+0000 7f2fb3f39640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fac0ca5d0 con 0x7f2fac0d2910 2023-12-02T00:29:43.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.589+0000 7f2fb14d4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fac0d3900 0x7f2fac0b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:43.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.589+0000 7f2fb14d4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fac0d3900 0x7f2fac0b4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38432/0 (socket says 172.21.15.46:38432) 2023-12-02T00:29:43.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.589+0000 7f2fb14d4640 1 -- 172.21.15.46:0/3935459302 learned_addr learned my addr 172.21.15.46:0/3935459302 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:43.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.589+0000 7f2fb24d6640 1 --2- 172.21.15.46:0/3935459302 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2fac15d330 0x7f2fac0c77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:43.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.589+0000 7f2fb1cd5640 1 --2- 172.21.15.46:0/3935459302 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2fac0d2910 0x7f2fac0b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:43.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.589+0000 7f2fb14d4640 1 -- 172.21.15.46:0/3935459302 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2fac0d2910 msgr2=0x7f2fac0b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:43.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.589+0000 7f2fb14d4640 1 --2- 172.21.15.46:0/3935459302 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2fac0d2910 0x7f2fac0b3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:43.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.589+0000 7f2fb14d4640 1 -- 172.21.15.46:0/3935459302 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2fac15d330 msgr2=0x7f2fac0c77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:43.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.589+0000 7f2fb14d4640 1 --2- 172.21.15.46:0/3935459302 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2fac15d330 0x7f2fac0c77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:43.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.589+0000 7f2fb14d4640 1 -- 172.21.15.46:0/3935459302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa4009b30 con 0x7f2fac0d3900 2023-12-02T00:29:43.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.589+0000 7f2fb14d4640 1 --2- 172.21.15.46:0/3935459302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fac0d3900 0x7f2fac0b4270 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c00c430 tx=0x7f2f9c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:43.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.589+0000 7f2fa27fc640 1 -- 172.21.15.46:0/3935459302 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c011790 con 0x7f2fac0d3900 2023-12-02T00:29:43.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.589+0000 7f2fa27fc640 1 -- 172.21.15.46:0/3935459302 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f9c0118f0 con 0x7f2fac0d3900 2023-12-02T00:29:43.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.589+0000 7f2fa27fc640 1 -- 172.21.15.46:0/3935459302 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c011ce0 con 0x7f2fac0d3900 2023-12-02T00:29:43.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.589+0000 7f2fb3f39640 1 -- 172.21.15.46:0/3935459302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fac0c6cd0 con 0x7f2fac0d3900 2023-12-02T00:29:43.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.589+0000 7f2fb3f39640 1 -- 172.21.15.46:0/3935459302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fac0ca8b0 con 0x7f2fac0d3900 2023-12-02T00:29:43.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.590+0000 7f2fb3f39640 1 -- 172.21.15.46:0/3935459302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fac0c74d0 con 0x7f2fac0d3900 2023-12-02T00:29:43.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.591+0000 7f2fa27fc640 1 -- 172.21.15.46:0/3935459302 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2f9c01c460 con 0x7f2fac0d3900 2023-12-02T00:29:43.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.591+0000 7f2fa27fc640 1 --2- 172.21.15.46:0/3935459302 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2f9006ae70 0x7f2f9006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:43.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.591+0000 7f2fa27fc640 1 -- 172.21.15.46:0/3935459302 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5502+0+0 (secure 0 0 0) 0x7f2f9c0892d0 con 0x7f2fac0d3900 2023-12-02T00:29:43.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.591+0000 7f2fb1cd5640 1 --2- 172.21.15.46:0/3935459302 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2f9006ae70 0x7f2f9006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:43.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.592+0000 7f2fb1cd5640 1 --2- 172.21.15.46:0/3935459302 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2f9006ae70 0x7f2f9006d330 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2fa800aac0 tx=0x7f2fa8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:43.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.595+0000 7f2fa27fc640 1 -- 172.21.15.46:0/3935459302 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2fac0c74d0 con 0x7f2fac0d3900 2023-12-02T00:29:43.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:43.779+0000 7f2fb3f39640 1 -- 172.21.15.46:0/3935459302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2fac0c74d0 con 0x7f2fac0d3900 2023-12-02T00:29:44.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:44.449+0000 7f2fa27fc640 1 -- 172.21.15.46:0/3935459302 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v26) v1 ==== 169+0+0 (secure 0 0 0) 0x7f2fac0c74d0 con 0x7f2fac0d3900 2023-12-02T00:29:44.450 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2023-12-02T00:29:44.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:44.453+0000 7f2fb3f39640 1 -- 172.21.15.46:0/3935459302 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2f9006ae70 msgr2=0x7f2f9006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:44.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:44.453+0000 7f2fb3f39640 1 --2- 172.21.15.46:0/3935459302 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2f9006ae70 0x7f2f9006d330 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2fa800aac0 tx=0x7f2fa8008040 comp rx=0 tx=0).stop 2023-12-02T00:29:44.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:44.453+0000 7f2fb3f39640 1 -- 172.21.15.46:0/3935459302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fac0d3900 msgr2=0x7f2fac0b4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:44.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:44.453+0000 7f2fb3f39640 1 --2- 172.21.15.46:0/3935459302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fac0d3900 0x7f2fac0b4270 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c00c430 tx=0x7f2f9c00c8f0 comp rx=0 tx=0).stop 2023-12-02T00:29:44.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:44.453+0000 7f2fb3f39640 1 -- 172.21.15.46:0/3935459302 shutdown_connections 2023-12-02T00:29:44.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:44.453+0000 7f2fb3f39640 1 --2- 172.21.15.46:0/3935459302 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2f9006ae70 0x7f2f9006d330 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:44.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:44.453+0000 7f2fb3f39640 1 --2- 172.21.15.46:0/3935459302 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2fac15d330 0x7f2fac0c77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:44.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:44.453+0000 7f2fb3f39640 1 --2- 172.21.15.46:0/3935459302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fac0d3900 0x7f2fac0b4270 secure :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c00c430 tx=0x7f2f9c00c8f0 comp rx=0 tx=0).stop 2023-12-02T00:29:44.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:44.453+0000 7f2fb3f39640 1 --2- 172.21.15.46:0/3935459302 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2fac0d2910 0x7f2fac0b3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:44.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:44.453+0000 7f2fb3f39640 1 -- 172.21.15.46:0/3935459302 >> 172.21.15.46:0/3935459302 conn(0x7f2fac072020 msgr2=0x7f2fac0d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:44.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:44.453+0000 7f2fb3f39640 1 -- 172.21.15.46:0/3935459302 shutdown_connections 2023-12-02T00:29:44.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:44.453+0000 7f2fb3f39640 1 -- 172.21.15.46:0/3935459302 wait complete. 2023-12-02T00:29:44.471 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-02T00:29:44.471 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-02T00:29:44.471 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-02T00:29:44.472 INFO:teuthology.orchestra.run:waiting for 300 2023-12-02T00:29:44.472 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-02T00:29:44.472 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-02T00:29:44.472 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-02T00:29:44.472 INFO:teuthology.orchestra.run:waiting for 300 2023-12-02T00:29:44.472 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-02T00:29:44.472 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-02T00:29:44.473 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-02T00:29:44.473 INFO:teuthology.orchestra.run:waiting for 300 2023-12-02T00:29:44.473 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-02T00:29:44.473 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-02T00:29:44.473 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-02T00:29:44.473 INFO:teuthology.orchestra.run:waiting for 300 2023-12-02T00:29:44.482 INFO:tasks.ceph.mds.a:Stopped 2023-12-02T00:29:44.483 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-02T00:29:44.483 INFO:tasks.ceph.mds.c:Stopped 2023-12-02T00:29:44.484 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-02T00:29:44.485 INFO:tasks.ceph.mds.d:Stopped 2023-12-02T00:29:44.485 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-02T00:29:44.525 INFO:tasks.ceph.mds.a:Started 2023-12-02T00:29:44.526 DEBUG:teuthology.parallel:result is None 2023-12-02T00:29:44.527 INFO:tasks.ceph.mds.c:Started 2023-12-02T00:29:44.528 DEBUG:teuthology.parallel:result is None 2023-12-02T00:29:44.528 INFO:tasks.ceph.mds.d:Started 2023-12-02T00:29:44.528 DEBUG:teuthology.parallel:result is None 2023-12-02T00:29:44.606 INFO:tasks.ceph.mds.d.smithi162.stdout:starting mds.d at 2023-12-02T00:29:44.611 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2023-12-02T00:29:44.612 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2023-12-02T00:29:46.114 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-12-02T00:29:50.460 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-02T00:29:50.463 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-02T00:29:50.485 INFO:tasks.ceph.mds.b:Stopped 2023-12-02T00:29:50.486 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-02T00:29:50.509 INFO:tasks.ceph.mds.b:Started 2023-12-02T00:29:50.509 DEBUG:teuthology.parallel:result is None 2023-12-02T00:29:50.510 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-02T00:29:50.590 INFO:tasks.ceph.mds.b.smithi162.stdout:starting mds.b at 2023-12-02T00:29:50.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.624+0000 7fcdd4843640 1 Processor -- start 2023-12-02T00:29:50.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.624+0000 7fcdd4843640 1 -- start start 2023-12-02T00:29:50.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.625+0000 7fcdd4843640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcdd0144980 0x7fcdd015c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:50.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.625+0000 7fcdd4843640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdd0071770 con 0x7fcdd00b4370 2023-12-02T00:29:50.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.625+0000 7fcdd4843640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdd00ccb60 con 0x7fcdd00ceb70 2023-12-02T00:29:50.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.625+0000 7fcdd4843640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdd00ccdc0 con 0x7fcdd0144980 2023-12-02T00:29:50.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.625+0000 7fcdce59c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcdd0144980 0x7fcdd015c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:50.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.625+0000 7fcdced9d640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fcdd00b4370 0x7fcdd0071c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42814/0 (socket says 172.21.15.46:42814) 2023-12-02T00:29:50.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.625+0000 7fcdced9d640 1 -- 172.21.15.46:0/3994920112 learned_addr learned my addr 172.21.15.46:0/3994920112 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:50.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.626+0000 7fcdce59c640 1 -- 172.21.15.46:0/3994920112 >> v1:172.21.15.162:6789/0 conn(0x7fcdd00ceb70 legacy=0x7fcdd00cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:50.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.626+0000 7fcdce59c640 1 -- 172.21.15.46:0/3994920112 >> v1:172.21.15.46:6789/0 conn(0x7fcdd00b4370 legacy=0x7fcdd0071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:50.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.626+0000 7fcdce59c640 1 -- 172.21.15.46:0/3994920112 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd00cd0a0 con 0x7fcdd0144980 2023-12-02T00:29:50.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.626+0000 7fcdce59c640 1 --2- 172.21.15.46:0/3994920112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdd0144980 0x7fcdd015c7c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fcdc80097a0 tx=0x7fcdc802b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e759353906094ceb server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:50.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.627+0000 7fcdcd59a640 1 -- 172.21.15.46:0/3994920112 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc8030710 con 0x7fcdd0144980 2023-12-02T00:29:50.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.627+0000 7fcdcd59a640 1 -- 172.21.15.46:0/3994920112 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdc802e050 con 0x7fcdd0144980 2023-12-02T00:29:50.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.627+0000 7fcdcd59a640 1 -- 172.21.15.46:0/3994920112 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc8030a90 con 0x7fcdd0144980 2023-12-02T00:29:50.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.627+0000 7fcdd4843640 1 -- 172.21.15.46:0/3994920112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdd0144980 msgr2=0x7fcdd015c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:50.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.627+0000 7fcdd4843640 1 --2- 172.21.15.46:0/3994920112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdd0144980 0x7fcdd015c7c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fcdc80097a0 tx=0x7fcdc802b790 comp rx=0 tx=0).stop 2023-12-02T00:29:50.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.627+0000 7fcdd4843640 1 -- 172.21.15.46:0/3994920112 shutdown_connections 2023-12-02T00:29:50.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.627+0000 7fcdd4843640 1 --2- 172.21.15.46:0/3994920112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdd0144980 0x7fcdd015c7c0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:50.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.627+0000 7fcdd4843640 1 -- 172.21.15.46:0/3994920112 >> 172.21.15.46:0/3994920112 conn(0x7fcdd006a4d0 msgr2=0x7fcdd006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:50.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.627+0000 7fcdd4843640 1 -- 172.21.15.46:0/3994920112 shutdown_connections 2023-12-02T00:29:50.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.627+0000 7fcdd4843640 1 -- 172.21.15.46:0/3994920112 wait complete. 2023-12-02T00:29:50.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.629+0000 7fcdd4843640 1 Processor -- start 2023-12-02T00:29:50.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.629+0000 7fcdd4843640 1 -- start start 2023-12-02T00:29:50.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.629+0000 7fcdd4843640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcdd00b4370 0x7fcdd0138830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:50.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.629+0000 7fcdd4843640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdd00ceb70 0x7fcdd01584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:50.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.629+0000 7fcdd4843640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcdd0144980 0x7fcdd00b0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:50.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.629+0000 7fcdd4843640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd013cf50 con 0x7fcdd00ceb70 2023-12-02T00:29:50.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.629+0000 7fcdd4843640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd00af840 con 0x7fcdd00b4370 2023-12-02T00:29:50.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.629+0000 7fcdd4843640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd00afb20 con 0x7fcdd0144980 2023-12-02T00:29:50.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.629+0000 7fcdcdd9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdd00ceb70 0x7fcdd01584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:50.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.630+0000 7fcdcdd9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdd00ceb70 0x7fcdd01584b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38496/0 (socket says 172.21.15.46:38496) 2023-12-02T00:29:50.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.630+0000 7fcdcdd9b640 1 -- 172.21.15.46:0/2371599904 learned_addr learned my addr 172.21.15.46:0/2371599904 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:50.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.630+0000 7fcdced9d640 1 --2- 172.21.15.46:0/2371599904 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcdd0144980 0x7fcdd00b0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:50.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.630+0000 7fcdce59c640 1 --2- 172.21.15.46:0/2371599904 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcdd00b4370 0x7fcdd0138830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:50.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.630+0000 7fcdcdd9b640 1 -- 172.21.15.46:0/2371599904 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcdd0144980 msgr2=0x7fcdd00b0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:50.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.630+0000 7fcdcdd9b640 1 --2- 172.21.15.46:0/2371599904 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcdd0144980 0x7fcdd00b0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:50.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.630+0000 7fcdcdd9b640 1 -- 172.21.15.46:0/2371599904 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcdd00b4370 msgr2=0x7fcdd0138830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:50.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.630+0000 7fcdcdd9b640 1 --2- 172.21.15.46:0/2371599904 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcdd00b4370 0x7fcdd0138830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:50.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.630+0000 7fcdcdd9b640 1 -- 172.21.15.46:0/2371599904 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc8009400 con 0x7fcdd00ceb70 2023-12-02T00:29:50.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.630+0000 7fcdcdd9b640 1 --2- 172.21.15.46:0/2371599904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdd00ceb70 0x7fcdd01584b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fcdb800c450 tx=0x7fcdb800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:50.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.630+0000 7fcdc6ffd640 1 -- 172.21.15.46:0/2371599904 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb80052d0 con 0x7fcdd00ceb70 2023-12-02T00:29:50.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.630+0000 7fcdc6ffd640 1 -- 172.21.15.46:0/2371599904 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdb8005430 con 0x7fcdd00ceb70 2023-12-02T00:29:50.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.630+0000 7fcdd4843640 1 -- 172.21.15.46:0/2371599904 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdd00afe00 con 0x7fcdd00ceb70 2023-12-02T00:29:50.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.630+0000 7fcdc6ffd640 1 -- 172.21.15.46:0/2371599904 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb8012870 con 0x7fcdd00ceb70 2023-12-02T00:29:50.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.630+0000 7fcdd4843640 1 -- 172.21.15.46:0/2371599904 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdd0138d70 con 0x7fcdd00ceb70 2023-12-02T00:29:50.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.631+0000 7fcdd4843640 1 -- 172.21.15.46:0/2371599904 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdd0069400 con 0x7fcdd00ceb70 2023-12-02T00:29:50.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.632+0000 7fcdc6ffd640 1 -- 172.21.15.46:0/2371599904 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcdb80129d0 con 0x7fcdd00ceb70 2023-12-02T00:29:50.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.632+0000 7fcdc6ffd640 1 --2- 172.21.15.46:0/2371599904 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fcdb006ae40 0x7fcdb006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:50.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.632+0000 7fcdc6ffd640 1 -- 172.21.15.46:0/2371599904 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fcdb8088db0 con 0x7fcdd00ceb70 2023-12-02T00:29:50.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.633+0000 7fcdce59c640 1 --2- 172.21.15.46:0/2371599904 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fcdb006ae40 0x7fcdb006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:50.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.633+0000 7fcdce59c640 1 --2- 172.21.15.46:0/2371599904 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fcdb006ae40 0x7fcdb006d300 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8034000 tx=0x7fcdc802f000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:50.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.637+0000 7fcdc6ffd640 1 -- 172.21.15.46:0/2371599904 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcdb808d070 con 0x7fcdd00ceb70 2023-12-02T00:29:50.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:50.829+0000 7fcdd4843640 1 -- 172.21.15.46:0/2371599904 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fcdd006ab70 con 0x7fcdd00ceb70 2023-12-02T00:29:51.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.719+0000 7fcdc6ffd640 1 -- 172.21.15.46:0/2371599904 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v31) v1 ==== 97+0+0 (secure 0 0 0) 0x7fcdb8054170 con 0x7fcdd00ceb70 2023-12-02T00:29:51.721 INFO:teuthology.orchestra.run.smithi046.stderr: removed all blocklist entries 2023-12-02T00:29:51.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.722+0000 7fcdd4843640 1 -- 172.21.15.46:0/2371599904 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fcdb006ae40 msgr2=0x7fcdb006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:51.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.722+0000 7fcdd4843640 1 --2- 172.21.15.46:0/2371599904 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fcdb006ae40 0x7fcdb006d300 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8034000 tx=0x7fcdc802f000 comp rx=0 tx=0).stop 2023-12-02T00:29:51.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.722+0000 7fcdd4843640 1 -- 172.21.15.46:0/2371599904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdd00ceb70 msgr2=0x7fcdd01584b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:51.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.722+0000 7fcdd4843640 1 --2- 172.21.15.46:0/2371599904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdd00ceb70 0x7fcdd01584b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fcdb800c450 tx=0x7fcdb800c910 comp rx=0 tx=0).stop 2023-12-02T00:29:51.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.722+0000 7fcdd4843640 1 -- 172.21.15.46:0/2371599904 shutdown_connections 2023-12-02T00:29:51.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.722+0000 7fcdd4843640 1 --2- 172.21.15.46:0/2371599904 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fcdb006ae40 0x7fcdb006d300 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:51.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.722+0000 7fcdd4843640 1 --2- 172.21.15.46:0/2371599904 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcdd0144980 0x7fcdd00b0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:51.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.722+0000 7fcdd4843640 1 --2- 172.21.15.46:0/2371599904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdd00ceb70 0x7fcdd01584b0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:51.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.722+0000 7fcdd4843640 1 --2- 172.21.15.46:0/2371599904 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcdd00b4370 0x7fcdd0138830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:51.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.722+0000 7fcdd4843640 1 -- 172.21.15.46:0/2371599904 >> 172.21.15.46:0/2371599904 conn(0x7fcdd006a4d0 msgr2=0x7fcdd0159e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:51.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.723+0000 7fcdd4843640 1 -- 172.21.15.46:0/2371599904 shutdown_connections 2023-12-02T00:29:51.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.723+0000 7fcdd4843640 1 -- 172.21.15.46:0/2371599904 wait complete. 2023-12-02T00:29:51.737 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-02T00:29:51.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.848+0000 7fe2bdbe3640 1 Processor -- start 2023-12-02T00:29:51.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.848+0000 7fe2bdbe3640 1 -- start start 2023-12-02T00:29:51.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.849+0000 7fe2bdbe3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2b806ea50 0x7fe2b806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:51.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.849+0000 7fe2bdbe3640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe2b80d0ed0 0x7fe2b80d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:51.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.849+0000 7fe2bdbe3640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2b8161d20 con 0x7fe2b8160fb0 2023-12-02T00:29:51.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.849+0000 7fe2bdbe3640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2b8161b20 con 0x7fe2b80d0ed0 2023-12-02T00:29:51.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.849+0000 7fe2bdbe3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b806f370 con 0x7fe2b806ea50 2023-12-02T00:29:51.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.849+0000 7fe2b6ffd640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe2b8160fb0 0x7fe2b8161410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42828/0 (socket says 172.21.15.46:42828) 2023-12-02T00:29:51.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.849+0000 7fe2b6ffd640 1 -- 172.21.15.46:0/1161824700 learned_addr learned my addr 172.21.15.46:0/1161824700 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:51.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.849+0000 7fe2b7fff640 1 --2- 172.21.15.46:0/1161824700 >> v2:172.21.15.162:3300/0 conn(0x7fe2b80d0ed0 0x7fe2b80d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:51.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.849+0000 7fe2b77fe640 1 --2- 172.21.15.46:0/1161824700 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2b806ea50 0x7fe2b806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:51.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.849+0000 7fe2b67fc640 1 -- 172.21.15.46:0/1161824700 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3697597967 0 0) 0x7fe2b8161d20 con 0x7fe2b8160fb0 2023-12-02T00:29:51.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.849+0000 7fe2b67fc640 1 -- 172.21.15.46:0/1161824700 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2a0003520 con 0x7fe2b8160fb0 2023-12-02T00:29:51.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.850+0000 7fe2b67fc640 1 -- 172.21.15.46:0/1161824700 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2981096695 0 0) 0x7fe2a0003520 con 0x7fe2b8160fb0 2023-12-02T00:29:51.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.850+0000 7fe2b67fc640 1 -- 172.21.15.46:0/1161824700 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2b806ea50 msgr2=0x7fe2b806ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:51.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.850+0000 7fe2b67fc640 1 --2- 172.21.15.46:0/1161824700 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2b806ea50 0x7fe2b806ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:51.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.850+0000 7fe2b67fc640 1 -- 172.21.15.46:0/1161824700 >> v2:172.21.15.162:3300/0 conn(0x7fe2b80d0ed0 msgr2=0x7fe2b80d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:51.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.850+0000 7fe2b67fc640 1 --2- 172.21.15.46:0/1161824700 >> v2:172.21.15.162:3300/0 conn(0x7fe2b80d0ed0 0x7fe2b80d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:51.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.850+0000 7fe2b67fc640 1 -- 172.21.15.46:0/1161824700 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b815e760 con 0x7fe2b8160fb0 2023-12-02T00:29:51.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.850+0000 7fe2b67fc640 1 -- 172.21.15.46:0/1161824700 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7fe2ac001670 con 0x7fe2b8160fb0 2023-12-02T00:29:51.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.850+0000 7fe2b67fc640 1 -- 172.21.15.46:0/1161824700 >> v1:172.21.15.46:6789/0 conn(0x7fe2b8160fb0 legacy=0x7fe2b8161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:51.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.850+0000 7fe2b67fc640 1 --2- 172.21.15.46:0/1161824700 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2b806ea50 0x7fe2a0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:51.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.850+0000 7fe2b67fc640 1 --2- 172.21.15.46:0/1161824700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2a0003ef0 0x7fe2a00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:51.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.850+0000 7fe2b67fc640 1 --2- 172.21.15.46:0/1161824700 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2a0004d00 0x7fe2a0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:51.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.850+0000 7fe2b67fc640 1 -- 172.21.15.46:0/1161824700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b8161b20 con 0x7fe2a0003ef0 2023-12-02T00:29:51.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.850+0000 7fe2b67fc640 1 -- 172.21.15.46:0/1161824700 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b806f370 con 0x7fe2a0004d00 2023-12-02T00:29:51.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.850+0000 7fe2b67fc640 1 -- 172.21.15.46:0/1161824700 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a000b980 con 0x7fe2b806ea50 2023-12-02T00:29:51.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.850+0000 7fe2b6ffd640 1 --2- 172.21.15.46:0/1161824700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2a0003ef0 0x7fe2a00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:51.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.850+0000 7fe2b77fe640 1 --2- 172.21.15.46:0/1161824700 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2b806ea50 0x7fe2a0003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:51.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.850+0000 7fe2b7fff640 1 --2- 172.21.15.46:0/1161824700 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2a0004d00 0x7fe2a0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:51.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.850+0000 7fe2b6ffd640 1 -- 172.21.15.46:0/1161824700 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2b806ea50 msgr2=0x7fe2a0003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:51.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.850+0000 7fe2b6ffd640 1 --2- 172.21.15.46:0/1161824700 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2b806ea50 0x7fe2a0003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:51.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.850+0000 7fe2b6ffd640 1 -- 172.21.15.46:0/1161824700 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2a0004d00 msgr2=0x7fe2a0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:51.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.850+0000 7fe2b6ffd640 1 --2- 172.21.15.46:0/1161824700 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2a0004d00 0x7fe2a0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:51.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.850+0000 7fe2b6ffd640 1 -- 172.21.15.46:0/1161824700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a0003520 con 0x7fe2a0003ef0 2023-12-02T00:29:51.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.850+0000 7fe2b7fff640 1 --2- 172.21.15.46:0/1161824700 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2a0004d00 0x7fe2a0005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:51.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.851+0000 7fe2b6ffd640 1 --2- 172.21.15.46:0/1161824700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2a0003ef0 0x7fe2a00047c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac00c240 tx=0x7fe2a00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=795a713c5c18f91 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:51.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.851+0000 7fe2b67fc640 1 -- 172.21.15.46:0/1161824700 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac02cae0 con 0x7fe2a0003ef0 2023-12-02T00:29:51.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.851+0000 7fe2b67fc640 1 -- 172.21.15.46:0/1161824700 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2ac02cc40 con 0x7fe2a0003ef0 2023-12-02T00:29:51.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.851+0000 7fe2b67fc640 1 -- 172.21.15.46:0/1161824700 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac005b10 con 0x7fe2a0003ef0 2023-12-02T00:29:51.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.852+0000 7fe2bdbe3640 1 -- 172.21.15.46:0/1161824700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2a0003ef0 msgr2=0x7fe2a00047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:51.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.852+0000 7fe2bdbe3640 1 --2- 172.21.15.46:0/1161824700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2a0003ef0 0x7fe2a00047c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac00c240 tx=0x7fe2a00057f0 comp rx=0 tx=0).stop 2023-12-02T00:29:51.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.852+0000 7fe2b7fff640 1 -- 172.21.15.46:0/1161824700 reap_dead start 2023-12-02T00:29:51.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.852+0000 7fe2bdbe3640 1 -- 172.21.15.46:0/1161824700 shutdown_connections 2023-12-02T00:29:51.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.852+0000 7fe2bdbe3640 1 -- 172.21.15.46:0/1161824700 >> 172.21.15.46:0/1161824700 conn(0x7fe2b806a6b0 msgr2=0x7fe2b806aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:51.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.852+0000 7fe2bdbe3640 1 -- 172.21.15.46:0/1161824700 shutdown_connections 2023-12-02T00:29:51.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.852+0000 7fe2bdbe3640 1 -- 172.21.15.46:0/1161824700 wait complete. 2023-12-02T00:29:51.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.853+0000 7fe2bdbe3640 1 Processor -- start 2023-12-02T00:29:51.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.853+0000 7fe2bdbe3640 1 -- start start 2023-12-02T00:29:51.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.854+0000 7fe2bdbe3640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2a0003ef0 0x7fe2b813d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:51.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.854+0000 7fe2bdbe3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2b81e8f20 0x7fe2b806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:51.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.854+0000 7fe2bdbe3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2b806cb70 0x7fe2b8146c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:51.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.854+0000 7fe2bdbe3640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b81e8b00 con 0x7fe2b806cb70 2023-12-02T00:29:51.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.854+0000 7fe2bdbe3640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b806cff0 con 0x7fe2a0003ef0 2023-12-02T00:29:51.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.854+0000 7fe2bdbe3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b8158fb0 con 0x7fe2b81e8f20 2023-12-02T00:29:51.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.854+0000 7fe2b7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2b806cb70 0x7fe2b8146c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:51.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.854+0000 7fe2b7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2b806cb70 0x7fe2b8146c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38506/0 (socket says 172.21.15.46:38506) 2023-12-02T00:29:51.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.854+0000 7fe2b7fff640 1 -- 172.21.15.46:0/3244486715 learned_addr learned my addr 172.21.15.46:0/3244486715 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:51.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.854+0000 7fe2b77fe640 1 --2- 172.21.15.46:0/3244486715 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2a0003ef0 0x7fe2b813d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:51.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.854+0000 7fe2b6ffd640 1 --2- 172.21.15.46:0/3244486715 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2b81e8f20 0x7fe2b806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:51.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.854+0000 7fe2b7fff640 1 -- 172.21.15.46:0/3244486715 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2b81e8f20 msgr2=0x7fe2b806c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:51.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.854+0000 7fe2b7fff640 1 --2- 172.21.15.46:0/3244486715 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2b81e8f20 0x7fe2b806c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:51.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.854+0000 7fe2b7fff640 1 -- 172.21.15.46:0/3244486715 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2a0003ef0 msgr2=0x7fe2b813d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:51.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.854+0000 7fe2b7fff640 1 --2- 172.21.15.46:0/3244486715 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2a0003ef0 0x7fe2b813d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:51.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.854+0000 7fe2b7fff640 1 -- 172.21.15.46:0/3244486715 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b8158cd0 con 0x7fe2b806cb70 2023-12-02T00:29:51.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.855+0000 7fe2b7fff640 1 --2- 172.21.15.46:0/3244486715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2b806cb70 0x7fe2b8146c00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe2b806b430 tx=0x7fe2b00101c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:51.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.855+0000 7fe29ffff640 1 -- 172.21.15.46:0/3244486715 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b000e050 con 0x7fe2b806cb70 2023-12-02T00:29:51.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.855+0000 7fe29ffff640 1 -- 172.21.15.46:0/3244486715 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b000f040 con 0x7fe2b806cb70 2023-12-02T00:29:51.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.855+0000 7fe2bdbe3640 1 -- 172.21.15.46:0/3244486715 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2b81e8440 con 0x7fe2b806cb70 2023-12-02T00:29:51.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.855+0000 7fe29ffff640 1 -- 172.21.15.46:0/3244486715 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0010d90 con 0x7fe2b806cb70 2023-12-02T00:29:51.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.855+0000 7fe2bdbe3640 1 -- 172.21.15.46:0/3244486715 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2b813df20 con 0x7fe2b806cb70 2023-12-02T00:29:51.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.856+0000 7fe2bdbe3640 1 -- 172.21.15.46:0/3244486715 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2b81e8c40 con 0x7fe2b806cb70 2023-12-02T00:29:51.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.856+0000 7fe29ffff640 1 -- 172.21.15.46:0/3244486715 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe2b0015440 con 0x7fe2b806cb70 2023-12-02T00:29:51.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.857+0000 7fe29ffff640 1 --2- 172.21.15.46:0/3244486715 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe2a406ae70 0x7fe2a406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:51.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.857+0000 7fe29ffff640 1 -- 172.21.15.46:0/3244486715 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe2b00897a0 con 0x7fe2b806cb70 2023-12-02T00:29:51.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.857+0000 7fe2b77fe640 1 --2- 172.21.15.46:0/3244486715 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe2a406ae70 0x7fe2a406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:51.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.858+0000 7fe2b77fe640 1 --2- 172.21.15.46:0/3244486715 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe2a406ae70 0x7fe2a406d330 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe2b80d0c90 tx=0x7fe2a8009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:51.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:51.861+0000 7fe29ffff640 1 -- 172.21.15.46:0/3244486715 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe2b81e8c40 con 0x7fe2b806cb70 2023-12-02T00:29:52.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.107+0000 7fe2bdbe3640 1 -- 172.21.15.46:0/3244486715 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fe2b81e8c40 con 0x7fe2b806cb70 2023-12-02T00:29:52.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.107+0000 7fe29ffff640 1 -- 172.21.15.46:0/3244486715 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fe2b81e8c40 con 0x7fe2b806cb70 2023-12-02T00:29:52.108 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:29:52.108 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-12-02T00:29:52.109 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_dump": [ 2023-12-02T00:29:52.109 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-02T00:29:52.109 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.a", 2023-12-02T00:29:52.109 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAIemplYQrZMBAAqrkez6r//VUITan4SKdf4w==", 2023-12-02T00:29:52.109 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-02T00:29:52.109 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-12-02T00:29:52.109 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-12-02T00:29:52.109 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-12-02T00:29:52.109 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-12-02T00:29:52.109 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:29:52.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:29:52.109 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-02T00:29:52.109 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.b", 2023-12-02T00:29:52.109 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAJemplKoSRAhAAuQHpGMNCUSFEtCtWMvY8yA==", 2023-12-02T00:29:52.109 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-02T00:29:52.110 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-12-02T00:29:52.110 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-12-02T00:29:52.110 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-12-02T00:29:52.110 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-12-02T00:29:52.110 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:29:52.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:29:52.110 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-02T00:29:52.110 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.c", 2023-12-02T00:29:52.110 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAIempl69PfOBAAQOgkrli+yM4wpJabPb3QxQ==", 2023-12-02T00:29:52.110 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-02T00:29:52.110 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-12-02T00:29:52.110 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-12-02T00:29:52.110 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-12-02T00:29:52.110 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-12-02T00:29:52.111 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:29:52.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:29:52.111 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-02T00:29:52.111 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.d", 2023-12-02T00:29:52.111 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAJemplsTHqCRAAi7WJBcwYYDl0/qZkMy7GsQ==", 2023-12-02T00:29:52.111 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-02T00:29:52.111 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-12-02T00:29:52.111 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-12-02T00:29:52.111 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-12-02T00:29:52.111 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-12-02T00:29:52.111 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:29:52.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:29:52.111 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-02T00:29:52.111 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.0", 2023-12-02T00:29:52.112 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAKempluFfMMhAAmbD1GpzugHbxobJBl3RtkQ==", 2023-12-02T00:29:52.112 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-02T00:29:52.112 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-12-02T00:29:52.112 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-12-02T00:29:52.112 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-12-02T00:29:52.112 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:29:52.112 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:29:52.112 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-02T00:29:52.112 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.1", 2023-12-02T00:29:52.112 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQANempleNzmMRAALK5ExII/uG0A8pXBsc86pg==", 2023-12-02T00:29:52.112 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-02T00:29:52.112 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-12-02T00:29:52.112 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-12-02T00:29:52.112 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-12-02T00:29:52.112 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:29:52.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:29:52.113 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-02T00:29:52.113 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.2", 2023-12-02T00:29:52.113 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAQemplKDzyMBAAfPwOg3WNbFSBUZUqpG1l7w==", 2023-12-02T00:29:52.113 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-02T00:29:52.113 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-12-02T00:29:52.113 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-12-02T00:29:52.113 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-12-02T00:29:52.113 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:29:52.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:29:52.113 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-02T00:29:52.113 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.3", 2023-12-02T00:29:52.113 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQATemplIbgAMBAAbynDehkgSKMaPHXYgwK2cA==", 2023-12-02T00:29:52.113 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-02T00:29:52.113 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-12-02T00:29:52.114 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-12-02T00:29:52.114 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-12-02T00:29:52.114 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:29:52.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:29:52.114 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-02T00:29:52.114 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.4", 2023-12-02T00:29:52.114 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAaempl/vxEABAAThzqmuy54tDx5dYyTilSzg==", 2023-12-02T00:29:52.114 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-02T00:29:52.114 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-12-02T00:29:52.114 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-12-02T00:29:52.114 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-12-02T00:29:52.114 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:29:52.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:29:52.115 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-02T00:29:52.115 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.5", 2023-12-02T00:29:52.115 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAcemplWd7NOhAAO0eGoj+K+P0F4QvV4SMUxg==", 2023-12-02T00:29:52.115 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-02T00:29:52.115 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-12-02T00:29:52.115 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-12-02T00:29:52.115 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-12-02T00:29:52.115 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:29:52.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:29:52.115 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-02T00:29:52.116 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.6", 2023-12-02T00:29:52.116 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAfemplz38cORAAmb09RTXtiWf2J4q0jbYEug==", 2023-12-02T00:29:52.116 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-02T00:29:52.116 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-12-02T00:29:52.116 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-12-02T00:29:52.116 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-12-02T00:29:52.116 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:29:52.116 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:29:52.116 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-02T00:29:52.116 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.7", 2023-12-02T00:29:52.116 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAiempl+Md9NxAA5bPhqhkvQPRZ59k3OE9R9w==", 2023-12-02T00:29:52.116 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-02T00:29:52.116 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-12-02T00:29:52.116 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-12-02T00:29:52.116 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-12-02T00:29:52.117 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:29:52.117 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:29:52.117 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-02T00:29:52.117 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.0", 2023-12-02T00:29:52.117 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAJempl5Lx1DhAAjpK6ajoQ5jQiUr/EMpFeLg==", 2023-12-02T00:29:52.117 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-02T00:29:52.117 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-12-02T00:29:52.117 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2023-12-02T00:29:52.117 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2023-12-02T00:29:52.118 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2023-12-02T00:29:52.118 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:29:52.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:29:52.118 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-02T00:29:52.118 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.1", 2023-12-02T00:29:52.119 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAJempl0gj4FRAAHZtOS7DxQQOSmYsu54uPZg==", 2023-12-02T00:29:52.119 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-02T00:29:52.119 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-12-02T00:29:52.119 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2023-12-02T00:29:52.119 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2023-12-02T00:29:52.119 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2023-12-02T00:29:52.120 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:29:52.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:29:52.120 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-02T00:29:52.120 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.2", 2023-12-02T00:29:52.120 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAJempl6t9OEhAACGdJ4lIgFImVMyS+vbiyrw==", 2023-12-02T00:29:52.120 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-02T00:29:52.121 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-12-02T00:29:52.121 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2023-12-02T00:29:52.121 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2023-12-02T00:29:52.121 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2023-12-02T00:29:52.121 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:29:52.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:29:52.122 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-02T00:29:52.122 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.3", 2023-12-02T00:29:52.122 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAJemplapXEGxAAz74ppCDNM8zW2+ozYOiifQ==", 2023-12-02T00:29:52.122 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-02T00:29:52.122 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-12-02T00:29:52.122 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2023-12-02T00:29:52.123 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2023-12-02T00:29:52.123 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2023-12-02T00:29:52.123 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:29:52.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:29:52.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-02T00:29:52.123 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.admin", 2023-12-02T00:29:52.124 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAIemplkFiEEhAAmwW5ztv7dNWpU/S9WKtyww==", 2023-12-02T00:29:52.124 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-02T00:29:52.124 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2023-12-02T00:29:52.124 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-12-02T00:29:52.124 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-12-02T00:29:52.124 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-12-02T00:29:52.125 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:29:52.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:29:52.125 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-02T00:29:52.125 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.bootstrap-mds", 2023-12-02T00:29:52.125 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAoempl65ZuNxAAbcBrPWnRq/DTBUNELtZwFw==", 2023-12-02T00:29:52.125 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-02T00:29:52.125 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile bootstrap-mds" 2023-12-02T00:29:52.125 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:29:52.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:29:52.126 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-02T00:29:52.126 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.bootstrap-mgr", 2023-12-02T00:29:52.126 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAoemplHaBuNxAAySmA45P+cYsMcUoVA8uS3g==", 2023-12-02T00:29:52.126 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-02T00:29:52.126 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile bootstrap-mgr" 2023-12-02T00:29:52.127 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:29:52.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:29:52.127 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-02T00:29:52.127 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.bootstrap-osd", 2023-12-02T00:29:52.127 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAoemplWqduNxAAFu7gpIIVHQtwKe/xYGAw9g==", 2023-12-02T00:29:52.127 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-02T00:29:52.127 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile bootstrap-osd" 2023-12-02T00:29:52.128 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:29:52.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:29:52.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-02T00:29:52.128 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.bootstrap-rbd", 2023-12-02T00:29:52.128 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAoempldq5uNxAAKmKjPwn4pC8TbCXL6xVdGA==", 2023-12-02T00:29:52.128 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-02T00:29:52.128 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile bootstrap-rbd" 2023-12-02T00:29:52.129 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:29:52.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:29:52.129 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-02T00:29:52.129 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-12-02T00:29:52.129 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAoemplD7ZuNxAAn/pWaOaF7UZ9EaxzaG5UQg==", 2023-12-02T00:29:52.129 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-02T00:29:52.129 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-12-02T00:29:52.129 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:29:52.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:29:52.130 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-02T00:29:52.130 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.bootstrap-rgw", 2023-12-02T00:29:52.130 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAoempl571uNxAAIGMsNqACoxqeaqYKV3/ljA==", 2023-12-02T00:29:52.130 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-02T00:29:52.130 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile bootstrap-rgw" 2023-12-02T00:29:52.130 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:29:52.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:29:52.131 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-02T00:29:52.131 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.x", 2023-12-02T00:29:52.131 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAIemplbBH1LBAAxs6g6VawCdt6IxZJn2yrQA==", 2023-12-02T00:29:52.131 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-02T00:29:52.131 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2023-12-02T00:29:52.131 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2023-12-02T00:29:52.131 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-12-02T00:29:52.131 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:29:52.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:29:52.132 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-02T00:29:52.132 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.y", 2023-12-02T00:29:52.132 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAIemplE8PaKBAAwleK4eGvEOffn5EzY+drMQ==", 2023-12-02T00:29:52.132 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-02T00:29:52.132 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2023-12-02T00:29:52.132 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2023-12-02T00:29:52.133 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-12-02T00:29:52.133 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:29:52.133 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:29:52.133 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2023-12-02T00:29:52.133 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-12-02T00:29:52.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.109+0000 7fe2bdbe3640 1 -- 172.21.15.46:0/3244486715 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe2a406ae70 msgr2=0x7fe2a406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:52.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.109+0000 7fe2bdbe3640 1 --2- 172.21.15.46:0/3244486715 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe2a406ae70 0x7fe2a406d330 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe2b80d0c90 tx=0x7fe2a8009190 comp rx=0 tx=0).stop 2023-12-02T00:29:52.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.109+0000 7fe2bdbe3640 1 -- 172.21.15.46:0/3244486715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2b806cb70 msgr2=0x7fe2b8146c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:52.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.109+0000 7fe2bdbe3640 1 --2- 172.21.15.46:0/3244486715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2b806cb70 0x7fe2b8146c00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe2b806b430 tx=0x7fe2b00101c0 comp rx=0 tx=0).stop 2023-12-02T00:29:52.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.109+0000 7fe2bdbe3640 1 -- 172.21.15.46:0/3244486715 shutdown_connections 2023-12-02T00:29:52.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.109+0000 7fe2bdbe3640 1 --2- 172.21.15.46:0/3244486715 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe2a406ae70 0x7fe2a406d330 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:52.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.109+0000 7fe2bdbe3640 1 --2- 172.21.15.46:0/3244486715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2b806cb70 0x7fe2b8146c00 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:52.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.109+0000 7fe2bdbe3640 1 --2- 172.21.15.46:0/3244486715 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2b81e8f20 0x7fe2b806c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:52.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.109+0000 7fe2bdbe3640 1 --2- 172.21.15.46:0/3244486715 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2a0003ef0 0x7fe2b813d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:52.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.109+0000 7fe2bdbe3640 1 -- 172.21.15.46:0/3244486715 >> 172.21.15.46:0/3244486715 conn(0x7fe2b806a6b0 msgr2=0x7fe2b815cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:52.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.109+0000 7fe2bdbe3640 1 -- 172.21.15.46:0/3244486715 shutdown_connections 2023-12-02T00:29:52.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.109+0000 7fe2bdbe3640 1 -- 172.21.15.46:0/3244486715 wait complete. 2023-12-02T00:29:52.137 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-12-02T00:29:52.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.244+0000 7f5f009ea640 1 Processor -- start 2023-12-02T00:29:52.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.244+0000 7f5f009ea640 1 -- start start 2023-12-02T00:29:52.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.245+0000 7f5f009ea640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5efc1415f0 0x7f5efc141e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:52.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.245+0000 7f5f009ea640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5efc0b2eb0 con 0x7f5efc06bbf0 2023-12-02T00:29:52.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.245+0000 7f5f009ea640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5efc0b3190 con 0x7f5efc144a00 2023-12-02T00:29:52.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.245+0000 7f5f009ea640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc0b3470 con 0x7f5efc1415f0 2023-12-02T00:29:52.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.245+0000 7f5efad9d640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5efc06bbf0 0x7f5efc0b3f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42838/0 (socket says 172.21.15.46:42838) 2023-12-02T00:29:52.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.245+0000 7f5efad9d640 1 -- 172.21.15.46:0/1643127441 learned_addr learned my addr 172.21.15.46:0/1643127441 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:52.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.245+0000 7f5efa59c640 1 --2- 172.21.15.46:0/1643127441 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5efc1415f0 0x7f5efc141e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:52.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.245+0000 7f5ef959a640 1 -- 172.21.15.46:0/1643127441 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 912513854 0 0) 0x7f5efc0b2eb0 con 0x7f5efc06bbf0 2023-12-02T00:29:52.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.245+0000 7f5ef959a640 1 -- 172.21.15.46:0/1643127441 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ee8003520 con 0x7f5efc06bbf0 2023-12-02T00:29:52.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.245+0000 7f5ef959a640 1 -- 172.21.15.46:0/1643127441 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3447542672 0 0) 0x7f5ee8003520 con 0x7f5efc06bbf0 2023-12-02T00:29:52.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.245+0000 7f5ef959a640 1 -- 172.21.15.46:0/1643127441 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5efc1415f0 msgr2=0x7f5efc141e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:52.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.245+0000 7f5ef959a640 1 --2- 172.21.15.46:0/1643127441 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5efc1415f0 0x7f5efc141e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:52.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.245+0000 7f5ef959a640 1 -- 172.21.15.46:0/1643127441 >> v1:172.21.15.162:6789/0 conn(0x7f5efc144a00 legacy=0x7f5efc06b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:52.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.245+0000 7f5ef959a640 1 -- 172.21.15.46:0/1643127441 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5efc0b3750 con 0x7f5efc06bbf0 2023-12-02T00:29:52.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.245+0000 7f5ef959a640 1 -- 172.21.15.46:0/1643127441 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7f5eec0019f0 con 0x7f5efc06bbf0 2023-12-02T00:29:52.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.245+0000 7f5ef959a640 1 -- 172.21.15.46:0/1643127441 >> v1:172.21.15.46:6789/0 conn(0x7f5efc06bbf0 legacy=0x7f5efc0b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:52.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.245+0000 7f5ef959a640 1 --2- 172.21.15.46:0/1643127441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ee80039a0 0x7f5ee8003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:52.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.245+0000 7f5ef959a640 1 --2- 172.21.15.46:0/1643127441 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ee8004350 0x7f5ee8004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:52.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.245+0000 7f5ef959a640 1 --2- 172.21.15.46:0/1643127441 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5efc1415f0 0x7f5ee8005080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:52.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.245+0000 7f5ef959a640 1 -- 172.21.15.46:0/1643127441 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc0b3470 con 0x7f5ee80039a0 2023-12-02T00:29:52.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.245+0000 7f5ef959a640 1 -- 172.21.15.46:0/1643127441 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee800b980 con 0x7f5ee8004350 2023-12-02T00:29:52.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.245+0000 7f5ef959a640 1 -- 172.21.15.46:0/1643127441 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee800bb40 con 0x7f5efc1415f0 2023-12-02T00:29:52.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.246+0000 7f5efa59c640 1 --2- 172.21.15.46:0/1643127441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ee80039a0 0x7f5ee8003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:52.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.246+0000 7f5ef9d9b640 1 --2- 172.21.15.46:0/1643127441 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ee8004350 0x7f5ee8004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:52.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.246+0000 7f5efad9d640 1 --2- 172.21.15.46:0/1643127441 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5efc1415f0 0x7f5ee8005080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:52.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.246+0000 7f5efa59c640 1 -- 172.21.15.46:0/1643127441 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5efc1415f0 msgr2=0x7f5ee8005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:52.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.246+0000 7f5efa59c640 1 --2- 172.21.15.46:0/1643127441 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5efc1415f0 0x7f5ee8005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:52.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.246+0000 7f5efa59c640 1 -- 172.21.15.46:0/1643127441 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ee8004350 msgr2=0x7f5ee8004820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:52.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.246+0000 7f5efa59c640 1 --2- 172.21.15.46:0/1643127441 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ee8004350 0x7f5ee8004820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:52.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.246+0000 7f5efa59c640 1 -- 172.21.15.46:0/1643127441 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee8003520 con 0x7f5ee80039a0 2023-12-02T00:29:52.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.247+0000 7f5efa59c640 1 --2- 172.21.15.46:0/1643127441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ee80039a0 0x7f5ee8003d80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4009ac0 tx=0x7f5ee80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a887f2ac489895d server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:52.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.247+0000 7f5ef959a640 1 -- 172.21.15.46:0/1643127441 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef402c6b0 con 0x7f5ee80039a0 2023-12-02T00:29:52.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.247+0000 7f5ef959a640 1 -- 172.21.15.46:0/1643127441 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ef402c810 con 0x7f5ee80039a0 2023-12-02T00:29:52.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.247+0000 7f5ef959a640 1 -- 172.21.15.46:0/1643127441 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef402cba0 con 0x7f5ee80039a0 2023-12-02T00:29:52.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.247+0000 7f5f009ea640 1 -- 172.21.15.46:0/1643127441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ee80039a0 msgr2=0x7f5ee8003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:52.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.247+0000 7f5f009ea640 1 --2- 172.21.15.46:0/1643127441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ee80039a0 0x7f5ee8003d80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4009ac0 tx=0x7f5ee80057f0 comp rx=0 tx=0).stop 2023-12-02T00:29:52.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.247+0000 7f5efad9d640 1 -- 172.21.15.46:0/1643127441 reap_dead start 2023-12-02T00:29:52.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.247+0000 7f5f009ea640 1 -- 172.21.15.46:0/1643127441 shutdown_connections 2023-12-02T00:29:52.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.247+0000 7f5f009ea640 1 -- 172.21.15.46:0/1643127441 >> 172.21.15.46:0/1643127441 conn(0x7f5efc070e70 msgr2=0x7f5efc071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:52.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.247+0000 7f5f009ea640 1 -- 172.21.15.46:0/1643127441 shutdown_connections 2023-12-02T00:29:52.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.247+0000 7f5f009ea640 1 -- 172.21.15.46:0/1643127441 wait complete. 2023-12-02T00:29:52.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.249+0000 7f5f009ea640 1 Processor -- start 2023-12-02T00:29:52.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.249+0000 7f5f009ea640 1 -- start start 2023-12-02T00:29:52.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.249+0000 7f5f009ea640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ee80039a0 0x7f5efc15d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:52.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.249+0000 7f5f009ea640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5efc0c5a70 0x7f5efc0c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:52.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.249+0000 7f5f009ea640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5efc0c4a80 0x7f5efc0c4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:52.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.249+0000 7f5f009ea640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc0af330 con 0x7f5efc0c4a80 2023-12-02T00:29:52.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.249+0000 7f5f009ea640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc0c5440 con 0x7f5ee80039a0 2023-12-02T00:29:52.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.249+0000 7f5f009ea640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc0ae720 con 0x7f5efc0c5a70 2023-12-02T00:29:52.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.250+0000 7f5efad9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5efc0c4a80 0x7f5efc0c4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:52.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.250+0000 7f5efad9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5efc0c4a80 0x7f5efc0c4f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38524/0 (socket says 172.21.15.46:38524) 2023-12-02T00:29:52.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.250+0000 7f5efad9d640 1 -- 172.21.15.46:0/4097940460 learned_addr learned my addr 172.21.15.46:0/4097940460 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:52.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.250+0000 7f5efa59c640 1 --2- 172.21.15.46:0/4097940460 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ee80039a0 0x7f5efc15d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:52.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.250+0000 7f5ef9d9b640 1 --2- 172.21.15.46:0/4097940460 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5efc0c5a70 0x7f5efc0c5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:52.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.250+0000 7f5efad9d640 1 -- 172.21.15.46:0/4097940460 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5efc0c5a70 msgr2=0x7f5efc0c5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:52.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.250+0000 7f5efad9d640 1 --2- 172.21.15.46:0/4097940460 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5efc0c5a70 0x7f5efc0c5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:52.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.250+0000 7f5efad9d640 1 -- 172.21.15.46:0/4097940460 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ee80039a0 msgr2=0x7f5efc15d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:52.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.250+0000 7f5efad9d640 1 --2- 172.21.15.46:0/4097940460 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ee80039a0 0x7f5efc15d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:52.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.250+0000 7f5efad9d640 1 -- 172.21.15.46:0/4097940460 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5efc0aea00 con 0x7f5efc0c4a80 2023-12-02T00:29:52.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.250+0000 7f5efa59c640 1 --2- 172.21.15.46:0/4097940460 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ee80039a0 0x7f5efc15d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:52.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.250+0000 7f5efad9d640 1 --2- 172.21.15.46:0/4097940460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5efc0c4a80 0x7f5efc0c4f00 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5efc066ad0 tx=0x7f5eec00a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:52.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.250+0000 7f5ef2ffd640 1 -- 172.21.15.46:0/4097940460 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eec007af0 con 0x7f5efc0c4a80 2023-12-02T00:29:52.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.250+0000 7f5ef2ffd640 1 -- 172.21.15.46:0/4097940460 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eec00ae30 con 0x7f5efc0c4a80 2023-12-02T00:29:52.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.250+0000 7f5f009ea640 1 -- 172.21.15.46:0/4097940460 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5efc0aece0 con 0x7f5efc0c4a80 2023-12-02T00:29:52.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.250+0000 7f5ef2ffd640 1 -- 172.21.15.46:0/4097940460 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eec013610 con 0x7f5efc0c4a80 2023-12-02T00:29:52.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.250+0000 7f5f009ea640 1 -- 172.21.15.46:0/4097940460 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5efc15d5d0 con 0x7f5efc0c4a80 2023-12-02T00:29:52.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.251+0000 7f5f009ea640 1 -- 172.21.15.46:0/4097940460 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5efc0b3190 con 0x7f5efc0c4a80 2023-12-02T00:29:52.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.252+0000 7f5ef2ffd640 1 -- 172.21.15.46:0/4097940460 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5eec013770 con 0x7f5efc0c4a80 2023-12-02T00:29:52.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.252+0000 7f5ef2ffd640 1 --2- 172.21.15.46:0/4097940460 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5edc06ae40 0x7f5edc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:52.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.252+0000 7f5ef2ffd640 1 -- 172.21.15.46:0/4097940460 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5eec089990 con 0x7f5efc0c4a80 2023-12-02T00:29:52.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.252+0000 7f5efa59c640 1 --2- 172.21.15.46:0/4097940460 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5edc06ae40 0x7f5edc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:52.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.253+0000 7f5efa59c640 1 --2- 172.21.15.46:0/4097940460 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5edc06ae40 0x7f5edc06d300 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4009ac0 tx=0x7f5ef4030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:52.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.256+0000 7f5ef2ffd640 1 -- 172.21.15.46:0/4097940460 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5eec054d90 con 0x7f5efc0c4a80 2023-12-02T00:29:52.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.506+0000 7f5f009ea640 1 -- 172.21.15.46:0/4097940460 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f5efc069400 con 0x7f5efc0c4a80 2023-12-02T00:29:52.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.508+0000 7f5ef2ffd640 1 -- 172.21.15.46:0/4097940460 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f5eec0583b0 con 0x7f5efc0c4a80 2023-12-02T00:29:52.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.510+0000 7f5f009ea640 1 -- 172.21.15.46:0/4097940460 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5edc06ae40 msgr2=0x7f5edc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:52.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.510+0000 7f5f009ea640 1 --2- 172.21.15.46:0/4097940460 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5edc06ae40 0x7f5edc06d300 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4009ac0 tx=0x7f5ef4030000 comp rx=0 tx=0).stop 2023-12-02T00:29:52.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.510+0000 7f5f009ea640 1 -- 172.21.15.46:0/4097940460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5efc0c4a80 msgr2=0x7f5efc0c4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:52.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.510+0000 7f5f009ea640 1 --2- 172.21.15.46:0/4097940460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5efc0c4a80 0x7f5efc0c4f00 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5efc066ad0 tx=0x7f5eec00a6a0 comp rx=0 tx=0).stop 2023-12-02T00:29:52.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.510+0000 7f5f009ea640 1 -- 172.21.15.46:0/4097940460 shutdown_connections 2023-12-02T00:29:52.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.510+0000 7f5f009ea640 1 --2- 172.21.15.46:0/4097940460 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5edc06ae40 0x7f5edc06d300 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:52.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.510+0000 7f5f009ea640 1 --2- 172.21.15.46:0/4097940460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5efc0c4a80 0x7f5efc0c4f00 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:52.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.510+0000 7f5f009ea640 1 --2- 172.21.15.46:0/4097940460 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5efc0c5a70 0x7f5efc0c5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:52.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.510+0000 7f5f009ea640 1 --2- 172.21.15.46:0/4097940460 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ee80039a0 0x7f5efc15d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:52.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.510+0000 7f5f009ea640 1 -- 172.21.15.46:0/4097940460 >> 172.21.15.46:0/4097940460 conn(0x7f5efc070e70 msgr2=0x7f5efc06a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:52.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.510+0000 7f5f009ea640 1 -- 172.21.15.46:0/4097940460 shutdown_connections 2023-12-02T00:29:52.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.511+0000 7f5f009ea640 1 -- 172.21.15.46:0/4097940460 wait complete. 2023-12-02T00:29:52.522 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-12-02T00:29:52.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.629+0000 7f3e760e5640 1 Processor -- start 2023-12-02T00:29:52.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.629+0000 7f3e760e5640 1 -- start start 2023-12-02T00:29:52.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.630+0000 7f3e760e5640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3e70146460 0x7f3e700b49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:52.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.630+0000 7f3e760e5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e7006f1c0 0x7f3e7006baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:52.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.630+0000 7f3e760e5640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e70146fd0 con 0x7f3e70146840 2023-12-02T00:29:52.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.630+0000 7f3e760e5640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e7006c180 con 0x7f3e70146460 2023-12-02T00:29:52.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.630+0000 7f3e760e5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7006c460 con 0x7f3e7006f1c0 2023-12-02T00:29:52.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.630+0000 7f3e6f7fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3e70146460 0x7f3e700b49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:52.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.630+0000 7f3e6f7fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3e70146460 0x7f3e700b49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50498/0 (socket says 172.21.15.46:50498) 2023-12-02T00:29:52.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.630+0000 7f3e6f7fe640 1 -- 172.21.15.46:0/2130009562 learned_addr learned my addr 172.21.15.46:0/2130009562 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:52.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.630+0000 7f3e6ffff640 1 --2- 172.21.15.46:0/2130009562 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e7006f1c0 0x7f3e7006baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:52.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.630+0000 7f3e6f7fe640 1 -- 172.21.15.46:0/2130009562 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e7006f1c0 msgr2=0x7f3e7006baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:52.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.630+0000 7f3e6f7fe640 1 --2- 172.21.15.46:0/2130009562 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e7006f1c0 0x7f3e7006baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:52.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.630+0000 7f3e6f7fe640 1 -- 172.21.15.46:0/2130009562 >> v1:172.21.15.162:6789/0 conn(0x7f3e70146840 legacy=0x7f3e7006eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:52.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.630+0000 7f3e6f7fe640 1 -- 172.21.15.46:0/2130009562 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e7006c740 con 0x7f3e70146460 2023-12-02T00:29:52.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.631+0000 7f3e6f7fe640 1 --2- 172.21.15.46:0/2130009562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e70146460 0x7f3e700b49d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3e60009870 tx=0x7f3e6002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68fee866ca9d5512 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:52.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.631+0000 7f3e6e7fc640 1 -- 172.21.15.46:0/2130009562 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e60031640 con 0x7f3e70146460 2023-12-02T00:29:52.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.631+0000 7f3e6e7fc640 1 -- 172.21.15.46:0/2130009562 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e600317a0 con 0x7f3e70146460 2023-12-02T00:29:52.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.631+0000 7f3e6e7fc640 1 -- 172.21.15.46:0/2130009562 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e60031b30 con 0x7f3e70146460 2023-12-02T00:29:52.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.632+0000 7f3e760e5640 1 -- 172.21.15.46:0/2130009562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e70146460 msgr2=0x7f3e700b49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:52.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.632+0000 7f3e760e5640 1 --2- 172.21.15.46:0/2130009562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e70146460 0x7f3e700b49d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3e60009870 tx=0x7f3e6002b8f0 comp rx=0 tx=0).stop 2023-12-02T00:29:52.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.632+0000 7f3e760e5640 1 -- 172.21.15.46:0/2130009562 shutdown_connections 2023-12-02T00:29:52.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.632+0000 7f3e760e5640 1 --2- 172.21.15.46:0/2130009562 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e7006f1c0 0x7f3e7006baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:52.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.632+0000 7f3e760e5640 1 --2- 172.21.15.46:0/2130009562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e70146460 0x7f3e700b49d0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:52.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.632+0000 7f3e760e5640 1 -- 172.21.15.46:0/2130009562 >> 172.21.15.46:0/2130009562 conn(0x7f3e7006a6b0 msgr2=0x7f3e7006aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:52.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.632+0000 7f3e760e5640 1 -- 172.21.15.46:0/2130009562 shutdown_connections 2023-12-02T00:29:52.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.632+0000 7f3e760e5640 1 -- 172.21.15.46:0/2130009562 wait complete. 2023-12-02T00:29:52.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.633+0000 7f3e760e5640 1 Processor -- start 2023-12-02T00:29:52.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.633+0000 7f3e760e5640 1 -- start start 2023-12-02T00:29:52.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.634+0000 7f3e760e5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e7006f1c0 0x7f3e701f2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:52.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.634+0000 7f3e760e5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e70146460 0x7f3e7013ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:52.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.634+0000 7f3e760e5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e70146840 0x7f3e701f6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:52.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.634+0000 7f3e760e5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e700ca780 con 0x7f3e70146840 2023-12-02T00:29:52.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.634+0000 7f3e760e5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7013f5d0 con 0x7f3e7006f1c0 2023-12-02T00:29:52.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.634+0000 7f3e760e5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7013f8b0 con 0x7f3e70146460 2023-12-02T00:29:52.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.634+0000 7f3e6ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e70146840 0x7f3e701f6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:52.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.634+0000 7f3e6ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e70146840 0x7f3e701f6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50508/0 (socket says 172.21.15.46:50508) 2023-12-02T00:29:52.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.634+0000 7f3e6ffff640 1 -- 172.21.15.46:0/3999260862 learned_addr learned my addr 172.21.15.46:0/3999260862 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:52.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.634+0000 7f3e6f7fe640 1 --2- 172.21.15.46:0/3999260862 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e7006f1c0 0x7f3e701f2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:52.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.634+0000 7f3e6effd640 1 --2- 172.21.15.46:0/3999260862 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e70146460 0x7f3e7013ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:52.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.634+0000 7f3e6ffff640 1 -- 172.21.15.46:0/3999260862 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e70146460 msgr2=0x7f3e7013ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:52.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.634+0000 7f3e6ffff640 1 --2- 172.21.15.46:0/3999260862 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e70146460 0x7f3e7013ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:52.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.634+0000 7f3e6ffff640 1 -- 172.21.15.46:0/3999260862 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e7006f1c0 msgr2=0x7f3e701f2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:52.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.634+0000 7f3e6ffff640 1 --2- 172.21.15.46:0/3999260862 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e7006f1c0 0x7f3e701f2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:52.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.634+0000 7f3e6ffff640 1 -- 172.21.15.46:0/3999260862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e600094d0 con 0x7f3e70146840 2023-12-02T00:29:52.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.634+0000 7f3e6ffff640 1 --2- 172.21.15.46:0/3999260862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e70146840 0x7f3e701f6390 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3e6800cd20 tx=0x7f3e6800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:52.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.634+0000 7f3e57fff640 1 -- 172.21.15.46:0/3999260862 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e68003b90 con 0x7f3e70146840 2023-12-02T00:29:52.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.634+0000 7f3e57fff640 1 -- 172.21.15.46:0/3999260862 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e68003cf0 con 0x7f3e70146840 2023-12-02T00:29:52.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.635+0000 7f3e760e5640 1 -- 172.21.15.46:0/3999260862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e7013fb90 con 0x7f3e70146840 2023-12-02T00:29:52.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.635+0000 7f3e57fff640 1 -- 172.21.15.46:0/3999260862 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e68004240 con 0x7f3e70146840 2023-12-02T00:29:52.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.635+0000 7f3e760e5640 1 -- 172.21.15.46:0/3999260862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e701f2ac0 con 0x7f3e70146840 2023-12-02T00:29:52.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.636+0000 7f3e760e5640 1 -- 172.21.15.46:0/3999260862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e700ca930 con 0x7f3e70146840 2023-12-02T00:29:52.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.636+0000 7f3e57fff640 1 -- 172.21.15.46:0/3999260862 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3e680043a0 con 0x7f3e70146840 2023-12-02T00:29:52.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.636+0000 7f3e57fff640 1 --2- 172.21.15.46:0/3999260862 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3e5c06ae70 0x7f3e5c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:52.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.636+0000 7f3e57fff640 1 -- 172.21.15.46:0/3999260862 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3e68012070 con 0x7f3e70146840 2023-12-02T00:29:52.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.637+0000 7f3e6f7fe640 1 --2- 172.21.15.46:0/3999260862 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3e5c06ae70 0x7f3e5c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:52.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.637+0000 7f3e6f7fe640 1 --2- 172.21.15.46:0/3999260862 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3e5c06ae70 0x7f3e5c06d330 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3e60036000 tx=0x7f3e6003c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:52.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.640+0000 7f3e57fff640 1 -- 172.21.15.46:0/3999260862 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3e700ca930 con 0x7f3e70146840 2023-12-02T00:29:52.839 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-12-02T00:29:52.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.886+0000 7f3e760e5640 1 -- 172.21.15.46:0/3999260862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f3e70069400 con 0x7f3e70146840 2023-12-02T00:29:52.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.888+0000 7f3e57fff640 1 -- 172.21.15.46:0/3999260862 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f3e68054f80 con 0x7f3e70146840 2023-12-02T00:29:52.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.890+0000 7f3e760e5640 1 -- 172.21.15.46:0/3999260862 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3e5c06ae70 msgr2=0x7f3e5c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:52.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.890+0000 7f3e760e5640 1 --2- 172.21.15.46:0/3999260862 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3e5c06ae70 0x7f3e5c06d330 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3e60036000 tx=0x7f3e6003c040 comp rx=0 tx=0).stop 2023-12-02T00:29:52.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.890+0000 7f3e760e5640 1 -- 172.21.15.46:0/3999260862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e70146840 msgr2=0x7f3e701f6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:52.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.890+0000 7f3e760e5640 1 --2- 172.21.15.46:0/3999260862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e70146840 0x7f3e701f6390 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3e6800cd20 tx=0x7f3e6800a660 comp rx=0 tx=0).stop 2023-12-02T00:29:52.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.890+0000 7f3e760e5640 1 -- 172.21.15.46:0/3999260862 shutdown_connections 2023-12-02T00:29:52.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.890+0000 7f3e760e5640 1 --2- 172.21.15.46:0/3999260862 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3e5c06ae70 0x7f3e5c06d330 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:52.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.890+0000 7f3e760e5640 1 --2- 172.21.15.46:0/3999260862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e70146840 0x7f3e701f6390 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:52.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.890+0000 7f3e760e5640 1 --2- 172.21.15.46:0/3999260862 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e70146460 0x7f3e7013ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:52.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.890+0000 7f3e760e5640 1 --2- 172.21.15.46:0/3999260862 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e7006f1c0 0x7f3e701f2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:52.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.890+0000 7f3e760e5640 1 -- 172.21.15.46:0/3999260862 >> 172.21.15.46:0/3999260862 conn(0x7f3e7006a6b0 msgr2=0x7f3e70161410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:52.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.890+0000 7f3e760e5640 1 -- 172.21.15.46:0/3999260862 shutdown_connections 2023-12-02T00:29:52.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:52.890+0000 7f3e760e5640 1 -- 172.21.15.46:0/3999260862 wait complete. 2023-12-02T00:29:52.902 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-12-02T00:29:53.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.009+0000 7f5f01539640 1 Processor -- start 2023-12-02T00:29:53.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.009+0000 7f5f01539640 1 -- start start 2023-12-02T00:29:53.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.009+0000 7f5f01539640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5efc06f1c0 0x7f5efc0b49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:53.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.009+0000 7f5f01539640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5efc146fd0 con 0x7f5efc146840 2023-12-02T00:29:53.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.009+0000 7f5f01539640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5efc06c180 con 0x7f5efc146460 2023-12-02T00:29:53.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.009+0000 7f5f01539640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc06c460 con 0x7f5efc06f1c0 2023-12-02T00:29:53.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.010+0000 7f5efb7fe640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5efc146840 0x7f5efc15db20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57046/0 (socket says 172.21.15.46:57046) 2023-12-02T00:29:53.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.010+0000 7f5efb7fe640 1 -- 172.21.15.46:0/1755045075 learned_addr learned my addr 172.21.15.46:0/1755045075 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:53.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.010+0000 7f5ef3fff640 1 --2- 172.21.15.46:0/1755045075 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5efc06f1c0 0x7f5efc0b49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:53.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.010+0000 7f5efa7fc640 1 -- 172.21.15.46:0/1755045075 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3112474323 0 0) 0x7f5efc146fd0 con 0x7f5efc146840 2023-12-02T00:29:53.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.010+0000 7f5efa7fc640 1 -- 172.21.15.46:0/1755045075 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ee8003520 con 0x7f5efc146840 2023-12-02T00:29:53.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.010+0000 7f5efa7fc640 1 -- 172.21.15.46:0/1755045075 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 719134671 0 0) 0x7f5efc06c180 con 0x7f5efc146460 2023-12-02T00:29:53.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.010+0000 7f5efa7fc640 1 -- 172.21.15.46:0/1755045075 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5efc146fd0 con 0x7f5efc146460 2023-12-02T00:29:53.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.010+0000 7f5ef3fff640 1 -- 172.21.15.46:0/1755045075 >> v1:172.21.15.162:6789/0 conn(0x7f5efc146460 legacy=0x7f5efc06eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:53.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.010+0000 7f5ef3fff640 1 -- 172.21.15.46:0/1755045075 >> v1:172.21.15.46:6789/0 conn(0x7f5efc146840 legacy=0x7f5efc15db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:53.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.010+0000 7f5ef3fff640 1 -- 172.21.15.46:0/1755045075 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5efc06c740 con 0x7f5efc06f1c0 2023-12-02T00:29:53.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.011+0000 7f5ef3fff640 1 --2- 172.21.15.46:0/1755045075 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5efc06f1c0 0x7f5efc0b49d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5ee40097d0 tx=0x7f5ee402b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7623c006f19ad873 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:53.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.011+0000 7f5efa7fc640 1 -- 172.21.15.46:0/1755045075 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee4031890 con 0x7f5efc06f1c0 2023-12-02T00:29:53.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.011+0000 7f5efa7fc640 1 -- 172.21.15.46:0/1755045075 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee40319f0 con 0x7f5efc06f1c0 2023-12-02T00:29:53.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.011+0000 7f5efa7fc640 1 -- 172.21.15.46:0/1755045075 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee4031d60 con 0x7f5efc06f1c0 2023-12-02T00:29:53.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.012+0000 7f5f01539640 1 -- 172.21.15.46:0/1755045075 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5efc06f1c0 msgr2=0x7f5efc0b49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:53.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.012+0000 7f5f01539640 1 --2- 172.21.15.46:0/1755045075 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5efc06f1c0 0x7f5efc0b49d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5ee40097d0 tx=0x7f5ee402b930 comp rx=0 tx=0).stop 2023-12-02T00:29:53.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.012+0000 7f5f01539640 1 -- 172.21.15.46:0/1755045075 shutdown_connections 2023-12-02T00:29:53.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.012+0000 7f5f01539640 1 --2- 172.21.15.46:0/1755045075 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5efc06f1c0 0x7f5efc0b49d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:53.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.012+0000 7f5f01539640 1 -- 172.21.15.46:0/1755045075 >> 172.21.15.46:0/1755045075 conn(0x7f5efc06a6b0 msgr2=0x7f5efc06aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:53.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.012+0000 7f5f01539640 1 -- 172.21.15.46:0/1755045075 shutdown_connections 2023-12-02T00:29:53.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.012+0000 7f5f01539640 1 -- 172.21.15.46:0/1755045075 wait complete. 2023-12-02T00:29:53.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.013+0000 7f5f01539640 1 Processor -- start 2023-12-02T00:29:53.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.013+0000 7f5f01539640 1 -- start start 2023-12-02T00:29:53.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.014+0000 7f5f01539640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5efc06f1c0 0x7f5efc0ca370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:53.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.014+0000 7f5f01539640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5efc146460 0x7f5efc15d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:53.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.014+0000 7f5f01539640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5efc146840 0x7f5efc1f6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:53.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.014+0000 7f5f01539640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc15d570 con 0x7f5efc146840 2023-12-02T00:29:53.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.014+0000 7f5f01539640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc1f74b0 con 0x7f5efc146460 2023-12-02T00:29:53.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.014+0000 7f5f01539640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc0caa00 con 0x7f5efc06f1c0 2023-12-02T00:29:53.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.014+0000 7f5efb7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5efc146840 0x7f5efc1f6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:53.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.014+0000 7f5efb7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5efc146840 0x7f5efc1f6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50512/0 (socket says 172.21.15.46:50512) 2023-12-02T00:29:53.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.014+0000 7f5efb7fe640 1 -- 172.21.15.46:0/1827445476 learned_addr learned my addr 172.21.15.46:0/1827445476 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:53.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.014+0000 7f5efaffd640 1 --2- 172.21.15.46:0/1827445476 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5efc06f1c0 0x7f5efc0ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:53.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.014+0000 7f5ef3fff640 1 --2- 172.21.15.46:0/1827445476 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5efc146460 0x7f5efc15d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:53.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.014+0000 7f5efb7fe640 1 -- 172.21.15.46:0/1827445476 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5efc06f1c0 msgr2=0x7f5efc0ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:53.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.014+0000 7f5efb7fe640 1 --2- 172.21.15.46:0/1827445476 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5efc06f1c0 0x7f5efc0ca370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:53.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.014+0000 7f5efb7fe640 1 -- 172.21.15.46:0/1827445476 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5efc146460 msgr2=0x7f5efc15d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:53.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.014+0000 7f5efb7fe640 1 --2- 172.21.15.46:0/1827445476 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5efc146460 0x7f5efc15d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:53.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.014+0000 7f5efb7fe640 1 -- 172.21.15.46:0/1827445476 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee4009400 con 0x7f5efc146840 2023-12-02T00:29:53.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.014+0000 7f5efaffd640 1 --2- 172.21.15.46:0/1827445476 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5efc06f1c0 0x7f5efc0ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:53.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.015+0000 7f5efb7fe640 1 --2- 172.21.15.46:0/1827445476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5efc146840 0x7f5efc1f6f70 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5ef400ca80 tx=0x7f5ef400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:53.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.015+0000 7f5ef37fe640 1 -- 172.21.15.46:0/1827445476 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef400abd0 con 0x7f5efc146840 2023-12-02T00:29:53.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.015+0000 7f5ef37fe640 1 -- 172.21.15.46:0/1827445476 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ef400ad30 con 0x7f5efc146840 2023-12-02T00:29:53.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.015+0000 7f5ef37fe640 1 -- 172.21.15.46:0/1827445476 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef400aea0 con 0x7f5efc146840 2023-12-02T00:29:53.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.015+0000 7f5f01539640 1 -- 172.21.15.46:0/1827445476 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5efc0cace0 con 0x7f5efc146840 2023-12-02T00:29:53.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.015+0000 7f5f01539640 1 -- 172.21.15.46:0/1827445476 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5efc0cb2a0 con 0x7f5efc146840 2023-12-02T00:29:53.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.016+0000 7f5f01539640 1 -- 172.21.15.46:0/1827445476 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5efc0d0ac0 con 0x7f5efc146840 2023-12-02T00:29:53.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.016+0000 7f5ef37fe640 1 -- 172.21.15.46:0/1827445476 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5ef4007070 con 0x7f5efc146840 2023-12-02T00:29:53.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.017+0000 7f5ef37fe640 1 --2- 172.21.15.46:0/1827445476 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5edc06ae40 0x7f5edc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:53.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.017+0000 7f5ef37fe640 1 -- 172.21.15.46:0/1827445476 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5ef40891d0 con 0x7f5efc146840 2023-12-02T00:29:53.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.017+0000 7f5efaffd640 1 --2- 172.21.15.46:0/1827445476 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5edc06ae40 0x7f5edc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:53.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.018+0000 7f5efaffd640 1 --2- 172.21.15.46:0/1827445476 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5edc06ae40 0x7f5edc06d300 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5efc1455e0 tx=0x7f5eec009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:53.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.021+0000 7f5ef37fe640 1 -- 172.21.15.46:0/1827445476 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5ef4058060 con 0x7f5efc146840 2023-12-02T00:29:53.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.263+0000 7f5f01539640 1 -- 172.21.15.46:0/1827445476 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f5efc069400 con 0x7f5efc146840 2023-12-02T00:29:53.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.265+0000 7f5ef37fe640 1 -- 172.21.15.46:0/1827445476 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f5ef4057bf0 con 0x7f5efc146840 2023-12-02T00:29:53.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.267+0000 7f5f01539640 1 -- 172.21.15.46:0/1827445476 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5edc06ae40 msgr2=0x7f5edc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:53.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.267+0000 7f5f01539640 1 --2- 172.21.15.46:0/1827445476 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5edc06ae40 0x7f5edc06d300 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5efc1455e0 tx=0x7f5eec009040 comp rx=0 tx=0).stop 2023-12-02T00:29:53.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.267+0000 7f5f01539640 1 -- 172.21.15.46:0/1827445476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5efc146840 msgr2=0x7f5efc1f6f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:53.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.267+0000 7f5f01539640 1 --2- 172.21.15.46:0/1827445476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5efc146840 0x7f5efc1f6f70 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5ef400ca80 tx=0x7f5ef400cf40 comp rx=0 tx=0).stop 2023-12-02T00:29:53.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.267+0000 7f5f01539640 1 -- 172.21.15.46:0/1827445476 shutdown_connections 2023-12-02T00:29:53.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.267+0000 7f5f01539640 1 --2- 172.21.15.46:0/1827445476 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5edc06ae40 0x7f5edc06d300 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:53.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.267+0000 7f5f01539640 1 --2- 172.21.15.46:0/1827445476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5efc146840 0x7f5efc1f6f70 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:53.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.267+0000 7f5f01539640 1 --2- 172.21.15.46:0/1827445476 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5efc146460 0x7f5efc15d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:53.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.267+0000 7f5f01539640 1 --2- 172.21.15.46:0/1827445476 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5efc06f1c0 0x7f5efc0ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:53.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.267+0000 7f5f01539640 1 -- 172.21.15.46:0/1827445476 >> 172.21.15.46:0/1827445476 conn(0x7f5efc06a6b0 msgr2=0x7f5efc0b4510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:53.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.267+0000 7f5f01539640 1 -- 172.21.15.46:0/1827445476 shutdown_connections 2023-12-02T00:29:53.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.267+0000 7f5f01539640 1 -- 172.21.15.46:0/1827445476 wait complete. 2023-12-02T00:29:53.279 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-12-02T00:29:53.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.385+0000 7f6239d4c640 1 Processor -- start 2023-12-02T00:29:53.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.385+0000 7f6239d4c640 1 -- start start 2023-12-02T00:29:53.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.386+0000 7f6239d4c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f62340ceb70 0x7f62340cc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:53.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.386+0000 7f6239d4c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f62340b4370 0x7f62340b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:53.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.386+0000 7f6239d4c640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6234071770 con 0x7f6234144980 2023-12-02T00:29:53.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.386+0000 7f6239d4c640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f623415ced0 con 0x7f62340b4370 2023-12-02T00:29:53.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.386+0000 7f6239d4c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62340b4ce0 con 0x7f62340ceb70 2023-12-02T00:29:53.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.386+0000 7f6233fff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f62340b4370 0x7f62340b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:53.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.386+0000 7f62337fe640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6234144980 0x7f623415c7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57050/0 (socket says 172.21.15.46:57050) 2023-12-02T00:29:53.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.386+0000 7f62337fe640 1 -- 172.21.15.46:0/3114941471 learned_addr learned my addr 172.21.15.46:0/3114941471 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:53.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.386+0000 7f6232ffd640 1 --2- 172.21.15.46:0/3114941471 >> v2:172.21.15.162:3300/0 conn(0x7f62340ceb70 0x7f62340cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:53.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.387+0000 7f6233fff640 1 -- 172.21.15.46:0/3114941471 >> v2:172.21.15.162:3300/0 conn(0x7f62340ceb70 msgr2=0x7f62340cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:53.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.387+0000 7f6233fff640 1 --2- 172.21.15.46:0/3114941471 >> v2:172.21.15.162:3300/0 conn(0x7f62340ceb70 0x7f62340cc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:53.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.387+0000 7f6233fff640 1 -- 172.21.15.46:0/3114941471 >> v1:172.21.15.46:6789/0 conn(0x7f6234144980 legacy=0x7f623415c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:53.389 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.387+0000 7f6233fff640 1 -- 172.21.15.46:0/3114941471 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62340ccdc0 con 0x7f62340b4370 2023-12-02T00:29:53.389 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.387+0000 7f62327fc640 1 -- 172.21.15.46:0/3114941471 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1863805163 0 0) 0x7f6234071770 con 0x7f6234144980 2023-12-02T00:29:53.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.387+0000 7f6233fff640 1 --2- 172.21.15.46:0/3114941471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62340b4370 0x7f62340b47a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f623415b600 tx=0x7f622c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc54d328d895d339 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:53.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.388+0000 7f62327fc640 1 -- 172.21.15.46:0/3114941471 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622c02cc80 con 0x7f62340b4370 2023-12-02T00:29:53.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.388+0000 7f62327fc640 1 -- 172.21.15.46:0/3114941471 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f622c02cde0 con 0x7f62340b4370 2023-12-02T00:29:53.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.388+0000 7f62327fc640 1 -- 172.21.15.46:0/3114941471 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622c034660 con 0x7f62340b4370 2023-12-02T00:29:53.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.388+0000 7f6239d4c640 1 -- 172.21.15.46:0/3114941471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62340b4370 msgr2=0x7f62340b47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:53.391 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.388+0000 7f6239d4c640 1 --2- 172.21.15.46:0/3114941471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62340b4370 0x7f62340b47a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f623415b600 tx=0x7f622c02bed0 comp rx=0 tx=0).stop 2023-12-02T00:29:53.391 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.388+0000 7f6239d4c640 1 -- 172.21.15.46:0/3114941471 shutdown_connections 2023-12-02T00:29:53.391 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.388+0000 7f6239d4c640 1 --2- 172.21.15.46:0/3114941471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62340b4370 0x7f62340b47a0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:53.391 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.388+0000 7f6239d4c640 1 --2- 172.21.15.46:0/3114941471 >> v2:172.21.15.162:3300/0 conn(0x7f62340ceb70 0x7f62340cc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:53.391 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.388+0000 7f6239d4c640 1 -- 172.21.15.46:0/3114941471 >> 172.21.15.46:0/3114941471 conn(0x7f623406a4d0 msgr2=0x7f623406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:53.391 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.388+0000 7f6239d4c640 1 -- 172.21.15.46:0/3114941471 shutdown_connections 2023-12-02T00:29:53.391 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.388+0000 7f6239d4c640 1 -- 172.21.15.46:0/3114941471 wait complete. 2023-12-02T00:29:53.391 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.389+0000 7f6239d4c640 1 Processor -- start 2023-12-02T00:29:53.391 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.389+0000 7f6239d4c640 1 -- start start 2023-12-02T00:29:53.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.390+0000 7f6239d4c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62340b4370 0x7f6234154b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:53.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.390+0000 7f6239d4c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f62340ceb70 0x7f62341f3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:53.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.390+0000 7f6239d4c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6234144980 0x7f62341f4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:53.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.390+0000 7f6239d4c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62340c9750 con 0x7f62340b4370 2023-12-02T00:29:53.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.390+0000 7f6239d4c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6234155080 con 0x7f6234144980 2023-12-02T00:29:53.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.390+0000 7f6239d4c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62340c8dd0 con 0x7f62340ceb70 2023-12-02T00:29:53.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.390+0000 7f62337fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62340b4370 0x7f6234154b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:53.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.390+0000 7f62337fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62340b4370 0x7f6234154b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50530/0 (socket says 172.21.15.46:50530) 2023-12-02T00:29:53.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.390+0000 7f62337fe640 1 -- 172.21.15.46:0/1077966375 learned_addr learned my addr 172.21.15.46:0/1077966375 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:53.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.391+0000 7f6233fff640 1 --2- 172.21.15.46:0/1077966375 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6234144980 0x7f62341f4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:53.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.391+0000 7f6232ffd640 1 --2- 172.21.15.46:0/1077966375 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f62340ceb70 0x7f62341f3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:53.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.391+0000 7f62337fe640 1 -- 172.21.15.46:0/1077966375 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f62340ceb70 msgr2=0x7f62341f3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:53.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.391+0000 7f62337fe640 1 --2- 172.21.15.46:0/1077966375 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f62340ceb70 0x7f62341f3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:53.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.391+0000 7f62337fe640 1 -- 172.21.15.46:0/1077966375 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6234144980 msgr2=0x7f62341f4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:53.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.391+0000 7f62337fe640 1 --2- 172.21.15.46:0/1077966375 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6234144980 0x7f62341f4500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:53.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.391+0000 7f62337fe640 1 -- 172.21.15.46:0/1077966375 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f622c009a70 con 0x7f62340b4370 2023-12-02T00:29:53.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.391+0000 7f62337fe640 1 --2- 172.21.15.46:0/1077966375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62340b4370 0x7f6234154b40 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f622400c430 tx=0x7f622400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:53.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.391+0000 7f621bfff640 1 -- 172.21.15.46:0/1077966375 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62240074e0 con 0x7f62340b4370 2023-12-02T00:29:53.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.391+0000 7f621bfff640 1 -- 172.21.15.46:0/1077966375 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6224007660 con 0x7f62340b4370 2023-12-02T00:29:53.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.391+0000 7f6239d4c640 1 -- 172.21.15.46:0/1077966375 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f623415b8b0 con 0x7f62340b4370 2023-12-02T00:29:53.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.391+0000 7f621bfff640 1 -- 172.21.15.46:0/1077966375 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6224004a20 con 0x7f62340b4370 2023-12-02T00:29:53.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.391+0000 7f6239d4c640 1 -- 172.21.15.46:0/1077966375 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62340c9370 con 0x7f62340b4370 2023-12-02T00:29:53.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.392+0000 7f6239d4c640 1 -- 172.21.15.46:0/1077966375 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6234069400 con 0x7f62340b4370 2023-12-02T00:29:53.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.392+0000 7f621bfff640 1 -- 172.21.15.46:0/1077966375 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6224004b80 con 0x7f62340b4370 2023-12-02T00:29:53.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.393+0000 7f621bfff640 1 --2- 172.21.15.46:0/1077966375 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f622006ae40 0x7f622006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:53.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.393+0000 7f621bfff640 1 -- 172.21.15.46:0/1077966375 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6224089210 con 0x7f62340b4370 2023-12-02T00:29:53.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.393+0000 7f6232ffd640 1 --2- 172.21.15.46:0/1077966375 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f622006ae40 0x7f622006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:53.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.394+0000 7f6232ffd640 1 --2- 172.21.15.46:0/1077966375 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f622006ae40 0x7f622006d300 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f623415bd00 tx=0x7f6228008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:53.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.397+0000 7f621bfff640 1 -- 172.21.15.46:0/1077966375 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f62240544f0 con 0x7f62340b4370 2023-12-02T00:29:53.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.643+0000 7f6239d4c640 1 -- 172.21.15.46:0/1077966375 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f62340704a0 con 0x7f62340b4370 2023-12-02T00:29:53.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.646+0000 7f621bfff640 1 -- 172.21.15.46:0/1077966375 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f6224057b10 con 0x7f62340b4370 2023-12-02T00:29:53.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.648+0000 7f6239d4c640 1 -- 172.21.15.46:0/1077966375 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f622006ae40 msgr2=0x7f622006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:53.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.648+0000 7f6239d4c640 1 --2- 172.21.15.46:0/1077966375 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f622006ae40 0x7f622006d300 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f623415bd00 tx=0x7f6228008040 comp rx=0 tx=0).stop 2023-12-02T00:29:53.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.648+0000 7f6239d4c640 1 -- 172.21.15.46:0/1077966375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62340b4370 msgr2=0x7f6234154b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:53.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.648+0000 7f6239d4c640 1 --2- 172.21.15.46:0/1077966375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62340b4370 0x7f6234154b40 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f622400c430 tx=0x7f622400c8f0 comp rx=0 tx=0).stop 2023-12-02T00:29:53.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.648+0000 7f6239d4c640 1 -- 172.21.15.46:0/1077966375 shutdown_connections 2023-12-02T00:29:53.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.648+0000 7f6239d4c640 1 --2- 172.21.15.46:0/1077966375 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f622006ae40 0x7f622006d300 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:53.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.648+0000 7f6239d4c640 1 --2- 172.21.15.46:0/1077966375 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6234144980 0x7f62341f4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:53.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.648+0000 7f6239d4c640 1 --2- 172.21.15.46:0/1077966375 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f62340ceb70 0x7f62341f3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:53.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.648+0000 7f6239d4c640 1 --2- 172.21.15.46:0/1077966375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62340b4370 0x7f6234154b40 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:53.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.648+0000 7f6239d4c640 1 -- 172.21.15.46:0/1077966375 >> 172.21.15.46:0/1077966375 conn(0x7f623406a4d0 msgr2=0x7f623406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:53.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.648+0000 7f6239d4c640 1 -- 172.21.15.46:0/1077966375 shutdown_connections 2023-12-02T00:29:53.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.648+0000 7f6239d4c640 1 -- 172.21.15.46:0/1077966375 wait complete. 2023-12-02T00:29:53.660 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-12-02T00:29:53.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.766+0000 7f9a364c2640 1 Processor -- start 2023-12-02T00:29:53.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.766+0000 7f9a364c2640 1 -- start start 2023-12-02T00:29:53.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.767+0000 7f9a364c2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a300d3900 0x7f9a300b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:53.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.767+0000 7f9a364c2640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a300b2b10 con 0x7f9a3015d330 2023-12-02T00:29:53.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.767+0000 7f9a364c2640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a300d1590 con 0x7f9a300d2910 2023-12-02T00:29:53.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.767+0000 7f9a364c2640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a300d1870 con 0x7f9a300d3900 2023-12-02T00:29:53.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.767+0000 7f9a2ffff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9a3015d330 0x7f9a300b8b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57058/0 (socket says 172.21.15.46:57058) 2023-12-02T00:29:53.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.767+0000 7f9a2ffff640 1 -- 172.21.15.46:0/2927850546 learned_addr learned my addr 172.21.15.46:0/2927850546 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:53.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.768+0000 7f9a2effd640 1 -- 172.21.15.46:0/2927850546 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3370947026 0 0) 0x7f9a300b2b10 con 0x7f9a3015d330 2023-12-02T00:29:53.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.768+0000 7f9a2effd640 1 -- 172.21.15.46:0/2927850546 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a18003520 con 0x7f9a3015d330 2023-12-02T00:29:53.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.768+0000 7f9a34a5f640 1 --2- 172.21.15.46:0/2927850546 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a300d3900 0x7f9a300b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:53.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.768+0000 7f9a2effd640 1 -- 172.21.15.46:0/2927850546 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1679223249 0 0) 0x7f9a18003520 con 0x7f9a3015d330 2023-12-02T00:29:53.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.768+0000 7f9a2effd640 1 -- 172.21.15.46:0/2927850546 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a300d3900 msgr2=0x7f9a300b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:53.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.768+0000 7f9a2effd640 1 --2- 172.21.15.46:0/2927850546 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a300d3900 0x7f9a300b6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:53.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.768+0000 7f9a2effd640 1 -- 172.21.15.46:0/2927850546 >> v1:172.21.15.162:6789/0 conn(0x7f9a300d2910 legacy=0x7f9a300d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:53.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.768+0000 7f9a2effd640 1 -- 172.21.15.46:0/2927850546 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a300d1b50 con 0x7f9a3015d330 2023-12-02T00:29:53.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.768+0000 7f9a2effd640 1 -- 172.21.15.46:0/2927850546 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7f9a20001670 con 0x7f9a3015d330 2023-12-02T00:29:53.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.768+0000 7f9a2effd640 1 -- 172.21.15.46:0/2927850546 >> v1:172.21.15.46:6789/0 conn(0x7f9a3015d330 legacy=0x7f9a300b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:53.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.768+0000 7f9a2effd640 1 --2- 172.21.15.46:0/2927850546 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a180039a0 0x7f9a18003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:53.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.768+0000 7f9a2effd640 1 --2- 172.21.15.46:0/2927850546 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a300d3900 0x7f9a180047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:53.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.768+0000 7f9a2effd640 1 --2- 172.21.15.46:0/2927850546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a18004d00 0x7f9a18005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:53.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.768+0000 7f9a2effd640 1 -- 172.21.15.46:0/2927850546 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a300d1870 con 0x7f9a18004d00 2023-12-02T00:29:53.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.768+0000 7f9a2effd640 1 -- 172.21.15.46:0/2927850546 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1800bab0 con 0x7f9a180039a0 2023-12-02T00:29:53.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.768+0000 7f9a2effd640 1 -- 172.21.15.46:0/2927850546 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1800bc70 con 0x7f9a300d3900 2023-12-02T00:29:53.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.768+0000 7f9a34a5f640 1 --2- 172.21.15.46:0/2927850546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a18004d00 0x7f9a18005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:53.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.768+0000 7f9a2ffff640 1 --2- 172.21.15.46:0/2927850546 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a180039a0 0x7f9a18003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:53.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.768+0000 7f9a2f7fe640 1 --2- 172.21.15.46:0/2927850546 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a300d3900 0x7f9a180047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:53.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.768+0000 7f9a34a5f640 1 -- 172.21.15.46:0/2927850546 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a300d3900 msgr2=0x7f9a180047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:53.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.768+0000 7f9a34a5f640 1 --2- 172.21.15.46:0/2927850546 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a300d3900 0x7f9a180047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:53.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.768+0000 7f9a34a5f640 1 -- 172.21.15.46:0/2927850546 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a180039a0 msgr2=0x7f9a18003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:53.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.768+0000 7f9a34a5f640 1 --2- 172.21.15.46:0/2927850546 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a180039a0 0x7f9a18003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:53.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.768+0000 7f9a34a5f640 1 -- 172.21.15.46:0/2927850546 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a18003520 con 0x7f9a18004d00 2023-12-02T00:29:53.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.769+0000 7f9a34a5f640 1 --2- 172.21.15.46:0/2927850546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a18004d00 0x7f9a18005180 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9a28009fc0 tx=0x7f9a180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4860c6b071a7115c server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:53.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.769+0000 7f9a2effd640 1 -- 172.21.15.46:0/2927850546 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2802cc20 con 0x7f9a18004d00 2023-12-02T00:29:53.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.769+0000 7f9a2effd640 1 -- 172.21.15.46:0/2927850546 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a2800f040 con 0x7f9a18004d00 2023-12-02T00:29:53.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.769+0000 7f9a2effd640 1 -- 172.21.15.46:0/2927850546 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a28034440 con 0x7f9a18004d00 2023-12-02T00:29:53.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.770+0000 7f9a364c2640 1 -- 172.21.15.46:0/2927850546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a18004d00 msgr2=0x7f9a18005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:53.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.770+0000 7f9a364c2640 1 --2- 172.21.15.46:0/2927850546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a18004d00 0x7f9a18005180 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9a28009fc0 tx=0x7f9a180057f0 comp rx=0 tx=0).stop 2023-12-02T00:29:53.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.770+0000 7f9a34a5f640 1 -- 172.21.15.46:0/2927850546 reap_dead start 2023-12-02T00:29:53.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.770+0000 7f9a364c2640 1 -- 172.21.15.46:0/2927850546 shutdown_connections 2023-12-02T00:29:53.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.770+0000 7f9a364c2640 1 -- 172.21.15.46:0/2927850546 >> 172.21.15.46:0/2927850546 conn(0x7f9a30072020 msgr2=0x7f9a30072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:53.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.770+0000 7f9a364c2640 1 -- 172.21.15.46:0/2927850546 shutdown_connections 2023-12-02T00:29:53.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.770+0000 7f9a364c2640 1 -- 172.21.15.46:0/2927850546 wait complete. 2023-12-02T00:29:53.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.771+0000 7f9a364c2640 1 Processor -- start 2023-12-02T00:29:53.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.771+0000 7f9a364c2640 1 -- start start 2023-12-02T00:29:53.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.772+0000 7f9a364c2640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a18004d00 0x7f9a30152070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:53.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.772+0000 7f9a364c2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a300c96e0 0x7f9a30154ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:53.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.772+0000 7f9a364c2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a301557e0 0x7f9a30155c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:53.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.772+0000 7f9a364c2640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a301561a0 con 0x7f9a301557e0 2023-12-02T00:29:53.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.772+0000 7f9a364c2640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a301f2780 con 0x7f9a18004d00 2023-12-02T00:29:53.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.772+0000 7f9a364c2640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a30154170 con 0x7f9a300c96e0 2023-12-02T00:29:53.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.772+0000 7f9a34a5f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a301557e0 0x7f9a30155c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:53.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.772+0000 7f9a34a5f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a301557e0 0x7f9a30155c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50546/0 (socket says 172.21.15.46:50546) 2023-12-02T00:29:53.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.772+0000 7f9a34a5f640 1 -- 172.21.15.46:0/1750530148 learned_addr learned my addr 172.21.15.46:0/1750530148 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:53.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.772+0000 7f9a2ffff640 1 --2- 172.21.15.46:0/1750530148 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a18004d00 0x7f9a30152070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:53.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.772+0000 7f9a2f7fe640 1 --2- 172.21.15.46:0/1750530148 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a300c96e0 0x7f9a30154ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:53.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.772+0000 7f9a34a5f640 1 -- 172.21.15.46:0/1750530148 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a300c96e0 msgr2=0x7f9a30154ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:53.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.772+0000 7f9a34a5f640 1 --2- 172.21.15.46:0/1750530148 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a300c96e0 0x7f9a30154ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:53.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.772+0000 7f9a34a5f640 1 -- 172.21.15.46:0/1750530148 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a18004d00 msgr2=0x7f9a30152070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:53.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.772+0000 7f9a34a5f640 1 --2- 172.21.15.46:0/1750530148 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a18004d00 0x7f9a30152070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:53.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.772+0000 7f9a34a5f640 1 -- 172.21.15.46:0/1750530148 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a30154450 con 0x7f9a301557e0 2023-12-02T00:29:53.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.773+0000 7f9a34a5f640 1 --2- 172.21.15.46:0/1750530148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a301557e0 0x7f9a30155c60 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9a28037040 tx=0x7f9a28009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:53.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.773+0000 7f9a2cff9640 1 -- 172.21.15.46:0/1750530148 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a28034d50 con 0x7f9a301557e0 2023-12-02T00:29:53.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.773+0000 7f9a2cff9640 1 -- 172.21.15.46:0/1750530148 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a28003990 con 0x7f9a301557e0 2023-12-02T00:29:53.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.773+0000 7f9a364c2640 1 -- 172.21.15.46:0/1750530148 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a30154730 con 0x7f9a301557e0 2023-12-02T00:29:53.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.773+0000 7f9a2cff9640 1 -- 172.21.15.46:0/1750530148 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a28003d40 con 0x7f9a301557e0 2023-12-02T00:29:53.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.773+0000 7f9a364c2640 1 -- 172.21.15.46:0/1750530148 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a30153e40 con 0x7f9a301557e0 2023-12-02T00:29:53.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.774+0000 7f9a364c2640 1 -- 172.21.15.46:0/1750530148 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a300c6e80 con 0x7f9a301557e0 2023-12-02T00:29:53.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.774+0000 7f9a2cff9640 1 -- 172.21.15.46:0/1750530148 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9a28004030 con 0x7f9a301557e0 2023-12-02T00:29:53.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.775+0000 7f9a2cff9640 1 --2- 172.21.15.46:0/1750530148 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9a1c06ae70 0x7f9a1c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:53.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.775+0000 7f9a2cff9640 1 -- 172.21.15.46:0/1750530148 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9a280aab10 con 0x7f9a301557e0 2023-12-02T00:29:53.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.775+0000 7f9a2ffff640 1 --2- 172.21.15.46:0/1750530148 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9a1c06ae70 0x7f9a1c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:53.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.776+0000 7f9a2ffff640 1 --2- 172.21.15.46:0/1750530148 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9a1c06ae70 0x7f9a1c06d330 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9a20000c00 tx=0x7f9a20009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:53.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:53.779+0000 7f9a2cff9640 1 -- 172.21.15.46:0/1750530148 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9a300c6e80 con 0x7f9a301557e0 2023-12-02T00:29:54.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.021+0000 7f9a364c2640 1 -- 172.21.15.46:0/1750530148 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f9a300c6e80 con 0x7f9a301557e0 2023-12-02T00:29:54.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.024+0000 7f9a2cff9640 1 -- 172.21.15.46:0/1750530148 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7f9a300c6e80 con 0x7f9a301557e0 2023-12-02T00:29:54.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.025+0000 7f9a364c2640 1 -- 172.21.15.46:0/1750530148 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9a1c06ae70 msgr2=0x7f9a1c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:54.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.025+0000 7f9a364c2640 1 --2- 172.21.15.46:0/1750530148 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9a1c06ae70 0x7f9a1c06d330 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9a20000c00 tx=0x7f9a20009040 comp rx=0 tx=0).stop 2023-12-02T00:29:54.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.025+0000 7f9a364c2640 1 -- 172.21.15.46:0/1750530148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a301557e0 msgr2=0x7f9a30155c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:54.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.025+0000 7f9a364c2640 1 --2- 172.21.15.46:0/1750530148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a301557e0 0x7f9a30155c60 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9a28037040 tx=0x7f9a28009750 comp rx=0 tx=0).stop 2023-12-02T00:29:54.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.026+0000 7f9a364c2640 1 -- 172.21.15.46:0/1750530148 shutdown_connections 2023-12-02T00:29:54.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.026+0000 7f9a364c2640 1 --2- 172.21.15.46:0/1750530148 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9a1c06ae70 0x7f9a1c06d330 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:54.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.026+0000 7f9a364c2640 1 --2- 172.21.15.46:0/1750530148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a301557e0 0x7f9a30155c60 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:54.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.026+0000 7f9a364c2640 1 --2- 172.21.15.46:0/1750530148 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a300c96e0 0x7f9a30154ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:54.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.026+0000 7f9a364c2640 1 --2- 172.21.15.46:0/1750530148 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a18004d00 0x7f9a30152070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:54.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.026+0000 7f9a364c2640 1 -- 172.21.15.46:0/1750530148 >> 172.21.15.46:0/1750530148 conn(0x7f9a30072020 msgr2=0x7f9a3015e420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:54.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.026+0000 7f9a364c2640 1 -- 172.21.15.46:0/1750530148 shutdown_connections 2023-12-02T00:29:54.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.026+0000 7f9a364c2640 1 -- 172.21.15.46:0/1750530148 wait complete. 2023-12-02T00:29:54.037 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-12-02T00:29:54.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.143+0000 7f3a13697640 1 Processor -- start 2023-12-02T00:29:54.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.144+0000 7f3a13697640 1 -- start start 2023-12-02T00:29:54.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.144+0000 7f3a13697640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3a0c06ea50 0x7f3a0c06ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:54.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.144+0000 7f3a13697640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3a0c160fb0 0x7f3a0c161440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:54.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.144+0000 7f3a13697640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a0c161d20 con 0x7f3a0c0b3ff0 2023-12-02T00:29:54.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.145+0000 7f3a13697640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a0c0d1ee0 con 0x7f3a0c06ea50 2023-12-02T00:29:54.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.145+0000 7f3a13697640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a0c06f370 con 0x7f3a0c160fb0 2023-12-02T00:29:54.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.145+0000 7f3a11433640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3a0c06ea50 0x7f3a0c06ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:54.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.145+0000 7f3a11433640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3a0c06ea50 0x7f3a0c06ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50556/0 (socket says 172.21.15.46:50556) 2023-12-02T00:29:54.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.145+0000 7f3a11433640 1 -- 172.21.15.46:0/3776086590 learned_addr learned my addr 172.21.15.46:0/3776086590 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:54.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.145+0000 7f3a10c32640 1 --2- 172.21.15.46:0/3776086590 >> v2:172.21.15.162:3300/0 conn(0x7f3a0c160fb0 0x7f3a0c161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:54.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.145+0000 7f3a11433640 1 -- 172.21.15.46:0/3776086590 >> v2:172.21.15.162:3300/0 conn(0x7f3a0c160fb0 msgr2=0x7f3a0c161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:54.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.145+0000 7f3a11433640 1 --2- 172.21.15.46:0/3776086590 >> v2:172.21.15.162:3300/0 conn(0x7f3a0c160fb0 0x7f3a0c161440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:54.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.145+0000 7f3a11433640 1 -- 172.21.15.46:0/3776086590 >> v1:172.21.15.162:6789/0 conn(0x7f3a0c0b3ff0 legacy=0x7f3a0c0b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:54.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.145+0000 7f3a11433640 1 -- 172.21.15.46:0/3776086590 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a0c15e760 con 0x7f3a0c06ea50 2023-12-02T00:29:54.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.146+0000 7f3a11433640 1 --2- 172.21.15.46:0/3776086590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a0c06ea50 0x7f3a0c06ee30 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c0b5b30 tx=0x7f3a0802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef2b9237ee6eb3a7 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:54.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.146+0000 7f3a03fff640 1 -- 172.21.15.46:0/3776086590 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a080317e0 con 0x7f3a0c06ea50 2023-12-02T00:29:54.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.146+0000 7f3a03fff640 1 -- 172.21.15.46:0/3776086590 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a08031940 con 0x7f3a0c06ea50 2023-12-02T00:29:54.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.146+0000 7f3a03fff640 1 -- 172.21.15.46:0/3776086590 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a08031cd0 con 0x7f3a0c06ea50 2023-12-02T00:29:54.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.146+0000 7f3a13697640 1 -- 172.21.15.46:0/3776086590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a0c06ea50 msgr2=0x7f3a0c06ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:54.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.146+0000 7f3a13697640 1 --2- 172.21.15.46:0/3776086590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a0c06ea50 0x7f3a0c06ee30 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c0b5b30 tx=0x7f3a0802b880 comp rx=0 tx=0).stop 2023-12-02T00:29:54.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.146+0000 7f3a13697640 1 -- 172.21.15.46:0/3776086590 shutdown_connections 2023-12-02T00:29:54.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.146+0000 7f3a13697640 1 --2- 172.21.15.46:0/3776086590 >> v2:172.21.15.162:3300/0 conn(0x7f3a0c160fb0 0x7f3a0c161440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:54.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.146+0000 7f3a13697640 1 --2- 172.21.15.46:0/3776086590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a0c06ea50 0x7f3a0c06ee30 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:54.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.146+0000 7f3a13697640 1 -- 172.21.15.46:0/3776086590 >> 172.21.15.46:0/3776086590 conn(0x7f3a0c06a6b0 msgr2=0x7f3a0c06aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:54.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.147+0000 7f3a13697640 1 -- 172.21.15.46:0/3776086590 shutdown_connections 2023-12-02T00:29:54.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.147+0000 7f3a13697640 1 -- 172.21.15.46:0/3776086590 wait complete. 2023-12-02T00:29:54.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.148+0000 7f3a13697640 1 Processor -- start 2023-12-02T00:29:54.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.148+0000 7f3a13697640 1 -- start start 2023-12-02T00:29:54.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.148+0000 7f3a13697640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a0c06ea50 0x7f3a0c14a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:54.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.148+0000 7f3a13697640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a0c0b3ff0 0x7f3a0c156380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:54.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.148+0000 7f3a13697640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a0c160fb0 0x7f3a0c06cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:54.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.149+0000 7f3a13697640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0c155ef0 con 0x7f3a0c0b3ff0 2023-12-02T00:29:54.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.149+0000 7f3a13697640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0c14a830 con 0x7f3a0c06ea50 2023-12-02T00:29:54.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.149+0000 7f3a13697640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0c1568c0 con 0x7f3a0c160fb0 2023-12-02T00:29:54.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.149+0000 7f3a10c32640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a0c0b3ff0 0x7f3a0c156380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:54.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.149+0000 7f3a10c32640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a0c0b3ff0 0x7f3a0c156380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50564/0 (socket says 172.21.15.46:50564) 2023-12-02T00:29:54.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.149+0000 7f3a10c32640 1 -- 172.21.15.46:0/2494593868 learned_addr learned my addr 172.21.15.46:0/2494593868 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:54.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.149+0000 7f3a11433640 1 --2- 172.21.15.46:0/2494593868 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a0c06ea50 0x7f3a0c14a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:54.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.149+0000 7f3a11c34640 1 --2- 172.21.15.46:0/2494593868 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a0c160fb0 0x7f3a0c06cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:54.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.149+0000 7f3a10c32640 1 -- 172.21.15.46:0/2494593868 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a0c160fb0 msgr2=0x7f3a0c06cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:54.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.149+0000 7f3a10c32640 1 --2- 172.21.15.46:0/2494593868 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a0c160fb0 0x7f3a0c06cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:54.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.149+0000 7f3a10c32640 1 -- 172.21.15.46:0/2494593868 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a0c06ea50 msgr2=0x7f3a0c14a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:54.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.149+0000 7f3a10c32640 1 --2- 172.21.15.46:0/2494593868 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a0c06ea50 0x7f3a0c14a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:54.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.149+0000 7f3a10c32640 1 -- 172.21.15.46:0/2494593868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a080094d0 con 0x7f3a0c0b3ff0 2023-12-02T00:29:54.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.149+0000 7f3a10c32640 1 --2- 172.21.15.46:0/2494593868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a0c0b3ff0 0x7f3a0c156380 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f39fc00d830 tx=0x7f39fc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:54.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.149+0000 7f3a01ffb640 1 -- 172.21.15.46:0/2494593868 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39fc00b660 con 0x7f3a0c0b3ff0 2023-12-02T00:29:54.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.149+0000 7f3a01ffb640 1 -- 172.21.15.46:0/2494593868 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39fc00b7c0 con 0x7f3a0c0b3ff0 2023-12-02T00:29:54.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.149+0000 7f3a13697640 1 -- 172.21.15.46:0/2494593868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a0c154760 con 0x7f3a0c0b3ff0 2023-12-02T00:29:54.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.149+0000 7f3a01ffb640 1 -- 172.21.15.46:0/2494593868 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39fc00bb50 con 0x7f3a0c0b3ff0 2023-12-02T00:29:54.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.150+0000 7f3a13697640 1 -- 172.21.15.46:0/2494593868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a0c1558a0 con 0x7f3a0c0b3ff0 2023-12-02T00:29:54.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.151+0000 7f3a13697640 1 -- 172.21.15.46:0/2494593868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a0c1560a0 con 0x7f3a0c0b3ff0 2023-12-02T00:29:54.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.151+0000 7f3a01ffb640 1 -- 172.21.15.46:0/2494593868 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f39fc01e460 con 0x7f3a0c0b3ff0 2023-12-02T00:29:54.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.151+0000 7f3a01ffb640 1 --2- 172.21.15.46:0/2494593868 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f39f006ae40 0x7f39f006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:54.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.152+0000 7f3a01ffb640 1 -- 172.21.15.46:0/2494593868 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f39fc0899c0 con 0x7f3a0c0b3ff0 2023-12-02T00:29:54.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.152+0000 7f3a11433640 1 --2- 172.21.15.46:0/2494593868 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f39f006ae40 0x7f39f006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:54.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.152+0000 7f3a11433640 1 --2- 172.21.15.46:0/2494593868 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f39f006ae40 0x7f39f006d300 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3a08036000 tx=0x7f3a0803d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:54.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.156+0000 7f3a01ffb640 1 -- 172.21.15.46:0/2494593868 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3a0c1560a0 con 0x7f3a0c0b3ff0 2023-12-02T00:29:54.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.402+0000 7f3a13697640 1 -- 172.21.15.46:0/2494593868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f3a0c1560a0 con 0x7f3a0c0b3ff0 2023-12-02T00:29:54.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.403+0000 7f3a01ffb640 1 -- 172.21.15.46:0/2494593868 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7f3a0c1560a0 con 0x7f3a0c0b3ff0 2023-12-02T00:29:54.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.405+0000 7f3a13697640 1 -- 172.21.15.46:0/2494593868 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f39f006ae40 msgr2=0x7f39f006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:54.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.405+0000 7f3a13697640 1 --2- 172.21.15.46:0/2494593868 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f39f006ae40 0x7f39f006d300 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3a08036000 tx=0x7f3a0803d040 comp rx=0 tx=0).stop 2023-12-02T00:29:54.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.405+0000 7f3a13697640 1 -- 172.21.15.46:0/2494593868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a0c0b3ff0 msgr2=0x7f3a0c156380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:54.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.405+0000 7f3a13697640 1 --2- 172.21.15.46:0/2494593868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a0c0b3ff0 0x7f3a0c156380 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f39fc00d830 tx=0x7f39fc00dcf0 comp rx=0 tx=0).stop 2023-12-02T00:29:54.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.406+0000 7f3a13697640 1 -- 172.21.15.46:0/2494593868 shutdown_connections 2023-12-02T00:29:54.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.406+0000 7f3a13697640 1 --2- 172.21.15.46:0/2494593868 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f39f006ae40 0x7f39f006d300 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:54.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.406+0000 7f3a13697640 1 --2- 172.21.15.46:0/2494593868 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a0c160fb0 0x7f3a0c06cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:54.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.406+0000 7f3a13697640 1 --2- 172.21.15.46:0/2494593868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a0c0b3ff0 0x7f3a0c156380 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:54.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.406+0000 7f3a13697640 1 --2- 172.21.15.46:0/2494593868 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a0c06ea50 0x7f3a0c14a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:54.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.406+0000 7f3a13697640 1 -- 172.21.15.46:0/2494593868 >> 172.21.15.46:0/2494593868 conn(0x7f3a0c06a6b0 msgr2=0x7f3a0c15cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:54.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.406+0000 7f3a13697640 1 -- 172.21.15.46:0/2494593868 shutdown_connections 2023-12-02T00:29:54.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.406+0000 7f3a13697640 1 -- 172.21.15.46:0/2494593868 wait complete. 2023-12-02T00:29:54.417 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:29:54.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.522+0000 7f91ec341640 1 Processor -- start 2023-12-02T00:29:54.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.522+0000 7f91ec341640 1 -- start start 2023-12-02T00:29:54.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.523+0000 7f91ec341640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91e40d2910 0x7f91e40d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:54.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.523+0000 7f91ec341640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91e40b2b10 con 0x7f91e415d330 2023-12-02T00:29:54.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.523+0000 7f91ec341640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91e40d3b50 con 0x7f91e40d3730 2023-12-02T00:29:54.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.523+0000 7f91ec341640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91e40d1590 con 0x7f91e40d2910 2023-12-02T00:29:54.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.523+0000 7f91ea0dd640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f91e415d330 0x7f91e40b8b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57074/0 (socket says 172.21.15.46:57074) 2023-12-02T00:29:54.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.523+0000 7f91ea0dd640 1 -- 172.21.15.46:0/3192894302 learned_addr learned my addr 172.21.15.46:0/3192894302 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:54.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.524+0000 7f91e98dc640 1 --2- 172.21.15.46:0/3192894302 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91e40d2910 0x7f91e40d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:54.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.524+0000 7f91e90db640 1 -- 172.21.15.46:0/3192894302 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1201155473 0 0) 0x7f91e40b2b10 con 0x7f91e415d330 2023-12-02T00:29:54.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.524+0000 7f91e90db640 1 -- 172.21.15.46:0/3192894302 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91cc003520 con 0x7f91e415d330 2023-12-02T00:29:54.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.524+0000 7f91e90db640 1 -- 172.21.15.46:0/3192894302 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1989234167 0 0) 0x7f91e40d3b50 con 0x7f91e40d3730 2023-12-02T00:29:54.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.524+0000 7f91e90db640 1 -- 172.21.15.46:0/3192894302 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91e40b2b10 con 0x7f91e40d3730 2023-12-02T00:29:54.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.524+0000 7f91e90db640 1 -- 172.21.15.46:0/3192894302 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2259410992 0 0) 0x7f91cc003520 con 0x7f91e415d330 2023-12-02T00:29:54.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.524+0000 7f91e90db640 1 -- 172.21.15.46:0/3192894302 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91e40d2910 msgr2=0x7f91e40d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:54.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.524+0000 7f91e90db640 1 --2- 172.21.15.46:0/3192894302 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91e40d2910 0x7f91e40d31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:54.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.524+0000 7f91e90db640 1 -- 172.21.15.46:0/3192894302 >> v1:172.21.15.162:6789/0 conn(0x7f91e40d3730 legacy=0x7f91e40b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:54.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.524+0000 7f91e90db640 1 -- 172.21.15.46:0/3192894302 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e40d1870 con 0x7f91e415d330 2023-12-02T00:29:54.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.524+0000 7f91e90db640 1 -- 172.21.15.46:0/3192894302 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7f91d40019f0 con 0x7f91e415d330 2023-12-02T00:29:54.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.524+0000 7f91e90db640 1 -- 172.21.15.46:0/3192894302 >> v1:172.21.15.46:6789/0 conn(0x7f91e415d330 legacy=0x7f91e40b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:54.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.524+0000 7f91e90db640 1 --2- 172.21.15.46:0/3192894302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91cc003b30 0x7f91cc003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:54.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.524+0000 7f91e90db640 1 --2- 172.21.15.46:0/3192894302 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91cc0047c0 0x7f91cc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:54.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.524+0000 7f91e90db640 1 --2- 172.21.15.46:0/3192894302 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91e40d2910 0x7f91cc0051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:54.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.524+0000 7f91e90db640 1 -- 172.21.15.46:0/3192894302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e40d1590 con 0x7f91cc003b30 2023-12-02T00:29:54.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.524+0000 7f91e90db640 1 -- 172.21.15.46:0/3192894302 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc00bac0 con 0x7f91cc0047c0 2023-12-02T00:29:54.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.524+0000 7f91e90db640 1 -- 172.21.15.46:0/3192894302 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc00bc80 con 0x7f91e40d2910 2023-12-02T00:29:54.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.524+0000 7f91ea0dd640 1 --2- 172.21.15.46:0/3192894302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91cc003b30 0x7f91cc003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:54.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.524+0000 7f91e98dc640 1 --2- 172.21.15.46:0/3192894302 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91cc0047c0 0x7f91cc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:54.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.524+0000 7f91ea8de640 1 --2- 172.21.15.46:0/3192894302 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91e40d2910 0x7f91cc0051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:54.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.525+0000 7f91ea0dd640 1 -- 172.21.15.46:0/3192894302 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91e40d2910 msgr2=0x7f91cc0051a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:54.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.525+0000 7f91ea0dd640 1 --2- 172.21.15.46:0/3192894302 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91e40d2910 0x7f91cc0051a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:54.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.525+0000 7f91ea0dd640 1 -- 172.21.15.46:0/3192894302 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91cc0047c0 msgr2=0x7f91cc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:54.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.525+0000 7f91ea0dd640 1 --2- 172.21.15.46:0/3192894302 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91cc0047c0 0x7f91cc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:54.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.525+0000 7f91ea0dd640 1 -- 172.21.15.46:0/3192894302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91cc003520 con 0x7f91cc003b30 2023-12-02T00:29:54.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.525+0000 7f91ea8de640 1 --2- 172.21.15.46:0/3192894302 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91e40d2910 0x7f91cc0051a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:54.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.525+0000 7f91ea0dd640 1 --2- 172.21.15.46:0/3192894302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91cc003b30 0x7f91cc003f30 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f91d400a7c0 tx=0x7f91cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89c2e41e8d16f90 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:54.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.525+0000 7f91e90db640 1 -- 172.21.15.46:0/3192894302 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d4034630 con 0x7f91cc003b30 2023-12-02T00:29:54.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.525+0000 7f91e90db640 1 -- 172.21.15.46:0/3192894302 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d402f070 con 0x7f91cc003b30 2023-12-02T00:29:54.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.525+0000 7f91e90db640 1 -- 172.21.15.46:0/3192894302 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d40349f0 con 0x7f91cc003b30 2023-12-02T00:29:54.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.526+0000 7f91ec341640 1 -- 172.21.15.46:0/3192894302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91cc003b30 msgr2=0x7f91cc003f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:54.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.526+0000 7f91ec341640 1 --2- 172.21.15.46:0/3192894302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91cc003b30 0x7f91cc003f30 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f91d400a7c0 tx=0x7f91cc0057f0 comp rx=0 tx=0).stop 2023-12-02T00:29:54.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.526+0000 7f91ea8de640 1 -- 172.21.15.46:0/3192894302 reap_dead start 2023-12-02T00:29:54.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.526+0000 7f91ec341640 1 -- 172.21.15.46:0/3192894302 shutdown_connections 2023-12-02T00:29:54.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.526+0000 7f91ec341640 1 -- 172.21.15.46:0/3192894302 >> 172.21.15.46:0/3192894302 conn(0x7f91e4072020 msgr2=0x7f91e4072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:54.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.526+0000 7f91ec341640 1 -- 172.21.15.46:0/3192894302 shutdown_connections 2023-12-02T00:29:54.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.526+0000 7f91ec341640 1 -- 172.21.15.46:0/3192894302 wait complete. 2023-12-02T00:29:54.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.527+0000 7f91ec341640 1 Processor -- start 2023-12-02T00:29:54.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.527+0000 7f91ec341640 1 -- start start 2023-12-02T00:29:54.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.528+0000 7f91ec341640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91cc003b30 0x7f91e4153550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:54.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.528+0000 7f91ec341640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91e40b5980 0x7f91e41593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:54.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.528+0000 7f91ec341640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91e40b8740 0x7f91e41f26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:54.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.528+0000 7f91ec341640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e4157230 con 0x7f91e40b8740 2023-12-02T00:29:54.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.528+0000 7f91ec341640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e4159e30 con 0x7f91cc003b30 2023-12-02T00:29:54.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.528+0000 7f91ec341640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91e4157480 con 0x7f91e40b5980 2023-12-02T00:29:54.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.528+0000 7f91ea8de640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91e40b8740 0x7f91e41f26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:54.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.528+0000 7f91ea8de640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91e40b8740 0x7f91e41f26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50582/0 (socket says 172.21.15.46:50582) 2023-12-02T00:29:54.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.528+0000 7f91ea8de640 1 -- 172.21.15.46:0/3190596312 learned_addr learned my addr 172.21.15.46:0/3190596312 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:54.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.528+0000 7f91ea8de640 1 -- 172.21.15.46:0/3190596312 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91e40b5980 msgr2=0x7f91e41593b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:54.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.528+0000 7f91ea8de640 1 --2- 172.21.15.46:0/3190596312 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91e40b5980 0x7f91e41593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:54.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.528+0000 7f91ea8de640 1 -- 172.21.15.46:0/3190596312 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91cc003b30 msgr2=0x7f91e4153550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:54.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.528+0000 7f91ea8de640 1 --2- 172.21.15.46:0/3190596312 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91cc003b30 0x7f91e4153550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:54.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.528+0000 7f91ea8de640 1 -- 172.21.15.46:0/3190596312 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e4157760 con 0x7f91e40b8740 2023-12-02T00:29:54.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.529+0000 7f91ea8de640 1 --2- 172.21.15.46:0/3190596312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91e40b8740 0x7f91e41f26a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f91dc007ae0 tx=0x7f91dc00e890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:54.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.529+0000 7f91e2ffd640 1 -- 172.21.15.46:0/3190596312 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91dc0052e0 con 0x7f91e40b8740 2023-12-02T00:29:54.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.529+0000 7f91e2ffd640 1 -- 172.21.15.46:0/3190596312 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91dc005460 con 0x7f91e40b8740 2023-12-02T00:29:54.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.529+0000 7f91ec341640 1 -- 172.21.15.46:0/3190596312 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91e4157a40 con 0x7f91e40b8740 2023-12-02T00:29:54.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.529+0000 7f91e2ffd640 1 -- 172.21.15.46:0/3190596312 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91dc003830 con 0x7f91e40b8740 2023-12-02T00:29:54.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.529+0000 7f91ec341640 1 -- 172.21.15.46:0/3190596312 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91e415e400 con 0x7f91e40b8740 2023-12-02T00:29:54.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.530+0000 7f91ec341640 1 -- 172.21.15.46:0/3190596312 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91e4069400 con 0x7f91e40b8740 2023-12-02T00:29:54.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.530+0000 7f91e2ffd640 1 -- 172.21.15.46:0/3190596312 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f91dc003990 con 0x7f91e40b8740 2023-12-02T00:29:54.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.531+0000 7f91e2ffd640 1 --2- 172.21.15.46:0/3190596312 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f91d006ae70 0x7f91d006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:54.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.531+0000 7f91e2ffd640 1 -- 172.21.15.46:0/3190596312 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f91dc088980 con 0x7f91e40b8740 2023-12-02T00:29:54.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.531+0000 7f91ea0dd640 1 --2- 172.21.15.46:0/3190596312 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f91d006ae70 0x7f91d006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:54.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.532+0000 7f91ea0dd640 1 --2- 172.21.15.46:0/3190596312 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f91d006ae70 0x7f91d006d330 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f91d4034000 tx=0x7f91d4007000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:54.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.535+0000 7f91e2ffd640 1 -- 172.21.15.46:0/3190596312 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f91dc053d70 con 0x7f91e40b8740 2023-12-02T00:29:54.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.712+0000 7f91ec341640 1 -- 172.21.15.46:0/3190596312 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f91e415e5c0 con 0x7f91e40b8740 2023-12-02T00:29:54.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.713+0000 7f91e2ffd640 1 -- 172.21.15.46:0/3190596312 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+12833 (secure 0 0 0) 0x7f91dc019020 con 0x7f91e40b8740 2023-12-02T00:29:54.714 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:29:54.714 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":31,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:29:51.716274+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:29:54.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.715+0000 7f91ec341640 1 -- 172.21.15.46:0/3190596312 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f91d006ae70 msgr2=0x7f91d006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:54.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.715+0000 7f91ec341640 1 --2- 172.21.15.46:0/3190596312 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f91d006ae70 0x7f91d006d330 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f91d4034000 tx=0x7f91d4007000 comp rx=0 tx=0).stop 2023-12-02T00:29:54.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.715+0000 7f91ec341640 1 -- 172.21.15.46:0/3190596312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91e40b8740 msgr2=0x7f91e41f26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:54.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.715+0000 7f91ec341640 1 --2- 172.21.15.46:0/3190596312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91e40b8740 0x7f91e41f26a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f91dc007ae0 tx=0x7f91dc00e890 comp rx=0 tx=0).stop 2023-12-02T00:29:54.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.715+0000 7f91ec341640 1 -- 172.21.15.46:0/3190596312 shutdown_connections 2023-12-02T00:29:54.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.715+0000 7f91ec341640 1 --2- 172.21.15.46:0/3190596312 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f91d006ae70 0x7f91d006d330 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:54.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.715+0000 7f91ec341640 1 --2- 172.21.15.46:0/3190596312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91e40b8740 0x7f91e41f26a0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:54.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.715+0000 7f91ec341640 1 --2- 172.21.15.46:0/3190596312 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91e40b5980 0x7f91e41593b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:54.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.715+0000 7f91ec341640 1 --2- 172.21.15.46:0/3190596312 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91cc003b30 0x7f91e4153550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:54.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.715+0000 7f91ec341640 1 -- 172.21.15.46:0/3190596312 >> 172.21.15.46:0/3190596312 conn(0x7f91e4072020 msgr2=0x7f91e40b4950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:54.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.715+0000 7f91ec341640 1 -- 172.21.15.46:0/3190596312 shutdown_connections 2023-12-02T00:29:54.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.715+0000 7f91ec341640 1 -- 172.21.15.46:0/3190596312 wait complete. 2023-12-02T00:29:54.726 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-02T00:28:37.213645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-02T00:29:54.727 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-02T00:29:54.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.835+0000 7f42a89a9640 1 Processor -- start 2023-12-02T00:29:54.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.836+0000 7f42a89a9640 1 -- start start 2023-12-02T00:29:54.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.836+0000 7f42a89a9640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f42a4143490 0x7f42a40b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:54.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.836+0000 7f42a89a9640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f42a415abf0 0x7f42a40b1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:54.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.836+0000 7f42a89a9640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42a415e2a0 con 0x7f42a415e730 2023-12-02T00:29:54.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.836+0000 7f42a89a9640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42a415b520 con 0x7f42a415abf0 2023-12-02T00:29:54.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.837+0000 7f42a89a9640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42a4143870 con 0x7f42a4143490 2023-12-02T00:29:54.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.837+0000 7f42a2d9d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f42a415abf0 0x7f42a40b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:54.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.837+0000 7f42a2d9d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f42a415abf0 0x7f42a40b1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50588/0 (socket says 172.21.15.46:50588) 2023-12-02T00:29:54.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.837+0000 7f42a2d9d640 1 -- 172.21.15.46:0/3078110309 learned_addr learned my addr 172.21.15.46:0/3078110309 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:54.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.837+0000 7f42a259c640 1 --2- 172.21.15.46:0/3078110309 >> v2:172.21.15.162:3300/0 conn(0x7f42a4143490 0x7f42a40b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:54.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.837+0000 7f42a2d9d640 1 -- 172.21.15.46:0/3078110309 >> v2:172.21.15.162:3300/0 conn(0x7f42a4143490 msgr2=0x7f42a40b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:54.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.837+0000 7f42a2d9d640 1 --2- 172.21.15.46:0/3078110309 >> v2:172.21.15.162:3300/0 conn(0x7f42a4143490 0x7f42a40b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:54.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.837+0000 7f42a2d9d640 1 -- 172.21.15.46:0/3078110309 >> v1:172.21.15.162:6789/0 conn(0x7f42a415e730 legacy=0x7f42a415a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:54.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.837+0000 7f42a2d9d640 1 -- 172.21.15.46:0/3078110309 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a415b240 con 0x7f42a415abf0 2023-12-02T00:29:54.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.837+0000 7f42a259c640 1 --2- 172.21.15.46:0/3078110309 >> v2:172.21.15.162:3300/0 conn(0x7f42a4143490 0x7f42a40b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:54.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.838+0000 7f42a2d9d640 1 --2- 172.21.15.46:0/3078110309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42a415abf0 0x7f42a40b1230 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f42a406e620 tx=0x7f429c02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa81df80defe9fe server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:54.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.838+0000 7f42a159a640 1 -- 172.21.15.46:0/3078110309 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429c02cb20 con 0x7f42a415abf0 2023-12-02T00:29:54.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.838+0000 7f42a159a640 1 -- 172.21.15.46:0/3078110309 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f429c02cc80 con 0x7f42a415abf0 2023-12-02T00:29:54.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.838+0000 7f42a159a640 1 -- 172.21.15.46:0/3078110309 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429c0343f0 con 0x7f42a415abf0 2023-12-02T00:29:54.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.838+0000 7f42a89a9640 1 -- 172.21.15.46:0/3078110309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42a415abf0 msgr2=0x7f42a40b1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:54.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.838+0000 7f42a89a9640 1 --2- 172.21.15.46:0/3078110309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42a415abf0 0x7f42a40b1230 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f42a406e620 tx=0x7f429c02bf80 comp rx=0 tx=0).stop 2023-12-02T00:29:54.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.839+0000 7f42a89a9640 1 -- 172.21.15.46:0/3078110309 shutdown_connections 2023-12-02T00:29:54.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.839+0000 7f42a89a9640 1 --2- 172.21.15.46:0/3078110309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42a415abf0 0x7f42a40b1230 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:54.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.839+0000 7f42a89a9640 1 --2- 172.21.15.46:0/3078110309 >> v2:172.21.15.162:3300/0 conn(0x7f42a4143490 0x7f42a40b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:54.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.839+0000 7f42a89a9640 1 -- 172.21.15.46:0/3078110309 >> 172.21.15.46:0/3078110309 conn(0x7f42a406a4d0 msgr2=0x7f42a406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:54.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.839+0000 7f42a89a9640 1 -- 172.21.15.46:0/3078110309 shutdown_connections 2023-12-02T00:29:54.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.839+0000 7f42a89a9640 1 -- 172.21.15.46:0/3078110309 wait complete. 2023-12-02T00:29:54.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.840+0000 7f42a89a9640 1 Processor -- start 2023-12-02T00:29:54.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.840+0000 7f42a89a9640 1 -- start start 2023-12-02T00:29:54.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.841+0000 7f42a89a9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42a4143490 0x7f42a40c83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:54.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.841+0000 7f42a89a9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f42a415abf0 0x7f42a4070670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:54.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.841+0000 7f42a89a9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42a415e730 0x7f42a4159550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:54.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.841+0000 7f42a89a9640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a40c8c10 con 0x7f42a415e730 2023-12-02T00:29:54.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.841+0000 7f42a89a9640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a40c8ef0 con 0x7f42a4143490 2023-12-02T00:29:54.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.841+0000 7f42a89a9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42a40c91d0 con 0x7f42a415abf0 2023-12-02T00:29:54.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.841+0000 7f42a2d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42a415e730 0x7f42a4159550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:54.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.841+0000 7f42a2d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42a415e730 0x7f42a4159550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50602/0 (socket says 172.21.15.46:50602) 2023-12-02T00:29:54.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.841+0000 7f42a2d9d640 1 -- 172.21.15.46:0/1013912382 learned_addr learned my addr 172.21.15.46:0/1013912382 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:54.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.841+0000 7f42a259c640 1 --2- 172.21.15.46:0/1013912382 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42a4143490 0x7f42a40c83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:54.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.841+0000 7f42a1d9b640 1 --2- 172.21.15.46:0/1013912382 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f42a415abf0 0x7f42a4070670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:54.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.841+0000 7f42a2d9d640 1 -- 172.21.15.46:0/1013912382 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f42a415abf0 msgr2=0x7f42a4070670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:54.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.841+0000 7f42a2d9d640 1 --2- 172.21.15.46:0/1013912382 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f42a415abf0 0x7f42a4070670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:54.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.841+0000 7f42a2d9d640 1 -- 172.21.15.46:0/1013912382 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42a4143490 msgr2=0x7f42a40c83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:54.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.841+0000 7f42a2d9d640 1 --2- 172.21.15.46:0/1013912382 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42a4143490 0x7f42a40c83e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:54.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.841+0000 7f42a2d9d640 1 -- 172.21.15.46:0/1013912382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f429c009b30 con 0x7f42a415e730 2023-12-02T00:29:54.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.841+0000 7f42a2d9d640 1 --2- 172.21.15.46:0/1013912382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42a415e730 0x7f42a4159550 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f429c036040 tx=0x7f429c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:54.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.841+0000 7f4292ffd640 1 -- 172.21.15.46:0/1013912382 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429c007e70 con 0x7f42a415e730 2023-12-02T00:29:54.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.841+0000 7f4292ffd640 1 -- 172.21.15.46:0/1013912382 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f429c009510 con 0x7f42a415e730 2023-12-02T00:29:54.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.841+0000 7f42a89a9640 1 -- 172.21.15.46:0/1013912382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42a40c94b0 con 0x7f42a415e730 2023-12-02T00:29:54.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.841+0000 7f4292ffd640 1 -- 172.21.15.46:0/1013912382 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429c03e690 con 0x7f42a415e730 2023-12-02T00:29:54.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.841+0000 7f42a89a9640 1 -- 172.21.15.46:0/1013912382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42a40c8920 con 0x7f42a415e730 2023-12-02T00:29:54.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.842+0000 7f42a89a9640 1 -- 172.21.15.46:0/1013912382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42a40c99d0 con 0x7f42a415e730 2023-12-02T00:29:54.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.843+0000 7f4292ffd640 1 -- 172.21.15.46:0/1013912382 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f429c03e7f0 con 0x7f42a415e730 2023-12-02T00:29:54.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.843+0000 7f4292ffd640 1 --2- 172.21.15.46:0/1013912382 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f428406ae40 0x7f428406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:54.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.843+0000 7f4292ffd640 1 -- 172.21.15.46:0/1013912382 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f429c0abe90 con 0x7f42a415e730 2023-12-02T00:29:54.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.843+0000 7f42a259c640 1 --2- 172.21.15.46:0/1013912382 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f428406ae40 0x7f428406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:54.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.844+0000 7f42a259c640 1 --2- 172.21.15.46:0/1013912382 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f428406ae40 0x7f428406d300 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4294006fd0 tx=0x7f4294008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:54.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:54.847+0000 7f4292ffd640 1 -- 172.21.15.46:0/1013912382 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f429c0773b0 con 0x7f42a415e730 2023-12-02T00:29:55.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.022+0000 7f42a89a9640 1 -- 172.21.15.46:0/1013912382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f42a40ae7f0 con 0x7f42a415e730 2023-12-02T00:29:55.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.022+0000 7f4292ffd640 1 -- 172.21.15.46:0/1013912382 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v31) v1 ==== 93+0+10 (secure 0 0 0) 0x7f429c07a9d0 con 0x7f42a415e730 2023-12-02T00:29:55.023 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-12-02T00:29:55.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.024+0000 7f42a89a9640 1 -- 172.21.15.46:0/1013912382 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f428406ae40 msgr2=0x7f428406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:55.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.024+0000 7f42a89a9640 1 --2- 172.21.15.46:0/1013912382 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f428406ae40 0x7f428406d300 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4294006fd0 tx=0x7f4294008040 comp rx=0 tx=0).stop 2023-12-02T00:29:55.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.024+0000 7f42a89a9640 1 -- 172.21.15.46:0/1013912382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42a415e730 msgr2=0x7f42a4159550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:55.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.024+0000 7f42a89a9640 1 --2- 172.21.15.46:0/1013912382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42a415e730 0x7f42a4159550 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f429c036040 tx=0x7f429c004a40 comp rx=0 tx=0).stop 2023-12-02T00:29:55.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.024+0000 7f42a89a9640 1 -- 172.21.15.46:0/1013912382 shutdown_connections 2023-12-02T00:29:55.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.024+0000 7f42a89a9640 1 --2- 172.21.15.46:0/1013912382 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f428406ae40 0x7f428406d300 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:55.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.024+0000 7f42a89a9640 1 --2- 172.21.15.46:0/1013912382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42a415e730 0x7f42a4159550 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:55.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.024+0000 7f42a89a9640 1 --2- 172.21.15.46:0/1013912382 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f42a415abf0 0x7f42a4070670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:55.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.024+0000 7f42a89a9640 1 --2- 172.21.15.46:0/1013912382 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42a4143490 0x7f42a40c83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:55.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.024+0000 7f42a89a9640 1 -- 172.21.15.46:0/1013912382 >> 172.21.15.46:0/1013912382 conn(0x7f42a406a4d0 msgr2=0x7f42a40cdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:55.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.024+0000 7f42a89a9640 1 -- 172.21.15.46:0/1013912382 shutdown_connections 2023-12-02T00:29:55.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.024+0000 7f42a89a9640 1 -- 172.21.15.46:0/1013912382 wait complete. 2023-12-02T00:29:55.036 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-02T00:29:55.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.143+0000 7f3bd27be640 1 Processor -- start 2023-12-02T00:29:55.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.143+0000 7f3bd27be640 1 -- start start 2023-12-02T00:29:55.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.144+0000 7f3bd27be640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3bcc15d1d0 0x7f3bcc06b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:55.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.144+0000 7f3bd27be640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3bcc0ad640 0x7f3bcc06ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:55.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.144+0000 7f3bd27be640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3bcc0aeb80 0x7f3bcc160ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:55.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.144+0000 7f3bd27be640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bcc0adae0 con 0x7f3bcc15d1d0 2023-12-02T00:29:55.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.144+0000 7f3bd27be640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bcc0addc0 con 0x7f3bcc0aeb80 2023-12-02T00:29:55.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.144+0000 7f3bd27be640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc0ae0a0 con 0x7f3bcc0ad640 2023-12-02T00:29:55.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.144+0000 7f3bcbfff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3bcc15d1d0 0x7f3bcc06b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:55.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.144+0000 7f3bcbfff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3bcc15d1d0 0x7f3bcc06b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50618/0 (socket says 172.21.15.46:50618) 2023-12-02T00:29:55.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.144+0000 7f3bcbfff640 1 -- 172.21.15.46:0/2177307139 learned_addr learned my addr 172.21.15.46:0/2177307139 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:55.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.144+0000 7f3bd0d5b640 1 --2- 172.21.15.46:0/2177307139 >> v2:172.21.15.162:3300/0 conn(0x7f3bcc0aeb80 0x7f3bcc160ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:55.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.144+0000 7f3bcb7fe640 1 --2- 172.21.15.46:0/2177307139 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3bcc0ad640 0x7f3bcc06ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:55.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.144+0000 7f3bcbfff640 1 -- 172.21.15.46:0/2177307139 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3bcc0ad640 msgr2=0x7f3bcc06ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:55.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.144+0000 7f3bcbfff640 1 --2- 172.21.15.46:0/2177307139 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3bcc0ad640 0x7f3bcc06ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:55.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.144+0000 7f3bcbfff640 1 -- 172.21.15.46:0/2177307139 >> v2:172.21.15.162:3300/0 conn(0x7f3bcc0aeb80 msgr2=0x7f3bcc160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:55.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.144+0000 7f3bcbfff640 1 --2- 172.21.15.46:0/2177307139 >> v2:172.21.15.162:3300/0 conn(0x7f3bcc0aeb80 0x7f3bcc160ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:55.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.144+0000 7f3bcbfff640 1 -- 172.21.15.46:0/2177307139 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bcc0ae380 con 0x7f3bcc15d1d0 2023-12-02T00:29:55.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.144+0000 7f3bcb7fe640 1 --2- 172.21.15.46:0/2177307139 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3bcc0ad640 0x7f3bcc06ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:55.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.145+0000 7f3bcbfff640 1 --2- 172.21.15.46:0/2177307139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bcc15d1d0 0x7f3bcc06b4e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc06f410 tx=0x7f3bbc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=712c662f6f219490 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:55.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.145+0000 7f3bcaffd640 1 -- 172.21.15.46:0/2177307139 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc031870 con 0x7f3bcc15d1d0 2023-12-02T00:29:55.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.145+0000 7f3bcaffd640 1 -- 172.21.15.46:0/2177307139 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bbc0319d0 con 0x7f3bcc15d1d0 2023-12-02T00:29:55.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.145+0000 7f3bcaffd640 1 -- 172.21.15.46:0/2177307139 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc031d60 con 0x7f3bcc15d1d0 2023-12-02T00:29:55.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.146+0000 7f3bd27be640 1 -- 172.21.15.46:0/2177307139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bcc15d1d0 msgr2=0x7f3bcc06b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:55.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.146+0000 7f3bd27be640 1 --2- 172.21.15.46:0/2177307139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bcc15d1d0 0x7f3bcc06b4e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc06f410 tx=0x7f3bbc02b910 comp rx=0 tx=0).stop 2023-12-02T00:29:55.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.146+0000 7f3bd27be640 1 -- 172.21.15.46:0/2177307139 shutdown_connections 2023-12-02T00:29:55.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.146+0000 7f3bd27be640 1 --2- 172.21.15.46:0/2177307139 >> v2:172.21.15.162:3300/0 conn(0x7f3bcc0aeb80 0x7f3bcc160ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:55.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.146+0000 7f3bd27be640 1 --2- 172.21.15.46:0/2177307139 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3bcc0ad640 0x7f3bcc06ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:55.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.146+0000 7f3bd27be640 1 --2- 172.21.15.46:0/2177307139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bcc15d1d0 0x7f3bcc06b4e0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:55.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.146+0000 7f3bd27be640 1 -- 172.21.15.46:0/2177307139 >> 172.21.15.46:0/2177307139 conn(0x7f3bcc070e70 msgr2=0x7f3bcc071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:55.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.146+0000 7f3bd27be640 1 -- 172.21.15.46:0/2177307139 shutdown_connections 2023-12-02T00:29:55.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.146+0000 7f3bd27be640 1 -- 172.21.15.46:0/2177307139 wait complete. 2023-12-02T00:29:55.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.147+0000 7f3bd27be640 1 Processor -- start 2023-12-02T00:29:55.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.147+0000 7f3bd27be640 1 -- start start 2023-12-02T00:29:55.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.148+0000 7f3bd27be640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3bcc0ad640 0x7f3bcc1f4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:55.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.148+0000 7f3bd27be640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3bcc0aeb80 0x7f3bcc152140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:55.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.148+0000 7f3bd27be640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bcc15d1d0 0x7f3bcc0cdc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:55.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.148+0000 7f3bd27be640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc15b930 con 0x7f3bcc15d1d0 2023-12-02T00:29:55.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.148+0000 7f3bd27be640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc15b020 con 0x7f3bcc0aeb80 2023-12-02T00:29:55.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.148+0000 7f3bd27be640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc1f71c0 con 0x7f3bcc0ad640 2023-12-02T00:29:55.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.148+0000 7f3bd0d5b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bcc15d1d0 0x7f3bcc0cdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:55.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.148+0000 7f3bd0d5b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bcc15d1d0 0x7f3bcc0cdc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50620/0 (socket says 172.21.15.46:50620) 2023-12-02T00:29:55.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.148+0000 7f3bd0d5b640 1 -- 172.21.15.46:0/31836510 learned_addr learned my addr 172.21.15.46:0/31836510 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:55.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.148+0000 7f3bcb7fe640 1 --2- 172.21.15.46:0/31836510 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3bcc0aeb80 0x7f3bcc152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:55.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.148+0000 7f3bcbfff640 1 --2- 172.21.15.46:0/31836510 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3bcc0ad640 0x7f3bcc1f4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:55.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.148+0000 7f3bd0d5b640 1 -- 172.21.15.46:0/31836510 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3bcc0ad640 msgr2=0x7f3bcc1f4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:55.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.148+0000 7f3bd0d5b640 1 --2- 172.21.15.46:0/31836510 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3bcc0ad640 0x7f3bcc1f4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:55.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.148+0000 7f3bd0d5b640 1 -- 172.21.15.46:0/31836510 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3bcc0aeb80 msgr2=0x7f3bcc152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:55.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.148+0000 7f3bd0d5b640 1 --2- 172.21.15.46:0/31836510 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3bcc0aeb80 0x7f3bcc152140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:55.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.148+0000 7f3bd0d5b640 1 -- 172.21.15.46:0/31836510 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bbc0094d0 con 0x7f3bcc15d1d0 2023-12-02T00:29:55.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.148+0000 7f3bcb7fe640 1 --2- 172.21.15.46:0/31836510 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3bcc0aeb80 0x7f3bcc152140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:55.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.148+0000 7f3bcbfff640 1 --2- 172.21.15.46:0/31836510 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3bcc0ad640 0x7f3bcc1f4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:55.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.149+0000 7f3bd0d5b640 1 --2- 172.21.15.46:0/31836510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bcc15d1d0 0x7f3bcc0cdc10 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3bc400efc0 tx=0x7f3bc400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:55.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.149+0000 7f3bc8ff9640 1 -- 172.21.15.46:0/31836510 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc4014480 con 0x7f3bcc15d1d0 2023-12-02T00:29:55.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.149+0000 7f3bc8ff9640 1 -- 172.21.15.46:0/31836510 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc400f040 con 0x7f3bcc15d1d0 2023-12-02T00:29:55.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.149+0000 7f3bd27be640 1 -- 172.21.15.46:0/31836510 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bcc15b270 con 0x7f3bcc15d1d0 2023-12-02T00:29:55.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.149+0000 7f3bc8ff9640 1 -- 172.21.15.46:0/31836510 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc4014820 con 0x7f3bcc15d1d0 2023-12-02T00:29:55.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.149+0000 7f3bd27be640 1 -- 172.21.15.46:0/31836510 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bcc152680 con 0x7f3bcc15d1d0 2023-12-02T00:29:55.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.150+0000 7f3bd27be640 1 -- 172.21.15.46:0/31836510 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bcc0cc360 con 0x7f3bcc15d1d0 2023-12-02T00:29:55.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.150+0000 7f3bc8ff9640 1 -- 172.21.15.46:0/31836510 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3bc4014980 con 0x7f3bcc15d1d0 2023-12-02T00:29:55.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.150+0000 7f3bc8ff9640 1 --2- 172.21.15.46:0/31836510 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3bb806ae40 0x7f3bb806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:55.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.151+0000 7f3bc8ff9640 1 -- 172.21.15.46:0/31836510 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3bc408a0d0 con 0x7f3bcc15d1d0 2023-12-02T00:29:55.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.151+0000 7f3bcbfff640 1 --2- 172.21.15.46:0/31836510 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3bb806ae40 0x7f3bb806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:55.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.151+0000 7f3bcbfff640 1 --2- 172.21.15.46:0/31836510 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3bb806ae40 0x7f3bb806d300 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc036000 tx=0x7f3bbc03c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:55.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.155+0000 7f3bc8ff9640 1 -- 172.21.15.46:0/31836510 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3bc4055560 con 0x7f3bcc15d1d0 2023-12-02T00:29:55.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.334+0000 7f3bd27be640 1 -- 172.21.15.46:0/31836510 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f3bcc15ba70 con 0x7f3bcc15d1d0 2023-12-02T00:29:55.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.334+0000 7f3bc8ff9640 1 -- 172.21.15.46:0/31836510 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v31) v1 ==== 84+0+65 (secure 0 0 0) 0x7f3bcc15ba70 con 0x7f3bcc15d1d0 2023-12-02T00:29:55.335 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:29:55.335 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2023-12-02T00:29:55.335 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-02T00:29:55.336 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2023-12-02T00:29:55.336 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2023-12-02T00:29:55.336 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:29:55.336 INFO:teuthology.orchestra.run.smithi046.stdout:] 2023-12-02T00:29:55.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.336+0000 7f3bd27be640 1 -- 172.21.15.46:0/31836510 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3bb806ae40 msgr2=0x7f3bb806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:55.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.336+0000 7f3bd27be640 1 --2- 172.21.15.46:0/31836510 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3bb806ae40 0x7f3bb806d300 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc036000 tx=0x7f3bbc03c040 comp rx=0 tx=0).stop 2023-12-02T00:29:55.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.336+0000 7f3bd27be640 1 -- 172.21.15.46:0/31836510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bcc15d1d0 msgr2=0x7f3bcc0cdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:55.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.336+0000 7f3bd27be640 1 --2- 172.21.15.46:0/31836510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bcc15d1d0 0x7f3bcc0cdc10 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3bc400efc0 tx=0x7f3bc400c490 comp rx=0 tx=0).stop 2023-12-02T00:29:55.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.336+0000 7f3bd27be640 1 -- 172.21.15.46:0/31836510 shutdown_connections 2023-12-02T00:29:55.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.336+0000 7f3bd27be640 1 --2- 172.21.15.46:0/31836510 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3bb806ae40 0x7f3bb806d300 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:55.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.336+0000 7f3bd27be640 1 --2- 172.21.15.46:0/31836510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bcc15d1d0 0x7f3bcc0cdc10 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:55.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.336+0000 7f3bd27be640 1 --2- 172.21.15.46:0/31836510 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3bcc0aeb80 0x7f3bcc152140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:55.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.336+0000 7f3bd27be640 1 --2- 172.21.15.46:0/31836510 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3bcc0ad640 0x7f3bcc1f4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:55.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.336+0000 7f3bd27be640 1 -- 172.21.15.46:0/31836510 >> 172.21.15.46:0/31836510 conn(0x7f3bcc070e70 msgr2=0x7f3bcc06eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:55.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.336+0000 7f3bd27be640 1 -- 172.21.15.46:0/31836510 shutdown_connections 2023-12-02T00:29:55.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.336+0000 7f3bd27be640 1 -- 172.21.15.46:0/31836510 wait complete. 2023-12-02T00:29:55.348 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-02T00:29:55.348 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-02T00:29:55.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.458+0000 7f568bdbe640 1 Processor -- start 2023-12-02T00:29:55.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.458+0000 7f568bdbe640 1 -- start start 2023-12-02T00:29:55.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.459+0000 7f568bdbe640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f568406ea50 0x7f568406ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:55.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.459+0000 7f568bdbe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5684160fb0 0x7f5684161440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:55.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.459+0000 7f568bdbe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f56840b3ff0 0x7f56840b4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:55.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.459+0000 7f568bdbe640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5684161d20 con 0x7f56840b3ff0 2023-12-02T00:29:55.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.459+0000 7f568bdbe640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f568406f370 con 0x7f568406ea50 2023-12-02T00:29:55.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.459+0000 7f568bdbe640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f568415e760 con 0x7f5684160fb0 2023-12-02T00:29:55.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.459+0000 7f568a35b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f56840b3ff0 0x7f56840b4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:55.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.459+0000 7f568a35b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f56840b3ff0 0x7f56840b4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50630/0 (socket says 172.21.15.46:50630) 2023-12-02T00:29:55.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.459+0000 7f568a35b640 1 -- 172.21.15.46:0/4057132279 learned_addr learned my addr 172.21.15.46:0/4057132279 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:55.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.459+0000 7f5689b5a640 1 --2- 172.21.15.46:0/4057132279 >> v2:172.21.15.162:3300/0 conn(0x7f568406ea50 0x7f568406ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:55.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.459+0000 7f5689359640 1 --2- 172.21.15.46:0/4057132279 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5684160fb0 0x7f5684161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:55.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.459+0000 7f568a35b640 1 -- 172.21.15.46:0/4057132279 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5684160fb0 msgr2=0x7f5684161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:55.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.459+0000 7f568a35b640 1 --2- 172.21.15.46:0/4057132279 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5684160fb0 0x7f5684161440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:55.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.459+0000 7f568a35b640 1 -- 172.21.15.46:0/4057132279 >> v2:172.21.15.162:3300/0 conn(0x7f568406ea50 msgr2=0x7f568406ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:55.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.459+0000 7f568a35b640 1 --2- 172.21.15.46:0/4057132279 >> v2:172.21.15.162:3300/0 conn(0x7f568406ea50 0x7f568406ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:55.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.459+0000 7f568a35b640 1 -- 172.21.15.46:0/4057132279 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f568415d9a0 con 0x7f56840b3ff0 2023-12-02T00:29:55.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.459+0000 7f5689359640 1 --2- 172.21.15.46:0/4057132279 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5684160fb0 0x7f5684161440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:29:55.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.460+0000 7f568a35b640 1 --2- 172.21.15.46:0/4057132279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56840b3ff0 0x7f56840b4480 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f568406f680 tx=0x7f567c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f658ce352734cacb server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:55.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.460+0000 7f5688b58640 1 -- 172.21.15.46:0/4057132279 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567c02ca70 con 0x7f56840b3ff0 2023-12-02T00:29:55.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.460+0000 7f5688b58640 1 -- 172.21.15.46:0/4057132279 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f567c02cbd0 con 0x7f56840b3ff0 2023-12-02T00:29:55.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.460+0000 7f5688b58640 1 -- 172.21.15.46:0/4057132279 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567c0343f0 con 0x7f56840b3ff0 2023-12-02T00:29:55.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.461+0000 7f568bdbe640 1 -- 172.21.15.46:0/4057132279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56840b3ff0 msgr2=0x7f56840b4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:55.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.461+0000 7f568bdbe640 1 --2- 172.21.15.46:0/4057132279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56840b3ff0 0x7f56840b4480 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f568406f680 tx=0x7f567c02bed0 comp rx=0 tx=0).stop 2023-12-02T00:29:55.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.461+0000 7f568bdbe640 1 -- 172.21.15.46:0/4057132279 shutdown_connections 2023-12-02T00:29:55.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.461+0000 7f568bdbe640 1 --2- 172.21.15.46:0/4057132279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56840b3ff0 0x7f56840b4480 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:55.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.461+0000 7f568bdbe640 1 --2- 172.21.15.46:0/4057132279 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5684160fb0 0x7f5684161440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:55.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.461+0000 7f568bdbe640 1 --2- 172.21.15.46:0/4057132279 >> v2:172.21.15.162:3300/0 conn(0x7f568406ea50 0x7f568406ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:55.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.461+0000 7f568bdbe640 1 -- 172.21.15.46:0/4057132279 >> 172.21.15.46:0/4057132279 conn(0x7f568406a6b0 msgr2=0x7f568406aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:55.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.461+0000 7f568bdbe640 1 -- 172.21.15.46:0/4057132279 shutdown_connections 2023-12-02T00:29:55.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.461+0000 7f568bdbe640 1 -- 172.21.15.46:0/4057132279 wait complete. 2023-12-02T00:29:55.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.462+0000 7f568bdbe640 1 Processor -- start 2023-12-02T00:29:55.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.462+0000 7f568bdbe640 1 -- start start 2023-12-02T00:29:55.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.463+0000 7f568bdbe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f568406ea50 0x7f5684156660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:55.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.463+0000 7f568bdbe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56840b3ff0 0x7f5684157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:55.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.463+0000 7f568bdbe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5684160fb0 0x7f56841f2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:55.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.463+0000 7f568bdbe640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568415ae30 con 0x7f5684160fb0 2023-12-02T00:29:55.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.463+0000 7f568bdbe640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5684156ba0 con 0x7f568406ea50 2023-12-02T00:29:55.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.463+0000 7f568bdbe640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f568406b5e0 con 0x7f56840b3ff0 2023-12-02T00:29:55.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.463+0000 7f568a35b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5684160fb0 0x7f56841f2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:55.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.463+0000 7f568a35b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5684160fb0 0x7f56841f2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50642/0 (socket says 172.21.15.46:50642) 2023-12-02T00:29:55.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.463+0000 7f568a35b640 1 -- 172.21.15.46:0/1277427374 learned_addr learned my addr 172.21.15.46:0/1277427374 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:55.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.463+0000 7f5689b5a640 1 --2- 172.21.15.46:0/1277427374 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f568406ea50 0x7f5684156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:55.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.463+0000 7f5689359640 1 --2- 172.21.15.46:0/1277427374 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56840b3ff0 0x7f5684157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:55.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.463+0000 7f568a35b640 1 -- 172.21.15.46:0/1277427374 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56840b3ff0 msgr2=0x7f5684157ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:55.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.463+0000 7f568a35b640 1 --2- 172.21.15.46:0/1277427374 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56840b3ff0 0x7f5684157ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:55.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.463+0000 7f568a35b640 1 -- 172.21.15.46:0/1277427374 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f568406ea50 msgr2=0x7f5684156660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:55.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.463+0000 7f568a35b640 1 --2- 172.21.15.46:0/1277427374 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f568406ea50 0x7f5684156660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:55.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.463+0000 7f568a35b640 1 -- 172.21.15.46:0/1277427374 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f567c009b30 con 0x7f5684160fb0 2023-12-02T00:29:55.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.463+0000 7f568a35b640 1 --2- 172.21.15.46:0/1277427374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5684160fb0 0x7f56841f2a80 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f567c009f40 tx=0x7f567c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:55.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.463+0000 7f567a7fc640 1 -- 172.21.15.46:0/1277427374 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567c009510 con 0x7f5684160fb0 2023-12-02T00:29:55.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.463+0000 7f567a7fc640 1 -- 172.21.15.46:0/1277427374 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f567c03e450 con 0x7f5684160fb0 2023-12-02T00:29:55.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.463+0000 7f568bdbe640 1 -- 172.21.15.46:0/1277427374 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5684158400 con 0x7f5684160fb0 2023-12-02T00:29:55.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.463+0000 7f567a7fc640 1 -- 172.21.15.46:0/1277427374 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567c03e7e0 con 0x7f5684160fb0 2023-12-02T00:29:55.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.464+0000 7f568bdbe640 1 -- 172.21.15.46:0/1277427374 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5684158980 con 0x7f5684160fb0 2023-12-02T00:29:55.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.464+0000 7f568bdbe640 1 -- 172.21.15.46:0/1277427374 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5684069400 con 0x7f5684160fb0 2023-12-02T00:29:55.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.465+0000 7f567a7fc640 1 -- 172.21.15.46:0/1277427374 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f567c03e940 con 0x7f5684160fb0 2023-12-02T00:29:55.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.465+0000 7f567a7fc640 1 --2- 172.21.15.46:0/1277427374 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f566806ae40 0x7f566806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:55.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.465+0000 7f567a7fc640 1 -- 172.21.15.46:0/1277427374 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f567c0ac370 con 0x7f5684160fb0 2023-12-02T00:29:55.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.465+0000 7f5689b5a640 1 --2- 172.21.15.46:0/1277427374 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f566806ae40 0x7f566806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:55.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.466+0000 7f5689b5a640 1 --2- 172.21.15.46:0/1277427374 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f566806ae40 0x7f566806d300 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5680006fd0 tx=0x7f5680008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:55.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.469+0000 7f567a7fc640 1 -- 172.21.15.46:0/1277427374 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f567c0777f0 con 0x7f5684160fb0 2023-12-02T00:29:55.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.645+0000 7f568bdbe640 1 -- 172.21.15.46:0/1277427374 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f5684070d30 con 0x7f5684160fb0 2023-12-02T00:29:55.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.895+0000 7f567a7fc640 1 -- 172.21.15.46:0/1277427374 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v32) v1 ==== 138+0+0 (secure 0 0 0) 0x7f567c033e80 con 0x7f5684160fb0 2023-12-02T00:29:55.896 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2023-12-02T00:29:55.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.897+0000 7f568bdbe640 1 -- 172.21.15.46:0/1277427374 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f566806ae40 msgr2=0x7f566806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:55.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.897+0000 7f568bdbe640 1 --2- 172.21.15.46:0/1277427374 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f566806ae40 0x7f566806d300 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5680006fd0 tx=0x7f5680008040 comp rx=0 tx=0).stop 2023-12-02T00:29:55.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.897+0000 7f568bdbe640 1 -- 172.21.15.46:0/1277427374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5684160fb0 msgr2=0x7f56841f2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:55.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.897+0000 7f568bdbe640 1 --2- 172.21.15.46:0/1277427374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5684160fb0 0x7f56841f2a80 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f567c009f40 tx=0x7f567c004a40 comp rx=0 tx=0).stop 2023-12-02T00:29:55.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.897+0000 7f568bdbe640 1 -- 172.21.15.46:0/1277427374 shutdown_connections 2023-12-02T00:29:55.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.897+0000 7f568bdbe640 1 --2- 172.21.15.46:0/1277427374 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f566806ae40 0x7f566806d300 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:55.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.897+0000 7f568bdbe640 1 --2- 172.21.15.46:0/1277427374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5684160fb0 0x7f56841f2a80 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:55.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.897+0000 7f568bdbe640 1 --2- 172.21.15.46:0/1277427374 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56840b3ff0 0x7f5684157ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:55.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.897+0000 7f568bdbe640 1 --2- 172.21.15.46:0/1277427374 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f568406ea50 0x7f5684156660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:55.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.897+0000 7f568bdbe640 1 -- 172.21.15.46:0/1277427374 >> 172.21.15.46:0/1277427374 conn(0x7f568406a6b0 msgr2=0x7f568406aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:55.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.897+0000 7f568bdbe640 1 -- 172.21.15.46:0/1277427374 shutdown_connections 2023-12-02T00:29:55.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:55.897+0000 7f568bdbe640 1 -- 172.21.15.46:0/1277427374 wait complete. 2023-12-02T00:29:55.912 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-02T00:29:56.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.025+0000 7fecba34c640 1 Processor -- start 2023-12-02T00:29:56.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.025+0000 7fecba34c640 1 -- start start 2023-12-02T00:29:56.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.026+0000 7fecba34c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fecb40d3030 0x7fecb40d3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:56.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.026+0000 7fecba34c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fecb40cfc30 0x7fecb40d0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:56.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.026+0000 7fecba34c640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecb40d09a0 con 0x7fecb415d250 2023-12-02T00:29:56.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.026+0000 7fecba34c640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecb40cdf70 con 0x7fecb40cfc30 2023-12-02T00:29:56.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.026+0000 7fecba34c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecb40ce250 con 0x7fecb40d3030 2023-12-02T00:29:56.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.026+0000 7fecb88e9640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fecb40cfc30 0x7fecb40d0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:56.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.026+0000 7fecb88e9640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fecb40cfc30 0x7fecb40d0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50648/0 (socket says 172.21.15.46:50648) 2023-12-02T00:29:56.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.026+0000 7fecb88e9640 1 -- 172.21.15.46:0/3111355818 learned_addr learned my addr 172.21.15.46:0/3111355818 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:56.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.027+0000 7fecb37fe640 1 --2- 172.21.15.46:0/3111355818 >> v2:172.21.15.162:3300/0 conn(0x7fecb40d3030 0x7fecb40d3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:56.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.027+0000 7fecb88e9640 1 -- 172.21.15.46:0/3111355818 >> v2:172.21.15.162:3300/0 conn(0x7fecb40d3030 msgr2=0x7fecb40d3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:56.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.027+0000 7fecb88e9640 1 --2- 172.21.15.46:0/3111355818 >> v2:172.21.15.162:3300/0 conn(0x7fecb40d3030 0x7fecb40d3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:56.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.027+0000 7fecb88e9640 1 -- 172.21.15.46:0/3111355818 >> v1:172.21.15.162:6789/0 conn(0x7fecb415d250 legacy=0x7fecb40cea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:56.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.027+0000 7fecb88e9640 1 -- 172.21.15.46:0/3111355818 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb40ce530 con 0x7fecb40cfc30 2023-12-02T00:29:56.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.027+0000 7fecb37fe640 1 --2- 172.21.15.46:0/3111355818 >> v2:172.21.15.162:3300/0 conn(0x7fecb40d3030 0x7fecb40d3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:56.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.028+0000 7fecb88e9640 1 --2- 172.21.15.46:0/3111355818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecb40cfc30 0x7fecb40d0060 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fecb408bde0 tx=0x7fecac02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d29a33f1c92b4278 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:56.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.028+0000 7fecb2ffd640 1 -- 172.21.15.46:0/3111355818 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecac02cb20 con 0x7fecb40cfc30 2023-12-02T00:29:56.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.028+0000 7fecb2ffd640 1 -- 172.21.15.46:0/3111355818 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecac02cc80 con 0x7fecb40cfc30 2023-12-02T00:29:56.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.028+0000 7fecb2ffd640 1 -- 172.21.15.46:0/3111355818 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecac0343f0 con 0x7fecb40cfc30 2023-12-02T00:29:56.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.028+0000 7fecba34c640 1 -- 172.21.15.46:0/3111355818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecb40cfc30 msgr2=0x7fecb40d0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:56.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.028+0000 7fecba34c640 1 --2- 172.21.15.46:0/3111355818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecb40cfc30 0x7fecb40d0060 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fecb408bde0 tx=0x7fecac02bf80 comp rx=0 tx=0).stop 2023-12-02T00:29:56.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.028+0000 7fecba34c640 1 -- 172.21.15.46:0/3111355818 shutdown_connections 2023-12-02T00:29:56.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.028+0000 7fecba34c640 1 --2- 172.21.15.46:0/3111355818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecb40cfc30 0x7fecb40d0060 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:56.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.028+0000 7fecba34c640 1 --2- 172.21.15.46:0/3111355818 >> v2:172.21.15.162:3300/0 conn(0x7fecb40d3030 0x7fecb40d3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:56.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.028+0000 7fecba34c640 1 -- 172.21.15.46:0/3111355818 >> 172.21.15.46:0/3111355818 conn(0x7fecb408b000 msgr2=0x7fecb409c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:56.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.028+0000 7fecba34c640 1 -- 172.21.15.46:0/3111355818 shutdown_connections 2023-12-02T00:29:56.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.028+0000 7fecba34c640 1 -- 172.21.15.46:0/3111355818 wait complete. 2023-12-02T00:29:56.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.029+0000 7fecba34c640 1 Processor -- start 2023-12-02T00:29:56.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.030+0000 7fecba34c640 1 -- start start 2023-12-02T00:29:56.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.030+0000 7fecba34c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fecb40cfc30 0x7fecb41f5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:56.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.030+0000 7fecba34c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fecb40d3030 0x7fecb40b8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:56.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.030+0000 7fecba34c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecb415d250 0x7fecb415b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:56.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.030+0000 7fecba34c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb40ca480 con 0x7fecb415d250 2023-12-02T00:29:56.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.030+0000 7fecba34c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb41f6cb0 con 0x7fecb40d3030 2023-12-02T00:29:56.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.030+0000 7fecba34c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecb41f6410 con 0x7fecb40cfc30 2023-12-02T00:29:56.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.031+0000 7fecb88e9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecb415d250 0x7fecb415b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:56.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.031+0000 7fecb88e9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecb415d250 0x7fecb415b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50654/0 (socket says 172.21.15.46:50654) 2023-12-02T00:29:56.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.031+0000 7fecb88e9640 1 -- 172.21.15.46:0/73108738 learned_addr learned my addr 172.21.15.46:0/73108738 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:56.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.031+0000 7fecb37fe640 1 --2- 172.21.15.46:0/73108738 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fecb40d3030 0x7fecb40b8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:56.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.031+0000 7fecb3fff640 1 --2- 172.21.15.46:0/73108738 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fecb40cfc30 0x7fecb41f5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:56.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.031+0000 7fecb88e9640 1 -- 172.21.15.46:0/73108738 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fecb40cfc30 msgr2=0x7fecb41f5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:56.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.031+0000 7fecb88e9640 1 --2- 172.21.15.46:0/73108738 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fecb40cfc30 0x7fecb41f5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:56.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.031+0000 7fecb88e9640 1 -- 172.21.15.46:0/73108738 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fecb40d3030 msgr2=0x7fecb40b8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:56.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.031+0000 7fecb88e9640 1 --2- 172.21.15.46:0/73108738 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fecb40d3030 0x7fecb40b8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:56.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.031+0000 7fecb88e9640 1 -- 172.21.15.46:0/73108738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecac009b30 con 0x7fecb415d250 2023-12-02T00:29:56.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.031+0000 7fecb88e9640 1 --2- 172.21.15.46:0/73108738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecb415d250 0x7fecb415b130 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fecac036040 tx=0x7fecac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:56.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.031+0000 7fecb0ff9640 1 -- 172.21.15.46:0/73108738 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecac007e70 con 0x7fecb415d250 2023-12-02T00:29:56.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.031+0000 7fecb0ff9640 1 -- 172.21.15.46:0/73108738 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecac009510 con 0x7fecb415d250 2023-12-02T00:29:56.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.031+0000 7fecba34c640 1 -- 172.21.15.46:0/73108738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecb41f66f0 con 0x7fecb415d250 2023-12-02T00:29:56.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.031+0000 7fecb0ff9640 1 -- 172.21.15.46:0/73108738 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecac03e690 con 0x7fecb415d250 2023-12-02T00:29:56.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.031+0000 7fecba34c640 1 -- 172.21.15.46:0/73108738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecb40b9050 con 0x7fecb415d250 2023-12-02T00:29:56.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.032+0000 7fecba34c640 1 -- 172.21.15.46:0/73108738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecb4069400 con 0x7fecb415d250 2023-12-02T00:29:56.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.033+0000 7fecb0ff9640 1 -- 172.21.15.46:0/73108738 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fecac03e7f0 con 0x7fecb415d250 2023-12-02T00:29:56.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.033+0000 7fecb0ff9640 1 --2- 172.21.15.46:0/73108738 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7feca006ae40 0x7feca006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:56.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.033+0000 7fecb0ff9640 1 -- 172.21.15.46:0/73108738 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fecac0abdf0 con 0x7fecb415d250 2023-12-02T00:29:56.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.033+0000 7fecb3fff640 1 --2- 172.21.15.46:0/73108738 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7feca006ae40 0x7feca006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:56.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.034+0000 7fecb3fff640 1 --2- 172.21.15.46:0/73108738 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7feca006ae40 0x7feca006d300 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7feca8000f60 tx=0x7feca8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:56.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.038+0000 7fecb0ff9640 1 -- 172.21.15.46:0/73108738 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fecac0772e0 con 0x7fecb415d250 2023-12-02T00:29:56.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.214+0000 7fecba34c640 1 -- 172.21.15.46:0/73108738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fecb4073760 con 0x7fecb415d250 2023-12-02T00:29:56.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.900+0000 7fecb0ff9640 1 -- 172.21.15.46:0/73108738 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v33) v1 ==== 170+0+0 (secure 0 0 0) 0x7fecac07a900 con 0x7fecb415d250 2023-12-02T00:29:56.902 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2023-12-02T00:29:56.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.906+0000 7fecba34c640 1 -- 172.21.15.46:0/73108738 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7feca006ae40 msgr2=0x7feca006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:56.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.906+0000 7fecba34c640 1 --2- 172.21.15.46:0/73108738 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7feca006ae40 0x7feca006d300 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7feca8000f60 tx=0x7feca8009040 comp rx=0 tx=0).stop 2023-12-02T00:29:56.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.906+0000 7fecba34c640 1 -- 172.21.15.46:0/73108738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecb415d250 msgr2=0x7fecb415b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:56.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.906+0000 7fecba34c640 1 --2- 172.21.15.46:0/73108738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecb415d250 0x7fecb415b130 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fecac036040 tx=0x7fecac004a40 comp rx=0 tx=0).stop 2023-12-02T00:29:56.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.908+0000 7fecba34c640 1 -- 172.21.15.46:0/73108738 shutdown_connections 2023-12-02T00:29:56.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.908+0000 7fecba34c640 1 --2- 172.21.15.46:0/73108738 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7feca006ae40 0x7feca006d300 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:56.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.908+0000 7fecba34c640 1 --2- 172.21.15.46:0/73108738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecb415d250 0x7fecb415b130 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:56.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.908+0000 7fecba34c640 1 --2- 172.21.15.46:0/73108738 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fecb40d3030 0x7fecb40b8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:56.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.908+0000 7fecba34c640 1 --2- 172.21.15.46:0/73108738 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fecb40cfc30 0x7fecb41f5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:56.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.908+0000 7fecba34c640 1 -- 172.21.15.46:0/73108738 >> 172.21.15.46:0/73108738 conn(0x7fecb408b000 msgr2=0x7fecb41601b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:56.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.913+0000 7fecba34c640 1 -- 172.21.15.46:0/73108738 shutdown_connections 2023-12-02T00:29:56.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:56.913+0000 7fecba34c640 1 -- 172.21.15.46:0/73108738 wait complete. 2023-12-02T00:29:56.935 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-02T00:29:57.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.047+0000 7f15771b7640 1 Processor -- start 2023-12-02T00:29:57.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.047+0000 7f15771b7640 1 -- start start 2023-12-02T00:29:57.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.048+0000 7f15771b7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f15700b5370 0x7f15700cda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:57.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.048+0000 7f15771b7640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f15700d2040 0x7f15700cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:57.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.048+0000 7f15771b7640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15700ce4c0 con 0x7f157015d2d0 2023-12-02T00:29:57.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.048+0000 7f15771b7640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15700d1bb0 con 0x7f15700d2040 2023-12-02T00:29:57.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.048+0000 7f15771b7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f157015d6b0 con 0x7f15700b5370 2023-12-02T00:29:57.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.048+0000 7f1574f53640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f157015d2d0 0x7f15700b4c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57088/0 (socket says 172.21.15.46:57088) 2023-12-02T00:29:57.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.048+0000 7f1574f53640 1 -- 172.21.15.46:0/3225471940 learned_addr learned my addr 172.21.15.46:0/3225471940 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:57.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.048+0000 7f1575754640 1 --2- 172.21.15.46:0/3225471940 >> v2:172.21.15.162:3300/0 conn(0x7f15700d2040 0x7f15700cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:57.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.049+0000 7f156f7fe640 1 -- 172.21.15.46:0/3225471940 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2164684221 0 0) 0x7f1568001110 con 0x7f157015d2d0 2023-12-02T00:29:57.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.049+0000 7f156ffff640 1 --2- 172.21.15.46:0/3225471940 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f15700b5370 0x7f15700cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:57.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.049+0000 7f156f7fe640 1 -- 172.21.15.46:0/3225471940 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f155c003520 con 0x7f157015d2d0 2023-12-02T00:29:57.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.049+0000 7f156f7fe640 1 -- 172.21.15.46:0/3225471940 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 300388474 0 0) 0x7f155c003520 con 0x7f157015d2d0 2023-12-02T00:29:57.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.049+0000 7f156f7fe640 1 -- 172.21.15.46:0/3225471940 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f15700b5370 msgr2=0x7f15700cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:57.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.049+0000 7f156f7fe640 1 --2- 172.21.15.46:0/3225471940 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f15700b5370 0x7f15700cda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:57.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.049+0000 7f156f7fe640 1 -- 172.21.15.46:0/3225471940 >> v2:172.21.15.162:3300/0 conn(0x7f15700d2040 msgr2=0x7f15700cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:57.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.049+0000 7f156f7fe640 1 --2- 172.21.15.46:0/3225471940 >> v2:172.21.15.162:3300/0 conn(0x7f15700d2040 0x7f15700cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:57.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.049+0000 7f156f7fe640 1 -- 172.21.15.46:0/3225471940 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15700ce700 con 0x7f157015d2d0 2023-12-02T00:29:57.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.049+0000 7f156f7fe640 1 -- 172.21.15.46:0/3225471940 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7f1568001800 con 0x7f157015d2d0 2023-12-02T00:29:57.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.049+0000 7f156f7fe640 1 -- 172.21.15.46:0/3225471940 >> v1:172.21.15.46:6789/0 conn(0x7f157015d2d0 legacy=0x7f15700b4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:57.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.049+0000 7f156f7fe640 1 --2- 172.21.15.46:0/3225471940 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f15700b5370 0x7f155c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:57.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.049+0000 7f156f7fe640 1 --2- 172.21.15.46:0/3225471940 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f155c003ef0 0x7f155c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:57.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.049+0000 7f156f7fe640 1 --2- 172.21.15.46:0/3225471940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f155c004d00 0x7f155c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:57.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.049+0000 7f156f7fe640 1 -- 172.21.15.46:0/3225471940 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15700d1bb0 con 0x7f155c004d00 2023-12-02T00:29:57.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.049+0000 7f156f7fe640 1 -- 172.21.15.46:0/3225471940 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157015d6b0 con 0x7f155c003ef0 2023-12-02T00:29:57.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.049+0000 7f156f7fe640 1 -- 172.21.15.46:0/3225471940 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f155c00b980 con 0x7f15700b5370 2023-12-02T00:29:57.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.049+0000 7f1575754640 1 --2- 172.21.15.46:0/3225471940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f155c004d00 0x7f155c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:57.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.049+0000 7f1574f53640 1 --2- 172.21.15.46:0/3225471940 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f15700b5370 0x7f155c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:57.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.049+0000 7f156ffff640 1 --2- 172.21.15.46:0/3225471940 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f155c003ef0 0x7f155c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:57.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.049+0000 7f156ffff640 1 -- 172.21.15.46:0/3225471940 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f15700b5370 msgr2=0x7f155c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:57.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.049+0000 7f156ffff640 1 --2- 172.21.15.46:0/3225471940 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f15700b5370 0x7f155c003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:57.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.049+0000 7f156ffff640 1 -- 172.21.15.46:0/3225471940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f155c004d00 msgr2=0x7f155c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:57.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.049+0000 7f156ffff640 1 --2- 172.21.15.46:0/3225471940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f155c004d00 0x7f155c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:57.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.049+0000 7f156ffff640 1 -- 172.21.15.46:0/3225471940 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f155c003520 con 0x7f155c003ef0 2023-12-02T00:29:57.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.049+0000 7f1574f53640 1 --2- 172.21.15.46:0/3225471940 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f15700b5370 0x7f155c003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:29:57.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.050+0000 7f156ffff640 1 --2- 172.21.15.46:0/3225471940 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f155c003ef0 0x7f155c0047c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1560009fc0 tx=0x7f155c0057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89626574105e07ff server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:57.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.050+0000 7f156f7fe640 1 -- 172.21.15.46:0/3225471940 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156002cc00 con 0x7f155c003ef0 2023-12-02T00:29:57.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.050+0000 7f156f7fe640 1 -- 172.21.15.46:0/3225471940 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f156000f040 con 0x7f155c003ef0 2023-12-02T00:29:57.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.051+0000 7f156f7fe640 1 -- 172.21.15.46:0/3225471940 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1560034440 con 0x7f155c003ef0 2023-12-02T00:29:57.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.051+0000 7f15771b7640 1 -- 172.21.15.46:0/3225471940 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f155c003ef0 msgr2=0x7f155c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:57.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.051+0000 7f15771b7640 1 --2- 172.21.15.46:0/3225471940 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f155c003ef0 0x7f155c0047c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1560009fc0 tx=0x7f155c0057f0 comp rx=0 tx=0).stop 2023-12-02T00:29:57.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.051+0000 7f1575754640 1 -- 172.21.15.46:0/3225471940 reap_dead start 2023-12-02T00:29:57.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.051+0000 7f15771b7640 1 -- 172.21.15.46:0/3225471940 shutdown_connections 2023-12-02T00:29:57.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.051+0000 7f15771b7640 1 -- 172.21.15.46:0/3225471940 >> 172.21.15.46:0/3225471940 conn(0x7f1570072050 msgr2=0x7f1570072460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:57.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.051+0000 7f15771b7640 1 -- 172.21.15.46:0/3225471940 shutdown_connections 2023-12-02T00:29:57.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.051+0000 7f15771b7640 1 -- 172.21.15.46:0/3225471940 wait complete. 2023-12-02T00:29:57.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.052+0000 7f15771b7640 1 Processor -- start 2023-12-02T00:29:57.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.052+0000 7f15771b7640 1 -- start start 2023-12-02T00:29:57.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.053+0000 7f15771b7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f155c003ef0 0x7f157015c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:57.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.053+0000 7f15771b7640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15701e93d0 0x7f157015b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:57.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.053+0000 7f15771b7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f15700cc480 0x7f15700cc930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:57.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.053+0000 7f15771b7640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15700c6e90 con 0x7f155c003ef0 2023-12-02T00:29:57.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.053+0000 7f15771b7640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15700cce70 con 0x7f15701e93d0 2023-12-02T00:29:57.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.053+0000 7f15771b7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f157015c860 con 0x7f15700cc480 2023-12-02T00:29:57.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.053+0000 7f1574f53640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f155c003ef0 0x7f157015c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:57.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.053+0000 7f1574f53640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f155c003ef0 0x7f157015c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50668/0 (socket says 172.21.15.46:50668) 2023-12-02T00:29:57.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.053+0000 7f1574f53640 1 -- 172.21.15.46:0/3073184794 learned_addr learned my addr 172.21.15.46:0/3073184794 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:57.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.053+0000 7f156ffff640 1 --2- 172.21.15.46:0/3073184794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15701e93d0 0x7f157015b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:57.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.054+0000 7f1575754640 1 --2- 172.21.15.46:0/3073184794 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f15700cc480 0x7f15700cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:57.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.054+0000 7f1574f53640 1 -- 172.21.15.46:0/3073184794 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f15700cc480 msgr2=0x7f15700cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:57.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.054+0000 7f1574f53640 1 --2- 172.21.15.46:0/3073184794 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f15700cc480 0x7f15700cc930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:57.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.054+0000 7f1574f53640 1 -- 172.21.15.46:0/3073184794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15701e93d0 msgr2=0x7f157015b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:57.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.054+0000 7f1574f53640 1 --2- 172.21.15.46:0/3073184794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15701e93d0 0x7f157015b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:57.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.054+0000 7f1574f53640 1 -- 172.21.15.46:0/3073184794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15701605f0 con 0x7f155c003ef0 2023-12-02T00:29:57.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.054+0000 7f1574f53640 1 --2- 172.21.15.46:0/3073184794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f155c003ef0 0x7f157015c320 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1568007370 tx=0x7f1568007830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:57.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.054+0000 7f156d7fa640 1 -- 172.21.15.46:0/3073184794 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156800f9c0 con 0x7f155c003ef0 2023-12-02T00:29:57.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.054+0000 7f156d7fa640 1 -- 172.21.15.46:0/3073184794 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f156800fb20 con 0x7f155c003ef0 2023-12-02T00:29:57.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.054+0000 7f15771b7640 1 -- 172.21.15.46:0/3073184794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1570160030 con 0x7f155c003ef0 2023-12-02T00:29:57.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.054+0000 7f156d7fa640 1 -- 172.21.15.46:0/3073184794 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156800feb0 con 0x7f155c003ef0 2023-12-02T00:29:57.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.054+0000 7f15771b7640 1 -- 172.21.15.46:0/3073184794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f157015bfb0 con 0x7f155c003ef0 2023-12-02T00:29:57.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.055+0000 7f15771b7640 1 -- 172.21.15.46:0/3073184794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15700c6fd0 con 0x7f155c003ef0 2023-12-02T00:29:57.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.056+0000 7f156d7fa640 1 -- 172.21.15.46:0/3073184794 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f156801d460 con 0x7f155c003ef0 2023-12-02T00:29:57.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.056+0000 7f156d7fa640 1 --2- 172.21.15.46:0/3073184794 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f155006ae40 0x7f155006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:57.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.056+0000 7f156d7fa640 1 -- 172.21.15.46:0/3073184794 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f1568089fe0 con 0x7f155c003ef0 2023-12-02T00:29:57.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.056+0000 7f156ffff640 1 --2- 172.21.15.46:0/3073184794 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f155006ae40 0x7f155006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:57.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.057+0000 7f156ffff640 1 --2- 172.21.15.46:0/3073184794 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f155006ae40 0x7f155006d300 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1560036000 tx=0x7f156002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:57.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.060+0000 7f156d7fa640 1 -- 172.21.15.46:0/3073184794 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1568055320 con 0x7f155c003ef0 2023-12-02T00:29:57.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.277+0000 7f15771b7640 1 -- 172.21.15.46:0/3073184794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f15700c6fd0 con 0x7f155c003ef0 2023-12-02T00:29:57.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.902+0000 7f156d7fa640 1 -- 172.21.15.46:0/3073184794 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v11) v1 ==== 172+0+0 (secure 0 0 0) 0x7f15700c6fd0 con 0x7f155c003ef0 2023-12-02T00:29:57.903 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 4 and data pool 5 2023-12-02T00:29:57.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.906+0000 7f15771b7640 1 -- 172.21.15.46:0/3073184794 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f155006ae40 msgr2=0x7f155006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.906+0000 7f15771b7640 1 --2- 172.21.15.46:0/3073184794 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f155006ae40 0x7f155006d300 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1560036000 tx=0x7f156002e000 comp rx=0 tx=0).stop 2023-12-02T00:29:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.906+0000 7f15771b7640 1 -- 172.21.15.46:0/3073184794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f155c003ef0 msgr2=0x7f157015c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.906+0000 7f15771b7640 1 --2- 172.21.15.46:0/3073184794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f155c003ef0 0x7f157015c320 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1568007370 tx=0x7f1568007830 comp rx=0 tx=0).stop 2023-12-02T00:29:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.906+0000 7f15771b7640 1 -- 172.21.15.46:0/3073184794 shutdown_connections 2023-12-02T00:29:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.906+0000 7f15771b7640 1 --2- 172.21.15.46:0/3073184794 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f155006ae40 0x7f155006d300 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.906+0000 7f15771b7640 1 --2- 172.21.15.46:0/3073184794 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f15700cc480 0x7f15700cc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.906+0000 7f15771b7640 1 --2- 172.21.15.46:0/3073184794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15701e93d0 0x7f157015b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.906+0000 7f15771b7640 1 --2- 172.21.15.46:0/3073184794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f155c003ef0 0x7f157015c320 secure :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1568007370 tx=0x7f1568007830 comp rx=0 tx=0).stop 2023-12-02T00:29:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.906+0000 7f15771b7640 1 -- 172.21.15.46:0/3073184794 >> 172.21.15.46:0/3073184794 conn(0x7f1570072050 msgr2=0x7f15700d2500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:57.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.908+0000 7f15771b7640 1 -- 172.21.15.46:0/3073184794 shutdown_connections 2023-12-02T00:29:57.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:57.908+0000 7f15771b7640 1 -- 172.21.15.46:0/3073184794 wait complete. 2023-12-02T00:29:57.938 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:29:58.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.049+0000 7fa6cb9b7640 1 Processor -- start 2023-12-02T00:29:58.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.049+0000 7fa6cb9b7640 1 -- start start 2023-12-02T00:29:58.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.050+0000 7fa6cb9b7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6c4160fb0 0x7fa6c4161440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:58.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.050+0000 7fa6cb9b7640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa6c40d0ed0 0x7fa6c40d1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:58.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.050+0000 7fa6cb9b7640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6c4161d20 con 0x7fa6c406ea50 2023-12-02T00:29:58.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.050+0000 7fa6cb9b7640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6c415e760 con 0x7fa6c40d0ed0 2023-12-02T00:29:58.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.050+0000 7fa6cb9b7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c415d9a0 con 0x7fa6c4160fb0 2023-12-02T00:29:58.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.050+0000 7fa6c9753640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa6c406ea50 0x7fa6c406ee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57100/0 (socket says 172.21.15.46:57100) 2023-12-02T00:29:58.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.050+0000 7fa6c9753640 1 -- 172.21.15.46:0/479911474 learned_addr learned my addr 172.21.15.46:0/479911474 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:29:58.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.050+0000 7fa6c9f54640 1 --2- 172.21.15.46:0/479911474 >> v2:172.21.15.162:3300/0 conn(0x7fa6c40d0ed0 0x7fa6c40d1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:58.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.050+0000 7fa6c3fff640 1 -- 172.21.15.46:0/479911474 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2306604692 0 0) 0x7fa6c4161d20 con 0x7fa6c406ea50 2023-12-02T00:29:58.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.051+0000 7fa6c8f52640 1 --2- 172.21.15.46:0/479911474 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6c4160fb0 0x7fa6c4161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:58.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.051+0000 7fa6c3fff640 1 -- 172.21.15.46:0/479911474 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6ac003520 con 0x7fa6c406ea50 2023-12-02T00:29:58.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.051+0000 7fa6c3fff640 1 -- 172.21.15.46:0/479911474 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 372152189 0 0) 0x7fa6ac003520 con 0x7fa6c406ea50 2023-12-02T00:29:58.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.051+0000 7fa6c3fff640 1 -- 172.21.15.46:0/479911474 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6c4160fb0 msgr2=0x7fa6c4161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:58.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.051+0000 7fa6c3fff640 1 --2- 172.21.15.46:0/479911474 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6c4160fb0 0x7fa6c4161440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:58.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.051+0000 7fa6c3fff640 1 -- 172.21.15.46:0/479911474 >> v2:172.21.15.162:3300/0 conn(0x7fa6c40d0ed0 msgr2=0x7fa6c40d1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:58.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.051+0000 7fa6c3fff640 1 --2- 172.21.15.46:0/479911474 >> v2:172.21.15.162:3300/0 conn(0x7fa6c40d0ed0 0x7fa6c40d1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:58.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.051+0000 7fa6c3fff640 1 -- 172.21.15.46:0/479911474 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c415dc80 con 0x7fa6c406ea50 2023-12-02T00:29:58.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.051+0000 7fa6c3fff640 1 -- 172.21.15.46:0/479911474 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7fa6b40019f0 con 0x7fa6c406ea50 2023-12-02T00:29:58.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.051+0000 7fa6c3fff640 1 -- 172.21.15.46:0/479911474 >> v1:172.21.15.46:6789/0 conn(0x7fa6c406ea50 legacy=0x7fa6c406ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:58.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.051+0000 7fa6c3fff640 1 --2- 172.21.15.46:0/479911474 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6ac003970 0x7fa6ac003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:58.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.051+0000 7fa6c3fff640 1 --2- 172.21.15.46:0/479911474 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6c4160fb0 0x7fa6ac0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:58.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.051+0000 7fa6c3fff640 1 --2- 172.21.15.46:0/479911474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6ac004d00 0x7fa6ac005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:58.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.051+0000 7fa6c3fff640 1 -- 172.21.15.46:0/479911474 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c415e760 con 0x7fa6ac004d00 2023-12-02T00:29:58.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.051+0000 7fa6c3fff640 1 -- 172.21.15.46:0/479911474 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c415d9a0 con 0x7fa6ac003970 2023-12-02T00:29:58.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.051+0000 7fa6c3fff640 1 -- 172.21.15.46:0/479911474 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ac00b980 con 0x7fa6c4160fb0 2023-12-02T00:29:58.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.051+0000 7fa6c9f54640 1 --2- 172.21.15.46:0/479911474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6ac004d00 0x7fa6ac005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:58.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.051+0000 7fa6c9753640 1 --2- 172.21.15.46:0/479911474 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6ac003970 0x7fa6ac003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:58.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.051+0000 7fa6c8f52640 1 --2- 172.21.15.46:0/479911474 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6c4160fb0 0x7fa6ac0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:58.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.051+0000 7fa6c9f54640 1 -- 172.21.15.46:0/479911474 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6c4160fb0 msgr2=0x7fa6ac0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:58.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.051+0000 7fa6c9f54640 1 --2- 172.21.15.46:0/479911474 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6c4160fb0 0x7fa6ac0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:58.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.051+0000 7fa6c9f54640 1 -- 172.21.15.46:0/479911474 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6ac003970 msgr2=0x7fa6ac003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:58.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.051+0000 7fa6c9f54640 1 --2- 172.21.15.46:0/479911474 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6ac003970 0x7fa6ac003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:58.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.051+0000 7fa6c9f54640 1 -- 172.21.15.46:0/479911474 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ac003520 con 0x7fa6ac004d00 2023-12-02T00:29:58.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.051+0000 7fa6c9753640 1 --2- 172.21.15.46:0/479911474 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6ac003970 0x7fa6ac003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:29:58.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.052+0000 7fa6c9f54640 1 --2- 172.21.15.46:0/479911474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6ac004d00 0x7fa6ac005180 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc009af0 tx=0x7fa6ac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8424d8778d6e0c22 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:58.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.052+0000 7fa6c3fff640 1 -- 172.21.15.46:0/479911474 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc02c710 con 0x7fa6ac004d00 2023-12-02T00:29:58.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.052+0000 7fa6c3fff640 1 -- 172.21.15.46:0/479911474 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6bc02c870 con 0x7fa6ac004d00 2023-12-02T00:29:58.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.052+0000 7fa6c3fff640 1 -- 172.21.15.46:0/479911474 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc02cc00 con 0x7fa6ac004d00 2023-12-02T00:29:58.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.053+0000 7fa6cb9b7640 1 -- 172.21.15.46:0/479911474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6ac004d00 msgr2=0x7fa6ac005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:58.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.053+0000 7fa6cb9b7640 1 --2- 172.21.15.46:0/479911474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6ac004d00 0x7fa6ac005180 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc009af0 tx=0x7fa6ac0057f0 comp rx=0 tx=0).stop 2023-12-02T00:29:58.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.053+0000 7fa6c9f54640 1 -- 172.21.15.46:0/479911474 reap_dead start 2023-12-02T00:29:58.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.053+0000 7fa6cb9b7640 1 -- 172.21.15.46:0/479911474 shutdown_connections 2023-12-02T00:29:58.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.053+0000 7fa6cb9b7640 1 -- 172.21.15.46:0/479911474 >> 172.21.15.46:0/479911474 conn(0x7fa6c406a6b0 msgr2=0x7fa6c406aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:58.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.053+0000 7fa6cb9b7640 1 -- 172.21.15.46:0/479911474 shutdown_connections 2023-12-02T00:29:58.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.053+0000 7fa6cb9b7640 1 -- 172.21.15.46:0/479911474 wait complete. 2023-12-02T00:29:58.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.054+0000 7fa6cb9b7640 1 Processor -- start 2023-12-02T00:29:58.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.054+0000 7fa6cb9b7640 1 -- start start 2023-12-02T00:29:58.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.055+0000 7fa6cb9b7640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6c41494b0 0x7fa6c4159a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:58.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.055+0000 7fa6cb9b7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6c4149b50 0x7fa6c415fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:58.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.055+0000 7fa6cb9b7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6c4160290 0x7fa6c413d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:58.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.055+0000 7fa6cb9b7640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c4160710 con 0x7fa6c4160290 2023-12-02T00:29:58.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.055+0000 7fa6cb9b7640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c41e6a30 con 0x7fa6c41494b0 2023-12-02T00:29:58.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.055+0000 7fa6cb9b7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c4156e50 con 0x7fa6c4149b50 2023-12-02T00:29:58.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.055+0000 7fa6c9f54640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6c4160290 0x7fa6c413d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:58.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.055+0000 7fa6c9f54640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6c4160290 0x7fa6c413d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50682/0 (socket says 172.21.15.46:50682) 2023-12-02T00:29:58.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.055+0000 7fa6c9f54640 1 -- 172.21.15.46:0/3061565792 learned_addr learned my addr 172.21.15.46:0/3061565792 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:58.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.055+0000 7fa6c9753640 1 --2- 172.21.15.46:0/3061565792 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6c41494b0 0x7fa6c4159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:58.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.055+0000 7fa6c8f52640 1 --2- 172.21.15.46:0/3061565792 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6c4149b50 0x7fa6c415fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:58.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.055+0000 7fa6c9f54640 1 -- 172.21.15.46:0/3061565792 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6c4149b50 msgr2=0x7fa6c415fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:58.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.055+0000 7fa6c9f54640 1 --2- 172.21.15.46:0/3061565792 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6c4149b50 0x7fa6c415fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:58.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.055+0000 7fa6c9f54640 1 -- 172.21.15.46:0/3061565792 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6c41494b0 msgr2=0x7fa6c4159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:58.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.055+0000 7fa6c9f54640 1 --2- 172.21.15.46:0/3061565792 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6c41494b0 0x7fa6c4159a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:58.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.055+0000 7fa6c9f54640 1 -- 172.21.15.46:0/3061565792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c415a100 con 0x7fa6c4160290 2023-12-02T00:29:58.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.056+0000 7fa6c9f54640 1 --2- 172.21.15.46:0/3061565792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6c4160290 0x7fa6c413d870 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc009af0 tx=0x7fa6bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:58.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.056+0000 7fa6c1ffb640 1 -- 172.21.15.46:0/3061565792 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc02c710 con 0x7fa6c4160290 2023-12-02T00:29:58.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.056+0000 7fa6c1ffb640 1 -- 172.21.15.46:0/3061565792 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6bc003830 con 0x7fa6c4160290 2023-12-02T00:29:58.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.056+0000 7fa6cb9b7640 1 -- 172.21.15.46:0/3061565792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6c415a3e0 con 0x7fa6c4160290 2023-12-02T00:29:58.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.056+0000 7fa6c1ffb640 1 -- 172.21.15.46:0/3061565792 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc003be0 con 0x7fa6c4160290 2023-12-02T00:29:58.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.056+0000 7fa6cb9b7640 1 -- 172.21.15.46:0/3061565792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6c41e6c80 con 0x7fa6c4160290 2023-12-02T00:29:58.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.057+0000 7fa6cb9b7640 1 -- 172.21.15.46:0/3061565792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6c4069400 con 0x7fa6c4160290 2023-12-02T00:29:58.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.057+0000 7fa6c1ffb640 1 -- 172.21.15.46:0/3061565792 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa6bc003d40 con 0x7fa6c4160290 2023-12-02T00:29:58.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.058+0000 7fa6c1ffb640 1 --2- 172.21.15.46:0/3061565792 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa6b006ae40 0x7fa6b006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:58.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.058+0000 7fa6c1ffb640 1 -- 172.21.15.46:0/3061565792 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa6bc0aad40 con 0x7fa6c4160290 2023-12-02T00:29:58.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.058+0000 7fa6c9753640 1 --2- 172.21.15.46:0/3061565792 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa6b006ae40 0x7fa6b006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:58.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.058+0000 7fa6c9753640 1 --2- 172.21.15.46:0/3061565792 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa6b006ae40 0x7fa6b006d300 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa6b40013d0 tx=0x7fa6b4006040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:58.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.062+0000 7fa6c1ffb640 1 -- 172.21.15.46:0/3061565792 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa6bc075ff0 con 0x7fa6c4160290 2023-12-02T00:29:58.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.237+0000 7fa6cb9b7640 1 -- 172.21.15.46:0/3061565792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa6c4070d30 con 0x7fa6c4160290 2023-12-02T00:29:58.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.238+0000 7fa6c1ffb640 1 -- 172.21.15.46:0/3061565792 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16558 (secure 0 0 0) 0x7fa6bc042090 con 0x7fa6c4160290 2023-12-02T00:29:58.240 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:29:58.240 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":34,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:29:57.278843+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-02T00:29:55.676572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-02T00:29:56.244869+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:29:58.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.240+0000 7fa6cb9b7640 1 -- 172.21.15.46:0/3061565792 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa6b006ae40 msgr2=0x7fa6b006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:58.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.240+0000 7fa6cb9b7640 1 --2- 172.21.15.46:0/3061565792 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa6b006ae40 0x7fa6b006d300 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa6b40013d0 tx=0x7fa6b4006040 comp rx=0 tx=0).stop 2023-12-02T00:29:58.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.240+0000 7fa6cb9b7640 1 -- 172.21.15.46:0/3061565792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6c4160290 msgr2=0x7fa6c413d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:58.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.240+0000 7fa6cb9b7640 1 --2- 172.21.15.46:0/3061565792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6c4160290 0x7fa6c413d870 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc009af0 tx=0x7fa6bc004730 comp rx=0 tx=0).stop 2023-12-02T00:29:58.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.240+0000 7fa6cb9b7640 1 -- 172.21.15.46:0/3061565792 shutdown_connections 2023-12-02T00:29:58.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.240+0000 7fa6cb9b7640 1 --2- 172.21.15.46:0/3061565792 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa6b006ae40 0x7fa6b006d300 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:58.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.240+0000 7fa6cb9b7640 1 --2- 172.21.15.46:0/3061565792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6c4160290 0x7fa6c413d870 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:58.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.240+0000 7fa6cb9b7640 1 --2- 172.21.15.46:0/3061565792 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6c4149b50 0x7fa6c415fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:58.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.240+0000 7fa6cb9b7640 1 --2- 172.21.15.46:0/3061565792 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6c41494b0 0x7fa6c4159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:58.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.240+0000 7fa6cb9b7640 1 -- 172.21.15.46:0/3061565792 >> 172.21.15.46:0/3061565792 conn(0x7fa6c406a6b0 msgr2=0x7fa6c415c790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:58.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.240+0000 7fa6cb9b7640 1 -- 172.21.15.46:0/3061565792 shutdown_connections 2023-12-02T00:29:58.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.240+0000 7fa6cb9b7640 1 -- 172.21.15.46:0/3061565792 wait complete. 2023-12-02T00:29:58.253 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:29:58.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.358+0000 7fa38042b640 1 Processor -- start 2023-12-02T00:29:58.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.358+0000 7fa38042b640 1 -- start start 2023-12-02T00:29:58.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.359+0000 7fa38042b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa37815d330 0x7fa3780b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:58.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.359+0000 7fa38042b640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa3780d2910 0x7fa3780d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:58.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.359+0000 7fa38042b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa3780d2d90 0x7fa3780d3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:58.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.359+0000 7fa38042b640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3780b2b10 con 0x7fa3780d2d90 2023-12-02T00:29:58.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.359+0000 7fa38042b640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3780d1590 con 0x7fa3780d2910 2023-12-02T00:29:58.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.359+0000 7fa38042b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3780d1870 con 0x7fa37815d330 2023-12-02T00:29:58.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.359+0000 7fa37e9c8640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa3780d2d90 0x7fa3780d3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:58.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.359+0000 7fa37e9c8640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa3780d2d90 0x7fa3780d3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50696/0 (socket says 172.21.15.46:50696) 2023-12-02T00:29:58.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.359+0000 7fa37e9c8640 1 -- 172.21.15.46:0/2647885053 learned_addr learned my addr 172.21.15.46:0/2647885053 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:58.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.360+0000 7fa37d9c6640 1 --2- 172.21.15.46:0/2647885053 >> v2:172.21.15.162:3300/0 conn(0x7fa3780d2910 0x7fa3780d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:58.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.360+0000 7fa37e1c7640 1 --2- 172.21.15.46:0/2647885053 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa37815d330 0x7fa3780b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:58.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.360+0000 7fa37e9c8640 1 -- 172.21.15.46:0/2647885053 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa37815d330 msgr2=0x7fa3780b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:58.362 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.360+0000 7fa37e9c8640 1 --2- 172.21.15.46:0/2647885053 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa37815d330 0x7fa3780b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:58.362 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.360+0000 7fa37e9c8640 1 -- 172.21.15.46:0/2647885053 >> v2:172.21.15.162:3300/0 conn(0x7fa3780d2910 msgr2=0x7fa3780d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:29:58.362 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.360+0000 7fa37e9c8640 1 --2- 172.21.15.46:0/2647885053 >> v2:172.21.15.162:3300/0 conn(0x7fa3780d2910 0x7fa3780d31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:58.362 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.360+0000 7fa37e9c8640 1 -- 172.21.15.46:0/2647885053 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3780d1b50 con 0x7fa3780d2d90 2023-12-02T00:29:58.369 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.360+0000 7fa37e9c8640 1 --2- 172.21.15.46:0/2647885053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3780d2d90 0x7fa3780d3730 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa3780d4360 tx=0x7fa37002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3258007195aaec3f server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:58.369 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.361+0000 7fa37d1c5640 1 -- 172.21.15.46:0/2647885053 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37002c9e0 con 0x7fa3780d2d90 2023-12-02T00:29:58.369 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.361+0000 7fa37d1c5640 1 -- 172.21.15.46:0/2647885053 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa37002cb40 con 0x7fa3780d2d90 2023-12-02T00:29:58.369 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.361+0000 7fa37d1c5640 1 -- 172.21.15.46:0/2647885053 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3700343f0 con 0x7fa3780d2d90 2023-12-02T00:29:58.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.361+0000 7fa38042b640 1 -- 172.21.15.46:0/2647885053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3780d2d90 msgr2=0x7fa3780d3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:58.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.361+0000 7fa38042b640 1 --2- 172.21.15.46:0/2647885053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3780d2d90 0x7fa3780d3730 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa3780d4360 tx=0x7fa37002be40 comp rx=0 tx=0).stop 2023-12-02T00:29:58.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.361+0000 7fa38042b640 1 -- 172.21.15.46:0/2647885053 shutdown_connections 2023-12-02T00:29:58.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.361+0000 7fa38042b640 1 --2- 172.21.15.46:0/2647885053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3780d2d90 0x7fa3780d3730 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:58.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.361+0000 7fa38042b640 1 --2- 172.21.15.46:0/2647885053 >> v2:172.21.15.162:3300/0 conn(0x7fa3780d2910 0x7fa3780d31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:58.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.361+0000 7fa38042b640 1 --2- 172.21.15.46:0/2647885053 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa37815d330 0x7fa3780b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:58.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.361+0000 7fa38042b640 1 -- 172.21.15.46:0/2647885053 >> 172.21.15.46:0/2647885053 conn(0x7fa378072020 msgr2=0x7fa378072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:58.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.361+0000 7fa38042b640 1 -- 172.21.15.46:0/2647885053 shutdown_connections 2023-12-02T00:29:58.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.361+0000 7fa38042b640 1 -- 172.21.15.46:0/2647885053 wait complete. 2023-12-02T00:29:58.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.362+0000 7fa38042b640 1 Processor -- start 2023-12-02T00:29:58.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.362+0000 7fa38042b640 1 -- start start 2023-12-02T00:29:58.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.363+0000 7fa38042b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa3780d2910 0x7fa37815e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:58.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.363+0000 7fa38042b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3780d2d90 0x7fa3780b1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:58.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.363+0000 7fa38042b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa37815d330 0x7fa3780ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:58.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.363+0000 7fa38042b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37815fcc0 con 0x7fa3780d2d90 2023-12-02T00:29:58.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.363+0000 7fa38042b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3780cd420 con 0x7fa37815d330 2023-12-02T00:29:58.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.363+0000 7fa38042b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa37815f040 con 0x7fa3780d2910 2023-12-02T00:29:58.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.363+0000 7fa37d9c6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3780d2d90 0x7fa3780b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:58.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.363+0000 7fa37d9c6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3780d2d90 0x7fa3780b1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50704/0 (socket says 172.21.15.46:50704) 2023-12-02T00:29:58.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.363+0000 7fa37d9c6640 1 -- 172.21.15.46:0/3307583891 learned_addr learned my addr 172.21.15.46:0/3307583891 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:58.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.363+0000 7fa37e9c8640 1 --2- 172.21.15.46:0/3307583891 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa37815d330 0x7fa3780ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:58.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.363+0000 7fa37e1c7640 1 --2- 172.21.15.46:0/3307583891 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa3780d2910 0x7fa37815e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:58.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.364+0000 7fa37d9c6640 1 -- 172.21.15.46:0/3307583891 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa3780d2910 msgr2=0x7fa37815e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:58.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.364+0000 7fa37d9c6640 1 --2- 172.21.15.46:0/3307583891 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa3780d2910 0x7fa37815e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:58.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.364+0000 7fa37d9c6640 1 -- 172.21.15.46:0/3307583891 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa37815d330 msgr2=0x7fa3780ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:58.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.364+0000 7fa37d9c6640 1 --2- 172.21.15.46:0/3307583891 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa37815d330 0x7fa3780ccee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:58.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.364+0000 7fa37d9c6640 1 -- 172.21.15.46:0/3307583891 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa370009b30 con 0x7fa3780d2d90 2023-12-02T00:29:58.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.364+0000 7fa37e1c7640 1 --2- 172.21.15.46:0/3307583891 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa3780d2910 0x7fa37815e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:29:58.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.364+0000 7fa37d9c6640 1 --2- 172.21.15.46:0/3307583891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3780d2d90 0x7fa3780b1670 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa36800d800 tx=0x7fa36800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:58.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.364+0000 7fa36effd640 1 -- 172.21.15.46:0/3307583891 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3680096a0 con 0x7fa3780d2d90 2023-12-02T00:29:58.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.364+0000 7fa36effd640 1 -- 172.21.15.46:0/3307583891 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa368009800 con 0x7fa3780d2d90 2023-12-02T00:29:58.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.364+0000 7fa38042b640 1 -- 172.21.15.46:0/3307583891 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa37815f320 con 0x7fa3780d2d90 2023-12-02T00:29:58.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.364+0000 7fa36effd640 1 -- 172.21.15.46:0/3307583891 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa368009b90 con 0x7fa3780d2d90 2023-12-02T00:29:58.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.364+0000 7fa38042b640 1 -- 172.21.15.46:0/3307583891 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa37815f8e0 con 0x7fa3780d2d90 2023-12-02T00:29:58.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.365+0000 7fa38042b640 1 -- 172.21.15.46:0/3307583891 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3780d4080 con 0x7fa3780d2d90 2023-12-02T00:29:58.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.365+0000 7fa36effd640 1 -- 172.21.15.46:0/3307583891 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa368009cf0 con 0x7fa3780d2d90 2023-12-02T00:29:58.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.366+0000 7fa36effd640 1 --2- 172.21.15.46:0/3307583891 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa35c06ae70 0x7fa35c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:58.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.366+0000 7fa36effd640 1 -- 172.21.15.46:0/3307583891 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa368089b70 con 0x7fa3780d2d90 2023-12-02T00:29:58.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.366+0000 7fa37e1c7640 1 --2- 172.21.15.46:0/3307583891 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa35c06ae70 0x7fa35c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:58.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.367+0000 7fa37e1c7640 1 --2- 172.21.15.46:0/3307583891 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa35c06ae70 0x7fa35c06d330 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa374006fd0 tx=0x7fa374008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:58.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.370+0000 7fa36effd640 1 -- 172.21.15.46:0/3307583891 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa368054cb0 con 0x7fa3780d2d90 2023-12-02T00:29:58.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.545+0000 7fa38042b640 1 -- 172.21.15.46:0/3307583891 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa37815c840 con 0x7fa3780d2d90 2023-12-02T00:29:58.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.547+0000 7fa36effd640 1 -- 172.21.15.46:0/3307583891 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16558 (secure 0 0 0) 0x7fa36800b440 con 0x7fa3780d2d90 2023-12-02T00:29:58.548 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:29:58.548 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":34,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:29:57.278843+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-02T00:29:55.676572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-02T00:29:56.244869+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:29:58.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.549+0000 7fa38042b640 1 -- 172.21.15.46:0/3307583891 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa35c06ae70 msgr2=0x7fa35c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:58.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.549+0000 7fa38042b640 1 --2- 172.21.15.46:0/3307583891 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa35c06ae70 0x7fa35c06d330 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa374006fd0 tx=0x7fa374008040 comp rx=0 tx=0).stop 2023-12-02T00:29:58.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.549+0000 7fa38042b640 1 -- 172.21.15.46:0/3307583891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3780d2d90 msgr2=0x7fa3780b1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:58.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.549+0000 7fa38042b640 1 --2- 172.21.15.46:0/3307583891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3780d2d90 0x7fa3780b1670 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa36800d800 tx=0x7fa36800dcc0 comp rx=0 tx=0).stop 2023-12-02T00:29:58.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.549+0000 7fa38042b640 1 -- 172.21.15.46:0/3307583891 shutdown_connections 2023-12-02T00:29:58.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.549+0000 7fa38042b640 1 --2- 172.21.15.46:0/3307583891 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa35c06ae70 0x7fa35c06d330 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:58.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.549+0000 7fa38042b640 1 --2- 172.21.15.46:0/3307583891 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa37815d330 0x7fa3780ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:58.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.549+0000 7fa38042b640 1 --2- 172.21.15.46:0/3307583891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3780d2d90 0x7fa3780b1670 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:58.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.549+0000 7fa38042b640 1 --2- 172.21.15.46:0/3307583891 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa3780d2910 0x7fa37815e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:58.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.549+0000 7fa38042b640 1 -- 172.21.15.46:0/3307583891 >> 172.21.15.46:0/3307583891 conn(0x7fa378072020 msgr2=0x7fa378072400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:58.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.549+0000 7fa38042b640 1 -- 172.21.15.46:0/3307583891 shutdown_connections 2023-12-02T00:29:58.552 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.549+0000 7fa38042b640 1 -- 172.21.15.46:0/3307583891 wait complete. 2023-12-02T00:29:58.561 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-02T00:29:58.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.669+0000 7fec977d0640 1 Processor -- start 2023-12-02T00:29:58.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.669+0000 7fec977d0640 1 -- start start 2023-12-02T00:29:58.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.670+0000 7fec977d0640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fec9015d330 0x7fec900b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:58.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.670+0000 7fec977d0640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec900b2b10 con 0x7fec900d2910 2023-12-02T00:29:58.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.670+0000 7fec977d0640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec900d1590 con 0x7fec900d3900 2023-12-02T00:29:58.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.670+0000 7fec977d0640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec900d1870 con 0x7fec9015d330 2023-12-02T00:29:58.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.670+0000 7fec9556c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fec9015d330 0x7fec900b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:58.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.670+0000 7fec9556c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fec9015d330 0x7fec900b8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50712/0 (socket says 172.21.15.46:50712) 2023-12-02T00:29:58.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.670+0000 7fec94d6b640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fec900d2910 0x7fec900d31f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57102/0 (socket says 172.21.15.46:57102) 2023-12-02T00:29:58.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.670+0000 7fec9556c640 1 -- 172.21.15.46:0/2314715212 learned_addr learned my addr 172.21.15.46:0/2314715212 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:58.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.671+0000 7fec9556c640 1 -- 172.21.15.46:0/2314715212 >> v1:172.21.15.162:6789/0 conn(0x7fec900d3900 legacy=0x7fec900b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:58.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.671+0000 7fec9556c640 1 -- 172.21.15.46:0/2314715212 >> v1:172.21.15.46:6789/0 conn(0x7fec900d2910 legacy=0x7fec900d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:58.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.671+0000 7fec9556c640 1 -- 172.21.15.46:0/2314715212 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec900d1b50 con 0x7fec9015d330 2023-12-02T00:29:58.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.671+0000 7fec9556c640 1 --2- 172.21.15.46:0/2314715212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec9015d330 0x7fec900b8b80 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fec88009870 tx=0x7fec8802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d4f3f736661c7c9 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:58.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.672+0000 7fec8ffff640 1 -- 172.21.15.46:0/2314715212 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec88030860 con 0x7fec9015d330 2023-12-02T00:29:58.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.672+0000 7fec8ffff640 1 -- 172.21.15.46:0/2314715212 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec880309c0 con 0x7fec9015d330 2023-12-02T00:29:58.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.672+0000 7fec8ffff640 1 -- 172.21.15.46:0/2314715212 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec88030d50 con 0x7fec9015d330 2023-12-02T00:29:58.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.672+0000 7fec977d0640 1 -- 172.21.15.46:0/2314715212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec9015d330 msgr2=0x7fec900b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:58.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.672+0000 7fec977d0640 1 --2- 172.21.15.46:0/2314715212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec9015d330 0x7fec900b8b80 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fec88009870 tx=0x7fec8802b9a0 comp rx=0 tx=0).stop 2023-12-02T00:29:58.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.672+0000 7fec977d0640 1 -- 172.21.15.46:0/2314715212 shutdown_connections 2023-12-02T00:29:58.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.672+0000 7fec977d0640 1 --2- 172.21.15.46:0/2314715212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec9015d330 0x7fec900b8b80 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:58.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.672+0000 7fec977d0640 1 -- 172.21.15.46:0/2314715212 >> 172.21.15.46:0/2314715212 conn(0x7fec90072020 msgr2=0x7fec90072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:58.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.672+0000 7fec977d0640 1 -- 172.21.15.46:0/2314715212 shutdown_connections 2023-12-02T00:29:58.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.672+0000 7fec977d0640 1 -- 172.21.15.46:0/2314715212 wait complete. 2023-12-02T00:29:58.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.673+0000 7fec977d0640 1 Processor -- start 2023-12-02T00:29:58.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.674+0000 7fec977d0640 1 -- start start 2023-12-02T00:29:58.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.674+0000 7fec977d0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec900d2910 0x7fec9015a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:58.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.674+0000 7fec977d0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec900d3900 0x7fec900cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:58.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.674+0000 7fec977d0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fec9015d330 0x7fec9015fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:58.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.674+0000 7fec977d0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec900cc390 con 0x7fec900d3900 2023-12-02T00:29:58.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.674+0000 7fec977d0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9015ac40 con 0x7fec900d2910 2023-12-02T00:29:58.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.674+0000 7fec977d0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec9015af20 con 0x7fec9015d330 2023-12-02T00:29:58.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.674+0000 7fec94d6b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec900d3900 0x7fec900cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:58.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.674+0000 7fec94d6b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec900d3900 0x7fec900cbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50718/0 (socket says 172.21.15.46:50718) 2023-12-02T00:29:58.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.674+0000 7fec94d6b640 1 -- 172.21.15.46:0/89751931 learned_addr learned my addr 172.21.15.46:0/89751931 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:58.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.675+0000 7fec9556c640 1 --2- 172.21.15.46:0/89751931 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec900d2910 0x7fec9015a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:58.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.675+0000 7fec94d6b640 1 -- 172.21.15.46:0/89751931 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fec9015d330 msgr2=0x7fec9015fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:58.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.675+0000 7fec94d6b640 1 --2- 172.21.15.46:0/89751931 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fec9015d330 0x7fec9015fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:58.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.675+0000 7fec94d6b640 1 -- 172.21.15.46:0/89751931 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec900d2910 msgr2=0x7fec9015a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:58.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.675+0000 7fec94d6b640 1 --2- 172.21.15.46:0/89751931 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec900d2910 0x7fec9015a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:58.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.675+0000 7fec94d6b640 1 -- 172.21.15.46:0/89751931 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec880094d0 con 0x7fec900d3900 2023-12-02T00:29:58.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.675+0000 7fec94d6b640 1 --2- 172.21.15.46:0/89751931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec900d3900 0x7fec900cbe50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fec8400c4d0 tx=0x7fec8400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:58.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.675+0000 7fec8dffb640 1 -- 172.21.15.46:0/89751931 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec84007480 con 0x7fec900d3900 2023-12-02T00:29:58.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.675+0000 7fec8dffb640 1 -- 172.21.15.46:0/89751931 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec840075e0 con 0x7fec900d3900 2023-12-02T00:29:58.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.675+0000 7fec977d0640 1 -- 172.21.15.46:0/89751931 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec9015b200 con 0x7fec900d3900 2023-12-02T00:29:58.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.675+0000 7fec8dffb640 1 -- 172.21.15.46:0/89751931 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec840049e0 con 0x7fec900d3900 2023-12-02T00:29:58.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.675+0000 7fec977d0640 1 -- 172.21.15.46:0/89751931 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec9015b7c0 con 0x7fec900d3900 2023-12-02T00:29:58.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.676+0000 7fec977d0640 1 -- 172.21.15.46:0/89751931 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec9015b9e0 con 0x7fec900d3900 2023-12-02T00:29:58.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.676+0000 7fec8dffb640 1 -- 172.21.15.46:0/89751931 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fec84004b40 con 0x7fec900d3900 2023-12-02T00:29:58.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.677+0000 7fec8dffb640 1 --2- 172.21.15.46:0/89751931 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fec7c06ae40 0x7fec7c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:58.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.677+0000 7fec8dffb640 1 -- 172.21.15.46:0/89751931 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fec84088b70 con 0x7fec900d3900 2023-12-02T00:29:58.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.677+0000 7fec9556c640 1 --2- 172.21.15.46:0/89751931 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fec7c06ae40 0x7fec7c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:58.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.677+0000 7fec9556c640 1 --2- 172.21.15.46:0/89751931 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fec7c06ae40 0x7fec7c06d300 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fec88036000 tx=0x7fec8803b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:58.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.681+0000 7fec8dffb640 1 -- 172.21.15.46:0/89751931 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fec84053c20 con 0x7fec900d3900 2023-12-02T00:29:58.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.897+0000 7fec977d0640 1 -- 172.21.15.46:0/89751931 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fec90069400 con 0x7fec900d3900 2023-12-02T00:29:58.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.915+0000 7fec8dffb640 1 -- 172.21.15.46:0/89751931 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v12) v1 ==== 118+0+0 (secure 0 0 0) 0x7fec840180b0 con 0x7fec900d3900 2023-12-02T00:29:58.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.918+0000 7fec977d0640 1 -- 172.21.15.46:0/89751931 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fec7c06ae40 msgr2=0x7fec7c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:58.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.918+0000 7fec977d0640 1 --2- 172.21.15.46:0/89751931 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fec7c06ae40 0x7fec7c06d300 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fec88036000 tx=0x7fec8803b040 comp rx=0 tx=0).stop 2023-12-02T00:29:58.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.918+0000 7fec977d0640 1 -- 172.21.15.46:0/89751931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec900d3900 msgr2=0x7fec900cbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:58.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.918+0000 7fec977d0640 1 --2- 172.21.15.46:0/89751931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec900d3900 0x7fec900cbe50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fec8400c4d0 tx=0x7fec8400c990 comp rx=0 tx=0).stop 2023-12-02T00:29:58.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.918+0000 7fec977d0640 1 -- 172.21.15.46:0/89751931 shutdown_connections 2023-12-02T00:29:58.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.918+0000 7fec977d0640 1 --2- 172.21.15.46:0/89751931 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fec7c06ae40 0x7fec7c06d300 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:58.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.918+0000 7fec977d0640 1 --2- 172.21.15.46:0/89751931 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fec9015d330 0x7fec9015fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:58.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.918+0000 7fec977d0640 1 --2- 172.21.15.46:0/89751931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec900d3900 0x7fec900cbe50 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:58.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.918+0000 7fec977d0640 1 --2- 172.21.15.46:0/89751931 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec900d2910 0x7fec9015a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:58.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.918+0000 7fec977d0640 1 -- 172.21.15.46:0/89751931 >> 172.21.15.46:0/89751931 conn(0x7fec90072020 msgr2=0x7fec90072400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:58.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.918+0000 7fec977d0640 1 -- 172.21.15.46:0/89751931 shutdown_connections 2023-12-02T00:29:58.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:58.918+0000 7fec977d0640 1 -- 172.21.15.46:0/89751931 wait complete. 2023-12-02T00:29:58.931 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-02T00:29:58.931 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-02T00:29:59.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.039+0000 7fe0a05be640 1 Processor -- start 2023-12-02T00:29:59.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.039+0000 7fe0a05be640 1 -- start start 2023-12-02T00:29:59.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.040+0000 7fe0a05be640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe0980d3900 0x7fe0980b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:59.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.040+0000 7fe0a05be640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0980b2b10 con 0x7fe09815d330 2023-12-02T00:29:59.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.040+0000 7fe0a05be640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0980d1590 con 0x7fe0980d2910 2023-12-02T00:29:59.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.040+0000 7fe0a05be640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0980d1870 con 0x7fe0980d3900 2023-12-02T00:29:59.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.040+0000 7fe09eb5b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe0980d3900 0x7fe0980b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:59.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.040+0000 7fe09eb5b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe0980d3900 0x7fe0980b6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50728/0 (socket says 172.21.15.46:50728) 2023-12-02T00:29:59.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.040+0000 7fe09eb5b640 1 -- 172.21.15.46:0/3551686358 learned_addr learned my addr 172.21.15.46:0/3551686358 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:59.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.040+0000 7fe09e35a640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe09815d330 0x7fe0980b8b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57114/0 (socket says 172.21.15.46:57114) 2023-12-02T00:29:59.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.040+0000 7fe09eb5b640 1 -- 172.21.15.46:0/3551686358 >> v1:172.21.15.162:6789/0 conn(0x7fe0980d2910 legacy=0x7fe0980d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:59.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.040+0000 7fe09eb5b640 1 -- 172.21.15.46:0/3551686358 >> v1:172.21.15.46:6789/0 conn(0x7fe09815d330 legacy=0x7fe0980b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:59.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.040+0000 7fe09eb5b640 1 -- 172.21.15.46:0/3551686358 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0980d1b50 con 0x7fe0980d3900 2023-12-02T00:29:59.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.040+0000 7fe09d358640 1 -- 172.21.15.46:0/3551686358 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3816636086 0 0) 0x7fe0980b2b10 con 0x7fe09815d330 2023-12-02T00:29:59.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.041+0000 7fe09eb5b640 1 --2- 172.21.15.46:0/3551686358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0980d3900 0x7fe0980b6390 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe094009e50 tx=0x7fe09402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23e111c00b25099a server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:59.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.041+0000 7fe09d358640 1 -- 172.21.15.46:0/3551686358 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09402cb20 con 0x7fe0980d3900 2023-12-02T00:29:59.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.041+0000 7fe09d358640 1 -- 172.21.15.46:0/3551686358 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe09402cc80 con 0x7fe0980d3900 2023-12-02T00:29:59.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.041+0000 7fe09d358640 1 -- 172.21.15.46:0/3551686358 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0940343f0 con 0x7fe0980d3900 2023-12-02T00:29:59.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.042+0000 7fe0a05be640 1 -- 172.21.15.46:0/3551686358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0980d3900 msgr2=0x7fe0980b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:59.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.042+0000 7fe0a05be640 1 --2- 172.21.15.46:0/3551686358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0980d3900 0x7fe0980b6390 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe094009e50 tx=0x7fe09402bf80 comp rx=0 tx=0).stop 2023-12-02T00:29:59.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.042+0000 7fe0a05be640 1 -- 172.21.15.46:0/3551686358 shutdown_connections 2023-12-02T00:29:59.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.042+0000 7fe0a05be640 1 --2- 172.21.15.46:0/3551686358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0980d3900 0x7fe0980b6390 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:59.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.042+0000 7fe0a05be640 1 -- 172.21.15.46:0/3551686358 >> 172.21.15.46:0/3551686358 conn(0x7fe098072020 msgr2=0x7fe098072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:59.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.042+0000 7fe0a05be640 1 -- 172.21.15.46:0/3551686358 shutdown_connections 2023-12-02T00:29:59.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.042+0000 7fe0a05be640 1 -- 172.21.15.46:0/3551686358 wait complete. 2023-12-02T00:29:59.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.043+0000 7fe0a05be640 1 Processor -- start 2023-12-02T00:29:59.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.043+0000 7fe0a05be640 1 -- start start 2023-12-02T00:29:59.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.044+0000 7fe0a05be640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0980d2910 0x7fe0980b4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:59.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.044+0000 7fe0a05be640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0980d3900 0x7fe0980c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:59.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.044+0000 7fe0a05be640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe09815d330 0x7fe0980ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:59.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.044+0000 7fe0a05be640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0980cc860 con 0x7fe0980d2910 2023-12-02T00:29:59.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.044+0000 7fe0a05be640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0980cbc20 con 0x7fe0980d3900 2023-12-02T00:29:59.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.044+0000 7fe0a05be640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0980cbec0 con 0x7fe09815d330 2023-12-02T00:29:59.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.044+0000 7fe09e35a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0980d2910 0x7fe0980b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:59.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.044+0000 7fe09e35a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0980d2910 0x7fe0980b4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50732/0 (socket says 172.21.15.46:50732) 2023-12-02T00:29:59.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.044+0000 7fe09e35a640 1 -- 172.21.15.46:0/3303290049 learned_addr learned my addr 172.21.15.46:0/3303290049 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:29:59.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.044+0000 7fe09e35a640 1 -- 172.21.15.46:0/3303290049 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe09815d330 msgr2=0x7fe0980ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:59.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.044+0000 7fe09e35a640 1 --2- 172.21.15.46:0/3303290049 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe09815d330 0x7fe0980ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:59.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.044+0000 7fe09e35a640 1 -- 172.21.15.46:0/3303290049 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0980d3900 msgr2=0x7fe0980c8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:59.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.044+0000 7fe09e35a640 1 --2- 172.21.15.46:0/3303290049 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0980d3900 0x7fe0980c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:59.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.044+0000 7fe09e35a640 1 -- 172.21.15.46:0/3303290049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe094009b30 con 0x7fe0980d2910 2023-12-02T00:29:59.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.045+0000 7fe09e35a640 1 --2- 172.21.15.46:0/3303290049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0980d2910 0x7fe0980b4030 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe09000c4a0 tx=0x7fe09000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:59.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.045+0000 7fe08effd640 1 -- 172.21.15.46:0/3303290049 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0900074e0 con 0x7fe0980d2910 2023-12-02T00:29:59.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.045+0000 7fe08effd640 1 -- 172.21.15.46:0/3303290049 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe090007640 con 0x7fe0980d2910 2023-12-02T00:29:59.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.045+0000 7fe0a05be640 1 -- 172.21.15.46:0/3303290049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0980cc1a0 con 0x7fe0980d2910 2023-12-02T00:29:59.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.045+0000 7fe08effd640 1 -- 172.21.15.46:0/3303290049 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe090004a00 con 0x7fe0980d2910 2023-12-02T00:29:59.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.045+0000 7fe0a05be640 1 -- 172.21.15.46:0/3303290049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0980c94e0 con 0x7fe0980d2910 2023-12-02T00:29:59.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.046+0000 7fe0a05be640 1 -- 172.21.15.46:0/3303290049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0980cc9a0 con 0x7fe0980d2910 2023-12-02T00:29:59.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.046+0000 7fe08effd640 1 -- 172.21.15.46:0/3303290049 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe090004b60 con 0x7fe0980d2910 2023-12-02T00:29:59.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.047+0000 7fe08effd640 1 --2- 172.21.15.46:0/3303290049 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe07c06ae40 0x7fe07c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:29:59.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.047+0000 7fe08effd640 1 -- 172.21.15.46:0/3303290049 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe090088b40 con 0x7fe0980d2910 2023-12-02T00:29:59.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.047+0000 7fe09db59640 1 --2- 172.21.15.46:0/3303290049 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe07c06ae40 0x7fe07c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:29:59.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.047+0000 7fe09db59640 1 --2- 172.21.15.46:0/3303290049 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe07c06ae40 0x7fe07c06d300 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe088000f60 tx=0x7fe0880079b0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:29:59.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.051+0000 7fe08effd640 1 -- 172.21.15.46:0/3303290049 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe090053be0 con 0x7fe0980d2910 2023-12-02T00:29:59.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.264+0000 7fe0a05be640 1 -- 172.21.15.46:0/3303290049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe09806bc40 con 0x7fe0980d2910 2023-12-02T00:29:59.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.920+0000 7fe08effd640 1 -- 172.21.15.46:0/3303290049 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v13) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe090018020 con 0x7fe0980d2910 2023-12-02T00:29:59.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.923+0000 7fe0a05be640 1 -- 172.21.15.46:0/3303290049 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe07c06ae40 msgr2=0x7fe07c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:59.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.923+0000 7fe0a05be640 1 --2- 172.21.15.46:0/3303290049 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe07c06ae40 0x7fe07c06d300 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe088000f60 tx=0x7fe0880079b0 comp rx=0 tx=0).stop 2023-12-02T00:29:59.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.923+0000 7fe0a05be640 1 -- 172.21.15.46:0/3303290049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0980d2910 msgr2=0x7fe0980b4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:29:59.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.923+0000 7fe0a05be640 1 --2- 172.21.15.46:0/3303290049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0980d2910 0x7fe0980b4030 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe09000c4a0 tx=0x7fe09000c960 comp rx=0 tx=0).stop 2023-12-02T00:29:59.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.923+0000 7fe0a05be640 1 -- 172.21.15.46:0/3303290049 shutdown_connections 2023-12-02T00:29:59.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.923+0000 7fe0a05be640 1 --2- 172.21.15.46:0/3303290049 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe07c06ae40 0x7fe07c06d300 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:59.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.923+0000 7fe0a05be640 1 --2- 172.21.15.46:0/3303290049 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe09815d330 0x7fe0980ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:59.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.923+0000 7fe0a05be640 1 --2- 172.21.15.46:0/3303290049 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0980d3900 0x7fe0980c8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:59.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.923+0000 7fe0a05be640 1 --2- 172.21.15.46:0/3303290049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0980d2910 0x7fe0980b4030 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:29:59.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.923+0000 7fe0a05be640 1 -- 172.21.15.46:0/3303290049 >> 172.21.15.46:0/3303290049 conn(0x7fe098072020 msgr2=0x7fe0980cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:29:59.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.923+0000 7fe0a05be640 1 -- 172.21.15.46:0/3303290049 shutdown_connections 2023-12-02T00:29:59.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:29:59.923+0000 7fe0a05be640 1 -- 172.21.15.46:0/3303290049 wait complete. 2023-12-02T00:29:59.937 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-02T00:29:59.938 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-02T00:30:00.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.049+0000 7fc00b7be640 1 Processor -- start 2023-12-02T00:30:00.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.050+0000 7fc00b7be640 1 -- start start 2023-12-02T00:30:00.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.050+0000 7fc00b7be640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc00415d330 0x7fc0040b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:00.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.050+0000 7fc00b7be640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0040d2910 0x7fc0040d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:00.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.050+0000 7fc00b7be640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc0040d3730 0x7fc0040b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:00.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.050+0000 7fc00b7be640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0040b2b10 con 0x7fc00415d330 2023-12-02T00:30:00.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.050+0000 7fc00b7be640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0040d3b50 con 0x7fc0040d3730 2023-12-02T00:30:00.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.050+0000 7fc00b7be640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0040d1590 con 0x7fc0040d2910 2023-12-02T00:30:00.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.051+0000 7fc00955a640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc00415d330 0x7fc0040b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:00.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.051+0000 7fc00955a640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc00415d330 0x7fc0040b8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50742/0 (socket says 172.21.15.46:50742) 2023-12-02T00:30:00.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.051+0000 7fc00955a640 1 -- 172.21.15.46:0/2776928481 learned_addr learned my addr 172.21.15.46:0/2776928481 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:00.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.051+0000 7fc009d5b640 1 --2- 172.21.15.46:0/2776928481 >> v2:172.21.15.162:3300/0 conn(0x7fc0040d3730 0x7fc0040b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:00.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.051+0000 7fc008d59640 1 --2- 172.21.15.46:0/2776928481 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0040d2910 0x7fc0040d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:00.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.051+0000 7fc00955a640 1 -- 172.21.15.46:0/2776928481 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0040d2910 msgr2=0x7fc0040d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:00.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.051+0000 7fc00955a640 1 --2- 172.21.15.46:0/2776928481 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0040d2910 0x7fc0040d31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:00.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.051+0000 7fc00955a640 1 -- 172.21.15.46:0/2776928481 >> v2:172.21.15.162:3300/0 conn(0x7fc0040d3730 msgr2=0x7fc0040b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:00.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.051+0000 7fc00955a640 1 --2- 172.21.15.46:0/2776928481 >> v2:172.21.15.162:3300/0 conn(0x7fc0040d3730 0x7fc0040b6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:00.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.051+0000 7fc00955a640 1 -- 172.21.15.46:0/2776928481 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0040d1870 con 0x7fc00415d330 2023-12-02T00:30:00.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.051+0000 7fc009d5b640 1 --2- 172.21.15.46:0/2776928481 >> v2:172.21.15.162:3300/0 conn(0x7fc0040d3730 0x7fc0040b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:30:00.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.052+0000 7fc00955a640 1 --2- 172.21.15.46:0/2776928481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc00415d330 0x7fc0040b8b80 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc0040d1510 tx=0x7fc00002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfa1f511453b9d51 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:00.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.052+0000 7fbffbfff640 1 -- 172.21.15.46:0/2776928481 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0000317e0 con 0x7fc00415d330 2023-12-02T00:30:00.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.052+0000 7fbffbfff640 1 -- 172.21.15.46:0/2776928481 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc000031940 con 0x7fc00415d330 2023-12-02T00:30:00.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.052+0000 7fbffbfff640 1 -- 172.21.15.46:0/2776928481 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc000031cd0 con 0x7fc00415d330 2023-12-02T00:30:00.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.052+0000 7fc00b7be640 1 -- 172.21.15.46:0/2776928481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc00415d330 msgr2=0x7fc0040b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:00.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.052+0000 7fc00b7be640 1 --2- 172.21.15.46:0/2776928481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc00415d330 0x7fc0040b8b80 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc0040d1510 tx=0x7fc00002b880 comp rx=0 tx=0).stop 2023-12-02T00:30:00.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.052+0000 7fc00b7be640 1 -- 172.21.15.46:0/2776928481 shutdown_connections 2023-12-02T00:30:00.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.052+0000 7fc00b7be640 1 --2- 172.21.15.46:0/2776928481 >> v2:172.21.15.162:3300/0 conn(0x7fc0040d3730 0x7fc0040b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:00.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.052+0000 7fc00b7be640 1 --2- 172.21.15.46:0/2776928481 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0040d2910 0x7fc0040d31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:00.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.052+0000 7fc00b7be640 1 --2- 172.21.15.46:0/2776928481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc00415d330 0x7fc0040b8b80 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:00.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.052+0000 7fc00b7be640 1 -- 172.21.15.46:0/2776928481 >> 172.21.15.46:0/2776928481 conn(0x7fc004072020 msgr2=0x7fc004072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:00.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.053+0000 7fc00b7be640 1 -- 172.21.15.46:0/2776928481 shutdown_connections 2023-12-02T00:30:00.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.053+0000 7fc00b7be640 1 -- 172.21.15.46:0/2776928481 wait complete. 2023-12-02T00:30:00.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.054+0000 7fc00b7be640 1 Processor -- start 2023-12-02T00:30:00.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.054+0000 7fc00b7be640 1 -- start start 2023-12-02T00:30:00.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.054+0000 7fc00b7be640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0040d2910 0x7fc00415b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:00.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.054+0000 7fc00b7be640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0040d3730 0x7fc0040cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:00.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.054+0000 7fc00b7be640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc00415d330 0x7fc0040ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:00.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.054+0000 7fc00b7be640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00415bc20 con 0x7fc00415d330 2023-12-02T00:30:00.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.054+0000 7fc00b7be640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00415eed0 con 0x7fc0040d3730 2023-12-02T00:30:00.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.054+0000 7fc00b7be640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc00415e2c0 con 0x7fc0040d2910 2023-12-02T00:30:00.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.055+0000 7fc009d5b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc00415d330 0x7fc0040ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:00.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.055+0000 7fc009d5b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc00415d330 0x7fc0040ccd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50752/0 (socket says 172.21.15.46:50752) 2023-12-02T00:30:00.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.055+0000 7fc009d5b640 1 -- 172.21.15.46:0/2859918314 learned_addr learned my addr 172.21.15.46:0/2859918314 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:00.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.055+0000 7fc008d59640 1 --2- 172.21.15.46:0/2859918314 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0040d3730 0x7fc0040cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:00.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.055+0000 7fc00955a640 1 --2- 172.21.15.46:0/2859918314 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0040d2910 0x7fc00415b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:00.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.055+0000 7fc009d5b640 1 -- 172.21.15.46:0/2859918314 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0040d2910 msgr2=0x7fc00415b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:00.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.055+0000 7fc009d5b640 1 --2- 172.21.15.46:0/2859918314 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0040d2910 0x7fc00415b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:00.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.055+0000 7fc009d5b640 1 -- 172.21.15.46:0/2859918314 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0040d3730 msgr2=0x7fc0040cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:00.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.055+0000 7fc009d5b640 1 --2- 172.21.15.46:0/2859918314 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0040d3730 0x7fc0040cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:00.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.055+0000 7fc009d5b640 1 -- 172.21.15.46:0/2859918314 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0000094d0 con 0x7fc00415d330 2023-12-02T00:30:00.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.055+0000 7fc008d59640 1 --2- 172.21.15.46:0/2859918314 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0040d3730 0x7fc0040cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:30:00.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.055+0000 7fc009d5b640 1 --2- 172.21.15.46:0/2859918314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc00415d330 0x7fc0040ccd20 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fbffc007ef0 tx=0x7fbffc00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:00.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.055+0000 7fbff9ffb640 1 -- 172.21.15.46:0/2859918314 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc003b90 con 0x7fc00415d330 2023-12-02T00:30:00.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.055+0000 7fbff9ffb640 1 -- 172.21.15.46:0/2859918314 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbffc00f040 con 0x7fc00415d330 2023-12-02T00:30:00.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.055+0000 7fc00b7be640 1 -- 172.21.15.46:0/2859918314 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc00415e5a0 con 0x7fc00415d330 2023-12-02T00:30:00.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.055+0000 7fbff9ffb640 1 -- 172.21.15.46:0/2859918314 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc00de80 con 0x7fc00415d330 2023-12-02T00:30:00.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.056+0000 7fc00b7be640 1 -- 172.21.15.46:0/2859918314 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc00415eb60 con 0x7fc00415d330 2023-12-02T00:30:00.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.056+0000 7fc00b7be640 1 -- 172.21.15.46:0/2859918314 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc004069400 con 0x7fc00415d330 2023-12-02T00:30:00.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.057+0000 7fbff9ffb640 1 -- 172.21.15.46:0/2859918314 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbffc004030 con 0x7fc00415d330 2023-12-02T00:30:00.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.057+0000 7fbff9ffb640 1 --2- 172.21.15.46:0/2859918314 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fbfe806ae40 0x7fbfe806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:00.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.057+0000 7fbff9ffb640 1 -- 172.21.15.46:0/2859918314 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbffc0897b0 con 0x7fc00415d330 2023-12-02T00:30:00.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.057+0000 7fc00955a640 1 --2- 172.21.15.46:0/2859918314 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fbfe806ae40 0x7fbfe806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:00.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.058+0000 7fc00955a640 1 --2- 172.21.15.46:0/2859918314 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fbfe806ae40 0x7fbfe806d300 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc000036000 tx=0x7fc00003d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:00.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.061+0000 7fbff9ffb640 1 -- 172.21.15.46:0/2859918314 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbffc0549a0 con 0x7fc00415d330 2023-12-02T00:30:00.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.273+0000 7fc00b7be640 1 -- 172.21.15.46:0/2859918314 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc00415f010 con 0x7fc00415d330 2023-12-02T00:30:00.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.274+0000 7fbff9ffb640 1 -- 172.21.15.46:0/2859918314 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+4917 (secure 0 0 0) 0x7fbffc057fc0 con 0x7fc00415d330 2023-12-02T00:30:00.275 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:30:00.275 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4811,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/3097254664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3097254664},{"type":"v1","addr":"172.21.15.162:6835","nonce":3097254664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":4826,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/3489476943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3489476943},{"type":"v1","addr":"172.21.15.162:6837","nonce":3489476943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5274,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1283185308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":1283185308},{"type":"v1","addr":"172.21.15.46:6835","nonce":1283185308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-02T00:29:57.279169+0000","modified":"2023-12-02T00:29:59.266062+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5277},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5277":{"gid":5277,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":5,"addr":"172.21.15.46:6834/2036053042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2036053042},{"type":"v1","addr":"172.21.15.46:6834","nonce":2036053042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-02T00:30:00.275 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 13 2023-12-02T00:30:00.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.276+0000 7fc00b7be640 1 -- 172.21.15.46:0/2859918314 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fbfe806ae40 msgr2=0x7fbfe806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:00.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.276+0000 7fc00b7be640 1 --2- 172.21.15.46:0/2859918314 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fbfe806ae40 0x7fbfe806d300 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc000036000 tx=0x7fc00003d040 comp rx=0 tx=0).stop 2023-12-02T00:30:00.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.276+0000 7fc00b7be640 1 -- 172.21.15.46:0/2859918314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc00415d330 msgr2=0x7fc0040ccd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:00.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.276+0000 7fc00b7be640 1 --2- 172.21.15.46:0/2859918314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc00415d330 0x7fc0040ccd20 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fbffc007ef0 tx=0x7fbffc00d4a0 comp rx=0 tx=0).stop 2023-12-02T00:30:00.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.276+0000 7fc00b7be640 1 -- 172.21.15.46:0/2859918314 shutdown_connections 2023-12-02T00:30:00.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.276+0000 7fc00b7be640 1 --2- 172.21.15.46:0/2859918314 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fbfe806ae40 0x7fbfe806d300 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:00.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.276+0000 7fc00b7be640 1 --2- 172.21.15.46:0/2859918314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc00415d330 0x7fc0040ccd20 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:00.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.276+0000 7fc00b7be640 1 --2- 172.21.15.46:0/2859918314 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0040d3730 0x7fc0040cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:00.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.276+0000 7fc00b7be640 1 --2- 172.21.15.46:0/2859918314 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0040d2910 0x7fc00415b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:00.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.276+0000 7fc00b7be640 1 -- 172.21.15.46:0/2859918314 >> 172.21.15.46:0/2859918314 conn(0x7fc004072020 msgr2=0x7fc0040b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:00.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.276+0000 7fc00b7be640 1 -- 172.21.15.46:0/2859918314 shutdown_connections 2023-12-02T00:30:00.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.276+0000 7fc00b7be640 1 -- 172.21.15.46:0/2859918314 wait complete. 2023-12-02T00:30:00.288 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:30:00.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.398+0000 7f8fa1a96640 1 Processor -- start 2023-12-02T00:30:00.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.398+0000 7f8fa1a96640 1 -- start start 2023-12-02T00:30:00.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.399+0000 7f8fa1a96640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8f9c0ceb70 0x7f8f9c0cc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:00.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.399+0000 7f8fa1a96640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f9c0b4370 0x7f8f9c0b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:00.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.399+0000 7f8fa1a96640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f9c071770 con 0x7f8f9c144980 2023-12-02T00:30:00.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.399+0000 7f8fa1a96640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f9c15ced0 con 0x7f8f9c0ceb70 2023-12-02T00:30:00.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.399+0000 7f8fa1a96640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9c0b4ce0 con 0x7f8f9c0b4370 2023-12-02T00:30:00.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.399+0000 7f8f9b7fe640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8f9c144980 0x7f8f9c15c7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57126/0 (socket says 172.21.15.46:57126) 2023-12-02T00:30:00.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.399+0000 7f8f9b7fe640 1 -- 172.21.15.46:0/3934282870 learned_addr learned my addr 172.21.15.46:0/3934282870 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:30:00.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.399+0000 7f8f9affd640 1 --2- 172.21.15.46:0/3934282870 >> v2:172.21.15.162:3300/0 conn(0x7f8f9c0ceb70 0x7f8f9c0cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:00.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.399+0000 7f8f9a7fc640 1 -- 172.21.15.46:0/3934282870 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 311480126 0 0) 0x7f8f9c071770 con 0x7f8f9c144980 2023-12-02T00:30:00.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.400+0000 7f8f9bfff640 1 --2- 172.21.15.46:0/3934282870 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f9c0b4370 0x7f8f9c0b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:00.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.400+0000 7f8f9a7fc640 1 -- 172.21.15.46:0/3934282870 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f84003520 con 0x7f8f9c144980 2023-12-02T00:30:00.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.400+0000 7f8f9a7fc640 1 -- 172.21.15.46:0/3934282870 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1158162772 0 0) 0x7f8f84003520 con 0x7f8f9c144980 2023-12-02T00:30:00.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.400+0000 7f8f9a7fc640 1 -- 172.21.15.46:0/3934282870 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f9c0b4370 msgr2=0x7f8f9c0b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:00.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.400+0000 7f8f9a7fc640 1 --2- 172.21.15.46:0/3934282870 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f9c0b4370 0x7f8f9c0b47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:00.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.400+0000 7f8f9a7fc640 1 -- 172.21.15.46:0/3934282870 >> v2:172.21.15.162:3300/0 conn(0x7f8f9c0ceb70 msgr2=0x7f8f9c0cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:00.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.400+0000 7f8f9a7fc640 1 --2- 172.21.15.46:0/3934282870 >> v2:172.21.15.162:3300/0 conn(0x7f8f9c0ceb70 0x7f8f9c0cc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:00.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.400+0000 7f8f9a7fc640 1 -- 172.21.15.46:0/3934282870 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f9c0ccdc0 con 0x7f8f9c144980 2023-12-02T00:30:00.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.400+0000 7f8f9affd640 1 --2- 172.21.15.46:0/3934282870 >> v2:172.21.15.162:3300/0 conn(0x7f8f9c0ceb70 0x7f8f9c0cc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:30:00.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.400+0000 7f8f9a7fc640 1 -- 172.21.15.46:0/3934282870 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7f8f8c001670 con 0x7f8f9c144980 2023-12-02T00:30:00.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.400+0000 7f8f9a7fc640 1 -- 172.21.15.46:0/3934282870 >> v1:172.21.15.46:6789/0 conn(0x7f8f9c144980 legacy=0x7f8f9c15c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:00.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.400+0000 7f8f9a7fc640 1 --2- 172.21.15.46:0/3934282870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f84003970 0x7f8f84003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:00.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.400+0000 7f8f9a7fc640 1 --2- 172.21.15.46:0/3934282870 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f9c0b4370 0x7f8f840047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:00.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.400+0000 7f8f9a7fc640 1 --2- 172.21.15.46:0/3934282870 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f84004d00 0x7f8f84005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:00.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.400+0000 7f8f9a7fc640 1 -- 172.21.15.46:0/3934282870 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9c15ced0 con 0x7f8f84003970 2023-12-02T00:30:00.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.400+0000 7f8f9a7fc640 1 -- 172.21.15.46:0/3934282870 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9c0b4ce0 con 0x7f8f84004d00 2023-12-02T00:30:00.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.400+0000 7f8f9a7fc640 1 -- 172.21.15.46:0/3934282870 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8400ba30 con 0x7f8f9c0b4370 2023-12-02T00:30:00.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.400+0000 7f8f9b7fe640 1 --2- 172.21.15.46:0/3934282870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f84003970 0x7f8f84003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:00.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.400+0000 7f8f9bfff640 1 --2- 172.21.15.46:0/3934282870 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f84004d00 0x7f8f84005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:00.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.400+0000 7f8f9affd640 1 --2- 172.21.15.46:0/3934282870 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f9c0b4370 0x7f8f840047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:00.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.400+0000 7f8f9b7fe640 1 -- 172.21.15.46:0/3934282870 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f9c0b4370 msgr2=0x7f8f840047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:00.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.400+0000 7f8f9b7fe640 1 --2- 172.21.15.46:0/3934282870 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f9c0b4370 0x7f8f840047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:00.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.400+0000 7f8f9b7fe640 1 -- 172.21.15.46:0/3934282870 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f84004d00 msgr2=0x7f8f84005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:00.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.400+0000 7f8f9b7fe640 1 --2- 172.21.15.46:0/3934282870 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f84004d00 0x7f8f84005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:00.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.400+0000 7f8f9b7fe640 1 -- 172.21.15.46:0/3934282870 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f84003520 con 0x7f8f84003970 2023-12-02T00:30:00.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.400+0000 7f8f9affd640 1 --2- 172.21.15.46:0/3934282870 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f9c0b4370 0x7f8f840047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:30:00.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.401+0000 7f8f9b7fe640 1 --2- 172.21.15.46:0/3934282870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f84003970 0x7f8f84003d90 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c00c2c0 tx=0x7f8f840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=579fdf6ea38dc391 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:00.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.401+0000 7f8f9a7fc640 1 -- 172.21.15.46:0/3934282870 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c005ad0 con 0x7f8f84003970 2023-12-02T00:30:00.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.401+0000 7f8f9a7fc640 1 -- 172.21.15.46:0/3934282870 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f8c005c30 con 0x7f8f84003970 2023-12-02T00:30:00.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.401+0000 7f8f9a7fc640 1 -- 172.21.15.46:0/3934282870 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c02c8b0 con 0x7f8f84003970 2023-12-02T00:30:00.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.402+0000 7f8fa1a96640 1 -- 172.21.15.46:0/3934282870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f84003970 msgr2=0x7f8f84003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:00.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.402+0000 7f8fa1a96640 1 --2- 172.21.15.46:0/3934282870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f84003970 0x7f8f84003d90 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c00c2c0 tx=0x7f8f840057f0 comp rx=0 tx=0).stop 2023-12-02T00:30:00.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.402+0000 7f8f9bfff640 1 -- 172.21.15.46:0/3934282870 reap_dead start 2023-12-02T00:30:00.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.402+0000 7f8fa1a96640 1 -- 172.21.15.46:0/3934282870 shutdown_connections 2023-12-02T00:30:00.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.402+0000 7f8fa1a96640 1 -- 172.21.15.46:0/3934282870 >> 172.21.15.46:0/3934282870 conn(0x7f8f9c06a4d0 msgr2=0x7f8f9c06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:00.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.402+0000 7f8fa1a96640 1 -- 172.21.15.46:0/3934282870 shutdown_connections 2023-12-02T00:30:00.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.402+0000 7f8fa1a96640 1 -- 172.21.15.46:0/3934282870 wait complete. 2023-12-02T00:30:00.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.403+0000 7f8fa1a96640 1 Processor -- start 2023-12-02T00:30:00.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.403+0000 7f8fa1a96640 1 -- start start 2023-12-02T00:30:00.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.404+0000 7f8fa1a96640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f84003970 0x7f8f9c135b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:00.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.404+0000 7f8fa1a96640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f9c1f0fb0 0x7f8f9c158690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:00.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.404+0000 7f8fa1a96640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f9c0c8010 0x7f8f9c1f3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:00.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.404+0000 7f8fa1a96640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9c1f1430 con 0x7f8f9c0c8010 2023-12-02T00:30:00.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.404+0000 7f8fa1a96640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9c158bd0 con 0x7f8f84003970 2023-12-02T00:30:00.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.404+0000 7f8fa1a96640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9c1361a0 con 0x7f8f9c1f0fb0 2023-12-02T00:30:00.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.404+0000 7f8f9bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f9c0c8010 0x7f8f9c1f3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:00.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.404+0000 7f8f9bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f9c0c8010 0x7f8f9c1f3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50778/0 (socket says 172.21.15.46:50778) 2023-12-02T00:30:00.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.404+0000 7f8f9bfff640 1 -- 172.21.15.46:0/536052262 learned_addr learned my addr 172.21.15.46:0/536052262 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:00.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.404+0000 7f8f9b7fe640 1 --2- 172.21.15.46:0/536052262 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f84003970 0x7f8f9c135b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:00.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.404+0000 7f8f9affd640 1 --2- 172.21.15.46:0/536052262 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f9c1f0fb0 0x7f8f9c158690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:00.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.404+0000 7f8f9bfff640 1 -- 172.21.15.46:0/536052262 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f9c1f0fb0 msgr2=0x7f8f9c158690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:00.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.404+0000 7f8f9bfff640 1 --2- 172.21.15.46:0/536052262 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f9c1f0fb0 0x7f8f9c158690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:00.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.404+0000 7f8f9bfff640 1 -- 172.21.15.46:0/536052262 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f84003970 msgr2=0x7f8f9c135b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:00.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.404+0000 7f8f9bfff640 1 --2- 172.21.15.46:0/536052262 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f84003970 0x7f8f9c135b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:00.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.404+0000 7f8f9bfff640 1 -- 172.21.15.46:0/536052262 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f9c136480 con 0x7f8f9c0c8010 2023-12-02T00:30:00.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.404+0000 7f8f9affd640 1 --2- 172.21.15.46:0/536052262 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f9c1f0fb0 0x7f8f9c158690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:30:00.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.404+0000 7f8f9bfff640 1 --2- 172.21.15.46:0/536052262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f9c0c8010 0x7f8f9c1f3950 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c06fac0 tx=0x7f8f94007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:00.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.405+0000 7f8f83fff640 1 -- 172.21.15.46:0/536052262 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f940046a0 con 0x7f8f9c0c8010 2023-12-02T00:30:00.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.405+0000 7f8f83fff640 1 -- 172.21.15.46:0/536052262 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f94004800 con 0x7f8f9c0c8010 2023-12-02T00:30:00.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.405+0000 7f8fa1a96640 1 -- 172.21.15.46:0/536052262 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f9c136760 con 0x7f8f9c0c8010 2023-12-02T00:30:00.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.405+0000 7f8f83fff640 1 -- 172.21.15.46:0/536052262 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f94015660 con 0x7f8f9c0c8010 2023-12-02T00:30:00.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.405+0000 7f8fa1a96640 1 -- 172.21.15.46:0/536052262 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f9c136d20 con 0x7f8f9c0c8010 2023-12-02T00:30:00.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.406+0000 7f8fa1a96640 1 -- 172.21.15.46:0/536052262 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f9c136ee0 con 0x7f8f9c0c8010 2023-12-02T00:30:00.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.406+0000 7f8f83fff640 1 -- 172.21.15.46:0/536052262 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8f940157c0 con 0x7f8f9c0c8010 2023-12-02T00:30:00.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.406+0000 7f8f83fff640 1 --2- 172.21.15.46:0/536052262 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8f8806ae70 0x7f8f8806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:00.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.407+0000 7f8f83fff640 1 -- 172.21.15.46:0/536052262 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8f94089bc0 con 0x7f8f9c0c8010 2023-12-02T00:30:00.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.407+0000 7f8f9b7fe640 1 --2- 172.21.15.46:0/536052262 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8f8806ae70 0x7f8f8806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:00.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.407+0000 7f8f9b7fe640 1 --2- 172.21.15.46:0/536052262 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8f8806ae70 0x7f8f8806d330 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c034000 tx=0x7f8f8c007000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:00.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.411+0000 7f8f83fff640 1 -- 172.21.15.46:0/536052262 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8f94054c60 con 0x7f8f9c0c8010 2023-12-02T00:30:00.468 INFO:tasks.ceph.mds.a.smithi046.stderr:2023-12-02T00:30:00.467+0000 7fc9c4aea640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-02T00:30:00.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.587+0000 7f8fa1a96640 1 -- 172.21.15.46:0/536052262 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8f9c0b1c20 con 0x7f8f9c0c8010 2023-12-02T00:30:00.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.588+0000 7f8f83fff640 1 -- 172.21.15.46:0/536052262 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16545 (secure 0 0 0) 0x7f8f94058280 con 0x7f8f9c0c8010 2023-12-02T00:30:00.589 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:30:00.589 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":35,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:29:58.915429+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-02T00:29:55.676572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-02T00:29:56.244869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:30:00.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.590+0000 7f8fa1a96640 1 -- 172.21.15.46:0/536052262 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8f8806ae70 msgr2=0x7f8f8806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:00.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.590+0000 7f8fa1a96640 1 --2- 172.21.15.46:0/536052262 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8f8806ae70 0x7f8f8806d330 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c034000 tx=0x7f8f8c007000 comp rx=0 tx=0).stop 2023-12-02T00:30:00.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.590+0000 7f8fa1a96640 1 -- 172.21.15.46:0/536052262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f9c0c8010 msgr2=0x7f8f9c1f3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:00.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.590+0000 7f8fa1a96640 1 --2- 172.21.15.46:0/536052262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f9c0c8010 0x7f8f9c1f3950 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c06fac0 tx=0x7f8f94007700 comp rx=0 tx=0).stop 2023-12-02T00:30:00.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.590+0000 7f8fa1a96640 1 -- 172.21.15.46:0/536052262 shutdown_connections 2023-12-02T00:30:00.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.590+0000 7f8fa1a96640 1 --2- 172.21.15.46:0/536052262 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8f8806ae70 0x7f8f8806d330 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:00.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.590+0000 7f8fa1a96640 1 --2- 172.21.15.46:0/536052262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f9c0c8010 0x7f8f9c1f3950 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:00.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.590+0000 7f8fa1a96640 1 --2- 172.21.15.46:0/536052262 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f9c1f0fb0 0x7f8f9c158690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:00.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.590+0000 7f8fa1a96640 1 --2- 172.21.15.46:0/536052262 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f84003970 0x7f8f9c135b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:00.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.590+0000 7f8fa1a96640 1 -- 172.21.15.46:0/536052262 >> 172.21.15.46:0/536052262 conn(0x7f8f9c06a4d0 msgr2=0x7f8f9c15e080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:00.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.590+0000 7f8fa1a96640 1 -- 172.21.15.46:0/536052262 shutdown_connections 2023-12-02T00:30:00.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.590+0000 7f8fa1a96640 1 -- 172.21.15.46:0/536052262 wait complete. 2023-12-02T00:30:00.602 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-02T00:30:00.602 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-02T00:30:00.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.708+0000 7fa572348640 1 Processor -- start 2023-12-02T00:30:00.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.708+0000 7fa572348640 1 -- start start 2023-12-02T00:30:00.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.709+0000 7fa572348640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa56c06f210 0x7fa56c140800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:00.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.709+0000 7fa572348640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa56c140d40 0x7fa56c13f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:00.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.709+0000 7fa572348640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa56c15ba40 con 0x7fa56c15c2f0 2023-12-02T00:30:00.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.709+0000 7fa572348640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa56c15c6d0 con 0x7fa56c140d40 2023-12-02T00:30:00.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.709+0000 7fa572348640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa56c13e530 con 0x7fa56c06f210 2023-12-02T00:30:00.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.709+0000 7fa5708e5640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa56c140d40 0x7fa56c13f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:00.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.709+0000 7fa56bfff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa56c15c2f0 0x7fa56c06eb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57128/0 (socket says 172.21.15.46:57128) 2023-12-02T00:30:00.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.709+0000 7fa56bfff640 1 -- 172.21.15.46:0/3065489200 learned_addr learned my addr 172.21.15.46:0/3065489200 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:30:00.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.709+0000 7fa5708e5640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa56c140d40 0x7fa56c13f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50794/0 (socket says 172.21.15.46:50794) 2023-12-02T00:30:00.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.709+0000 7fa56b7fe640 1 --2- 172.21.15.46:0/3065489200 >> v2:172.21.15.162:3300/0 conn(0x7fa56c06f210 0x7fa56c140800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:00.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.710+0000 7fa5708e5640 1 -- 172.21.15.46:0/3065489200 >> v2:172.21.15.162:3300/0 conn(0x7fa56c06f210 msgr2=0x7fa56c140800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:00.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.710+0000 7fa5708e5640 1 --2- 172.21.15.46:0/3065489200 >> v2:172.21.15.162:3300/0 conn(0x7fa56c06f210 0x7fa56c140800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:00.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.710+0000 7fa5708e5640 1 -- 172.21.15.46:0/3065489200 >> v1:172.21.15.46:6789/0 conn(0x7fa56c15c2f0 legacy=0x7fa56c06eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:00.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.710+0000 7fa5708e5640 1 -- 172.21.15.46:0/3065489200 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa56c141190 con 0x7fa56c140d40 2023-12-02T00:30:00.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.710+0000 7fa56affd640 1 -- 172.21.15.46:0/3065489200 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1305266671 0 0) 0x7fa56c15ba40 con 0x7fa56c15c2f0 2023-12-02T00:30:00.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.710+0000 7fa5708e5640 1 --2- 172.21.15.46:0/3065489200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa56c140d40 0x7fa56c13f330 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa56c0d20f0 tx=0x7fa56402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=544341b5c6127da4 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:00.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.711+0000 7fa56affd640 1 -- 172.21.15.46:0/3065489200 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56402cc80 con 0x7fa56c140d40 2023-12-02T00:30:00.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.711+0000 7fa56affd640 1 -- 172.21.15.46:0/3065489200 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa56402cde0 con 0x7fa56c140d40 2023-12-02T00:30:00.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.711+0000 7fa56affd640 1 -- 172.21.15.46:0/3065489200 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa564034660 con 0x7fa56c140d40 2023-12-02T00:30:00.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.711+0000 7fa572348640 1 -- 172.21.15.46:0/3065489200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa56c140d40 msgr2=0x7fa56c13f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:00.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.711+0000 7fa572348640 1 --2- 172.21.15.46:0/3065489200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa56c140d40 0x7fa56c13f330 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa56c0d20f0 tx=0x7fa56402bed0 comp rx=0 tx=0).stop 2023-12-02T00:30:00.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.711+0000 7fa572348640 1 -- 172.21.15.46:0/3065489200 shutdown_connections 2023-12-02T00:30:00.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.711+0000 7fa572348640 1 --2- 172.21.15.46:0/3065489200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa56c140d40 0x7fa56c13f330 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:00.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.711+0000 7fa572348640 1 --2- 172.21.15.46:0/3065489200 >> v2:172.21.15.162:3300/0 conn(0x7fa56c06f210 0x7fa56c140800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:00.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.711+0000 7fa572348640 1 -- 172.21.15.46:0/3065489200 >> 172.21.15.46:0/3065489200 conn(0x7fa56c06a6b0 msgr2=0x7fa56c06aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:00.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.711+0000 7fa572348640 1 -- 172.21.15.46:0/3065489200 shutdown_connections 2023-12-02T00:30:00.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.711+0000 7fa572348640 1 -- 172.21.15.46:0/3065489200 wait complete. 2023-12-02T00:30:00.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.712+0000 7fa572348640 1 Processor -- start 2023-12-02T00:30:00.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.712+0000 7fa572348640 1 -- start start 2023-12-02T00:30:00.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.713+0000 7fa572348640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa56c06f210 0x7fa56c153360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:00.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.713+0000 7fa572348640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa56c140d40 0x7fa56c0b1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:00.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.713+0000 7fa572348640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa56c15c2f0 0x7fa56c1f61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:00.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.713+0000 7fa572348640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c0b21f0 con 0x7fa56c140d40 2023-12-02T00:30:00.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.713+0000 7fa572348640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c0b1c30 con 0x7fa56c06f210 2023-12-02T00:30:00.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.713+0000 7fa572348640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c0b1f10 con 0x7fa56c15c2f0 2023-12-02T00:30:00.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.713+0000 7fa56b7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa56c140d40 0x7fa56c0b1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:00.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.713+0000 7fa56b7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa56c140d40 0x7fa56c0b1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50804/0 (socket says 172.21.15.46:50804) 2023-12-02T00:30:00.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.713+0000 7fa56b7fe640 1 -- 172.21.15.46:0/2877061393 learned_addr learned my addr 172.21.15.46:0/2877061393 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:00.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.713+0000 7fa56bfff640 1 --2- 172.21.15.46:0/2877061393 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa56c06f210 0x7fa56c153360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:00.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.713+0000 7fa5708e5640 1 --2- 172.21.15.46:0/2877061393 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa56c15c2f0 0x7fa56c1f61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:00.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.714+0000 7fa56b7fe640 1 -- 172.21.15.46:0/2877061393 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa56c15c2f0 msgr2=0x7fa56c1f61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:00.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.714+0000 7fa56b7fe640 1 --2- 172.21.15.46:0/2877061393 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa56c15c2f0 0x7fa56c1f61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:00.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.714+0000 7fa56b7fe640 1 -- 172.21.15.46:0/2877061393 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa56c06f210 msgr2=0x7fa56c153360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:00.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.714+0000 7fa56b7fe640 1 --2- 172.21.15.46:0/2877061393 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa56c06f210 0x7fa56c153360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:00.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.714+0000 7fa56b7fe640 1 -- 172.21.15.46:0/2877061393 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa564009a70 con 0x7fa56c140d40 2023-12-02T00:30:00.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.714+0000 7fa5708e5640 1 --2- 172.21.15.46:0/2877061393 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa56c15c2f0 0x7fa56c1f61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:30:00.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.714+0000 7fa56b7fe640 1 --2- 172.21.15.46:0/2877061393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa56c140d40 0x7fa56c0b1170 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fa56000d800 tx=0x7fa56000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:00.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.714+0000 7fa568ff9640 1 -- 172.21.15.46:0/2877061393 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56000b6c0 con 0x7fa56c140d40 2023-12-02T00:30:00.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.714+0000 7fa568ff9640 1 -- 172.21.15.46:0/2877061393 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa56000b820 con 0x7fa56c140d40 2023-12-02T00:30:00.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.714+0000 7fa572348640 1 -- 172.21.15.46:0/2877061393 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa56c06b970 con 0x7fa56c140d40 2023-12-02T00:30:00.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.714+0000 7fa568ff9640 1 -- 172.21.15.46:0/2877061393 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56000bbb0 con 0x7fa56c140d40 2023-12-02T00:30:00.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.714+0000 7fa572348640 1 -- 172.21.15.46:0/2877061393 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa56c155450 con 0x7fa56c140d40 2023-12-02T00:30:00.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.715+0000 7fa572348640 1 -- 172.21.15.46:0/2877061393 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa56c069400 con 0x7fa56c140d40 2023-12-02T00:30:00.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.715+0000 7fa568ff9640 1 -- 172.21.15.46:0/2877061393 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa56000bd10 con 0x7fa56c140d40 2023-12-02T00:30:00.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.716+0000 7fa568ff9640 1 --2- 172.21.15.46:0/2877061393 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa55806ae40 0x7fa55806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:00.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.716+0000 7fa568ff9640 1 -- 172.21.15.46:0/2877061393 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa560089bb0 con 0x7fa56c140d40 2023-12-02T00:30:00.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.716+0000 7fa56bfff640 1 --2- 172.21.15.46:0/2877061393 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa55806ae40 0x7fa55806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:00.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.717+0000 7fa56bfff640 1 --2- 172.21.15.46:0/2877061393 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa55806ae40 0x7fa55806d300 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa55c000f60 tx=0x7fa55c007c00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:00.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.720+0000 7fa568ff9640 1 -- 172.21.15.46:0/2877061393 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa560054c60 con 0x7fa56c140d40 2023-12-02T00:30:00.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.892+0000 7fa572348640 1 -- 172.21.15.46:0/2877061393 --> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa56c155b00 con 0x7fa55806ae40 2023-12-02T00:30:00.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.900+0000 7fa568ff9640 1 -- 172.21.15.46:0/2877061393 <== mgr.4108 v2:172.21.15.162:6832/3248644722 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+276860 (secure 0 0 0) 0x7fa56c155b00 con 0x7fa55806ae40 2023-12-02T00:30:00.904 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:30:00.905 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-12-02T00:30:00.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.903+0000 7fa572348640 1 -- 172.21.15.46:0/2877061393 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa55806ae40 msgr2=0x7fa55806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:00.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.903+0000 7fa572348640 1 --2- 172.21.15.46:0/2877061393 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa55806ae40 0x7fa55806d300 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa55c000f60 tx=0x7fa55c007c00 comp rx=0 tx=0).stop 2023-12-02T00:30:00.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.903+0000 7fa572348640 1 -- 172.21.15.46:0/2877061393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa56c140d40 msgr2=0x7fa56c0b1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:00.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.903+0000 7fa572348640 1 --2- 172.21.15.46:0/2877061393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa56c140d40 0x7fa56c0b1170 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fa56000d800 tx=0x7fa56000dcc0 comp rx=0 tx=0).stop 2023-12-02T00:30:00.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.903+0000 7fa572348640 1 -- 172.21.15.46:0/2877061393 shutdown_connections 2023-12-02T00:30:00.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.903+0000 7fa572348640 1 --2- 172.21.15.46:0/2877061393 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa55806ae40 0x7fa55806d300 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:00.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.903+0000 7fa572348640 1 --2- 172.21.15.46:0/2877061393 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa56c15c2f0 0x7fa56c1f61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:00.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.903+0000 7fa572348640 1 --2- 172.21.15.46:0/2877061393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa56c140d40 0x7fa56c0b1170 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:00.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.903+0000 7fa572348640 1 --2- 172.21.15.46:0/2877061393 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa56c06f210 0x7fa56c153360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:00.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.903+0000 7fa572348640 1 -- 172.21.15.46:0/2877061393 >> 172.21.15.46:0/2877061393 conn(0x7fa56c06a6b0 msgr2=0x7fa56c06aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:00.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.903+0000 7fa572348640 1 -- 172.21.15.46:0/2877061393 shutdown_connections 2023-12-02T00:30:00.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:00.903+0000 7fa572348640 1 -- 172.21.15.46:0/2877061393 wait complete. 2023-12-02T00:30:00.915 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":70,"stamp":"2023-12-02T00:29:59.174705+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236708,"kb_used_data":3896,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738012,"statfs":{"total":773094113280,"available":772851724288,"internally_reserved":0,"allocated":3989504,"data_stored":2243716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000809"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.903298+0000","last_change":"2023-12-02T00:29:55.903298+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.903298+0000","last_undegraded":"2023-12-02T00:29:55.903298+0000","last_fullsized":"2023-12-02T00:29:55.903298+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:55.891930+0000","last_change":"2023-12-02T00:29:55.891930+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.891930+0000","last_undegraded":"2023-12-02T00:29:55.891930+0000","last_fullsized":"2023-12-02T00:29:55.891930+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.900712+0000","last_change":"2023-12-02T00:29:55.900712+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.900712+0000","last_undegraded":"2023-12-02T00:29:55.900712+0000","last_fullsized":"2023-12-02T00:29:55.900712+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.901454+0000","last_change":"2023-12-02T00:29:55.901454+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.901454+0000","last_undegraded":"2023-12-02T00:29:55.901454+0000","last_fullsized":"2023-12-02T00:29:55.901454+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.900095+0000","last_change":"2023-12-02T00:29:55.900095+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.900095+0000","last_undegraded":"2023-12-02T00:29:55.900095+0000","last_fullsized":"2023-12-02T00:29:55.900095+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.898451+0000","last_change":"2023-12-02T00:29:55.898451+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.898451+0000","last_undegraded":"2023-12-02T00:29:55.898451+0000","last_fullsized":"2023-12-02T00:29:55.898451+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.900139+0000","last_change":"2023-12-02T00:29:55.900139+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.900139+0000","last_undegraded":"2023-12-02T00:29:55.900139+0000","last_fullsized":"2023-12-02T00:29:55.900139+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.898680+0000","last_change":"2023-12-02T00:29:55.898680+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.898680+0000","last_undegraded":"2023-12-02T00:29:55.898680+0000","last_fullsized":"2023-12-02T00:29:55.898680+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.899737+0000","last_change":"2023-12-02T00:29:55.899737+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.899737+0000","last_undegraded":"2023-12-02T00:29:55.899737+0000","last_fullsized":"2023-12-02T00:29:55.899737+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:55.891930+0000","last_change":"2023-12-02T00:29:55.891930+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.891930+0000","last_undegraded":"2023-12-02T00:29:55.891930+0000","last_fullsized":"2023-12-02T00:29:55.891930+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.898306+0000","last_change":"2023-12-02T00:29:55.898306+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.898306+0000","last_undegraded":"2023-12-02T00:29:55.898306+0000","last_fullsized":"2023-12-02T00:29:55.898306+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.899642+0000","last_change":"2023-12-02T00:29:55.899642+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.899642+0000","last_undegraded":"2023-12-02T00:29:55.899642+0000","last_fullsized":"2023-12-02T00:29:55.899642+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.899635+0000","last_change":"2023-12-02T00:29:55.899635+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.899635+0000","last_undegraded":"2023-12-02T00:29:55.899635+0000","last_fullsized":"2023-12-02T00:29:55.899635+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.902337+0000","last_change":"2023-12-02T00:29:55.902337+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.902337+0000","last_undegraded":"2023-12-02T00:29:55.902337+0000","last_fullsized":"2023-12-02T00:29:55.902337+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.899585+0000","last_change":"2023-12-02T00:29:55.899585+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.899585+0000","last_undegraded":"2023-12-02T00:29:55.899585+0000","last_fullsized":"2023-12-02T00:29:55.899585+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.899587+0000","last_change":"2023-12-02T00:29:55.899587+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.899587+0000","last_undegraded":"2023-12-02T00:29:55.899587+0000","last_fullsized":"2023-12-02T00:29:55.899587+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.902333+0000","last_change":"2023-12-02T00:29:55.902333+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.902333+0000","last_undegraded":"2023-12-02T00:29:55.902333+0000","last_fullsized":"2023-12-02T00:29:55.902333+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.898944+0000","last_change":"2023-12-02T00:29:55.898944+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.898944+0000","last_undegraded":"2023-12-02T00:29:55.898944+0000","last_fullsized":"2023-12-02T00:29:55.898944+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.899297+0000","last_change":"2023-12-02T00:29:55.899297+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.899297+0000","last_undegraded":"2023-12-02T00:29:55.899297+0000","last_fullsized":"2023-12-02T00:29:55.899297+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.898740+0000","last_change":"2023-12-02T00:29:55.898740+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.898740+0000","last_undegraded":"2023-12-02T00:29:55.898740+0000","last_fullsized":"2023-12-02T00:29:55.898740+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:55.891930+0000","last_change":"2023-12-02T00:29:55.891930+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.891930+0000","last_undegraded":"2023-12-02T00:29:55.891930+0000","last_fullsized":"2023-12-02T00:29:55.891930+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.903215+0000","last_change":"2023-12-02T00:29:55.903215+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.903215+0000","last_undegraded":"2023-12-02T00:29:55.903215+0000","last_fullsized":"2023-12-02T00:29:55.903215+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.900528+0000","last_change":"2023-12-02T00:29:55.900528+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.900528+0000","last_undegraded":"2023-12-02T00:29:55.900528+0000","last_fullsized":"2023-12-02T00:29:55.900528+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.898308+0000","last_change":"2023-12-02T00:29:55.898308+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.898308+0000","last_undegraded":"2023-12-02T00:29:55.898308+0000","last_fullsized":"2023-12-02T00:29:55.898308+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.901881+0000","last_change":"2023-12-02T00:29:55.901881+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.901881+0000","last_undegraded":"2023-12-02T00:29:55.901881+0000","last_fullsized":"2023-12-02T00:29:55.901881+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:55.891930+0000","last_change":"2023-12-02T00:29:55.891930+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.891930+0000","last_undegraded":"2023-12-02T00:29:55.891930+0000","last_fullsized":"2023-12-02T00:29:55.891930+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.901839+0000","last_change":"2023-12-02T00:29:55.901839+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.901839+0000","last_undegraded":"2023-12-02T00:29:55.901839+0000","last_fullsized":"2023-12-02T00:29:55.901839+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:55.891930+0000","last_change":"2023-12-02T00:29:55.891930+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.891930+0000","last_undegraded":"2023-12-02T00:29:55.891930+0000","last_fullsized":"2023-12-02T00:29:55.891930+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.901835+0000","last_change":"2023-12-02T00:29:55.901835+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.901835+0000","last_undegraded":"2023-12-02T00:29:55.901835+0000","last_fullsized":"2023-12-02T00:29:55.901835+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.898934+0000","last_change":"2023-12-02T00:29:55.898934+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.898934+0000","last_undegraded":"2023-12-02T00:29:55.898934+0000","last_fullsized":"2023-12-02T00:29:55.898934+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.899146+0000","last_change":"2023-12-02T00:29:55.899146+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.899146+0000","last_undegraded":"2023-12-02T00:29:55.899146+0000","last_fullsized":"2023-12-02T00:29:55.899146+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.903298+0000","last_change":"2023-12-02T00:29:55.903298+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.903298+0000","last_undegraded":"2023-12-02T00:29:55.903298+0000","last_fullsized":"2023-12-02T00:29:55.903298+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":175,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-12-02T00:29:52.806248+0000","last_change":"2023-12-02T00:28:39.379380+0000","last_active":"2023-12-02T00:29:52.806248+0000","last_peered":"2023-12-02T00:29:52.806248+0000","last_clean":"2023-12-02T00:29:52.806248+0000","last_became_active":"2023-12-02T00:28:39.378255+0000","last_became_peered":"2023-12-02T00:28:39.378255+0000","last_unstale":"2023-12-02T00:29:52.806248+0000","last_undegraded":"2023-12-02T00:29:52.806248+0000","last_fullsized":"2023-12-02T00:29:52.806248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:38.177345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:38.177345+0000","last_clean_scrub_stamp":"2023-12-02T00:28:38.177345+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:21:08.582412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":54,"acting":54,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":721941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.45900000000000002}]},{"osd":1,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.50700000000000001}]},{"osd":2,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.61599999999999999}]},{"osd":3,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.436}]},{"osd":4,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.57899999999999996}]},{"osd":5,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.59799999999999998}]},{"osd":6,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.68100000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":360448,"data_stored":137390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.59699999999999998}]},{"osd":1,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.46500000000000002}]},{"osd":2,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.443},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.497}]},{"osd":3,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.629}]},{"osd":4,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.56799999999999995}]},{"osd":5,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.64800000000000002}]},{"osd":7,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.72999999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":131573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.49099999999999999}]},{"osd":1,"last update":"Sat Dec 2 00:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.41799999999999998}]},{"osd":2,"last update":"Sat Dec 2 00:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.749}]},{"osd":3,"last update":"Sat Dec 2 00:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.63300000000000001}]},{"osd":4,"last update":"Sat Dec 2 00:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.66900000000000004}]},{"osd":6,"last update":"Sat Dec 2 00:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.77800000000000002}]},{"osd":7,"last update":"Sat Dec 2 00:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.76700000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":131573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.58099999999999996}]},{"osd":1,"last update":"Sat Dec 2 00:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.47199999999999998}]},{"osd":2,"last update":"Sat Dec 2 00:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.443},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.64200000000000002}]},{"osd":3,"last update":"Sat Dec 2 00:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.55800000000000005}]},{"osd":5,"last update":"Sat Dec 2 00:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.42199999999999999}]},{"osd":6,"last update":"Sat Dec 2 00:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.629},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.51900000000000002}]},{"osd":7,"last update":"Sat Dec 2 00:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.59999999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31532,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340308,"statfs":{"total":96636764160,"available":96604475392,"internally_reserved":0,"allocated":360448,"data_stored":137390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.28599999999999998}]},{"osd":1,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.36599999999999999}]},{"osd":2,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.30599999999999999}]},{"osd":4,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.52700000000000002}]},{"osd":5,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.64300000000000002}]},{"osd":6,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.58899999999999997}]},{"osd":7,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.45200000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31520,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340320,"statfs":{"total":96636764160,"available":96604487680,"internally_reserved":0,"allocated":344064,"data_stored":130954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.22}]},{"osd":1,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.503,"5min":0.503,"15min":0.503},"last":0.317}]},{"osd":3,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.42399999999999999}]},{"osd":4,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.55700000000000005}]},{"osd":5,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.442},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.59299999999999997}]},{"osd":6,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.52100000000000002}]},{"osd":7,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.72699999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31520,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340320,"statfs":{"total":96636764160,"available":96604487680,"internally_reserved":0,"allocated":348160,"data_stored":131573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.378}]},{"osd":2,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.41499999999999998}]},{"osd":3,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.34399999999999997}]},{"osd":4,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.56999999999999995}]},{"osd":5,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.437}]},{"osd":6,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.54900000000000004}]},{"osd":7,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.45300000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":937984,"data_stored":721322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.58399999999999996}]},{"osd":2,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.71699999999999997}]},{"osd":3,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.67400000000000004}]},{"osd":4,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.74199999999999999}]},{"osd":5,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.69599999999999995}]},{"osd":6,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.82099999999999995}]},{"osd":7,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.78000000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-02T00:30:00.921 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-02T00:30:01.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.061+0000 7f7926bbb640 1 Processor -- start 2023-12-02T00:30:01.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.061+0000 7f7926bbb640 1 -- start start 2023-12-02T00:30:01.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.061+0000 7f7926bbb640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f792015d330 0x7f79200b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:01.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.061+0000 7f7926bbb640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f79200d3900 0x7f79200b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:01.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.061+0000 7f7926bbb640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79200b2b10 con 0x7f79200d2910 2023-12-02T00:30:01.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.061+0000 7f7926bbb640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79200d1590 con 0x7f79200d3900 2023-12-02T00:30:01.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.061+0000 7f7926bbb640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79200d1870 con 0x7f792015d330 2023-12-02T00:30:01.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.062+0000 7f7925158640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f79200d3900 0x7f79200b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:01.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.062+0000 7f791ffff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f79200d2910 0x7f79200d31f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57132/0 (socket says 172.21.15.46:57132) 2023-12-02T00:30:01.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.062+0000 7f791ffff640 1 -- 172.21.15.46:0/4162848785 learned_addr learned my addr 172.21.15.46:0/4162848785 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:30:01.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.062+0000 7f7924957640 1 --2- 172.21.15.46:0/4162848785 >> v2:172.21.15.162:3300/0 conn(0x7f792015d330 0x7f79200b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:01.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.062+0000 7f7925158640 1 -- 172.21.15.46:0/4162848785 >> v2:172.21.15.162:3300/0 conn(0x7f792015d330 msgr2=0x7f79200b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:01.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.062+0000 7f7925158640 1 --2- 172.21.15.46:0/4162848785 >> v2:172.21.15.162:3300/0 conn(0x7f792015d330 0x7f79200b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:01.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.062+0000 7f7925158640 1 -- 172.21.15.46:0/4162848785 >> v1:172.21.15.46:6789/0 conn(0x7f79200d2910 legacy=0x7f79200d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:01.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.062+0000 7f7925158640 1 -- 172.21.15.46:0/4162848785 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79200d1b50 con 0x7f79200d3900 2023-12-02T00:30:01.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.062+0000 7f791f7fe640 1 -- 172.21.15.46:0/4162848785 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3126700498 0 0) 0x7f79200b2b10 con 0x7f79200d2910 2023-12-02T00:30:01.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.063+0000 7f7925158640 1 --2- 172.21.15.46:0/4162848785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79200d3900 0x7f79200b6390 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f79200d0d10 tx=0x7f790802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be7af210796b3ad8 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:01.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.063+0000 7f791f7fe640 1 -- 172.21.15.46:0/4162848785 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790802e070 con 0x7f79200d3900 2023-12-02T00:30:01.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.063+0000 7f791f7fe640 1 -- 172.21.15.46:0/4162848785 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79080315a0 con 0x7f79200d3900 2023-12-02T00:30:01.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.063+0000 7f791f7fe640 1 -- 172.21.15.46:0/4162848785 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7908031930 con 0x7f79200d3900 2023-12-02T00:30:01.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.063+0000 7f7926bbb640 1 -- 172.21.15.46:0/4162848785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79200d3900 msgr2=0x7f79200b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:01.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.063+0000 7f7926bbb640 1 --2- 172.21.15.46:0/4162848785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79200d3900 0x7f79200b6390 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f79200d0d10 tx=0x7f790802b840 comp rx=0 tx=0).stop 2023-12-02T00:30:01.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.063+0000 7f7926bbb640 1 -- 172.21.15.46:0/4162848785 shutdown_connections 2023-12-02T00:30:01.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.063+0000 7f7926bbb640 1 --2- 172.21.15.46:0/4162848785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79200d3900 0x7f79200b6390 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:01.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.063+0000 7f7926bbb640 1 --2- 172.21.15.46:0/4162848785 >> v2:172.21.15.162:3300/0 conn(0x7f792015d330 0x7f79200b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:01.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.063+0000 7f7926bbb640 1 -- 172.21.15.46:0/4162848785 >> 172.21.15.46:0/4162848785 conn(0x7f7920072020 msgr2=0x7f7920072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:01.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.063+0000 7f7926bbb640 1 -- 172.21.15.46:0/4162848785 shutdown_connections 2023-12-02T00:30:01.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.063+0000 7f7926bbb640 1 -- 172.21.15.46:0/4162848785 wait complete. 2023-12-02T00:30:01.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.064+0000 7f7926bbb640 1 Processor -- start 2023-12-02T00:30:01.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.065+0000 7f7926bbb640 1 -- start start 2023-12-02T00:30:01.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.065+0000 7f7926bbb640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79200d2910 0x7f7920152990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:01.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.065+0000 7f7926bbb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79200d3900 0x7f79201f2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:01.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.065+0000 7f7926bbb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f792015d330 0x7f79200c8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:01.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.065+0000 7f7926bbb640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79201f3200 con 0x7f792015d330 2023-12-02T00:30:01.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.065+0000 7f7926bbb640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79200c95d0 con 0x7f79200d2910 2023-12-02T00:30:01.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.065+0000 7f7926bbb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f792015eec0 con 0x7f79200d3900 2023-12-02T00:30:01.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.065+0000 7f7925158640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f792015d330 0x7f79200c8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:01.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.066+0000 7f7925158640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f792015d330 0x7f79200c8ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50826/0 (socket says 172.21.15.46:50826) 2023-12-02T00:30:01.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.066+0000 7f7925158640 1 -- 172.21.15.46:0/3042353650 learned_addr learned my addr 172.21.15.46:0/3042353650 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:01.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.066+0000 7f7924957640 1 --2- 172.21.15.46:0/3042353650 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79200d2910 0x7f7920152990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:01.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.066+0000 7f791ffff640 1 --2- 172.21.15.46:0/3042353650 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79200d3900 0x7f79201f2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:01.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.066+0000 7f7925158640 1 -- 172.21.15.46:0/3042353650 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79200d3900 msgr2=0x7f79201f2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:01.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.066+0000 7f7925158640 1 --2- 172.21.15.46:0/3042353650 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79200d3900 0x7f79201f2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:01.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.066+0000 7f7925158640 1 -- 172.21.15.46:0/3042353650 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79200d2910 msgr2=0x7f7920152990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:01.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.066+0000 7f7925158640 1 --2- 172.21.15.46:0/3042353650 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79200d2910 0x7f7920152990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:01.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.066+0000 7f7925158640 1 -- 172.21.15.46:0/3042353650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7908009400 con 0x7f792015d330 2023-12-02T00:30:01.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.066+0000 7f791ffff640 1 --2- 172.21.15.46:0/3042353650 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79200d3900 0x7f79201f2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:30:01.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.066+0000 7f7925158640 1 --2- 172.21.15.46:0/3042353650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f792015d330 0x7f79200c8ab0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f790802be50 tx=0x7f79080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:01.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.066+0000 7f791d7fa640 1 -- 172.21.15.46:0/3042353650 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7908032840 con 0x7f792015d330 2023-12-02T00:30:01.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.066+0000 7f791d7fa640 1 -- 172.21.15.46:0/3042353650 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79080329a0 con 0x7f792015d330 2023-12-02T00:30:01.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.066+0000 7f7926bbb640 1 -- 172.21.15.46:0/3042353650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f792015e590 con 0x7f792015d330 2023-12-02T00:30:01.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.066+0000 7f791d7fa640 1 -- 172.21.15.46:0/3042353650 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7908032d30 con 0x7f792015d330 2023-12-02T00:30:01.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.066+0000 7f7926bbb640 1 -- 172.21.15.46:0/3042353650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f792015eb50 con 0x7f792015d330 2023-12-02T00:30:01.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.067+0000 7f7926bbb640 1 -- 172.21.15.46:0/3042353650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7920069400 con 0x7f792015d330 2023-12-02T00:30:01.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.068+0000 7f791d7fa640 1 -- 172.21.15.46:0/3042353650 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f790804e430 con 0x7f792015d330 2023-12-02T00:30:01.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.068+0000 7f791d7fa640 1 --2- 172.21.15.46:0/3042353650 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f790006ae70 0x7f790006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:01.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.068+0000 7f791d7fa640 1 -- 172.21.15.46:0/3042353650 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f79080ab460 con 0x7f792015d330 2023-12-02T00:30:01.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.068+0000 7f7924957640 1 --2- 172.21.15.46:0/3042353650 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f790006ae70 0x7f790006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:01.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.069+0000 7f7924957640 1 --2- 172.21.15.46:0/3042353650 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f790006ae70 0x7f790006d330 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7918003b80 tx=0x7f79180098d0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:01.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.073+0000 7f791d7fa640 1 -- 172.21.15.46:0/3042353650 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f79080765a0 con 0x7f792015d330 2023-12-02T00:30:01.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.243+0000 7f7926bbb640 1 -- 172.21.15.46:0/3042353650 --> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7920072400 con 0x7f790006ae70 2023-12-02T00:30:01.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.251+0000 7f791d7fa640 1 -- 172.21.15.46:0/3042353650 <== mgr.4108 v2:172.21.15.162:6832/3248644722 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+276860 (secure 0 0 0) 0x7f7920072400 con 0x7f790006ae70 2023-12-02T00:30:01.254 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:30:01.256 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-12-02T00:30:01.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.253+0000 7f7926bbb640 1 -- 172.21.15.46:0/3042353650 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f790006ae70 msgr2=0x7f790006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:01.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.253+0000 7f7926bbb640 1 --2- 172.21.15.46:0/3042353650 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f790006ae70 0x7f790006d330 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7918003b80 tx=0x7f79180098d0 comp rx=0 tx=0).stop 2023-12-02T00:30:01.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.253+0000 7f7926bbb640 1 -- 172.21.15.46:0/3042353650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f792015d330 msgr2=0x7f79200c8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:01.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.253+0000 7f7926bbb640 1 --2- 172.21.15.46:0/3042353650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f792015d330 0x7f79200c8ab0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f790802be50 tx=0x7f79080046c0 comp rx=0 tx=0).stop 2023-12-02T00:30:01.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.254+0000 7f7926bbb640 1 -- 172.21.15.46:0/3042353650 shutdown_connections 2023-12-02T00:30:01.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.254+0000 7f7926bbb640 1 --2- 172.21.15.46:0/3042353650 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f790006ae70 0x7f790006d330 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:01.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.254+0000 7f7926bbb640 1 --2- 172.21.15.46:0/3042353650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f792015d330 0x7f79200c8ab0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:01.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.254+0000 7f7926bbb640 1 --2- 172.21.15.46:0/3042353650 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79200d3900 0x7f79201f2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:01.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.254+0000 7f7926bbb640 1 --2- 172.21.15.46:0/3042353650 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79200d2910 0x7f7920152990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:01.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.254+0000 7f7926bbb640 1 -- 172.21.15.46:0/3042353650 >> 172.21.15.46:0/3042353650 conn(0x7f7920072020 msgr2=0x7f79200d0050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:01.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.254+0000 7f7926bbb640 1 -- 172.21.15.46:0/3042353650 shutdown_connections 2023-12-02T00:30:01.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.254+0000 7f7926bbb640 1 -- 172.21.15.46:0/3042353650 wait complete. 2023-12-02T00:30:01.265 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2023-12-02T00:30:01.175151+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236708,"kb_used_data":3896,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738012,"statfs":{"total":773094113280,"available":772851724288,"internally_reserved":0,"allocated":3989504,"data_stored":2243716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.278482"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.903298+0000","last_change":"2023-12-02T00:29:55.903298+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.903298+0000","last_undegraded":"2023-12-02T00:29:55.903298+0000","last_fullsized":"2023-12-02T00:29:55.903298+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:55.891930+0000","last_change":"2023-12-02T00:29:55.891930+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.891930+0000","last_undegraded":"2023-12-02T00:29:55.891930+0000","last_fullsized":"2023-12-02T00:29:55.891930+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.900712+0000","last_change":"2023-12-02T00:29:55.900712+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.900712+0000","last_undegraded":"2023-12-02T00:29:55.900712+0000","last_fullsized":"2023-12-02T00:29:55.900712+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.901454+0000","last_change":"2023-12-02T00:29:55.901454+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.901454+0000","last_undegraded":"2023-12-02T00:29:55.901454+0000","last_fullsized":"2023-12-02T00:29:55.901454+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.900095+0000","last_change":"2023-12-02T00:29:55.900095+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.900095+0000","last_undegraded":"2023-12-02T00:29:55.900095+0000","last_fullsized":"2023-12-02T00:29:55.900095+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.898451+0000","last_change":"2023-12-02T00:29:55.898451+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.898451+0000","last_undegraded":"2023-12-02T00:29:55.898451+0000","last_fullsized":"2023-12-02T00:29:55.898451+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.900139+0000","last_change":"2023-12-02T00:29:55.900139+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.900139+0000","last_undegraded":"2023-12-02T00:29:55.900139+0000","last_fullsized":"2023-12-02T00:29:55.900139+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.898680+0000","last_change":"2023-12-02T00:29:55.898680+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.898680+0000","last_undegraded":"2023-12-02T00:29:55.898680+0000","last_fullsized":"2023-12-02T00:29:55.898680+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.899737+0000","last_change":"2023-12-02T00:29:55.899737+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.899737+0000","last_undegraded":"2023-12-02T00:29:55.899737+0000","last_fullsized":"2023-12-02T00:29:55.899737+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:55.891930+0000","last_change":"2023-12-02T00:29:55.891930+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.891930+0000","last_undegraded":"2023-12-02T00:29:55.891930+0000","last_fullsized":"2023-12-02T00:29:55.891930+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.898306+0000","last_change":"2023-12-02T00:29:55.898306+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.898306+0000","last_undegraded":"2023-12-02T00:29:55.898306+0000","last_fullsized":"2023-12-02T00:29:55.898306+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.899642+0000","last_change":"2023-12-02T00:29:55.899642+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.899642+0000","last_undegraded":"2023-12-02T00:29:55.899642+0000","last_fullsized":"2023-12-02T00:29:55.899642+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.899635+0000","last_change":"2023-12-02T00:29:55.899635+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.899635+0000","last_undegraded":"2023-12-02T00:29:55.899635+0000","last_fullsized":"2023-12-02T00:29:55.899635+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.902337+0000","last_change":"2023-12-02T00:29:55.902337+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.902337+0000","last_undegraded":"2023-12-02T00:29:55.902337+0000","last_fullsized":"2023-12-02T00:29:55.902337+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.899585+0000","last_change":"2023-12-02T00:29:55.899585+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.899585+0000","last_undegraded":"2023-12-02T00:29:55.899585+0000","last_fullsized":"2023-12-02T00:29:55.899585+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.899587+0000","last_change":"2023-12-02T00:29:55.899587+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.899587+0000","last_undegraded":"2023-12-02T00:29:55.899587+0000","last_fullsized":"2023-12-02T00:29:55.899587+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.902333+0000","last_change":"2023-12-02T00:29:55.902333+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.902333+0000","last_undegraded":"2023-12-02T00:29:55.902333+0000","last_fullsized":"2023-12-02T00:29:55.902333+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.898944+0000","last_change":"2023-12-02T00:29:55.898944+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.898944+0000","last_undegraded":"2023-12-02T00:29:55.898944+0000","last_fullsized":"2023-12-02T00:29:55.898944+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.899297+0000","last_change":"2023-12-02T00:29:55.899297+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.899297+0000","last_undegraded":"2023-12-02T00:29:55.899297+0000","last_fullsized":"2023-12-02T00:29:55.899297+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.898740+0000","last_change":"2023-12-02T00:29:55.898740+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.898740+0000","last_undegraded":"2023-12-02T00:29:55.898740+0000","last_fullsized":"2023-12-02T00:29:55.898740+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:55.891930+0000","last_change":"2023-12-02T00:29:55.891930+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.891930+0000","last_undegraded":"2023-12-02T00:29:55.891930+0000","last_fullsized":"2023-12-02T00:29:55.891930+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.903215+0000","last_change":"2023-12-02T00:29:55.903215+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.903215+0000","last_undegraded":"2023-12-02T00:29:55.903215+0000","last_fullsized":"2023-12-02T00:29:55.903215+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.900528+0000","last_change":"2023-12-02T00:29:55.900528+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.900528+0000","last_undegraded":"2023-12-02T00:29:55.900528+0000","last_fullsized":"2023-12-02T00:29:55.900528+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.898308+0000","last_change":"2023-12-02T00:29:55.898308+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.898308+0000","last_undegraded":"2023-12-02T00:29:55.898308+0000","last_fullsized":"2023-12-02T00:29:55.898308+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.901881+0000","last_change":"2023-12-02T00:29:55.901881+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.901881+0000","last_undegraded":"2023-12-02T00:29:55.901881+0000","last_fullsized":"2023-12-02T00:29:55.901881+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:55.891930+0000","last_change":"2023-12-02T00:29:55.891930+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.891930+0000","last_undegraded":"2023-12-02T00:29:55.891930+0000","last_fullsized":"2023-12-02T00:29:55.891930+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.901839+0000","last_change":"2023-12-02T00:29:55.901839+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.901839+0000","last_undegraded":"2023-12-02T00:29:55.901839+0000","last_fullsized":"2023-12-02T00:29:55.901839+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:55.891930+0000","last_change":"2023-12-02T00:29:55.891930+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.891930+0000","last_undegraded":"2023-12-02T00:29:55.891930+0000","last_fullsized":"2023-12-02T00:29:55.891930+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.901835+0000","last_change":"2023-12-02T00:29:55.901835+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.901835+0000","last_undegraded":"2023-12-02T00:29:55.901835+0000","last_fullsized":"2023-12-02T00:29:55.901835+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.898934+0000","last_change":"2023-12-02T00:29:55.898934+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.898934+0000","last_undegraded":"2023-12-02T00:29:55.898934+0000","last_fullsized":"2023-12-02T00:29:55.898934+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.899146+0000","last_change":"2023-12-02T00:29:55.899146+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.899146+0000","last_undegraded":"2023-12-02T00:29:55.899146+0000","last_fullsized":"2023-12-02T00:29:55.899146+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.903298+0000","last_change":"2023-12-02T00:29:55.903298+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.903298+0000","last_undegraded":"2023-12-02T00:29:55.903298+0000","last_fullsized":"2023-12-02T00:29:55.903298+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":175,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-12-02T00:29:52.806248+0000","last_change":"2023-12-02T00:28:39.379380+0000","last_active":"2023-12-02T00:29:52.806248+0000","last_peered":"2023-12-02T00:29:52.806248+0000","last_clean":"2023-12-02T00:29:52.806248+0000","last_became_active":"2023-12-02T00:28:39.378255+0000","last_became_peered":"2023-12-02T00:28:39.378255+0000","last_unstale":"2023-12-02T00:29:52.806248+0000","last_undegraded":"2023-12-02T00:29:52.806248+0000","last_fullsized":"2023-12-02T00:29:52.806248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:38.177345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:38.177345+0000","last_clean_scrub_stamp":"2023-12-02T00:28:38.177345+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:21:08.582412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":54,"acting":54,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":721941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.45900000000000002}]},{"osd":1,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.50700000000000001}]},{"osd":2,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.61599999999999999}]},{"osd":3,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.436}]},{"osd":4,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.57899999999999996}]},{"osd":5,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.59799999999999998}]},{"osd":6,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.68100000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":360448,"data_stored":137390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.59699999999999998}]},{"osd":1,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.46500000000000002}]},{"osd":2,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.443},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.497}]},{"osd":3,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.629}]},{"osd":4,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.56799999999999995}]},{"osd":5,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.64800000000000002}]},{"osd":7,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.72999999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":131573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.49099999999999999}]},{"osd":1,"last update":"Sat Dec 2 00:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.41799999999999998}]},{"osd":2,"last update":"Sat Dec 2 00:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.749}]},{"osd":3,"last update":"Sat Dec 2 00:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.63300000000000001}]},{"osd":4,"last update":"Sat Dec 2 00:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.66900000000000004}]},{"osd":6,"last update":"Sat Dec 2 00:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.77800000000000002}]},{"osd":7,"last update":"Sat Dec 2 00:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.76700000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":131573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.58099999999999996}]},{"osd":1,"last update":"Sat Dec 2 00:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.47199999999999998}]},{"osd":2,"last update":"Sat Dec 2 00:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.443},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.64200000000000002}]},{"osd":3,"last update":"Sat Dec 2 00:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.55800000000000005}]},{"osd":5,"last update":"Sat Dec 2 00:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.42199999999999999}]},{"osd":6,"last update":"Sat Dec 2 00:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.629},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.51900000000000002}]},{"osd":7,"last update":"Sat Dec 2 00:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.59999999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31532,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340308,"statfs":{"total":96636764160,"available":96604475392,"internally_reserved":0,"allocated":360448,"data_stored":137390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.28599999999999998}]},{"osd":1,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.36599999999999999}]},{"osd":2,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.30599999999999999}]},{"osd":4,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.52700000000000002}]},{"osd":5,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.64300000000000002}]},{"osd":6,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.58899999999999997}]},{"osd":7,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.45200000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31520,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340320,"statfs":{"total":96636764160,"available":96604487680,"internally_reserved":0,"allocated":344064,"data_stored":130954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.22}]},{"osd":1,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.503,"5min":0.503,"15min":0.503},"last":0.317}]},{"osd":3,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.42399999999999999}]},{"osd":4,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.55700000000000005}]},{"osd":5,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.442},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.59299999999999997}]},{"osd":6,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.52100000000000002}]},{"osd":7,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.72699999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31520,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340320,"statfs":{"total":96636764160,"available":96604487680,"internally_reserved":0,"allocated":348160,"data_stored":131573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.378}]},{"osd":2,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.41499999999999998}]},{"osd":3,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.34399999999999997}]},{"osd":4,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.56999999999999995}]},{"osd":5,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.437}]},{"osd":6,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.54900000000000004}]},{"osd":7,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.45300000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":937984,"data_stored":721322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.58399999999999996}]},{"osd":2,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.71699999999999997}]},{"osd":3,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.67400000000000004}]},{"osd":4,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.74199999999999999}]},{"osd":5,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.69599999999999995}]},{"osd":6,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.82099999999999995}]},{"osd":7,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.78000000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-02T00:30:01.270 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-02T00:30:01.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.411+0000 7fb4799f1640 1 Processor -- start 2023-12-02T00:30:01.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.412+0000 7fb4799f1640 1 -- start start 2023-12-02T00:30:01.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.412+0000 7fb4799f1640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb474142270 0x7fb4741426c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:01.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.412+0000 7fb4799f1640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb474161d20 con 0x7fb47406ea50 2023-12-02T00:30:01.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.412+0000 7fb4799f1640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb47415e760 con 0x7fb474160fb0 2023-12-02T00:30:01.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.412+0000 7fb4799f1640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4740d1ee0 con 0x7fb474142270 2023-12-02T00:30:01.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.412+0000 7fb472ffd640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb47406ea50 0x7fb47406ee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57142/0 (socket says 172.21.15.46:57142) 2023-12-02T00:30:01.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.412+0000 7fb472ffd640 1 -- 172.21.15.46:0/3453450673 learned_addr learned my addr 172.21.15.46:0/3453450673 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:30:01.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.413+0000 7fb4737fe640 1 --2- 172.21.15.46:0/3453450673 >> v2:172.21.15.162:3300/0 conn(0x7fb474142270 0x7fb4741426c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:01.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.413+0000 7fb471ffb640 1 -- 172.21.15.46:0/3453450673 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 370295990 0 0) 0x7fb474161d20 con 0x7fb47406ea50 2023-12-02T00:30:01.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.413+0000 7fb471ffb640 1 -- 172.21.15.46:0/3453450673 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb45c003520 con 0x7fb47406ea50 2023-12-02T00:30:01.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.413+0000 7fb471ffb640 1 -- 172.21.15.46:0/3453450673 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2244047689 0 0) 0x7fb45c003520 con 0x7fb47406ea50 2023-12-02T00:30:01.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.413+0000 7fb471ffb640 1 -- 172.21.15.46:0/3453450673 >> v2:172.21.15.162:3300/0 conn(0x7fb474142270 msgr2=0x7fb4741426c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:01.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.413+0000 7fb471ffb640 1 --2- 172.21.15.46:0/3453450673 >> v2:172.21.15.162:3300/0 conn(0x7fb474142270 0x7fb4741426c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:01.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.413+0000 7fb471ffb640 1 -- 172.21.15.46:0/3453450673 >> v1:172.21.15.162:6789/0 conn(0x7fb474160fb0 legacy=0x7fb474161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:01.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.413+0000 7fb471ffb640 1 -- 172.21.15.46:0/3453450673 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb47415d9a0 con 0x7fb47406ea50 2023-12-02T00:30:01.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.413+0000 7fb471ffb640 1 -- 172.21.15.46:0/3453450673 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7fb4640019f0 con 0x7fb47406ea50 2023-12-02T00:30:01.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.413+0000 7fb471ffb640 1 -- 172.21.15.46:0/3453450673 >> v1:172.21.15.46:6789/0 conn(0x7fb47406ea50 legacy=0x7fb47406ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:01.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.413+0000 7fb471ffb640 1 --2- 172.21.15.46:0/3453450673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb45c003970 0x7fb45c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:01.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.413+0000 7fb471ffb640 1 --2- 172.21.15.46:0/3453450673 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb45c0047c0 0x7fb45c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:01.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.413+0000 7fb471ffb640 1 --2- 172.21.15.46:0/3453450673 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb45c005180 0x7fb45c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:01.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.413+0000 7fb471ffb640 1 -- 172.21.15.46:0/3453450673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4740d1ee0 con 0x7fb45c003970 2023-12-02T00:30:01.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.413+0000 7fb471ffb640 1 -- 172.21.15.46:0/3453450673 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb45c00c120 con 0x7fb45c0047c0 2023-12-02T00:30:01.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.413+0000 7fb471ffb640 1 -- 172.21.15.46:0/3453450673 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb45c00c2e0 con 0x7fb45c005180 2023-12-02T00:30:01.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.413+0000 7fb472ffd640 1 --2- 172.21.15.46:0/3453450673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb45c003970 0x7fb45c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:01.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.413+0000 7fb4727fc640 1 --2- 172.21.15.46:0/3453450673 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb45c0047c0 0x7fb45c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:01.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.413+0000 7fb4737fe640 1 --2- 172.21.15.46:0/3453450673 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb45c005180 0x7fb45c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:01.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.414+0000 7fb472ffd640 1 -- 172.21.15.46:0/3453450673 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb45c005180 msgr2=0x7fb45c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:01.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.414+0000 7fb472ffd640 1 --2- 172.21.15.46:0/3453450673 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb45c005180 0x7fb45c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:01.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.414+0000 7fb472ffd640 1 -- 172.21.15.46:0/3453450673 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb45c0047c0 msgr2=0x7fb45c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:01.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.414+0000 7fb472ffd640 1 --2- 172.21.15.46:0/3453450673 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb45c0047c0 0x7fb45c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:01.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.414+0000 7fb472ffd640 1 -- 172.21.15.46:0/3453450673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb45c003520 con 0x7fb45c003970 2023-12-02T00:30:01.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.414+0000 7fb4737fe640 1 --2- 172.21.15.46:0/3453450673 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb45c005180 0x7fb45c00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:30:01.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.414+0000 7fb4737fe640 1 -- 172.21.15.46:0/3453450673 reap_dead start 2023-12-02T00:30:01.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.414+0000 7fb472ffd640 1 --2- 172.21.15.46:0/3453450673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb45c003970 0x7fb45c003d90 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb46400a830 tx=0x7fb45c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8470bafa78468f4e server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:01.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.414+0000 7fb471ffb640 1 -- 172.21.15.46:0/3453450673 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4640072d0 con 0x7fb45c003970 2023-12-02T00:30:01.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.414+0000 7fb471ffb640 1 -- 172.21.15.46:0/3453450673 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb46402f070 con 0x7fb45c003970 2023-12-02T00:30:01.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.414+0000 7fb471ffb640 1 -- 172.21.15.46:0/3453450673 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4640076f0 con 0x7fb45c003970 2023-12-02T00:30:01.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.415+0000 7fb4799f1640 1 -- 172.21.15.46:0/3453450673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb45c003970 msgr2=0x7fb45c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:01.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.415+0000 7fb4799f1640 1 --2- 172.21.15.46:0/3453450673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb45c003970 0x7fb45c003d90 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb46400a830 tx=0x7fb45c0057f0 comp rx=0 tx=0).stop 2023-12-02T00:30:01.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.415+0000 7fb4799f1640 1 -- 172.21.15.46:0/3453450673 shutdown_connections 2023-12-02T00:30:01.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.415+0000 7fb4799f1640 1 --2- 172.21.15.46:0/3453450673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb45c003970 0x7fb45c003d90 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:01.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.415+0000 7fb4799f1640 1 -- 172.21.15.46:0/3453450673 >> 172.21.15.46:0/3453450673 conn(0x7fb47406a6b0 msgr2=0x7fb47406aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:01.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.415+0000 7fb4799f1640 1 -- 172.21.15.46:0/3453450673 shutdown_connections 2023-12-02T00:30:01.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.415+0000 7fb4799f1640 1 -- 172.21.15.46:0/3453450673 wait complete. 2023-12-02T00:30:01.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.416+0000 7fb4799f1640 1 Processor -- start 2023-12-02T00:30:01.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.416+0000 7fb4799f1640 1 -- start start 2023-12-02T00:30:01.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.417+0000 7fb4799f1640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb45c003970 0x7fb474156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:01.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.417+0000 7fb4799f1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4740b4750 0x7fb4740b2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:01.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.417+0000 7fb4799f1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb47415adf0 0x7fb47415c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:01.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.417+0000 7fb4799f1640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4740b42c0 con 0x7fb4740b4750 2023-12-02T00:30:01.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.417+0000 7fb4799f1640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb474157450 con 0x7fb45c003970 2023-12-02T00:30:01.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.417+0000 7fb4799f1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4740b36b0 con 0x7fb47415adf0 2023-12-02T00:30:01.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.417+0000 7fb4727fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4740b4750 0x7fb4740b2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:01.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.417+0000 7fb4727fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4740b4750 0x7fb4740b2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50852/0 (socket says 172.21.15.46:50852) 2023-12-02T00:30:01.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.417+0000 7fb4727fc640 1 -- 172.21.15.46:0/408029133 learned_addr learned my addr 172.21.15.46:0/408029133 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:01.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.417+0000 7fb4727fc640 1 -- 172.21.15.46:0/408029133 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb47415adf0 msgr2=0x7fb47415c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:01.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.417+0000 7fb4727fc640 1 --2- 172.21.15.46:0/408029133 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb47415adf0 0x7fb47415c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:01.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.417+0000 7fb4727fc640 1 -- 172.21.15.46:0/408029133 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb45c003970 msgr2=0x7fb474156f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:01.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.417+0000 7fb472ffd640 1 --2- 172.21.15.46:0/408029133 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb45c003970 0x7fb474156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:01.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.418+0000 7fb4727fc640 1 --2- 172.21.15.46:0/408029133 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb45c003970 0x7fb474156f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:01.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.418+0000 7fb4727fc640 1 -- 172.21.15.46:0/408029133 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4740b3990 con 0x7fb4740b4750 2023-12-02T00:30:01.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.418+0000 7fb472ffd640 1 --2- 172.21.15.46:0/408029133 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb45c003970 0x7fb474156f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:30:01.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.418+0000 7fb4727fc640 1 --2- 172.21.15.46:0/408029133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4740b4750 0x7fb4740b2df0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb46800f570 tx=0x7fb46800fa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:01.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.418+0000 7fb45bfff640 1 -- 172.21.15.46:0/408029133 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4680052e0 con 0x7fb4740b4750 2023-12-02T00:30:01.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.418+0000 7fb45bfff640 1 -- 172.21.15.46:0/408029133 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb468005440 con 0x7fb4740b4750 2023-12-02T00:30:01.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.418+0000 7fb4799f1640 1 -- 172.21.15.46:0/408029133 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4740b3c70 con 0x7fb4740b4750 2023-12-02T00:30:01.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.418+0000 7fb45bfff640 1 -- 172.21.15.46:0/408029133 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb468003810 con 0x7fb4740b4750 2023-12-02T00:30:01.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.418+0000 7fb4799f1640 1 -- 172.21.15.46:0/408029133 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4740b3330 con 0x7fb4740b4750 2023-12-02T00:30:01.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.419+0000 7fb4799f1640 1 -- 172.21.15.46:0/408029133 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4740b4430 con 0x7fb4740b4750 2023-12-02T00:30:01.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.419+0000 7fb45bfff640 1 -- 172.21.15.46:0/408029133 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb468003970 con 0x7fb4740b4750 2023-12-02T00:30:01.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.420+0000 7fb45bfff640 1 --2- 172.21.15.46:0/408029133 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb46006ae70 0x7fb46006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:01.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.420+0000 7fb45bfff640 1 -- 172.21.15.46:0/408029133 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb468089430 con 0x7fb4740b4750 2023-12-02T00:30:01.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.420+0000 7fb472ffd640 1 --2- 172.21.15.46:0/408029133 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb46006ae70 0x7fb46006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:01.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.420+0000 7fb472ffd640 1 --2- 172.21.15.46:0/408029133 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb46006ae70 0x7fb46006d330 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb464034000 tx=0x7fb464007000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:01.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.424+0000 7fb45bfff640 1 -- 172.21.15.46:0/408029133 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb4680544e0 con 0x7fb4740b4750 2023-12-02T00:30:01.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.595+0000 7fb4799f1640 1 -- 172.21.15.46:0/408029133 --> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb474073c20 con 0x7fb46006ae70 2023-12-02T00:30:01.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.602+0000 7fb45bfff640 1 -- 172.21.15.46:0/408029133 <== mgr.4108 v2:172.21.15.162:6832/3248644722 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+276860 (secure 0 0 0) 0x7fb474073c20 con 0x7fb46006ae70 2023-12-02T00:30:01.606 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:30:01.608 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-12-02T00:30:01.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.605+0000 7fb4799f1640 1 -- 172.21.15.46:0/408029133 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb46006ae70 msgr2=0x7fb46006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:01.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.605+0000 7fb4799f1640 1 --2- 172.21.15.46:0/408029133 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb46006ae70 0x7fb46006d330 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb464034000 tx=0x7fb464007000 comp rx=0 tx=0).stop 2023-12-02T00:30:01.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.605+0000 7fb4799f1640 1 -- 172.21.15.46:0/408029133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4740b4750 msgr2=0x7fb4740b2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:01.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.605+0000 7fb4799f1640 1 --2- 172.21.15.46:0/408029133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4740b4750 0x7fb4740b2df0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb46800f570 tx=0x7fb46800fa30 comp rx=0 tx=0).stop 2023-12-02T00:30:01.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.605+0000 7fb4799f1640 1 -- 172.21.15.46:0/408029133 shutdown_connections 2023-12-02T00:30:01.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.605+0000 7fb4799f1640 1 --2- 172.21.15.46:0/408029133 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb46006ae70 0x7fb46006d330 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:01.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.605+0000 7fb4799f1640 1 --2- 172.21.15.46:0/408029133 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb47415adf0 0x7fb47415c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:01.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.605+0000 7fb4799f1640 1 --2- 172.21.15.46:0/408029133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4740b4750 0x7fb4740b2df0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:01.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.605+0000 7fb4799f1640 1 --2- 172.21.15.46:0/408029133 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb45c003970 0x7fb474156f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:01.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.605+0000 7fb4799f1640 1 -- 172.21.15.46:0/408029133 >> 172.21.15.46:0/408029133 conn(0x7fb47406a6b0 msgr2=0x7fb47413ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:01.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.606+0000 7fb4799f1640 1 -- 172.21.15.46:0/408029133 shutdown_connections 2023-12-02T00:30:01.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.606+0000 7fb4799f1640 1 -- 172.21.15.46:0/408029133 wait complete. 2023-12-02T00:30:01.618 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2023-12-02T00:30:01.175151+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236708,"kb_used_data":3896,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738012,"statfs":{"total":773094113280,"available":772851724288,"internally_reserved":0,"allocated":3989504,"data_stored":2243716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.278482"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.903298+0000","last_change":"2023-12-02T00:29:55.903298+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.903298+0000","last_undegraded":"2023-12-02T00:29:55.903298+0000","last_fullsized":"2023-12-02T00:29:55.903298+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:55.891930+0000","last_change":"2023-12-02T00:29:55.891930+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.891930+0000","last_undegraded":"2023-12-02T00:29:55.891930+0000","last_fullsized":"2023-12-02T00:29:55.891930+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.900712+0000","last_change":"2023-12-02T00:29:55.900712+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.900712+0000","last_undegraded":"2023-12-02T00:29:55.900712+0000","last_fullsized":"2023-12-02T00:29:55.900712+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.901454+0000","last_change":"2023-12-02T00:29:55.901454+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.901454+0000","last_undegraded":"2023-12-02T00:29:55.901454+0000","last_fullsized":"2023-12-02T00:29:55.901454+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.900095+0000","last_change":"2023-12-02T00:29:55.900095+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.900095+0000","last_undegraded":"2023-12-02T00:29:55.900095+0000","last_fullsized":"2023-12-02T00:29:55.900095+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.898451+0000","last_change":"2023-12-02T00:29:55.898451+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.898451+0000","last_undegraded":"2023-12-02T00:29:55.898451+0000","last_fullsized":"2023-12-02T00:29:55.898451+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.900139+0000","last_change":"2023-12-02T00:29:55.900139+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.900139+0000","last_undegraded":"2023-12-02T00:29:55.900139+0000","last_fullsized":"2023-12-02T00:29:55.900139+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.898680+0000","last_change":"2023-12-02T00:29:55.898680+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.898680+0000","last_undegraded":"2023-12-02T00:29:55.898680+0000","last_fullsized":"2023-12-02T00:29:55.898680+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.899737+0000","last_change":"2023-12-02T00:29:55.899737+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.899737+0000","last_undegraded":"2023-12-02T00:29:55.899737+0000","last_fullsized":"2023-12-02T00:29:55.899737+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:55.891930+0000","last_change":"2023-12-02T00:29:55.891930+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.891930+0000","last_undegraded":"2023-12-02T00:29:55.891930+0000","last_fullsized":"2023-12-02T00:29:55.891930+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.898306+0000","last_change":"2023-12-02T00:29:55.898306+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.898306+0000","last_undegraded":"2023-12-02T00:29:55.898306+0000","last_fullsized":"2023-12-02T00:29:55.898306+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.899642+0000","last_change":"2023-12-02T00:29:55.899642+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.899642+0000","last_undegraded":"2023-12-02T00:29:55.899642+0000","last_fullsized":"2023-12-02T00:29:55.899642+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.899635+0000","last_change":"2023-12-02T00:29:55.899635+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.899635+0000","last_undegraded":"2023-12-02T00:29:55.899635+0000","last_fullsized":"2023-12-02T00:29:55.899635+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.902337+0000","last_change":"2023-12-02T00:29:55.902337+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.902337+0000","last_undegraded":"2023-12-02T00:29:55.902337+0000","last_fullsized":"2023-12-02T00:29:55.902337+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.899585+0000","last_change":"2023-12-02T00:29:55.899585+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.899585+0000","last_undegraded":"2023-12-02T00:29:55.899585+0000","last_fullsized":"2023-12-02T00:29:55.899585+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.899587+0000","last_change":"2023-12-02T00:29:55.899587+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.899587+0000","last_undegraded":"2023-12-02T00:29:55.899587+0000","last_fullsized":"2023-12-02T00:29:55.899587+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.902333+0000","last_change":"2023-12-02T00:29:55.902333+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.902333+0000","last_undegraded":"2023-12-02T00:29:55.902333+0000","last_fullsized":"2023-12-02T00:29:55.902333+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.898944+0000","last_change":"2023-12-02T00:29:55.898944+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.898944+0000","last_undegraded":"2023-12-02T00:29:55.898944+0000","last_fullsized":"2023-12-02T00:29:55.898944+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.899297+0000","last_change":"2023-12-02T00:29:55.899297+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.899297+0000","last_undegraded":"2023-12-02T00:29:55.899297+0000","last_fullsized":"2023-12-02T00:29:55.899297+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.898740+0000","last_change":"2023-12-02T00:29:55.898740+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.898740+0000","last_undegraded":"2023-12-02T00:29:55.898740+0000","last_fullsized":"2023-12-02T00:29:55.898740+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:55.891930+0000","last_change":"2023-12-02T00:29:55.891930+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.891930+0000","last_undegraded":"2023-12-02T00:29:55.891930+0000","last_fullsized":"2023-12-02T00:29:55.891930+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.903215+0000","last_change":"2023-12-02T00:29:55.903215+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.903215+0000","last_undegraded":"2023-12-02T00:29:55.903215+0000","last_fullsized":"2023-12-02T00:29:55.903215+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.900528+0000","last_change":"2023-12-02T00:29:55.900528+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.900528+0000","last_undegraded":"2023-12-02T00:29:55.900528+0000","last_fullsized":"2023-12-02T00:29:55.900528+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.898308+0000","last_change":"2023-12-02T00:29:55.898308+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.898308+0000","last_undegraded":"2023-12-02T00:29:55.898308+0000","last_fullsized":"2023-12-02T00:29:55.898308+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.901881+0000","last_change":"2023-12-02T00:29:55.901881+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.901881+0000","last_undegraded":"2023-12-02T00:29:55.901881+0000","last_fullsized":"2023-12-02T00:29:55.901881+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:55.891930+0000","last_change":"2023-12-02T00:29:55.891930+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.891930+0000","last_undegraded":"2023-12-02T00:29:55.891930+0000","last_fullsized":"2023-12-02T00:29:55.891930+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.901839+0000","last_change":"2023-12-02T00:29:55.901839+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.901839+0000","last_undegraded":"2023-12-02T00:29:55.901839+0000","last_fullsized":"2023-12-02T00:29:55.901839+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:55.891930+0000","last_change":"2023-12-02T00:29:55.891930+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.891930+0000","last_undegraded":"2023-12-02T00:29:55.891930+0000","last_fullsized":"2023-12-02T00:29:55.891930+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.901835+0000","last_change":"2023-12-02T00:29:55.901835+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.901835+0000","last_undegraded":"2023-12-02T00:29:55.901835+0000","last_fullsized":"2023-12-02T00:29:55.901835+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.898934+0000","last_change":"2023-12-02T00:29:55.898934+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.898934+0000","last_undegraded":"2023-12-02T00:29:55.898934+0000","last_fullsized":"2023-12-02T00:29:55.898934+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.899146+0000","last_change":"2023-12-02T00:29:55.899146+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.899146+0000","last_undegraded":"2023-12-02T00:29:55.899146+0000","last_fullsized":"2023-12-02T00:29:55.899146+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.903298+0000","last_change":"2023-12-02T00:29:55.903298+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.903298+0000","last_undegraded":"2023-12-02T00:29:55.903298+0000","last_fullsized":"2023-12-02T00:29:55.903298+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":175,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-12-02T00:29:52.806248+0000","last_change":"2023-12-02T00:28:39.379380+0000","last_active":"2023-12-02T00:29:52.806248+0000","last_peered":"2023-12-02T00:29:52.806248+0000","last_clean":"2023-12-02T00:29:52.806248+0000","last_became_active":"2023-12-02T00:28:39.378255+0000","last_became_peered":"2023-12-02T00:28:39.378255+0000","last_unstale":"2023-12-02T00:29:52.806248+0000","last_undegraded":"2023-12-02T00:29:52.806248+0000","last_fullsized":"2023-12-02T00:29:52.806248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:38.177345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:38.177345+0000","last_clean_scrub_stamp":"2023-12-02T00:28:38.177345+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:21:08.582412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":54,"acting":54,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":721941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.45900000000000002}]},{"osd":1,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.50700000000000001}]},{"osd":2,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.61599999999999999}]},{"osd":3,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.436}]},{"osd":4,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.57899999999999996}]},{"osd":5,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.59799999999999998}]},{"osd":6,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.68100000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":360448,"data_stored":137390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.59699999999999998}]},{"osd":1,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.46500000000000002}]},{"osd":2,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.443},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.497}]},{"osd":3,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.629}]},{"osd":4,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.56799999999999995}]},{"osd":5,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.64800000000000002}]},{"osd":7,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.72999999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":131573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.49099999999999999}]},{"osd":1,"last update":"Sat Dec 2 00:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.41799999999999998}]},{"osd":2,"last update":"Sat Dec 2 00:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.749}]},{"osd":3,"last update":"Sat Dec 2 00:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.63300000000000001}]},{"osd":4,"last update":"Sat Dec 2 00:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.66900000000000004}]},{"osd":6,"last update":"Sat Dec 2 00:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.77800000000000002}]},{"osd":7,"last update":"Sat Dec 2 00:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.76700000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":131573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.58099999999999996}]},{"osd":1,"last update":"Sat Dec 2 00:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.47199999999999998}]},{"osd":2,"last update":"Sat Dec 2 00:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.443},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.64200000000000002}]},{"osd":3,"last update":"Sat Dec 2 00:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.55800000000000005}]},{"osd":5,"last update":"Sat Dec 2 00:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.42199999999999999}]},{"osd":6,"last update":"Sat Dec 2 00:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.629},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.51900000000000002}]},{"osd":7,"last update":"Sat Dec 2 00:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.59999999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31532,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340308,"statfs":{"total":96636764160,"available":96604475392,"internally_reserved":0,"allocated":360448,"data_stored":137390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.28599999999999998}]},{"osd":1,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.36599999999999999}]},{"osd":2,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.30599999999999999}]},{"osd":4,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.52700000000000002}]},{"osd":5,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.64300000000000002}]},{"osd":6,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.58899999999999997}]},{"osd":7,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.45200000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31520,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340320,"statfs":{"total":96636764160,"available":96604487680,"internally_reserved":0,"allocated":344064,"data_stored":130954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.22}]},{"osd":1,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.503,"5min":0.503,"15min":0.503},"last":0.317}]},{"osd":3,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.42399999999999999}]},{"osd":4,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.55700000000000005}]},{"osd":5,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.442},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.59299999999999997}]},{"osd":6,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.52100000000000002}]},{"osd":7,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.72699999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31520,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340320,"statfs":{"total":96636764160,"available":96604487680,"internally_reserved":0,"allocated":348160,"data_stored":131573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.378}]},{"osd":2,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.41499999999999998}]},{"osd":3,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.34399999999999997}]},{"osd":4,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.56999999999999995}]},{"osd":5,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.437}]},{"osd":6,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.54900000000000004}]},{"osd":7,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.45300000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":937984,"data_stored":721322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.58399999999999996}]},{"osd":2,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.71699999999999997}]},{"osd":3,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.67400000000000004}]},{"osd":4,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.74199999999999999}]},{"osd":5,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.69599999999999995}]},{"osd":6,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.82099999999999995}]},{"osd":7,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.78000000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-02T00:30:01.623 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3f is not active+clean 2023-12-02T00:30:01.623 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.623 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3e is not active+clean 2023-12-02T00:30:01.623 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.623 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3d is not active+clean 2023-12-02T00:30:01.623 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.623 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3c is not active+clean 2023-12-02T00:30:01.623 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.623 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3b is not active+clean 2023-12-02T00:30:01.624 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.624 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3a is not active+clean 2023-12-02T00:30:01.624 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.624 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.39 is not active+clean 2023-12-02T00:30:01.624 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.624 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.38 is not active+clean 2023-12-02T00:30:01.624 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.624 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.37 is not active+clean 2023-12-02T00:30:01.624 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.624 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.36 is not active+clean 2023-12-02T00:30:01.624 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.624 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.35 is not active+clean 2023-12-02T00:30:01.624 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.625 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.34 is not active+clean 2023-12-02T00:30:01.625 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.625 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.33 is not active+clean 2023-12-02T00:30:01.625 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.625 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.32 is not active+clean 2023-12-02T00:30:01.625 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.625 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.31 is not active+clean 2023-12-02T00:30:01.625 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.625 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.30 is not active+clean 2023-12-02T00:30:01.625 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.625 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2f is not active+clean 2023-12-02T00:30:01.626 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.626 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2e is not active+clean 2023-12-02T00:30:01.626 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.626 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2d is not active+clean 2023-12-02T00:30:01.626 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.627 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2c is not active+clean 2023-12-02T00:30:01.627 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.627 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2b is not active+clean 2023-12-02T00:30:01.627 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.627 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2a is not active+clean 2023-12-02T00:30:01.628 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.628 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.29 is not active+clean 2023-12-02T00:30:01.628 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.628 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.28 is not active+clean 2023-12-02T00:30:01.628 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.629 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.27 is not active+clean 2023-12-02T00:30:01.629 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.629 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.26 is not active+clean 2023-12-02T00:30:01.630 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.630 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.25 is not active+clean 2023-12-02T00:30:01.630 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.630 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.24 is not active+clean 2023-12-02T00:30:01.630 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.630 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.23 is not active+clean 2023-12-02T00:30:01.630 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.631 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.22 is not active+clean 2023-12-02T00:30:01.631 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.631 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.21 is not active+clean 2023-12-02T00:30:01.631 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.631 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.20 is not active+clean 2023-12-02T00:30:01.631 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.631 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1e is not active+clean 2023-12-02T00:30:01.631 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.631 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1f is not active+clean 2023-12-02T00:30:01.631 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:29:55.903298+0000', 'last_change': '2023-12-02T00:29:55.903298+0000', 'last_active': '2023-12-02T00:29:55.891930+0000', 'last_peered': '2023-12-02T00:29:55.891930+0000', 'last_clean': '2023-12-02T00:29:55.891930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:55.903298+0000', 'last_undegraded': '2023-12-02T00:29:55.903298+0000', 'last_fullsized': '2023-12-02T00:29:55.903298+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-02T00:30:01.631 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1f is not active+clean 2023-12-02T00:30:01.632 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.632 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1e is not active+clean 2023-12-02T00:30:01.632 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:55.891930+0000', 'last_change': '2023-12-02T00:29:55.891930+0000', 'last_active': '2023-12-02T00:29:55.891930+0000', 'last_peered': '2023-12-02T00:29:55.891930+0000', 'last_clean': '2023-12-02T00:29:55.891930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:55.891930+0000', 'last_undegraded': '2023-12-02T00:29:55.891930+0000', 'last_fullsized': '2023-12-02T00:29:55.891930+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.632 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1c is not active+clean 2023-12-02T00:30:01.632 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.632 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1d is not active+clean 2023-12-02T00:30:01.632 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:29:55.900712+0000', 'last_change': '2023-12-02T00:29:55.900712+0000', 'last_active': '2023-12-02T00:29:55.891930+0000', 'last_peered': '2023-12-02T00:29:55.891930+0000', 'last_clean': '2023-12-02T00:29:55.891930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:55.900712+0000', 'last_undegraded': '2023-12-02T00:29:55.900712+0000', 'last_fullsized': '2023-12-02T00:29:55.900712+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-02T00:30:01.632 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1d is not active+clean 2023-12-02T00:30:01.632 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.632 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1c is not active+clean 2023-12-02T00:30:01.632 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:29:55.901454+0000', 'last_change': '2023-12-02T00:29:55.901454+0000', 'last_active': '2023-12-02T00:29:55.891930+0000', 'last_peered': '2023-12-02T00:29:55.891930+0000', 'last_clean': '2023-12-02T00:29:55.891930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:55.901454+0000', 'last_undegraded': '2023-12-02T00:29:55.901454+0000', 'last_fullsized': '2023-12-02T00:29:55.901454+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-02T00:30:01.632 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1a is not active+clean 2023-12-02T00:30:01.633 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.633 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1b is not active+clean 2023-12-02T00:30:01.633 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:29:55.900095+0000', 'last_change': '2023-12-02T00:29:55.900095+0000', 'last_active': '2023-12-02T00:29:55.891930+0000', 'last_peered': '2023-12-02T00:29:55.891930+0000', 'last_clean': '2023-12-02T00:29:55.891930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:55.900095+0000', 'last_undegraded': '2023-12-02T00:29:55.900095+0000', 'last_fullsized': '2023-12-02T00:29:55.900095+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-02T00:30:01.633 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1b is not active+clean 2023-12-02T00:30:01.633 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.633 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1a is not active+clean 2023-12-02T00:30:01.633 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:29:55.898451+0000', 'last_change': '2023-12-02T00:29:55.898451+0000', 'last_active': '2023-12-02T00:29:55.891930+0000', 'last_peered': '2023-12-02T00:29:55.891930+0000', 'last_clean': '2023-12-02T00:29:55.891930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:55.898451+0000', 'last_undegraded': '2023-12-02T00:29:55.898451+0000', 'last_fullsized': '2023-12-02T00:29:55.898451+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-02T00:30:01.633 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.18 is not active+clean 2023-12-02T00:30:01.633 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.633 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.19 is not active+clean 2023-12-02T00:30:01.633 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:29:55.900139+0000', 'last_change': '2023-12-02T00:29:55.900139+0000', 'last_active': '2023-12-02T00:29:55.891930+0000', 'last_peered': '2023-12-02T00:29:55.891930+0000', 'last_clean': '2023-12-02T00:29:55.891930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:55.900139+0000', 'last_undegraded': '2023-12-02T00:29:55.900139+0000', 'last_fullsized': '2023-12-02T00:29:55.900139+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-02T00:30:01.633 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.19 is not active+clean 2023-12-02T00:30:01.634 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.634 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.18 is not active+clean 2023-12-02T00:30:01.634 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:29:55.898680+0000', 'last_change': '2023-12-02T00:29:55.898680+0000', 'last_active': '2023-12-02T00:29:55.891930+0000', 'last_peered': '2023-12-02T00:29:55.891930+0000', 'last_clean': '2023-12-02T00:29:55.891930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:55.898680+0000', 'last_undegraded': '2023-12-02T00:29:55.898680+0000', 'last_fullsized': '2023-12-02T00:29:55.898680+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-02T00:30:01.634 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.16 is not active+clean 2023-12-02T00:30:01.634 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.634 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.17 is not active+clean 2023-12-02T00:30:01.634 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:29:55.899737+0000', 'last_change': '2023-12-02T00:29:55.899737+0000', 'last_active': '2023-12-02T00:29:55.891930+0000', 'last_peered': '2023-12-02T00:29:55.891930+0000', 'last_clean': '2023-12-02T00:29:55.891930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:55.899737+0000', 'last_undegraded': '2023-12-02T00:29:55.899737+0000', 'last_fullsized': '2023-12-02T00:29:55.899737+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-02T00:30:01.634 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.17 is not active+clean 2023-12-02T00:30:01.634 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.635 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.16 is not active+clean 2023-12-02T00:30:01.635 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:55.891930+0000', 'last_change': '2023-12-02T00:29:55.891930+0000', 'last_active': '2023-12-02T00:29:55.891930+0000', 'last_peered': '2023-12-02T00:29:55.891930+0000', 'last_clean': '2023-12-02T00:29:55.891930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:55.891930+0000', 'last_undegraded': '2023-12-02T00:29:55.891930+0000', 'last_fullsized': '2023-12-02T00:29:55.891930+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.635 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.14 is not active+clean 2023-12-02T00:30:01.635 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.635 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.15 is not active+clean 2023-12-02T00:30:01.635 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:29:55.898306+0000', 'last_change': '2023-12-02T00:29:55.898306+0000', 'last_active': '2023-12-02T00:29:55.891930+0000', 'last_peered': '2023-12-02T00:29:55.891930+0000', 'last_clean': '2023-12-02T00:29:55.891930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:55.898306+0000', 'last_undegraded': '2023-12-02T00:29:55.898306+0000', 'last_fullsized': '2023-12-02T00:29:55.898306+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-02T00:30:01.635 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2023-12-02T00:30:01.635 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.635 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.14 is not active+clean 2023-12-02T00:30:01.635 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:29:55.899642+0000', 'last_change': '2023-12-02T00:29:55.899642+0000', 'last_active': '2023-12-02T00:29:55.891930+0000', 'last_peered': '2023-12-02T00:29:55.891930+0000', 'last_clean': '2023-12-02T00:29:55.891930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:55.899642+0000', 'last_undegraded': '2023-12-02T00:29:55.899642+0000', 'last_fullsized': '2023-12-02T00:29:55.899642+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-02T00:30:01.635 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2023-12-02T00:30:01.635 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.636 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.13 is not active+clean 2023-12-02T00:30:01.636 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:29:55.899635+0000', 'last_change': '2023-12-02T00:29:55.899635+0000', 'last_active': '2023-12-02T00:29:55.891930+0000', 'last_peered': '2023-12-02T00:29:55.891930+0000', 'last_clean': '2023-12-02T00:29:55.891930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:55.899635+0000', 'last_undegraded': '2023-12-02T00:29:55.899635+0000', 'last_fullsized': '2023-12-02T00:29:55.899635+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-02T00:30:01.636 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.13 is not active+clean 2023-12-02T00:30:01.636 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.636 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.12 is not active+clean 2023-12-02T00:30:01.636 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:29:55.902337+0000', 'last_change': '2023-12-02T00:29:55.902337+0000', 'last_active': '2023-12-02T00:29:55.891930+0000', 'last_peered': '2023-12-02T00:29:55.891930+0000', 'last_clean': '2023-12-02T00:29:55.891930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:55.902337+0000', 'last_undegraded': '2023-12-02T00:29:55.902337+0000', 'last_fullsized': '2023-12-02T00:29:55.902337+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-02T00:30:01.636 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.10 is not active+clean 2023-12-02T00:30:01.636 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.636 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.11 is not active+clean 2023-12-02T00:30:01.636 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:29:55.899585+0000', 'last_change': '2023-12-02T00:29:55.899585+0000', 'last_active': '2023-12-02T00:29:55.891930+0000', 'last_peered': '2023-12-02T00:29:55.891930+0000', 'last_clean': '2023-12-02T00:29:55.891930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:55.899585+0000', 'last_undegraded': '2023-12-02T00:29:55.899585+0000', 'last_fullsized': '2023-12-02T00:29:55.899585+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-02T00:30:01.637 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.11 is not active+clean 2023-12-02T00:30:01.637 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.637 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.10 is not active+clean 2023-12-02T00:30:01.637 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:29:55.899587+0000', 'last_change': '2023-12-02T00:29:55.899587+0000', 'last_active': '2023-12-02T00:29:55.891930+0000', 'last_peered': '2023-12-02T00:29:55.891930+0000', 'last_clean': '2023-12-02T00:29:55.891930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:55.899587+0000', 'last_undegraded': '2023-12-02T00:29:55.899587+0000', 'last_fullsized': '2023-12-02T00:29:55.899587+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-02T00:30:01.637 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.e is not active+clean 2023-12-02T00:30:01.637 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.637 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.f is not active+clean 2023-12-02T00:30:01.637 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:29:55.902333+0000', 'last_change': '2023-12-02T00:29:55.902333+0000', 'last_active': '2023-12-02T00:29:55.891930+0000', 'last_peered': '2023-12-02T00:29:55.891930+0000', 'last_clean': '2023-12-02T00:29:55.891930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:55.902333+0000', 'last_undegraded': '2023-12-02T00:29:55.902333+0000', 'last_fullsized': '2023-12-02T00:29:55.902333+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-02T00:30:01.637 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.f is not active+clean 2023-12-02T00:30:01.637 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.637 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.e is not active+clean 2023-12-02T00:30:01.637 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:29:55.898944+0000', 'last_change': '2023-12-02T00:29:55.898944+0000', 'last_active': '2023-12-02T00:29:55.891930+0000', 'last_peered': '2023-12-02T00:29:55.891930+0000', 'last_clean': '2023-12-02T00:29:55.891930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:55.898944+0000', 'last_undegraded': '2023-12-02T00:29:55.898944+0000', 'last_fullsized': '2023-12-02T00:29:55.898944+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-02T00:30:01.638 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2023-12-02T00:30:01.638 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.638 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.d is not active+clean 2023-12-02T00:30:01.638 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:29:55.899297+0000', 'last_change': '2023-12-02T00:29:55.899297+0000', 'last_active': '2023-12-02T00:29:55.891930+0000', 'last_peered': '2023-12-02T00:29:55.891930+0000', 'last_clean': '2023-12-02T00:29:55.891930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:55.899297+0000', 'last_undegraded': '2023-12-02T00:29:55.899297+0000', 'last_fullsized': '2023-12-02T00:29:55.899297+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-02T00:30:01.638 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.d is not active+clean 2023-12-02T00:30:01.638 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.638 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.c is not active+clean 2023-12-02T00:30:01.639 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:29:55.898740+0000', 'last_change': '2023-12-02T00:29:55.898740+0000', 'last_active': '2023-12-02T00:29:55.891930+0000', 'last_peered': '2023-12-02T00:29:55.891930+0000', 'last_clean': '2023-12-02T00:29:55.891930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:55.898740+0000', 'last_undegraded': '2023-12-02T00:29:55.898740+0000', 'last_fullsized': '2023-12-02T00:29:55.898740+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-02T00:30:01.639 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.a is not active+clean 2023-12-02T00:30:01.640 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.640 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.b is not active+clean 2023-12-02T00:30:01.641 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:55.891930+0000', 'last_change': '2023-12-02T00:29:55.891930+0000', 'last_active': '2023-12-02T00:29:55.891930+0000', 'last_peered': '2023-12-02T00:29:55.891930+0000', 'last_clean': '2023-12-02T00:29:55.891930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:55.891930+0000', 'last_undegraded': '2023-12-02T00:29:55.891930+0000', 'last_fullsized': '2023-12-02T00:29:55.891930+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.641 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2023-12-02T00:30:01.641 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.641 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.a is not active+clean 2023-12-02T00:30:01.641 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:29:55.903215+0000', 'last_change': '2023-12-02T00:29:55.903215+0000', 'last_active': '2023-12-02T00:29:55.891930+0000', 'last_peered': '2023-12-02T00:29:55.891930+0000', 'last_clean': '2023-12-02T00:29:55.891930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:55.903215+0000', 'last_undegraded': '2023-12-02T00:29:55.903215+0000', 'last_fullsized': '2023-12-02T00:29:55.903215+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-02T00:30:01.641 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.8 is not active+clean 2023-12-02T00:30:01.642 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.642 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.9 is not active+clean 2023-12-02T00:30:01.642 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:29:55.900528+0000', 'last_change': '2023-12-02T00:29:55.900528+0000', 'last_active': '2023-12-02T00:29:55.891930+0000', 'last_peered': '2023-12-02T00:29:55.891930+0000', 'last_clean': '2023-12-02T00:29:55.891930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:55.900528+0000', 'last_undegraded': '2023-12-02T00:29:55.900528+0000', 'last_fullsized': '2023-12-02T00:29:55.900528+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-02T00:30:01.643 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.9 is not active+clean 2023-12-02T00:30:01.643 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.643 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.8 is not active+clean 2023-12-02T00:30:01.643 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:29:55.898308+0000', 'last_change': '2023-12-02T00:29:55.898308+0000', 'last_active': '2023-12-02T00:29:55.891930+0000', 'last_peered': '2023-12-02T00:29:55.891930+0000', 'last_clean': '2023-12-02T00:29:55.891930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:55.898308+0000', 'last_undegraded': '2023-12-02T00:29:55.898308+0000', 'last_fullsized': '2023-12-02T00:29:55.898308+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-02T00:30:01.643 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.6 is not active+clean 2023-12-02T00:30:01.643 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.644 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.7 is not active+clean 2023-12-02T00:30:01.644 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:29:55.901881+0000', 'last_change': '2023-12-02T00:29:55.901881+0000', 'last_active': '2023-12-02T00:29:55.891930+0000', 'last_peered': '2023-12-02T00:29:55.891930+0000', 'last_clean': '2023-12-02T00:29:55.891930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:55.901881+0000', 'last_undegraded': '2023-12-02T00:29:55.901881+0000', 'last_fullsized': '2023-12-02T00:29:55.901881+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-02T00:30:01.644 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.7 is not active+clean 2023-12-02T00:30:01.645 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.645 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.6 is not active+clean 2023-12-02T00:30:01.645 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:55.891930+0000', 'last_change': '2023-12-02T00:29:55.891930+0000', 'last_active': '2023-12-02T00:29:55.891930+0000', 'last_peered': '2023-12-02T00:29:55.891930+0000', 'last_clean': '2023-12-02T00:29:55.891930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:55.891930+0000', 'last_undegraded': '2023-12-02T00:29:55.891930+0000', 'last_fullsized': '2023-12-02T00:29:55.891930+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.645 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.5 is not active+clean 2023-12-02T00:30:01.646 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.646 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.4 is not active+clean 2023-12-02T00:30:01.646 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:29:55.901839+0000', 'last_change': '2023-12-02T00:29:55.901839+0000', 'last_active': '2023-12-02T00:29:55.891930+0000', 'last_peered': '2023-12-02T00:29:55.891930+0000', 'last_clean': '2023-12-02T00:29:55.891930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:55.901839+0000', 'last_undegraded': '2023-12-02T00:29:55.901839+0000', 'last_fullsized': '2023-12-02T00:29:55.901839+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-02T00:30:01.646 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2 is not active+clean 2023-12-02T00:30:01.646 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.647 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3 is not active+clean 2023-12-02T00:30:01.647 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:55.891930+0000', 'last_change': '2023-12-02T00:29:55.891930+0000', 'last_active': '2023-12-02T00:29:55.891930+0000', 'last_peered': '2023-12-02T00:29:55.891930+0000', 'last_clean': '2023-12-02T00:29:55.891930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:55.891930+0000', 'last_undegraded': '2023-12-02T00:29:55.891930+0000', 'last_fullsized': '2023-12-02T00:29:55.891930+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.647 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3 is not active+clean 2023-12-02T00:30:01.647 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.648 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2 is not active+clean 2023-12-02T00:30:01.648 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:29:55.901835+0000', 'last_change': '2023-12-02T00:29:55.901835+0000', 'last_active': '2023-12-02T00:29:55.891930+0000', 'last_peered': '2023-12-02T00:29:55.891930+0000', 'last_clean': '2023-12-02T00:29:55.891930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:55.901835+0000', 'last_undegraded': '2023-12-02T00:29:55.901835+0000', 'last_fullsized': '2023-12-02T00:29:55.901835+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-02T00:30:01.648 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.0 is not active+clean 2023-12-02T00:30:01.648 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.649 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1 is not active+clean 2023-12-02T00:30:01.649 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:29:55.898934+0000', 'last_change': '2023-12-02T00:29:55.898934+0000', 'last_active': '2023-12-02T00:29:55.891930+0000', 'last_peered': '2023-12-02T00:29:55.891930+0000', 'last_clean': '2023-12-02T00:29:55.891930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:55.898934+0000', 'last_undegraded': '2023-12-02T00:29:55.898934+0000', 'last_fullsized': '2023-12-02T00:29:55.898934+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-02T00:30:01.649 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1 is not active+clean 2023-12-02T00:30:01.649 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.649 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.0 is not active+clean 2023-12-02T00:30:01.649 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:29:55.899146+0000', 'last_change': '2023-12-02T00:29:55.899146+0000', 'last_active': '2023-12-02T00:29:55.891930+0000', 'last_peered': '2023-12-02T00:29:55.891930+0000', 'last_clean': '2023-12-02T00:29:55.891930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:55.899146+0000', 'last_undegraded': '2023-12-02T00:29:55.899146+0000', 'last_fullsized': '2023-12-02T00:29:55.899146+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-02T00:30:01.649 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2023-12-02T00:30:01.650 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-02T00:29:56.896331+0000', 'last_change': '2023-12-02T00:29:56.896331+0000', 'last_active': '2023-12-02T00:29:56.896331+0000', 'last_peered': '2023-12-02T00:29:56.896331+0000', 'last_clean': '2023-12-02T00:29:56.896331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:56.896331+0000', 'last_undegraded': '2023-12-02T00:29:56.896331+0000', 'last_fullsized': '2023-12-02T00:29:56.896331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:56.896331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-02T00:30:01.650 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.5 is not active+clean 2023-12-02T00:30:01.650 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:29:55.903298+0000', 'last_change': '2023-12-02T00:29:55.903298+0000', 'last_active': '2023-12-02T00:29:55.891930+0000', 'last_peered': '2023-12-02T00:29:55.891930+0000', 'last_clean': '2023-12-02T00:29:55.891930+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:29:55.903298+0000', 'last_undegraded': '2023-12-02T00:29:55.903298+0000', 'last_fullsized': '2023-12-02T00:29:55.903298+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'last_clean_scrub_stamp': '2023-12-02T00:29:55.891930+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-02T00:30:01.651 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-12-02T00:30:01.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.763+0000 7fadf6f9e640 1 Processor -- start 2023-12-02T00:30:01.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.763+0000 7fadf6f9e640 1 -- start start 2023-12-02T00:30:01.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.763+0000 7fadf6f9e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fadf015d330 0x7fadf00b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:01.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.763+0000 7fadf6f9e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fadf00d3900 0x7fadf00b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:01.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.763+0000 7fadf6f9e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadf00b2b10 con 0x7fadf00d2910 2023-12-02T00:30:01.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.763+0000 7fadf6f9e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadf00d1590 con 0x7fadf015d330 2023-12-02T00:30:01.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.763+0000 7fadf6f9e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadf00d1870 con 0x7fadf00d3900 2023-12-02T00:30:01.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.764+0000 7fadf4d3a640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fadf015d330 0x7fadf00b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:01.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.764+0000 7fadf553b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fadf00d3900 0x7fadf00b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:01.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.764+0000 7fadeffff640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fadf00d2910 0x7fadf00d31f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.46:58870/0 (socket says 172.21.15.46:58870) 2023-12-02T00:30:01.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.764+0000 7fadeffff640 1 -- 172.21.15.46:0/3571579292 learned_addr learned my addr 172.21.15.46:0/3571579292 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:30:01.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.764+0000 7fadef7fe640 1 -- 172.21.15.46:0/3571579292 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1151346859 0 0) 0x7fadf00b2b10 con 0x7fadf00d2910 2023-12-02T00:30:01.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.764+0000 7fadf553b640 1 -- 172.21.15.46:0/3571579292 >> v2:172.21.15.162:3300/0 conn(0x7fadf015d330 msgr2=0x7fadf00b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:01.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.764+0000 7fadf553b640 1 --2- 172.21.15.46:0/3571579292 >> v2:172.21.15.162:3300/0 conn(0x7fadf015d330 0x7fadf00b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:01.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.764+0000 7fadf553b640 1 -- 172.21.15.46:0/3571579292 >> v1:172.21.15.162:6789/0 conn(0x7fadf00d2910 legacy=0x7fadf00d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:01.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.764+0000 7fadf553b640 1 -- 172.21.15.46:0/3571579292 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf00d1b50 con 0x7fadf00d3900 2023-12-02T00:30:01.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.764+0000 7fadf4d3a640 1 --2- 172.21.15.46:0/3571579292 >> v2:172.21.15.162:3300/0 conn(0x7fadf015d330 0x7fadf00b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:30:01.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.765+0000 7fadf553b640 1 --2- 172.21.15.46:0/3571579292 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fadf00d3900 0x7fadf00b6390 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fade8009d90 tx=0x7fade802be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34f6cc931a07151e server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:01.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.765+0000 7fadef7fe640 1 -- 172.21.15.46:0/3571579292 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade802cbc0 con 0x7fadf00d3900 2023-12-02T00:30:01.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.765+0000 7fadef7fe640 1 -- 172.21.15.46:0/3571579292 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fade802cd20 con 0x7fadf00d3900 2023-12-02T00:30:01.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.765+0000 7fadef7fe640 1 -- 172.21.15.46:0/3571579292 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade8034660 con 0x7fadf00d3900 2023-12-02T00:30:01.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.766+0000 7fadf6f9e640 1 -- 172.21.15.46:0/3571579292 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fadf00d3900 msgr2=0x7fadf00b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:01.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.766+0000 7fadf6f9e640 1 --2- 172.21.15.46:0/3571579292 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fadf00d3900 0x7fadf00b6390 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fade8009d90 tx=0x7fade802be10 comp rx=0 tx=0).stop 2023-12-02T00:30:01.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.766+0000 7fadf6f9e640 1 -- 172.21.15.46:0/3571579292 shutdown_connections 2023-12-02T00:30:01.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.766+0000 7fadf6f9e640 1 --2- 172.21.15.46:0/3571579292 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fadf00d3900 0x7fadf00b6390 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:01.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.766+0000 7fadf6f9e640 1 --2- 172.21.15.46:0/3571579292 >> v2:172.21.15.162:3300/0 conn(0x7fadf015d330 0x7fadf00b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:01.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.766+0000 7fadf6f9e640 1 -- 172.21.15.46:0/3571579292 >> 172.21.15.46:0/3571579292 conn(0x7fadf0072020 msgr2=0x7fadf0072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:01.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.766+0000 7fadf6f9e640 1 -- 172.21.15.46:0/3571579292 shutdown_connections 2023-12-02T00:30:01.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.766+0000 7fadf6f9e640 1 -- 172.21.15.46:0/3571579292 wait complete. 2023-12-02T00:30:01.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.767+0000 7fadf6f9e640 1 Processor -- start 2023-12-02T00:30:01.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.767+0000 7fadf6f9e640 1 -- start start 2023-12-02T00:30:01.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.768+0000 7fadf6f9e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fadf00d2910 0x7fadf01565e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:01.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.768+0000 7fadf6f9e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fadf00d3900 0x7fadf015e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:01.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.768+0000 7fadf6f9e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fadf015d330 0x7fadf015eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:01.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.768+0000 7fadf6f9e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf01f2bc0 con 0x7fadf00d3900 2023-12-02T00:30:01.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.768+0000 7fadf6f9e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf00c8560 con 0x7fadf015d330 2023-12-02T00:30:01.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.768+0000 7fadf6f9e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadf00c7950 con 0x7fadf00d2910 2023-12-02T00:30:01.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.768+0000 7fadeffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fadf00d3900 0x7fadf015e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:01.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.768+0000 7fadeffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fadf00d3900 0x7fadf015e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50856/0 (socket says 172.21.15.46:50856) 2023-12-02T00:30:01.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.768+0000 7fadeffff640 1 -- 172.21.15.46:0/3408235887 learned_addr learned my addr 172.21.15.46:0/3408235887 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:01.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.768+0000 7fadf4d3a640 1 --2- 172.21.15.46:0/3408235887 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fadf00d2910 0x7fadf01565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:01.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.768+0000 7fadf553b640 1 --2- 172.21.15.46:0/3408235887 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fadf015d330 0x7fadf015eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:01.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.768+0000 7fadeffff640 1 -- 172.21.15.46:0/3408235887 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fadf00d2910 msgr2=0x7fadf01565e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:01.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.768+0000 7fadeffff640 1 --2- 172.21.15.46:0/3408235887 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fadf00d2910 0x7fadf01565e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:01.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.768+0000 7fadeffff640 1 -- 172.21.15.46:0/3408235887 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fadf015d330 msgr2=0x7fadf015eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:01.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.768+0000 7fadeffff640 1 --2- 172.21.15.46:0/3408235887 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fadf015d330 0x7fadf015eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:01.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.768+0000 7fadeffff640 1 -- 172.21.15.46:0/3408235887 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade4009400 con 0x7fadf00d3900 2023-12-02T00:30:01.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.768+0000 7fadeffff640 1 --2- 172.21.15.46:0/3408235887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fadf00d3900 0x7fadf015e620 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7faddc00c430 tx=0x7faddc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:01.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.769+0000 7faded7fa640 1 -- 172.21.15.46:0/3408235887 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc0074e0 con 0x7fadf00d3900 2023-12-02T00:30:01.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.769+0000 7faded7fa640 1 -- 172.21.15.46:0/3408235887 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faddc007660 con 0x7fadf00d3900 2023-12-02T00:30:01.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.769+0000 7fadf6f9e640 1 -- 172.21.15.46:0/3408235887 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fade8009a70 con 0x7fadf00d3900 2023-12-02T00:30:01.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.769+0000 7faded7fa640 1 -- 172.21.15.46:0/3408235887 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc004a20 con 0x7fadf00d3900 2023-12-02T00:30:01.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.769+0000 7fadf6f9e640 1 -- 172.21.15.46:0/3408235887 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadf00c7f10 con 0x7fadf00d3900 2023-12-02T00:30:01.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.770+0000 7fadf6f9e640 1 -- 172.21.15.46:0/3408235887 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadf0069400 con 0x7fadf00d3900 2023-12-02T00:30:01.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.770+0000 7faded7fa640 1 -- 172.21.15.46:0/3408235887 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faddc004b80 con 0x7fadf00d3900 2023-12-02T00:30:01.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.771+0000 7faded7fa640 1 --2- 172.21.15.46:0/3408235887 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fadd406ae70 0x7fadd406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:01.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.771+0000 7faded7fa640 1 -- 172.21.15.46:0/3408235887 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faddc089590 con 0x7fadf00d3900 2023-12-02T00:30:01.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.771+0000 7fadf4d3a640 1 --2- 172.21.15.46:0/3408235887 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fadd406ae70 0x7fadd406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:01.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.771+0000 7fadf4d3a640 1 --2- 172.21.15.46:0/3408235887 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fadd406ae70 0x7fadd406d330 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fade4009c60 tx=0x7fade4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:01.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:01.775+0000 7faded7fa640 1 -- 172.21.15.46:0/3408235887 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faddc054520 con 0x7fadf00d3900 2023-12-02T00:30:02.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.022+0000 7fadf6f9e640 1 -- 172.21.15.46:0/3408235887 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fadf00c86a0 con 0x7fadf00d3900 2023-12-02T00:30:02.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.022+0000 7faded7fa640 1 -- 172.21.15.46:0/3408235887 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1268 (secure 0 0 0) 0x7faddc057b40 con 0x7fadf00d3900 2023-12-02T00:30:02.023 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:30:02.024 INFO:teuthology.orchestra.run.smithi046.stdout:{"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":97,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":35,"num_osds":8,"num_up_osds":8,"osd_up_since":1701476915,"num_in_osds":8,"osd_in_since":1701476907,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":69},{"state_name":"creating+peering","count":27},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":242388992,"bytes_avail":772851724288,"bytes_total":773094113280,"unknown_pgs_ratio":0.71134018898010254,"inactive_pgs_ratio":0.27835050225257874},"fsmap":{"epoch":13,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"a","status":"up:active","gid":5277}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2023-12-02T00:29:45.171776+0000","services":{"mds":{"daemons":{"summary":"","d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-12-02T00:30:02.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.024+0000 7fadf6f9e640 1 -- 172.21.15.46:0/3408235887 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fadd406ae70 msgr2=0x7fadd406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:02.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.024+0000 7fadf6f9e640 1 --2- 172.21.15.46:0/3408235887 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fadd406ae70 0x7fadd406d330 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fade4009c60 tx=0x7fade4008040 comp rx=0 tx=0).stop 2023-12-02T00:30:02.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.024+0000 7fadf6f9e640 1 -- 172.21.15.46:0/3408235887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fadf00d3900 msgr2=0x7fadf015e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:02.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.024+0000 7fadf6f9e640 1 --2- 172.21.15.46:0/3408235887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fadf00d3900 0x7fadf015e620 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7faddc00c430 tx=0x7faddc00c8f0 comp rx=0 tx=0).stop 2023-12-02T00:30:02.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.025+0000 7fadf6f9e640 1 -- 172.21.15.46:0/3408235887 shutdown_connections 2023-12-02T00:30:02.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.025+0000 7fadf6f9e640 1 --2- 172.21.15.46:0/3408235887 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fadd406ae70 0x7fadd406d330 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:02.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.025+0000 7fadf6f9e640 1 --2- 172.21.15.46:0/3408235887 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fadf015d330 0x7fadf015eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:02.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.025+0000 7fadf6f9e640 1 --2- 172.21.15.46:0/3408235887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fadf00d3900 0x7fadf015e620 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:02.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.025+0000 7fadf6f9e640 1 --2- 172.21.15.46:0/3408235887 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fadf00d2910 0x7fadf01565e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:02.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.025+0000 7fadf6f9e640 1 -- 172.21.15.46:0/3408235887 >> 172.21.15.46:0/3408235887 conn(0x7fadf0072020 msgr2=0x7fadf00b54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:02.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.025+0000 7fadf6f9e640 1 -- 172.21.15.46:0/3408235887 shutdown_connections 2023-12-02T00:30:02.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.025+0000 7fadf6f9e640 1 -- 172.21.15.46:0/3408235887 wait complete. 2023-12-02T00:30:02.036 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-12-02T00:30:02.037 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-02T00:30:02.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.142+0000 7fdabd5b0640 1 Processor -- start 2023-12-02T00:30:02.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.142+0000 7fdabd5b0640 1 -- start start 2023-12-02T00:30:02.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.142+0000 7fdabd5b0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdab8142270 0x7fdab81426c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:02.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.142+0000 7fdabd5b0640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdab8161d20 con 0x7fdab806ea50 2023-12-02T00:30:02.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.142+0000 7fdabd5b0640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdab815e760 con 0x7fdab8160fb0 2023-12-02T00:30:02.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.142+0000 7fdabd5b0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdab80d1ee0 con 0x7fdab8142270 2023-12-02T00:30:02.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.143+0000 7fdab6ffd640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fdab806ea50 0x7fdab806ee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57154/0 (socket says 172.21.15.46:57154) 2023-12-02T00:30:02.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.143+0000 7fdab6ffd640 1 -- 172.21.15.46:0/97162617 learned_addr learned my addr 172.21.15.46:0/97162617 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:30:02.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.143+0000 7fdab77fe640 1 --2- 172.21.15.46:0/97162617 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdab8142270 0x7fdab81426c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:02.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.143+0000 7fdab5ffb640 1 -- 172.21.15.46:0/97162617 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1055414118 0 0) 0x7fdab8161d20 con 0x7fdab806ea50 2023-12-02T00:30:02.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.143+0000 7fdab5ffb640 1 -- 172.21.15.46:0/97162617 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdaa0003520 con 0x7fdab806ea50 2023-12-02T00:30:02.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.143+0000 7fdab5ffb640 1 -- 172.21.15.46:0/97162617 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 303771171 0 0) 0x7fdab815e760 con 0x7fdab8160fb0 2023-12-02T00:30:02.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.143+0000 7fdab5ffb640 1 -- 172.21.15.46:0/97162617 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdab8161d20 con 0x7fdab8160fb0 2023-12-02T00:30:02.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.143+0000 7fdab5ffb640 1 -- 172.21.15.46:0/97162617 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1258930995 0 0) 0x7fdaa0003520 con 0x7fdab806ea50 2023-12-02T00:30:02.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.143+0000 7fdab5ffb640 1 -- 172.21.15.46:0/97162617 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdab8142270 msgr2=0x7fdab81426c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:02.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.143+0000 7fdab5ffb640 1 --2- 172.21.15.46:0/97162617 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdab8142270 0x7fdab81426c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:02.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.143+0000 7fdab5ffb640 1 -- 172.21.15.46:0/97162617 >> v1:172.21.15.162:6789/0 conn(0x7fdab8160fb0 legacy=0x7fdab8161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:02.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.143+0000 7fdab5ffb640 1 -- 172.21.15.46:0/97162617 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab815d9a0 con 0x7fdab806ea50 2023-12-02T00:30:02.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.143+0000 7fdab5ffb640 1 -- 172.21.15.46:0/97162617 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7fdaa8001670 con 0x7fdab806ea50 2023-12-02T00:30:02.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.143+0000 7fdab5ffb640 1 -- 172.21.15.46:0/97162617 >> v1:172.21.15.46:6789/0 conn(0x7fdab806ea50 legacy=0x7fdab806ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:02.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.143+0000 7fdab5ffb640 1 --2- 172.21.15.46:0/97162617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdaa0003b30 0x7fdaa0003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:02.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.143+0000 7fdab5ffb640 1 --2- 172.21.15.46:0/97162617 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdab8142270 0x7fdaa00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:02.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.143+0000 7fdab5ffb640 1 --2- 172.21.15.46:0/97162617 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdaa0004d00 0x7fdaa0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:02.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.143+0000 7fdab5ffb640 1 -- 172.21.15.46:0/97162617 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab80d1ee0 con 0x7fdaa0003b30 2023-12-02T00:30:02.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.143+0000 7fdab5ffb640 1 -- 172.21.15.46:0/97162617 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa000ba30 con 0x7fdaa0004d00 2023-12-02T00:30:02.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.143+0000 7fdab5ffb640 1 -- 172.21.15.46:0/97162617 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa000bba0 con 0x7fdab8142270 2023-12-02T00:30:02.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.143+0000 7fdab6ffd640 1 --2- 172.21.15.46:0/97162617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdaa0003b30 0x7fdaa0003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:02.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.143+0000 7fdab67fc640 1 --2- 172.21.15.46:0/97162617 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdab8142270 0x7fdaa00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:02.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.144+0000 7fdab77fe640 1 --2- 172.21.15.46:0/97162617 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdaa0004d00 0x7fdaa0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:02.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.144+0000 7fdab6ffd640 1 -- 172.21.15.46:0/97162617 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdab8142270 msgr2=0x7fdaa00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:02.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.144+0000 7fdab6ffd640 1 --2- 172.21.15.46:0/97162617 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdab8142270 0x7fdaa00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:02.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.144+0000 7fdab6ffd640 1 -- 172.21.15.46:0/97162617 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdaa0004d00 msgr2=0x7fdaa0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:02.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.144+0000 7fdab6ffd640 1 --2- 172.21.15.46:0/97162617 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdaa0004d00 0x7fdaa0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:02.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.144+0000 7fdab6ffd640 1 -- 172.21.15.46:0/97162617 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa0003520 con 0x7fdaa0003b30 2023-12-02T00:30:02.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.144+0000 7fdab6ffd640 1 --2- 172.21.15.46:0/97162617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdaa0003b30 0x7fdaa0003f30 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fdaa800c160 tx=0x7fdaa00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e47e45ab2a2a60f1 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:02.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.145+0000 7fdab5ffb640 1 -- 172.21.15.46:0/97162617 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa8005ad0 con 0x7fdaa0003b30 2023-12-02T00:30:02.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.145+0000 7fdab5ffb640 1 -- 172.21.15.46:0/97162617 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaa8005c30 con 0x7fdaa0003b30 2023-12-02T00:30:02.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.145+0000 7fdab5ffb640 1 -- 172.21.15.46:0/97162617 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa802c850 con 0x7fdaa0003b30 2023-12-02T00:30:02.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.145+0000 7fdabd5b0640 1 -- 172.21.15.46:0/97162617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdaa0003b30 msgr2=0x7fdaa0003f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:02.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.145+0000 7fdabd5b0640 1 --2- 172.21.15.46:0/97162617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdaa0003b30 0x7fdaa0003f30 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fdaa800c160 tx=0x7fdaa00057f0 comp rx=0 tx=0).stop 2023-12-02T00:30:02.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.145+0000 7fdab77fe640 1 -- 172.21.15.46:0/97162617 reap_dead start 2023-12-02T00:30:02.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.145+0000 7fdabd5b0640 1 -- 172.21.15.46:0/97162617 shutdown_connections 2023-12-02T00:30:02.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.145+0000 7fdabd5b0640 1 -- 172.21.15.46:0/97162617 >> 172.21.15.46:0/97162617 conn(0x7fdab806a6b0 msgr2=0x7fdab806aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:02.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.145+0000 7fdabd5b0640 1 -- 172.21.15.46:0/97162617 shutdown_connections 2023-12-02T00:30:02.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.145+0000 7fdabd5b0640 1 -- 172.21.15.46:0/97162617 wait complete. 2023-12-02T00:30:02.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.146+0000 7fdabd5b0640 1 Processor -- start 2023-12-02T00:30:02.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.147+0000 7fdabd5b0640 1 -- start start 2023-12-02T00:30:02.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.147+0000 7fdabd5b0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdaa0003b30 0x7fdab80b1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:02.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.147+0000 7fdabd5b0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdab8157ec0 0x7fdab813cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:02.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.147+0000 7fdabd5b0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdab81f31c0 0x7fdab81f3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:02.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.147+0000 7fdabd5b0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab8156720 con 0x7fdab8157ec0 2023-12-02T00:30:02.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.147+0000 7fdabd5b0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab8158340 con 0x7fdaa0003b30 2023-12-02T00:30:02.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.147+0000 7fdabd5b0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdab80b2c10 con 0x7fdab81f31c0 2023-12-02T00:30:02.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.147+0000 7fdab67fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdab8157ec0 0x7fdab813cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:02.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.147+0000 7fdab67fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdab8157ec0 0x7fdab813cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50886/0 (socket says 172.21.15.46:50886) 2023-12-02T00:30:02.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.147+0000 7fdab67fc640 1 -- 172.21.15.46:0/4225269420 learned_addr learned my addr 172.21.15.46:0/4225269420 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:02.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.147+0000 7fdab6ffd640 1 --2- 172.21.15.46:0/4225269420 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdaa0003b30 0x7fdab80b1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:02.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.147+0000 7fdab77fe640 1 --2- 172.21.15.46:0/4225269420 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdab81f31c0 0x7fdab81f3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:02.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.148+0000 7fdab67fc640 1 -- 172.21.15.46:0/4225269420 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdab81f31c0 msgr2=0x7fdab81f3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:02.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.148+0000 7fdab67fc640 1 --2- 172.21.15.46:0/4225269420 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdab81f31c0 0x7fdab81f3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:02.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.148+0000 7fdab67fc640 1 -- 172.21.15.46:0/4225269420 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdaa0003b30 msgr2=0x7fdab80b1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:02.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.148+0000 7fdab67fc640 1 --2- 172.21.15.46:0/4225269420 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdaa0003b30 0x7fdab80b1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:02.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.148+0000 7fdab67fc640 1 -- 172.21.15.46:0/4225269420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab813d310 con 0x7fdab8157ec0 2023-12-02T00:30:02.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.148+0000 7fdab67fc640 1 --2- 172.21.15.46:0/4225269420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdab8157ec0 0x7fdab813cdd0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fdaac00f800 tx=0x7fdaac00fcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:02.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.148+0000 7fda9f7fe640 1 -- 172.21.15.46:0/4225269420 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaac00feb0 con 0x7fdab8157ec0 2023-12-02T00:30:02.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.148+0000 7fda9f7fe640 1 -- 172.21.15.46:0/4225269420 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaac0052e0 con 0x7fdab8157ec0 2023-12-02T00:30:02.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.148+0000 7fdabd5b0640 1 -- 172.21.15.46:0/4225269420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdab80b22e0 con 0x7fdab8157ec0 2023-12-02T00:30:02.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.148+0000 7fda9f7fe640 1 -- 172.21.15.46:0/4225269420 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaac00d930 con 0x7fdab8157ec0 2023-12-02T00:30:02.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.148+0000 7fdabd5b0640 1 -- 172.21.15.46:0/4225269420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdab80b28a0 con 0x7fdab8157ec0 2023-12-02T00:30:02.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.149+0000 7fdabd5b0640 1 -- 172.21.15.46:0/4225269420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdab80b2a60 con 0x7fdab8157ec0 2023-12-02T00:30:02.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.150+0000 7fda9f7fe640 1 -- 172.21.15.46:0/4225269420 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdaac00da90 con 0x7fdab8157ec0 2023-12-02T00:30:02.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.150+0000 7fda9f7fe640 1 --2- 172.21.15.46:0/4225269420 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fdaa406ae70 0x7fdaa406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:02.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.150+0000 7fda9f7fe640 1 -- 172.21.15.46:0/4225269420 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdaac089280 con 0x7fdab8157ec0 2023-12-02T00:30:02.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.150+0000 7fdab6ffd640 1 --2- 172.21.15.46:0/4225269420 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fdaa406ae70 0x7fdaa406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:02.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.151+0000 7fdab6ffd640 1 --2- 172.21.15.46:0/4225269420 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fdaa406ae70 0x7fdaa406d330 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8034000 tx=0x7fdaa8007000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:02.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.154+0000 7fda9f7fe640 1 -- 172.21.15.46:0/4225269420 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdaac0543e0 con 0x7fdab8157ec0 2023-12-02T00:30:02.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.329+0000 7fdabd5b0640 1 -- 172.21.15.46:0/4225269420 --> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdab815e240 con 0x7fdaa406ae70 2023-12-02T00:30:02.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.336+0000 7fda9f7fe640 1 -- 172.21.15.46:0/4225269420 <== mgr.4108 v2:172.21.15.162:6832/3248644722 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+276860 (secure 0 0 0) 0x7fdaa8001470 con 0x7fdaa406ae70 2023-12-02T00:30:02.340 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:30:02.341 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-12-02T00:30:02.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.339+0000 7fdabd5b0640 1 -- 172.21.15.46:0/4225269420 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fdaa406ae70 msgr2=0x7fdaa406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:02.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.339+0000 7fdabd5b0640 1 --2- 172.21.15.46:0/4225269420 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fdaa406ae70 0x7fdaa406d330 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8034000 tx=0x7fdaa8007000 comp rx=0 tx=0).stop 2023-12-02T00:30:02.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.339+0000 7fdabd5b0640 1 -- 172.21.15.46:0/4225269420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdab8157ec0 msgr2=0x7fdab813cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:02.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.339+0000 7fdabd5b0640 1 --2- 172.21.15.46:0/4225269420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdab8157ec0 0x7fdab813cdd0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fdaac00f800 tx=0x7fdaac00fcc0 comp rx=0 tx=0).stop 2023-12-02T00:30:02.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.339+0000 7fdabd5b0640 1 -- 172.21.15.46:0/4225269420 shutdown_connections 2023-12-02T00:30:02.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.339+0000 7fdabd5b0640 1 --2- 172.21.15.46:0/4225269420 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fdaa406ae70 0x7fdaa406d330 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:02.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.339+0000 7fdabd5b0640 1 --2- 172.21.15.46:0/4225269420 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdab81f31c0 0x7fdab81f3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:02.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.339+0000 7fdabd5b0640 1 --2- 172.21.15.46:0/4225269420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdab8157ec0 0x7fdab813cdd0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:02.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.340+0000 7fdabd5b0640 1 --2- 172.21.15.46:0/4225269420 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdaa0003b30 0x7fdab80b1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:02.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.340+0000 7fdabd5b0640 1 -- 172.21.15.46:0/4225269420 >> 172.21.15.46:0/4225269420 conn(0x7fdab806a6b0 msgr2=0x7fdab806ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:02.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.340+0000 7fdabd5b0640 1 -- 172.21.15.46:0/4225269420 shutdown_connections 2023-12-02T00:30:02.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:02.340+0000 7fdabd5b0640 1 -- 172.21.15.46:0/4225269420 wait complete. 2023-12-02T00:30:02.351 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2023-12-02T00:30:01.175151+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236708,"kb_used_data":3896,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738012,"statfs":{"total":773094113280,"available":772851724288,"internally_reserved":0,"allocated":3989504,"data_stored":2243716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.278482"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.903298+0000","last_change":"2023-12-02T00:29:55.903298+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.903298+0000","last_undegraded":"2023-12-02T00:29:55.903298+0000","last_fullsized":"2023-12-02T00:29:55.903298+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:55.891930+0000","last_change":"2023-12-02T00:29:55.891930+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.891930+0000","last_undegraded":"2023-12-02T00:29:55.891930+0000","last_fullsized":"2023-12-02T00:29:55.891930+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.900712+0000","last_change":"2023-12-02T00:29:55.900712+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.900712+0000","last_undegraded":"2023-12-02T00:29:55.900712+0000","last_fullsized":"2023-12-02T00:29:55.900712+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.901454+0000","last_change":"2023-12-02T00:29:55.901454+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.901454+0000","last_undegraded":"2023-12-02T00:29:55.901454+0000","last_fullsized":"2023-12-02T00:29:55.901454+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.900095+0000","last_change":"2023-12-02T00:29:55.900095+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.900095+0000","last_undegraded":"2023-12-02T00:29:55.900095+0000","last_fullsized":"2023-12-02T00:29:55.900095+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.898451+0000","last_change":"2023-12-02T00:29:55.898451+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.898451+0000","last_undegraded":"2023-12-02T00:29:55.898451+0000","last_fullsized":"2023-12-02T00:29:55.898451+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.900139+0000","last_change":"2023-12-02T00:29:55.900139+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.900139+0000","last_undegraded":"2023-12-02T00:29:55.900139+0000","last_fullsized":"2023-12-02T00:29:55.900139+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.898680+0000","last_change":"2023-12-02T00:29:55.898680+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.898680+0000","last_undegraded":"2023-12-02T00:29:55.898680+0000","last_fullsized":"2023-12-02T00:29:55.898680+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.899737+0000","last_change":"2023-12-02T00:29:55.899737+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.899737+0000","last_undegraded":"2023-12-02T00:29:55.899737+0000","last_fullsized":"2023-12-02T00:29:55.899737+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:55.891930+0000","last_change":"2023-12-02T00:29:55.891930+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.891930+0000","last_undegraded":"2023-12-02T00:29:55.891930+0000","last_fullsized":"2023-12-02T00:29:55.891930+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.898306+0000","last_change":"2023-12-02T00:29:55.898306+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.898306+0000","last_undegraded":"2023-12-02T00:29:55.898306+0000","last_fullsized":"2023-12-02T00:29:55.898306+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.899642+0000","last_change":"2023-12-02T00:29:55.899642+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.899642+0000","last_undegraded":"2023-12-02T00:29:55.899642+0000","last_fullsized":"2023-12-02T00:29:55.899642+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.899635+0000","last_change":"2023-12-02T00:29:55.899635+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.899635+0000","last_undegraded":"2023-12-02T00:29:55.899635+0000","last_fullsized":"2023-12-02T00:29:55.899635+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.902337+0000","last_change":"2023-12-02T00:29:55.902337+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.902337+0000","last_undegraded":"2023-12-02T00:29:55.902337+0000","last_fullsized":"2023-12-02T00:29:55.902337+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.899585+0000","last_change":"2023-12-02T00:29:55.899585+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.899585+0000","last_undegraded":"2023-12-02T00:29:55.899585+0000","last_fullsized":"2023-12-02T00:29:55.899585+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.899587+0000","last_change":"2023-12-02T00:29:55.899587+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.899587+0000","last_undegraded":"2023-12-02T00:29:55.899587+0000","last_fullsized":"2023-12-02T00:29:55.899587+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.902333+0000","last_change":"2023-12-02T00:29:55.902333+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.902333+0000","last_undegraded":"2023-12-02T00:29:55.902333+0000","last_fullsized":"2023-12-02T00:29:55.902333+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.898944+0000","last_change":"2023-12-02T00:29:55.898944+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.898944+0000","last_undegraded":"2023-12-02T00:29:55.898944+0000","last_fullsized":"2023-12-02T00:29:55.898944+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.899297+0000","last_change":"2023-12-02T00:29:55.899297+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.899297+0000","last_undegraded":"2023-12-02T00:29:55.899297+0000","last_fullsized":"2023-12-02T00:29:55.899297+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.898740+0000","last_change":"2023-12-02T00:29:55.898740+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.898740+0000","last_undegraded":"2023-12-02T00:29:55.898740+0000","last_fullsized":"2023-12-02T00:29:55.898740+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:55.891930+0000","last_change":"2023-12-02T00:29:55.891930+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.891930+0000","last_undegraded":"2023-12-02T00:29:55.891930+0000","last_fullsized":"2023-12-02T00:29:55.891930+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.903215+0000","last_change":"2023-12-02T00:29:55.903215+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.903215+0000","last_undegraded":"2023-12-02T00:29:55.903215+0000","last_fullsized":"2023-12-02T00:29:55.903215+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.900528+0000","last_change":"2023-12-02T00:29:55.900528+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.900528+0000","last_undegraded":"2023-12-02T00:29:55.900528+0000","last_fullsized":"2023-12-02T00:29:55.900528+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.898308+0000","last_change":"2023-12-02T00:29:55.898308+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.898308+0000","last_undegraded":"2023-12-02T00:29:55.898308+0000","last_fullsized":"2023-12-02T00:29:55.898308+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.901881+0000","last_change":"2023-12-02T00:29:55.901881+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.901881+0000","last_undegraded":"2023-12-02T00:29:55.901881+0000","last_fullsized":"2023-12-02T00:29:55.901881+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:55.891930+0000","last_change":"2023-12-02T00:29:55.891930+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.891930+0000","last_undegraded":"2023-12-02T00:29:55.891930+0000","last_fullsized":"2023-12-02T00:29:55.891930+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.901839+0000","last_change":"2023-12-02T00:29:55.901839+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.901839+0000","last_undegraded":"2023-12-02T00:29:55.901839+0000","last_fullsized":"2023-12-02T00:29:55.901839+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:55.891930+0000","last_change":"2023-12-02T00:29:55.891930+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.891930+0000","last_undegraded":"2023-12-02T00:29:55.891930+0000","last_fullsized":"2023-12-02T00:29:55.891930+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.901835+0000","last_change":"2023-12-02T00:29:55.901835+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.901835+0000","last_undegraded":"2023-12-02T00:29:55.901835+0000","last_fullsized":"2023-12-02T00:29:55.901835+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.898934+0000","last_change":"2023-12-02T00:29:55.898934+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.898934+0000","last_undegraded":"2023-12-02T00:29:55.898934+0000","last_fullsized":"2023-12-02T00:29:55.898934+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.899146+0000","last_change":"2023-12-02T00:29:55.899146+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.899146+0000","last_undegraded":"2023-12-02T00:29:55.899146+0000","last_fullsized":"2023-12-02T00:29:55.899146+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-02T00:29:56.896331+0000","last_change":"2023-12-02T00:29:56.896331+0000","last_active":"2023-12-02T00:29:56.896331+0000","last_peered":"2023-12-02T00:29:56.896331+0000","last_clean":"2023-12-02T00:29:56.896331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:56.896331+0000","last_undegraded":"2023-12-02T00:29:56.896331+0000","last_fullsized":"2023-12-02T00:29:56.896331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-02T00:29:55.903298+0000","last_change":"2023-12-02T00:29:55.903298+0000","last_active":"2023-12-02T00:29:55.891930+0000","last_peered":"2023-12-02T00:29:55.891930+0000","last_clean":"2023-12-02T00:29:55.891930+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:29:55.903298+0000","last_undegraded":"2023-12-02T00:29:55.903298+0000","last_fullsized":"2023-12-02T00:29:55.903298+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":175,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-12-02T00:29:52.806248+0000","last_change":"2023-12-02T00:28:39.379380+0000","last_active":"2023-12-02T00:29:52.806248+0000","last_peered":"2023-12-02T00:29:52.806248+0000","last_clean":"2023-12-02T00:29:52.806248+0000","last_became_active":"2023-12-02T00:28:39.378255+0000","last_became_peered":"2023-12-02T00:28:39.378255+0000","last_unstale":"2023-12-02T00:29:52.806248+0000","last_undegraded":"2023-12-02T00:29:52.806248+0000","last_fullsized":"2023-12-02T00:29:52.806248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:38.177345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:38.177345+0000","last_clean_scrub_stamp":"2023-12-02T00:28:38.177345+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:21:08.582412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":54,"acting":54,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":721941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.45900000000000002}]},{"osd":1,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.50700000000000001}]},{"osd":2,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.61599999999999999}]},{"osd":3,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.436}]},{"osd":4,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.57899999999999996}]},{"osd":5,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.59799999999999998}]},{"osd":6,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.68100000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":360448,"data_stored":137390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.59699999999999998}]},{"osd":1,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.46500000000000002}]},{"osd":2,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.443},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.497}]},{"osd":3,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.629}]},{"osd":4,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.56799999999999995}]},{"osd":5,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.64800000000000002}]},{"osd":7,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.72999999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":131573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.49099999999999999}]},{"osd":1,"last update":"Sat Dec 2 00:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.41799999999999998}]},{"osd":2,"last update":"Sat Dec 2 00:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.749}]},{"osd":3,"last update":"Sat Dec 2 00:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.63300000000000001}]},{"osd":4,"last update":"Sat Dec 2 00:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.66900000000000004}]},{"osd":6,"last update":"Sat Dec 2 00:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.77800000000000002}]},{"osd":7,"last update":"Sat Dec 2 00:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.76700000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":131573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.58099999999999996}]},{"osd":1,"last update":"Sat Dec 2 00:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.47199999999999998}]},{"osd":2,"last update":"Sat Dec 2 00:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.443},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.64200000000000002}]},{"osd":3,"last update":"Sat Dec 2 00:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.55800000000000005}]},{"osd":5,"last update":"Sat Dec 2 00:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.42199999999999999}]},{"osd":6,"last update":"Sat Dec 2 00:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.629},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.51900000000000002}]},{"osd":7,"last update":"Sat Dec 2 00:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.59999999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31532,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340308,"statfs":{"total":96636764160,"available":96604475392,"internally_reserved":0,"allocated":360448,"data_stored":137390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.28599999999999998}]},{"osd":1,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.36599999999999999}]},{"osd":2,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.30599999999999999}]},{"osd":4,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.52700000000000002}]},{"osd":5,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.64300000000000002}]},{"osd":6,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.58899999999999997}]},{"osd":7,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.45200000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31520,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340320,"statfs":{"total":96636764160,"available":96604487680,"internally_reserved":0,"allocated":344064,"data_stored":130954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.22}]},{"osd":1,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.503,"5min":0.503,"15min":0.503},"last":0.317}]},{"osd":3,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.42399999999999999}]},{"osd":4,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.55700000000000005}]},{"osd":5,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.442},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.59299999999999997}]},{"osd":6,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.52100000000000002}]},{"osd":7,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.72699999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31520,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340320,"statfs":{"total":96636764160,"available":96604487680,"internally_reserved":0,"allocated":348160,"data_stored":131573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.378}]},{"osd":2,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.41499999999999998}]},{"osd":3,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.34399999999999997}]},{"osd":4,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.56999999999999995}]},{"osd":5,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.437}]},{"osd":6,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.54900000000000004}]},{"osd":7,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.45300000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":937984,"data_stored":721322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.58399999999999996}]},{"osd":2,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.71699999999999997}]},{"osd":3,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.67400000000000004}]},{"osd":4,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.74199999999999999}]},{"osd":5,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.69599999999999995}]},{"osd":6,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.82099999999999995}]},{"osd":7,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.78000000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-02T00:30:05.356 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-02T00:30:05.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.474+0000 7f149a15a640 1 Processor -- start 2023-12-02T00:30:05.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.474+0000 7f149a15a640 1 -- start start 2023-12-02T00:30:05.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.475+0000 7f149a15a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f149406ea50 0x7f149406ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:05.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.475+0000 7f149a15a640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1494160fb0 0x7f1494161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:05.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.475+0000 7f149a15a640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f14940b3ff0 0x7f14940b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:05.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.475+0000 7f149a15a640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1494161d20 con 0x7f14940b3ff0 2023-12-02T00:30:05.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.475+0000 7f149a15a640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f149406f370 con 0x7f1494160fb0 2023-12-02T00:30:05.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.475+0000 7f149a15a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f149415e760 con 0x7f149406ea50 2023-12-02T00:30:05.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.475+0000 7f1493fff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f14940b3ff0 0x7f14940b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:05.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.475+0000 7f1493fff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f14940b3ff0 0x7f14940b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46108/0 (socket says 172.21.15.46:46108) 2023-12-02T00:30:05.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.476+0000 7f1493fff640 1 -- 172.21.15.46:0/2319402876 learned_addr learned my addr 172.21.15.46:0/2319402876 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:05.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.476+0000 7f14937fe640 1 --2- 172.21.15.46:0/2319402876 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f149406ea50 0x7f149406ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:05.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.476+0000 7f1492ffd640 1 --2- 172.21.15.46:0/2319402876 >> v2:172.21.15.162:3300/0 conn(0x7f1494160fb0 0x7f1494161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:05.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.476+0000 7f1493fff640 1 -- 172.21.15.46:0/2319402876 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f149406ea50 msgr2=0x7f149406ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:05.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.476+0000 7f1493fff640 1 --2- 172.21.15.46:0/2319402876 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f149406ea50 0x7f149406ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:05.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.476+0000 7f1493fff640 1 -- 172.21.15.46:0/2319402876 >> v2:172.21.15.162:3300/0 conn(0x7f1494160fb0 msgr2=0x7f1494161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:05.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.476+0000 7f1493fff640 1 --2- 172.21.15.46:0/2319402876 >> v2:172.21.15.162:3300/0 conn(0x7f1494160fb0 0x7f1494161410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:05.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.476+0000 7f1493fff640 1 -- 172.21.15.46:0/2319402876 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f149415d9a0 con 0x7f14940b3ff0 2023-12-02T00:30:05.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.476+0000 7f14937fe640 1 --2- 172.21.15.46:0/2319402876 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f149406ea50 0x7f149406ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:30:05.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.476+0000 7f1492ffd640 1 --2- 172.21.15.46:0/2319402876 >> v2:172.21.15.162:3300/0 conn(0x7f1494160fb0 0x7f1494161410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:30:05.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.477+0000 7f1493fff640 1 --2- 172.21.15.46:0/2319402876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14940b3ff0 0x7f14940b4460 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f14940d2120 tx=0x7f148c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ab7db78ee8ff83 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:05.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.477+0000 7f14927fc640 1 -- 172.21.15.46:0/2319402876 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148c02c9e0 con 0x7f14940b3ff0 2023-12-02T00:30:05.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.477+0000 7f14927fc640 1 -- 172.21.15.46:0/2319402876 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f148c02cb40 con 0x7f14940b3ff0 2023-12-02T00:30:05.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.477+0000 7f14927fc640 1 -- 172.21.15.46:0/2319402876 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148c0343f0 con 0x7f14940b3ff0 2023-12-02T00:30:05.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.477+0000 7f149a15a640 1 -- 172.21.15.46:0/2319402876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14940b3ff0 msgr2=0x7f14940b4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:05.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.477+0000 7f149a15a640 1 --2- 172.21.15.46:0/2319402876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14940b3ff0 0x7f14940b4460 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f14940d2120 tx=0x7f148c02be40 comp rx=0 tx=0).stop 2023-12-02T00:30:05.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.477+0000 7f149a15a640 1 -- 172.21.15.46:0/2319402876 shutdown_connections 2023-12-02T00:30:05.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.477+0000 7f149a15a640 1 --2- 172.21.15.46:0/2319402876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14940b3ff0 0x7f14940b4460 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:05.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.477+0000 7f149a15a640 1 --2- 172.21.15.46:0/2319402876 >> v2:172.21.15.162:3300/0 conn(0x7f1494160fb0 0x7f1494161410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:05.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.477+0000 7f149a15a640 1 --2- 172.21.15.46:0/2319402876 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f149406ea50 0x7f149406ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:05.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.477+0000 7f149a15a640 1 -- 172.21.15.46:0/2319402876 >> 172.21.15.46:0/2319402876 conn(0x7f149406a6b0 msgr2=0x7f149406aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:05.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.477+0000 7f149a15a640 1 -- 172.21.15.46:0/2319402876 shutdown_connections 2023-12-02T00:30:05.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.477+0000 7f149a15a640 1 -- 172.21.15.46:0/2319402876 wait complete. 2023-12-02T00:30:05.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.479+0000 7f149a15a640 1 Processor -- start 2023-12-02T00:30:05.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.479+0000 7f149a15a640 1 -- start start 2023-12-02T00:30:05.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.480+0000 7f149a15a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f149406ea50 0x7f1494157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:05.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.480+0000 7f149a15a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14940b3ff0 0x7f14940cf320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:05.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.480+0000 7f149a15a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1494160fb0 0x7f149415c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:05.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.480+0000 7f149a15a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14940cfc50 con 0x7f149406ea50 2023-12-02T00:30:05.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.480+0000 7f149a15a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14940cff30 con 0x7f1494160fb0 2023-12-02T00:30:05.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.480+0000 7f149a15a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14940d0210 con 0x7f14940b3ff0 2023-12-02T00:30:05.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.480+0000 7f14937fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f149406ea50 0x7f1494157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:05.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.480+0000 7f14937fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f149406ea50 0x7f1494157050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46118/0 (socket says 172.21.15.46:46118) 2023-12-02T00:30:05.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.480+0000 7f14937fe640 1 -- 172.21.15.46:0/184655405 learned_addr learned my addr 172.21.15.46:0/184655405 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:05.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.480+0000 7f1493fff640 1 --2- 172.21.15.46:0/184655405 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1494160fb0 0x7f149415c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:05.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.480+0000 7f1492ffd640 1 --2- 172.21.15.46:0/184655405 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14940b3ff0 0x7f14940cf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:05.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.480+0000 7f14937fe640 1 -- 172.21.15.46:0/184655405 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14940b3ff0 msgr2=0x7f14940cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:05.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.480+0000 7f14937fe640 1 --2- 172.21.15.46:0/184655405 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14940b3ff0 0x7f14940cf320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:05.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.480+0000 7f14937fe640 1 -- 172.21.15.46:0/184655405 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1494160fb0 msgr2=0x7f149415c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:05.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.480+0000 7f14937fe640 1 --2- 172.21.15.46:0/184655405 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1494160fb0 0x7f149415c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:05.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.480+0000 7f14937fe640 1 -- 172.21.15.46:0/184655405 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f148c009b30 con 0x7f149406ea50 2023-12-02T00:30:05.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.480+0000 7f14937fe640 1 --2- 172.21.15.46:0/184655405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f149406ea50 0x7f1494157050 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f148400c7f0 tx=0x7f148400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:05.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.481+0000 7f147bfff640 1 -- 172.21.15.46:0/184655405 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1484007820 con 0x7f149406ea50 2023-12-02T00:30:05.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.481+0000 7f147bfff640 1 -- 172.21.15.46:0/184655405 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1484007980 con 0x7f149406ea50 2023-12-02T00:30:05.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.481+0000 7f149a15a640 1 -- 172.21.15.46:0/184655405 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14940d04f0 con 0x7f149406ea50 2023-12-02T00:30:05.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.481+0000 7f147bfff640 1 -- 172.21.15.46:0/184655405 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1484007cf0 con 0x7f149406ea50 2023-12-02T00:30:05.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.481+0000 7f149a15a640 1 -- 172.21.15.46:0/184655405 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14940cf860 con 0x7f149406ea50 2023-12-02T00:30:05.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.482+0000 7f149a15a640 1 -- 172.21.15.46:0/184655405 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f149406ee30 con 0x7f149406ea50 2023-12-02T00:30:05.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.482+0000 7f147bfff640 1 -- 172.21.15.46:0/184655405 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f14840113f0 con 0x7f149406ea50 2023-12-02T00:30:05.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.483+0000 7f147bfff640 1 --2- 172.21.15.46:0/184655405 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f148006ae40 0x7f148006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:05.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.483+0000 7f147bfff640 1 -- 172.21.15.46:0/184655405 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f14840897e0 con 0x7f149406ea50 2023-12-02T00:30:05.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.483+0000 7f1492ffd640 1 --2- 172.21.15.46:0/184655405 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f148006ae40 0x7f148006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:05.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.483+0000 7f1492ffd640 1 --2- 172.21.15.46:0/184655405 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f148006ae40 0x7f148006d300 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f14940d0970 tx=0x7f1488008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:05.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.487+0000 7f147bfff640 1 -- 172.21.15.46:0/184655405 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f14840549d0 con 0x7f149406ea50 2023-12-02T00:30:05.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.660+0000 7f149a15a640 1 -- 172.21.15.46:0/184655405 --> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1494157a90 con 0x7f148006ae40 2023-12-02T00:30:05.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.668+0000 7f147bfff640 1 -- 172.21.15.46:0/184655405 <== mgr.4108 v2:172.21.15.162:6832/3248644722 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289319 (secure 0 0 0) 0x7f1494157a90 con 0x7f148006ae40 2023-12-02T00:30:05.672 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:30:05.673 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-12-02T00:30:05.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.671+0000 7f149a15a640 1 -- 172.21.15.46:0/184655405 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f148006ae40 msgr2=0x7f148006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:05.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.671+0000 7f149a15a640 1 --2- 172.21.15.46:0/184655405 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f148006ae40 0x7f148006d300 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f14940d0970 tx=0x7f1488008040 comp rx=0 tx=0).stop 2023-12-02T00:30:05.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.671+0000 7f149a15a640 1 -- 172.21.15.46:0/184655405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f149406ea50 msgr2=0x7f1494157050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:05.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.671+0000 7f149a15a640 1 --2- 172.21.15.46:0/184655405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f149406ea50 0x7f1494157050 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f148400c7f0 tx=0x7f148400ccb0 comp rx=0 tx=0).stop 2023-12-02T00:30:05.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.671+0000 7f149a15a640 1 -- 172.21.15.46:0/184655405 shutdown_connections 2023-12-02T00:30:05.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.671+0000 7f149a15a640 1 --2- 172.21.15.46:0/184655405 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f148006ae40 0x7f148006d300 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:05.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.671+0000 7f149a15a640 1 --2- 172.21.15.46:0/184655405 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1494160fb0 0x7f149415c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:05.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.671+0000 7f149a15a640 1 --2- 172.21.15.46:0/184655405 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14940b3ff0 0x7f14940cf320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:05.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.671+0000 7f149a15a640 1 --2- 172.21.15.46:0/184655405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f149406ea50 0x7f1494157050 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:05.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.671+0000 7f149a15a640 1 -- 172.21.15.46:0/184655405 >> 172.21.15.46:0/184655405 conn(0x7f149406a6b0 msgr2=0x7f14940b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:05.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.671+0000 7f149a15a640 1 -- 172.21.15.46:0/184655405 shutdown_connections 2023-12-02T00:30:05.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.672+0000 7f149a15a640 1 -- 172.21.15.46:0/184655405 wait complete. 2023-12-02T00:30:05.683 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":73,"stamp":"2023-12-02T00:30:05.176224+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":172,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237124,"kb_used_data":4320,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737596,"statfs":{"total":773094113280,"available":772851298304,"internally_reserved":0,"allocated":4423680,"data_stored":2438174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001960"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:58.927358+0000","last_change":"2023-12-02T00:29:57.924195+0000","last_active":"2023-12-02T00:29:58.927358+0000","last_peered":"2023-12-02T00:29:58.927358+0000","last_clean":"2023-12-02T00:29:58.927358+0000","last_became_active":"2023-12-02T00:29:57.923891+0000","last_became_peered":"2023-12-02T00:29:57.923891+0000","last_unstale":"2023-12-02T00:29:58.927358+0000","last_undegraded":"2023-12-02T00:29:58.927358+0000","last_fullsized":"2023-12-02T00:29:58.927358+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T01:36:06.225035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:59.731024+0000","last_change":"2023-12-02T00:29:57.921252+0000","last_active":"2023-12-02T00:29:59.731024+0000","last_peered":"2023-12-02T00:29:59.731024+0000","last_clean":"2023-12-02T00:29:59.731024+0000","last_became_active":"2023-12-02T00:29:57.920883+0000","last_became_peered":"2023-12-02T00:29:57.920883+0000","last_unstale":"2023-12-02T00:29:59.731024+0000","last_undegraded":"2023-12-02T00:29:59.731024+0000","last_fullsized":"2023-12-02T00:29:59.731024+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:11:12.543774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:58.927909+0000","last_change":"2023-12-02T00:29:57.930468+0000","last_active":"2023-12-02T00:29:58.927909+0000","last_peered":"2023-12-02T00:29:58.927909+0000","last_clean":"2023-12-02T00:29:58.927909+0000","last_became_active":"2023-12-02T00:29:57.930124+0000","last_became_peered":"2023-12-02T00:29:57.930124+0000","last_unstale":"2023-12-02T00:29:58.927909+0000","last_undegraded":"2023-12-02T00:29:58.927909+0000","last_fullsized":"2023-12-02T00:29:58.927909+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:11:12.543774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:58.921619+0000","last_change":"2023-12-02T00:29:57.924291+0000","last_active":"2023-12-02T00:29:58.921619+0000","last_peered":"2023-12-02T00:29:58.921619+0000","last_clean":"2023-12-02T00:29:58.921619+0000","last_became_active":"2023-12-02T00:29:57.924013+0000","last_became_peered":"2023-12-02T00:29:57.924013+0000","last_unstale":"2023-12-02T00:29:58.921619+0000","last_undegraded":"2023-12-02T00:29:58.921619+0000","last_fullsized":"2023-12-02T00:29:58.921619+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:56:01.736069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:59.208813+0000","last_change":"2023-12-02T00:29:57.929184+0000","last_active":"2023-12-02T00:29:59.208813+0000","last_peered":"2023-12-02T00:29:59.208813+0000","last_clean":"2023-12-02T00:29:59.208813+0000","last_became_active":"2023-12-02T00:29:57.928263+0000","last_became_peered":"2023-12-02T00:29:57.928263+0000","last_unstale":"2023-12-02T00:29:59.208813+0000","last_undegraded":"2023-12-02T00:29:59.208813+0000","last_fullsized":"2023-12-02T00:29:59.208813+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T01:34:59.514513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:59.730801+0000","last_change":"2023-12-02T00:29:57.923512+0000","last_active":"2023-12-02T00:29:59.730801+0000","last_peered":"2023-12-02T00:29:59.730801+0000","last_clean":"2023-12-02T00:29:59.730801+0000","last_became_active":"2023-12-02T00:29:57.923245+0000","last_became_peered":"2023-12-02T00:29:57.923245+0000","last_unstale":"2023-12-02T00:29:59.730801+0000","last_undegraded":"2023-12-02T00:29:59.730801+0000","last_fullsized":"2023-12-02T00:29:59.730801+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T02:53:46.303522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:59.731333+0000","last_change":"2023-12-02T00:29:57.914606+0000","last_active":"2023-12-02T00:29:59.731333+0000","last_peered":"2023-12-02T00:29:59.731333+0000","last_clean":"2023-12-02T00:29:59.731333+0000","last_became_active":"2023-12-02T00:29:57.913570+0000","last_became_peered":"2023-12-02T00:29:57.913570+0000","last_unstale":"2023-12-02T00:29:59.731333+0000","last_undegraded":"2023-12-02T00:29:59.731333+0000","last_fullsized":"2023-12-02T00:29:59.731333+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:12:11.434947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:59.731159+0000","last_change":"2023-12-02T00:29:57.915590+0000","last_active":"2023-12-02T00:29:59.731159+0000","last_peered":"2023-12-02T00:29:59.731159+0000","last_clean":"2023-12-02T00:29:59.731159+0000","last_became_active":"2023-12-02T00:29:57.915318+0000","last_became_peered":"2023-12-02T00:29:57.915318+0000","last_unstale":"2023-12-02T00:29:59.731159+0000","last_undegraded":"2023-12-02T00:29:59.731159+0000","last_fullsized":"2023-12-02T00:29:59.731159+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:57:03.990444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:58.927295+0000","last_change":"2023-12-02T00:29:57.923438+0000","last_active":"2023-12-02T00:29:58.927295+0000","last_peered":"2023-12-02T00:29:58.927295+0000","last_clean":"2023-12-02T00:29:58.927295+0000","last_became_active":"2023-12-02T00:29:57.923167+0000","last_became_peered":"2023-12-02T00:29:57.923167+0000","last_unstale":"2023-12-02T00:29:58.927295+0000","last_undegraded":"2023-12-02T00:29:58.927295+0000","last_fullsized":"2023-12-02T00:29:58.927295+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:09:03.223346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:58.921889+0000","last_change":"2023-12-02T00:29:57.935347+0000","last_active":"2023-12-02T00:29:58.921889+0000","last_peered":"2023-12-02T00:29:58.921889+0000","last_clean":"2023-12-02T00:29:58.921889+0000","last_became_active":"2023-12-02T00:29:57.935073+0000","last_became_peered":"2023-12-02T00:29:57.935073+0000","last_unstale":"2023-12-02T00:29:58.921889+0000","last_undegraded":"2023-12-02T00:29:58.921889+0000","last_fullsized":"2023-12-02T00:29:58.921889+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:30:31.799693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:59.731409+0000","last_change":"2023-12-02T00:29:57.930661+0000","last_active":"2023-12-02T00:29:59.731409+0000","last_peered":"2023-12-02T00:29:59.731409+0000","last_clean":"2023-12-02T00:29:59.731409+0000","last_became_active":"2023-12-02T00:29:57.930389+0000","last_became_peered":"2023-12-02T00:29:57.930389+0000","last_unstale":"2023-12-02T00:29:59.731409+0000","last_undegraded":"2023-12-02T00:29:59.731409+0000","last_fullsized":"2023-12-02T00:29:59.731409+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:32:48.460074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:58.927672+0000","last_change":"2023-12-02T00:29:57.929910+0000","last_active":"2023-12-02T00:29:58.927672+0000","last_peered":"2023-12-02T00:29:58.927672+0000","last_clean":"2023-12-02T00:29:58.927672+0000","last_became_active":"2023-12-02T00:29:57.929610+0000","last_became_peered":"2023-12-02T00:29:57.929610+0000","last_unstale":"2023-12-02T00:29:58.927672+0000","last_undegraded":"2023-12-02T00:29:58.927672+0000","last_fullsized":"2023-12-02T00:29:58.927672+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:12:11.434947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:59.208796+0000","last_change":"2023-12-02T00:29:57.925784+0000","last_active":"2023-12-02T00:29:59.208796+0000","last_peered":"2023-12-02T00:29:59.208796+0000","last_clean":"2023-12-02T00:29:59.208796+0000","last_became_active":"2023-12-02T00:29:57.925520+0000","last_became_peered":"2023-12-02T00:29:57.925520+0000","last_unstale":"2023-12-02T00:29:59.208796+0000","last_undegraded":"2023-12-02T00:29:59.208796+0000","last_fullsized":"2023-12-02T00:29:59.208796+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T12:15:19.575280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:59.730760+0000","last_change":"2023-12-02T00:29:57.922480+0000","last_active":"2023-12-02T00:29:59.730760+0000","last_peered":"2023-12-02T00:29:59.730760+0000","last_clean":"2023-12-02T00:29:59.730760+0000","last_became_active":"2023-12-02T00:29:57.921218+0000","last_became_peered":"2023-12-02T00:29:57.921218+0000","last_unstale":"2023-12-02T00:29:59.730760+0000","last_undegraded":"2023-12-02T00:29:59.730760+0000","last_fullsized":"2023-12-02T00:29:59.730760+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:12:11.434947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:58.927120+0000","last_change":"2023-12-02T00:29:57.918299+0000","last_active":"2023-12-02T00:29:58.927120+0000","last_peered":"2023-12-02T00:29:58.927120+0000","last_clean":"2023-12-02T00:29:58.927120+0000","last_became_active":"2023-12-02T00:29:57.917983+0000","last_became_peered":"2023-12-02T00:29:57.917983+0000","last_unstale":"2023-12-02T00:29:58.927120+0000","last_undegraded":"2023-12-02T00:29:58.927120+0000","last_fullsized":"2023-12-02T00:29:58.927120+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T02:59:31.950169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:59.731420+0000","last_change":"2023-12-02T00:29:57.921604+0000","last_active":"2023-12-02T00:29:59.731420+0000","last_peered":"2023-12-02T00:29:59.731420+0000","last_clean":"2023-12-02T00:29:59.731420+0000","last_became_active":"2023-12-02T00:29:57.921280+0000","last_became_peered":"2023-12-02T00:29:57.921280+0000","last_unstale":"2023-12-02T00:29:59.731420+0000","last_undegraded":"2023-12-02T00:29:59.731420+0000","last_fullsized":"2023-12-02T00:29:59.731420+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:11:12.543774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:59.731540+0000","last_change":"2023-12-02T00:29:57.912565+0000","last_active":"2023-12-02T00:29:59.731540+0000","last_peered":"2023-12-02T00:29:59.731540+0000","last_clean":"2023-12-02T00:29:59.731540+0000","last_became_active":"2023-12-02T00:29:57.912316+0000","last_became_peered":"2023-12-02T00:29:57.912316+0000","last_unstale":"2023-12-02T00:29:59.731540+0000","last_undegraded":"2023-12-02T00:29:59.731540+0000","last_fullsized":"2023-12-02T00:29:59.731540+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:18:11.139003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:59.731087+0000","last_change":"2023-12-02T00:29:57.928493+0000","last_active":"2023-12-02T00:29:59.731087+0000","last_peered":"2023-12-02T00:29:59.731087+0000","last_clean":"2023-12-02T00:29:59.731087+0000","last_became_active":"2023-12-02T00:29:57.928242+0000","last_became_peered":"2023-12-02T00:29:57.928242+0000","last_unstale":"2023-12-02T00:29:59.731087+0000","last_undegraded":"2023-12-02T00:29:59.731087+0000","last_fullsized":"2023-12-02T00:29:59.731087+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:00:13.079293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:59.731025+0000","last_change":"2023-12-02T00:29:57.932502+0000","last_active":"2023-12-02T00:29:59.731025+0000","last_peered":"2023-12-02T00:29:59.731025+0000","last_clean":"2023-12-02T00:29:59.731025+0000","last_became_active":"2023-12-02T00:29:57.932233+0000","last_became_peered":"2023-12-02T00:29:57.932233+0000","last_unstale":"2023-12-02T00:29:59.731025+0000","last_undegraded":"2023-12-02T00:29:59.731025+0000","last_fullsized":"2023-12-02T00:29:59.731025+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:54:25.311426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:59.208887+0000","last_change":"2023-12-02T00:29:57.927192+0000","last_active":"2023-12-02T00:29:59.208887+0000","last_peered":"2023-12-02T00:29:59.208887+0000","last_clean":"2023-12-02T00:29:59.208887+0000","last_became_active":"2023-12-02T00:29:57.926916+0000","last_became_peered":"2023-12-02T00:29:57.926916+0000","last_unstale":"2023-12-02T00:29:59.208887+0000","last_undegraded":"2023-12-02T00:29:59.208887+0000","last_fullsized":"2023-12-02T00:29:59.208887+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:43:18.542818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:58.928099+0000","last_change":"2023-12-02T00:29:57.921456+0000","last_active":"2023-12-02T00:29:58.928099+0000","last_peered":"2023-12-02T00:29:58.928099+0000","last_clean":"2023-12-02T00:29:58.928099+0000","last_became_active":"2023-12-02T00:29:57.921155+0000","last_became_peered":"2023-12-02T00:29:57.921155+0000","last_unstale":"2023-12-02T00:29:58.928099+0000","last_undegraded":"2023-12-02T00:29:58.928099+0000","last_fullsized":"2023-12-02T00:29:58.928099+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:30:31.799693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:58.928145+0000","last_change":"2023-12-02T00:29:57.930482+0000","last_active":"2023-12-02T00:29:58.928145+0000","last_peered":"2023-12-02T00:29:58.928145+0000","last_clean":"2023-12-02T00:29:58.928145+0000","last_became_active":"2023-12-02T00:29:57.929635+0000","last_became_peered":"2023-12-02T00:29:57.929635+0000","last_unstale":"2023-12-02T00:29:58.928145+0000","last_undegraded":"2023-12-02T00:29:58.928145+0000","last_fullsized":"2023-12-02T00:29:58.928145+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:15:52.725813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:59.617442+0000","last_change":"2023-12-02T00:29:57.929331+0000","last_active":"2023-12-02T00:29:59.617442+0000","last_peered":"2023-12-02T00:29:59.617442+0000","last_clean":"2023-12-02T00:29:59.617442+0000","last_became_active":"2023-12-02T00:29:57.929003+0000","last_became_peered":"2023-12-02T00:29:57.929003+0000","last_unstale":"2023-12-02T00:29:59.617442+0000","last_undegraded":"2023-12-02T00:29:59.617442+0000","last_fullsized":"2023-12-02T00:29:59.617442+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:25:27.359516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:58.926780+0000","last_change":"2023-12-02T00:29:57.917725+0000","last_active":"2023-12-02T00:29:58.926780+0000","last_peered":"2023-12-02T00:29:58.926780+0000","last_clean":"2023-12-02T00:29:58.926780+0000","last_became_active":"2023-12-02T00:29:57.917443+0000","last_became_peered":"2023-12-02T00:29:57.917443+0000","last_unstale":"2023-12-02T00:29:58.926780+0000","last_undegraded":"2023-12-02T00:29:58.926780+0000","last_fullsized":"2023-12-02T00:29:58.926780+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:24:47.248059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:58.922058+0000","last_change":"2023-12-02T00:29:57.930805+0000","last_active":"2023-12-02T00:29:58.922058+0000","last_peered":"2023-12-02T00:29:58.922058+0000","last_clean":"2023-12-02T00:29:58.922058+0000","last_became_active":"2023-12-02T00:29:57.930544+0000","last_became_peered":"2023-12-02T00:29:57.930544+0000","last_unstale":"2023-12-02T00:29:58.922058+0000","last_undegraded":"2023-12-02T00:29:58.922058+0000","last_fullsized":"2023-12-02T00:29:58.922058+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T01:52:39.055257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:59.731141+0000","last_change":"2023-12-02T00:29:57.916949+0000","last_active":"2023-12-02T00:29:59.731141+0000","last_peered":"2023-12-02T00:29:59.731141+0000","last_clean":"2023-12-02T00:29:59.731141+0000","last_became_active":"2023-12-02T00:29:57.916683+0000","last_became_peered":"2023-12-02T00:29:57.916683+0000","last_unstale":"2023-12-02T00:29:59.731141+0000","last_undegraded":"2023-12-02T00:29:59.731141+0000","last_fullsized":"2023-12-02T00:29:59.731141+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:16:15.161956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:58.921914+0000","last_change":"2023-12-02T00:29:57.927120+0000","last_active":"2023-12-02T00:29:58.921914+0000","last_peered":"2023-12-02T00:29:58.921914+0000","last_clean":"2023-12-02T00:29:58.921914+0000","last_became_active":"2023-12-02T00:29:57.926645+0000","last_became_peered":"2023-12-02T00:29:57.926645+0000","last_unstale":"2023-12-02T00:29:58.921914+0000","last_undegraded":"2023-12-02T00:29:58.921914+0000","last_fullsized":"2023-12-02T00:29:58.921914+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:15:52.725813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:58.927800+0000","last_change":"2023-12-02T00:29:57.920740+0000","last_active":"2023-12-02T00:29:58.927800+0000","last_peered":"2023-12-02T00:29:58.927800+0000","last_clean":"2023-12-02T00:29:58.927800+0000","last_became_active":"2023-12-02T00:29:57.920437+0000","last_became_peered":"2023-12-02T00:29:57.920437+0000","last_unstale":"2023-12-02T00:29:58.927800+0000","last_undegraded":"2023-12-02T00:29:58.927800+0000","last_fullsized":"2023-12-02T00:29:58.927800+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:32:48.460074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:58.928149+0000","last_change":"2023-12-02T00:29:57.930027+0000","last_active":"2023-12-02T00:29:58.928149+0000","last_peered":"2023-12-02T00:29:58.928149+0000","last_clean":"2023-12-02T00:29:58.928149+0000","last_became_active":"2023-12-02T00:29:57.929712+0000","last_became_peered":"2023-12-02T00:29:57.929712+0000","last_unstale":"2023-12-02T00:29:58.928149+0000","last_undegraded":"2023-12-02T00:29:58.928149+0000","last_fullsized":"2023-12-02T00:29:58.928149+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T09:23:38.713043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:59.731104+0000","last_change":"2023-12-02T00:29:57.912013+0000","last_active":"2023-12-02T00:29:59.731104+0000","last_peered":"2023-12-02T00:29:59.731104+0000","last_clean":"2023-12-02T00:29:59.731104+0000","last_became_active":"2023-12-02T00:29:57.911736+0000","last_became_peered":"2023-12-02T00:29:57.911736+0000","last_unstale":"2023-12-02T00:29:59.731104+0000","last_undegraded":"2023-12-02T00:29:59.731104+0000","last_fullsized":"2023-12-02T00:29:59.731104+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:24:47.248059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:58.927041+0000","last_change":"2023-12-02T00:29:57.914559+0000","last_active":"2023-12-02T00:29:58.927041+0000","last_peered":"2023-12-02T00:29:58.927041+0000","last_clean":"2023-12-02T00:29:58.927041+0000","last_became_active":"2023-12-02T00:29:57.914236+0000","last_became_peered":"2023-12-02T00:29:57.914236+0000","last_unstale":"2023-12-02T00:29:58.927041+0000","last_undegraded":"2023-12-02T00:29:58.927041+0000","last_fullsized":"2023-12-02T00:29:58.927041+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T10:41:59.077713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:59.731174+0000","last_change":"2023-12-02T00:29:57.921998+0000","last_active":"2023-12-02T00:29:59.731174+0000","last_peered":"2023-12-02T00:29:59.731174+0000","last_clean":"2023-12-02T00:29:59.731174+0000","last_became_active":"2023-12-02T00:29:57.921695+0000","last_became_peered":"2023-12-02T00:29:57.921695+0000","last_unstale":"2023-12-02T00:29:59.731174+0000","last_undegraded":"2023-12-02T00:29:59.731174+0000","last_fullsized":"2023-12-02T00:29:59.731174+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:57:01.750415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:59.617580+0000","last_change":"2023-12-02T00:29:57.929383+0000","last_active":"2023-12-02T00:29:59.617580+0000","last_peered":"2023-12-02T00:29:59.617580+0000","last_clean":"2023-12-02T00:29:59.617580+0000","last_became_active":"2023-12-02T00:29:57.929127+0000","last_became_peered":"2023-12-02T00:29:57.929127+0000","last_unstale":"2023-12-02T00:29:59.617580+0000","last_undegraded":"2023-12-02T00:29:59.617580+0000","last_fullsized":"2023-12-02T00:29:59.617580+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T09:28:13.357359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"34'4","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-02T00:29:57.941236+0000","last_change":"2023-12-02T00:29:56.920079+0000","last_active":"2023-12-02T00:29:57.941236+0000","last_peered":"2023-12-02T00:29:57.941236+0000","last_clean":"2023-12-02T00:29:57.941236+0000","last_became_active":"2023-12-02T00:29:56.919743+0000","last_became_peered":"2023-12-02T00:29:56.919743+0000","last_unstale":"2023-12-02T00:29:57.941236+0000","last_undegraded":"2023-12-02T00:29:57.941236+0000","last_fullsized":"2023-12-02T00:29:57.941236+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:43:17.538417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:58.927483+0000","last_change":"2023-12-02T00:29:57.919047+0000","last_active":"2023-12-02T00:29:58.927483+0000","last_peered":"2023-12-02T00:29:58.927483+0000","last_clean":"2023-12-02T00:29:58.927483+0000","last_became_active":"2023-12-02T00:29:57.918738+0000","last_became_peered":"2023-12-02T00:29:57.918738+0000","last_unstale":"2023-12-02T00:29:58.927483+0000","last_undegraded":"2023-12-02T00:29:58.927483+0000","last_fullsized":"2023-12-02T00:29:58.927483+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T09:28:13.357359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:59.617587+0000","last_change":"2023-12-02T00:29:56.916866+0000","last_active":"2023-12-02T00:29:59.617587+0000","last_peered":"2023-12-02T00:29:59.617587+0000","last_clean":"2023-12-02T00:29:59.617587+0000","last_became_active":"2023-12-02T00:29:56.916518+0000","last_became_peered":"2023-12-02T00:29:56.916518+0000","last_unstale":"2023-12-02T00:29:59.617587+0000","last_undegraded":"2023-12-02T00:29:59.617587+0000","last_fullsized":"2023-12-02T00:29:59.617587+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:22:01.548454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:58.928341+0000","last_change":"2023-12-02T00:29:57.933750+0000","last_active":"2023-12-02T00:29:58.928341+0000","last_peered":"2023-12-02T00:29:58.928341+0000","last_clean":"2023-12-02T00:29:58.928341+0000","last_became_active":"2023-12-02T00:29:57.933464+0000","last_became_peered":"2023-12-02T00:29:57.933464+0000","last_unstale":"2023-12-02T00:29:58.928341+0000","last_undegraded":"2023-12-02T00:29:58.928341+0000","last_fullsized":"2023-12-02T00:29:58.928341+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T01:52:39.055257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"34'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-02T00:29:57.941016+0000","last_change":"2023-12-02T00:29:56.915398+0000","last_active":"2023-12-02T00:29:57.941016+0000","last_peered":"2023-12-02T00:29:57.941016+0000","last_clean":"2023-12-02T00:29:57.941016+0000","last_became_active":"2023-12-02T00:29:56.915139+0000","last_became_peered":"2023-12-02T00:29:56.915139+0000","last_unstale":"2023-12-02T00:29:57.941016+0000","last_undegraded":"2023-12-02T00:29:57.941016+0000","last_fullsized":"2023-12-02T00:29:57.941016+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:14:44.794672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:58.921973+0000","last_change":"2023-12-02T00:29:57.927074+0000","last_active":"2023-12-02T00:29:58.921973+0000","last_peered":"2023-12-02T00:29:58.921973+0000","last_clean":"2023-12-02T00:29:58.921973+0000","last_became_active":"2023-12-02T00:29:57.926342+0000","last_became_peered":"2023-12-02T00:29:57.926342+0000","last_unstale":"2023-12-02T00:29:58.921973+0000","last_undegraded":"2023-12-02T00:29:58.921973+0000","last_fullsized":"2023-12-02T00:29:58.921973+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T09:23:38.713043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-02T00:29:57.932771+0000","last_change":"2023-12-02T00:29:56.917641+0000","last_active":"2023-12-02T00:29:57.932771+0000","last_peered":"2023-12-02T00:29:57.932771+0000","last_clean":"2023-12-02T00:29:57.932771+0000","last_became_active":"2023-12-02T00:29:56.917341+0000","last_became_peered":"2023-12-02T00:29:56.917341+0000","last_unstale":"2023-12-02T00:29:57.932771+0000","last_undegraded":"2023-12-02T00:29:57.932771+0000","last_fullsized":"2023-12-02T00:29:57.932771+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:45:03.336433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:59.731338+0000","last_change":"2023-12-02T00:29:57.921762+0000","last_active":"2023-12-02T00:29:59.731338+0000","last_peered":"2023-12-02T00:29:59.731338+0000","last_clean":"2023-12-02T00:29:59.731338+0000","last_became_active":"2023-12-02T00:29:57.921453+0000","last_became_peered":"2023-12-02T00:29:57.921453+0000","last_unstale":"2023-12-02T00:29:59.731338+0000","last_undegraded":"2023-12-02T00:29:59.731338+0000","last_fullsized":"2023-12-02T00:29:59.731338+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:45:14.018895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"34'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-02T00:29:57.939015+0000","last_change":"2023-12-02T00:29:56.917199+0000","last_active":"2023-12-02T00:29:57.939015+0000","last_peered":"2023-12-02T00:29:57.939015+0000","last_clean":"2023-12-02T00:29:57.939015+0000","last_became_active":"2023-12-02T00:29:56.916693+0000","last_became_peered":"2023-12-02T00:29:56.916693+0000","last_unstale":"2023-12-02T00:29:57.939015+0000","last_undegraded":"2023-12-02T00:29:57.939015+0000","last_fullsized":"2023-12-02T00:29:57.939015+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T01:36:05.220633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:58.927352+0000","last_change":"2023-12-02T00:29:57.924109+0000","last_active":"2023-12-02T00:29:58.927352+0000","last_peered":"2023-12-02T00:29:58.927352+0000","last_clean":"2023-12-02T00:29:58.927352+0000","last_became_active":"2023-12-02T00:29:57.921050+0000","last_became_peered":"2023-12-02T00:29:57.921050+0000","last_unstale":"2023-12-02T00:29:58.927352+0000","last_undegraded":"2023-12-02T00:29:58.927352+0000","last_fullsized":"2023-12-02T00:29:58.927352+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:57:01.750415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"34'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-02T00:29:57.938730+0000","last_change":"2023-12-02T00:29:56.917152+0000","last_active":"2023-12-02T00:29:57.938730+0000","last_peered":"2023-12-02T00:29:57.938730+0000","last_clean":"2023-12-02T00:29:57.938730+0000","last_became_active":"2023-12-02T00:29:56.916584+0000","last_became_peered":"2023-12-02T00:29:56.916584+0000","last_unstale":"2023-12-02T00:29:57.938730+0000","last_undegraded":"2023-12-02T00:29:57.938730+0000","last_fullsized":"2023-12-02T00:29:57.938730+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:22:01.548454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:58.927879+0000","last_change":"2023-12-02T00:29:57.919714+0000","last_active":"2023-12-02T00:29:58.927879+0000","last_peered":"2023-12-02T00:29:58.927879+0000","last_clean":"2023-12-02T00:29:58.927879+0000","last_became_active":"2023-12-02T00:29:57.919329+0000","last_became_peered":"2023-12-02T00:29:57.919329+0000","last_unstale":"2023-12-02T00:29:58.927879+0000","last_undegraded":"2023-12-02T00:29:58.927879+0000","last_fullsized":"2023-12-02T00:29:58.927879+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:14:45.799073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-02T00:29:57.906607+0000","last_change":"2023-12-02T00:29:56.917401+0000","last_active":"2023-12-02T00:29:57.906607+0000","last_peered":"2023-12-02T00:29:57.906607+0000","last_clean":"2023-12-02T00:29:57.906607+0000","last_became_active":"2023-12-02T00:29:56.917116+0000","last_became_peered":"2023-12-02T00:29:56.917116+0000","last_unstale":"2023-12-02T00:29:57.906607+0000","last_undegraded":"2023-12-02T00:29:57.906607+0000","last_fullsized":"2023-12-02T00:29:57.906607+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T01:36:05.220633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:58.921482+0000","last_change":"2023-12-02T00:29:57.925111+0000","last_active":"2023-12-02T00:29:58.921482+0000","last_peered":"2023-12-02T00:29:58.921482+0000","last_clean":"2023-12-02T00:29:58.921482+0000","last_became_active":"2023-12-02T00:29:57.924794+0000","last_became_peered":"2023-12-02T00:29:57.924794+0000","last_unstale":"2023-12-02T00:29:58.921482+0000","last_undegraded":"2023-12-02T00:29:58.921482+0000","last_fullsized":"2023-12-02T00:29:58.921482+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T09:33:07.733332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-02T00:29:57.916018+0000","last_change":"2023-12-02T00:29:56.912493+0000","last_active":"2023-12-02T00:29:57.916018+0000","last_peered":"2023-12-02T00:29:57.916018+0000","last_clean":"2023-12-02T00:29:57.916018+0000","last_became_active":"2023-12-02T00:29:56.912222+0000","last_became_peered":"2023-12-02T00:29:56.912222+0000","last_unstale":"2023-12-02T00:29:57.916018+0000","last_undegraded":"2023-12-02T00:29:57.916018+0000","last_fullsized":"2023-12-02T00:29:57.916018+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:44:41.457686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:58.927734+0000","last_change":"2023-12-02T00:29:57.920655+0000","last_active":"2023-12-02T00:29:58.927734+0000","last_peered":"2023-12-02T00:29:58.927734+0000","last_clean":"2023-12-02T00:29:58.927734+0000","last_became_active":"2023-12-02T00:29:57.920367+0000","last_became_peered":"2023-12-02T00:29:57.920367+0000","last_unstale":"2023-12-02T00:29:58.927734+0000","last_undegraded":"2023-12-02T00:29:58.927734+0000","last_fullsized":"2023-12-02T00:29:58.927734+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:45:14.018895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-02T00:29:57.909254+0000","last_change":"2023-12-02T00:29:56.921483+0000","last_active":"2023-12-02T00:29:57.909254+0000","last_peered":"2023-12-02T00:29:57.909254+0000","last_clean":"2023-12-02T00:29:57.909254+0000","last_became_active":"2023-12-02T00:29:56.921203+0000","last_became_peered":"2023-12-02T00:29:56.921203+0000","last_unstale":"2023-12-02T00:29:57.909254+0000","last_undegraded":"2023-12-02T00:29:57.909254+0000","last_fullsized":"2023-12-02T00:29:57.909254+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T02:59:30.945768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:59.731084+0000","last_change":"2023-12-02T00:29:57.932406+0000","last_active":"2023-12-02T00:29:59.731084+0000","last_peered":"2023-12-02T00:29:59.731084+0000","last_clean":"2023-12-02T00:29:59.731084+0000","last_became_active":"2023-12-02T00:29:57.932144+0000","last_became_peered":"2023-12-02T00:29:57.932144+0000","last_unstale":"2023-12-02T00:29:59.731084+0000","last_undegraded":"2023-12-02T00:29:59.731084+0000","last_fullsized":"2023-12-02T00:29:59.731084+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:18:11.139003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:59.617641+0000","last_change":"2023-12-02T00:29:56.917193+0000","last_active":"2023-12-02T00:29:59.617641+0000","last_peered":"2023-12-02T00:29:59.617641+0000","last_clean":"2023-12-02T00:29:59.617641+0000","last_became_active":"2023-12-02T00:29:56.916673+0000","last_became_peered":"2023-12-02T00:29:56.916673+0000","last_unstale":"2023-12-02T00:29:59.617641+0000","last_undegraded":"2023-12-02T00:29:59.617641+0000","last_fullsized":"2023-12-02T00:29:59.617641+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T02:14:08.136616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:59.731008+0000","last_change":"2023-12-02T00:29:57.916278+0000","last_active":"2023-12-02T00:29:59.731008+0000","last_peered":"2023-12-02T00:29:59.731008+0000","last_clean":"2023-12-02T00:29:59.731008+0000","last_became_active":"2023-12-02T00:29:57.916015+0000","last_became_peered":"2023-12-02T00:29:57.916015+0000","last_unstale":"2023-12-02T00:29:59.731008+0000","last_undegraded":"2023-12-02T00:29:59.731008+0000","last_fullsized":"2023-12-02T00:29:59.731008+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:32:48.460074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"34'3","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-02T00:29:57.941583+0000","last_change":"2023-12-02T00:29:56.920511+0000","last_active":"2023-12-02T00:29:57.941583+0000","last_peered":"2023-12-02T00:29:57.941583+0000","last_clean":"2023-12-02T00:29:57.941583+0000","last_became_active":"2023-12-02T00:29:56.920210+0000","last_became_peered":"2023-12-02T00:29:56.920210+0000","last_unstale":"2023-12-02T00:29:57.941583+0000","last_undegraded":"2023-12-02T00:29:57.941583+0000","last_fullsized":"2023-12-02T00:29:57.941583+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T09:23:37.708641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:58.927836+0000","last_change":"2023-12-02T00:29:57.930799+0000","last_active":"2023-12-02T00:29:58.927836+0000","last_peered":"2023-12-02T00:29:58.927836+0000","last_clean":"2023-12-02T00:29:58.927836+0000","last_became_active":"2023-12-02T00:29:57.930517+0000","last_became_peered":"2023-12-02T00:29:57.930517+0000","last_unstale":"2023-12-02T00:29:58.927836+0000","last_undegraded":"2023-12-02T00:29:58.927836+0000","last_fullsized":"2023-12-02T00:29:58.927836+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:54:25.311426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"34'6","reported_seq":18,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-02T00:29:57.943805+0000","last_change":"2023-12-02T00:29:56.916537+0000","last_active":"2023-12-02T00:29:57.943805+0000","last_peered":"2023-12-02T00:29:57.943805+0000","last_clean":"2023-12-02T00:29:57.943805+0000","last_became_active":"2023-12-02T00:29:56.916224+0000","last_became_peered":"2023-12-02T00:29:56.916224+0000","last_unstale":"2023-12-02T00:29:57.943805+0000","last_undegraded":"2023-12-02T00:29:57.943805+0000","last_fullsized":"2023-12-02T00:29:57.943805+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:24:46.243658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:58.921482+0000","last_change":"2023-12-02T00:29:57.925140+0000","last_active":"2023-12-02T00:29:58.921482+0000","last_peered":"2023-12-02T00:29:58.921482+0000","last_clean":"2023-12-02T00:29:58.921482+0000","last_became_active":"2023-12-02T00:29:57.924843+0000","last_became_peered":"2023-12-02T00:29:57.924843+0000","last_unstale":"2023-12-02T00:29:58.921482+0000","last_undegraded":"2023-12-02T00:29:58.921482+0000","last_fullsized":"2023-12-02T00:29:58.921482+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:09:03.223346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"34'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-02T00:29:57.941414+0000","last_change":"2023-12-02T00:29:56.915845+0000","last_active":"2023-12-02T00:29:57.941414+0000","last_peered":"2023-12-02T00:29:57.941414+0000","last_clean":"2023-12-02T00:29:57.941414+0000","last_became_active":"2023-12-02T00:29:56.912922+0000","last_became_peered":"2023-12-02T00:29:56.912922+0000","last_unstale":"2023-12-02T00:29:57.941414+0000","last_undegraded":"2023-12-02T00:29:57.941414+0000","last_fullsized":"2023-12-02T00:29:57.941414+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:19:19.312768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:59.730827+0000","last_change":"2023-12-02T00:29:57.927468+0000","last_active":"2023-12-02T00:29:59.730827+0000","last_peered":"2023-12-02T00:29:59.730827+0000","last_clean":"2023-12-02T00:29:59.730827+0000","last_became_active":"2023-12-02T00:29:57.922403+0000","last_became_peered":"2023-12-02T00:29:57.922403+0000","last_unstale":"2023-12-02T00:29:59.730827+0000","last_undegraded":"2023-12-02T00:29:59.730827+0000","last_fullsized":"2023-12-02T00:29:59.730827+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:45:14.018895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-02T00:29:57.929678+0000","last_change":"2023-12-02T00:29:56.916474+0000","last_active":"2023-12-02T00:29:57.929678+0000","last_peered":"2023-12-02T00:29:57.929678+0000","last_clean":"2023-12-02T00:29:57.929678+0000","last_became_active":"2023-12-02T00:29:56.916217+0000","last_became_peered":"2023-12-02T00:29:56.916217+0000","last_unstale":"2023-12-02T00:29:57.929678+0000","last_undegraded":"2023-12-02T00:29:57.929678+0000","last_fullsized":"2023-12-02T00:29:57.929678+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T09:28:12.352957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:59.731419+0000","last_change":"2023-12-02T00:29:57.922237+0000","last_active":"2023-12-02T00:29:59.731419+0000","last_peered":"2023-12-02T00:29:59.731419+0000","last_clean":"2023-12-02T00:29:59.731419+0000","last_became_active":"2023-12-02T00:29:57.921347+0000","last_became_peered":"2023-12-02T00:29:57.921347+0000","last_unstale":"2023-12-02T00:29:59.731419+0000","last_undegraded":"2023-12-02T00:29:59.731419+0000","last_fullsized":"2023-12-02T00:29:59.731419+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:54:25.311426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-02T00:29:57.906450+0000","last_change":"2023-12-02T00:29:56.920814+0000","last_active":"2023-12-02T00:29:57.906450+0000","last_peered":"2023-12-02T00:29:57.906450+0000","last_clean":"2023-12-02T00:29:57.906450+0000","last_became_active":"2023-12-02T00:29:56.920475+0000","last_became_peered":"2023-12-02T00:29:56.920475+0000","last_unstale":"2023-12-02T00:29:57.906450+0000","last_undegraded":"2023-12-02T00:29:57.906450+0000","last_fullsized":"2023-12-02T00:29:57.906450+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T10:41:58.073311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:58.927148+0000","last_change":"2023-12-02T00:29:57.920357+0000","last_active":"2023-12-02T00:29:58.927148+0000","last_peered":"2023-12-02T00:29:58.927148+0000","last_clean":"2023-12-02T00:29:58.927148+0000","last_became_active":"2023-12-02T00:29:57.919997+0000","last_became_peered":"2023-12-02T00:29:57.919997+0000","last_unstale":"2023-12-02T00:29:58.927148+0000","last_undegraded":"2023-12-02T00:29:58.927148+0000","last_fullsized":"2023-12-02T00:29:58.927148+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:25:27.359516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"34'8","reported_seq":20,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-02T00:29:57.942389+0000","last_change":"2023-12-02T00:29:56.921328+0000","last_active":"2023-12-02T00:29:57.942389+0000","last_peered":"2023-12-02T00:29:57.942389+0000","last_clean":"2023-12-02T00:29:57.942389+0000","last_became_active":"2023-12-02T00:29:56.921050+0000","last_became_peered":"2023-12-02T00:29:56.921050+0000","last_unstale":"2023-12-02T00:29:57.942389+0000","last_undegraded":"2023-12-02T00:29:57.942389+0000","last_fullsized":"2023-12-02T00:29:57.942389+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T02:14:08.136616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:58.927473+0000","last_change":"2023-12-02T00:29:57.925934+0000","last_active":"2023-12-02T00:29:58.927473+0000","last_peered":"2023-12-02T00:29:58.927473+0000","last_clean":"2023-12-02T00:29:58.927473+0000","last_became_active":"2023-12-02T00:29:57.925629+0000","last_became_peered":"2023-12-02T00:29:57.925629+0000","last_unstale":"2023-12-02T00:29:58.927473+0000","last_undegraded":"2023-12-02T00:29:58.927473+0000","last_fullsized":"2023-12-02T00:29:58.927473+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:56:01.736069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"34'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-02T00:29:57.938822+0000","last_change":"2023-12-02T00:29:56.918435+0000","last_active":"2023-12-02T00:29:57.938822+0000","last_peered":"2023-12-02T00:29:57.938822+0000","last_clean":"2023-12-02T00:29:57.938822+0000","last_became_active":"2023-12-02T00:29:56.918172+0000","last_became_peered":"2023-12-02T00:29:56.918172+0000","last_unstale":"2023-12-02T00:29:57.938822+0000","last_undegraded":"2023-12-02T00:29:57.938822+0000","last_fullsized":"2023-12-02T00:29:57.938822+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T10:41:58.073311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:58.927238+0000","last_change":"2023-12-02T00:29:57.923792+0000","last_active":"2023-12-02T00:29:58.927238+0000","last_peered":"2023-12-02T00:29:58.927238+0000","last_clean":"2023-12-02T00:29:58.927238+0000","last_became_active":"2023-12-02T00:29:57.922578+0000","last_became_peered":"2023-12-02T00:29:57.922578+0000","last_unstale":"2023-12-02T00:29:58.927238+0000","last_undegraded":"2023-12-02T00:29:58.927238+0000","last_fullsized":"2023-12-02T00:29:58.927238+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:57:03.990444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"34'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-02T00:29:57.941367+0000","last_change":"2023-12-02T00:29:56.915204+0000","last_active":"2023-12-02T00:29:57.941367+0000","last_peered":"2023-12-02T00:29:57.941367+0000","last_clean":"2023-12-02T00:29:57.941367+0000","last_became_active":"2023-12-02T00:29:56.914610+0000","last_became_peered":"2023-12-02T00:29:56.914610+0000","last_unstale":"2023-12-02T00:29:57.941367+0000","last_undegraded":"2023-12-02T00:29:57.941367+0000","last_fullsized":"2023-12-02T00:29:57.941367+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T10:41:58.073311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:58.921662+0000","last_change":"2023-12-02T00:29:57.923622+0000","last_active":"2023-12-02T00:29:58.921662+0000","last_peered":"2023-12-02T00:29:58.921662+0000","last_clean":"2023-12-02T00:29:58.921662+0000","last_became_active":"2023-12-02T00:29:57.923287+0000","last_became_peered":"2023-12-02T00:29:57.923287+0000","last_unstale":"2023-12-02T00:29:58.921662+0000","last_undegraded":"2023-12-02T00:29:58.921662+0000","last_fullsized":"2023-12-02T00:29:58.921662+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:14:45.799073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"34'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-02T00:29:57.938698+0000","last_change":"2023-12-02T00:29:56.916942+0000","last_active":"2023-12-02T00:29:57.938698+0000","last_peered":"2023-12-02T00:29:57.938698+0000","last_clean":"2023-12-02T00:29:57.938698+0000","last_became_active":"2023-12-02T00:29:56.915665+0000","last_became_peered":"2023-12-02T00:29:56.915665+0000","last_unstale":"2023-12-02T00:29:57.938698+0000","last_undegraded":"2023-12-02T00:29:57.938698+0000","last_fullsized":"2023-12-02T00:29:57.938698+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:25:26.355115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:59.208855+0000","last_change":"2023-12-02T00:29:57.924678+0000","last_active":"2023-12-02T00:29:59.208855+0000","last_peered":"2023-12-02T00:29:59.208855+0000","last_clean":"2023-12-02T00:29:59.208855+0000","last_became_active":"2023-12-02T00:29:57.924421+0000","last_became_peered":"2023-12-02T00:29:57.924421+0000","last_unstale":"2023-12-02T00:29:59.208855+0000","last_undegraded":"2023-12-02T00:29:59.208855+0000","last_fullsized":"2023-12-02T00:29:59.208855+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:44:42.462088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-02T00:29:57.916193+0000","last_change":"2023-12-02T00:29:56.917257+0000","last_active":"2023-12-02T00:29:57.916193+0000","last_peered":"2023-12-02T00:29:57.916193+0000","last_clean":"2023-12-02T00:29:57.916193+0000","last_became_active":"2023-12-02T00:29:56.916635+0000","last_became_peered":"2023-12-02T00:29:56.916635+0000","last_unstale":"2023-12-02T00:29:57.916193+0000","last_undegraded":"2023-12-02T00:29:57.916193+0000","last_fullsized":"2023-12-02T00:29:57.916193+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:43:17.538417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:59.208701+0000","last_change":"2023-12-02T00:29:57.921369+0000","last_active":"2023-12-02T00:29:59.208701+0000","last_peered":"2023-12-02T00:29:59.208701+0000","last_clean":"2023-12-02T00:29:59.208701+0000","last_became_active":"2023-12-02T00:29:57.921094+0000","last_became_peered":"2023-12-02T00:29:57.921094+0000","last_unstale":"2023-12-02T00:29:59.208701+0000","last_undegraded":"2023-12-02T00:29:59.208701+0000","last_fullsized":"2023-12-02T00:29:59.208701+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:12:10.772830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"34'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:59.617605+0000","last_change":"2023-12-02T00:29:56.914868+0000","last_active":"2023-12-02T00:29:59.617605+0000","last_peered":"2023-12-02T00:29:59.617605+0000","last_clean":"2023-12-02T00:29:59.617605+0000","last_became_active":"2023-12-02T00:29:56.914563+0000","last_became_peered":"2023-12-02T00:29:56.914563+0000","last_unstale":"2023-12-02T00:29:59.617605+0000","last_undegraded":"2023-12-02T00:29:59.617605+0000","last_fullsized":"2023-12-02T00:29:59.617605+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T01:34:58.510112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:59.208928+0000","last_change":"2023-12-02T00:29:57.919686+0000","last_active":"2023-12-02T00:29:59.208928+0000","last_peered":"2023-12-02T00:29:59.208928+0000","last_clean":"2023-12-02T00:29:59.208928+0000","last_became_active":"2023-12-02T00:29:57.919436+0000","last_became_peered":"2023-12-02T00:29:57.919436+0000","last_unstale":"2023-12-02T00:29:59.208928+0000","last_undegraded":"2023-12-02T00:29:59.208928+0000","last_fullsized":"2023-12-02T00:29:59.208928+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:22:02.552855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-02T00:29:57.907845+0000","last_change":"2023-12-02T00:29:56.920699+0000","last_active":"2023-12-02T00:29:57.907845+0000","last_peered":"2023-12-02T00:29:57.907845+0000","last_clean":"2023-12-02T00:29:57.907845+0000","last_became_active":"2023-12-02T00:29:56.920415+0000","last_became_peered":"2023-12-02T00:29:56.920415+0000","last_unstale":"2023-12-02T00:29:57.907845+0000","last_undegraded":"2023-12-02T00:29:57.907845+0000","last_fullsized":"2023-12-02T00:29:57.907845+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:44:41.457686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:59.208717+0000","last_change":"2023-12-02T00:29:57.925849+0000","last_active":"2023-12-02T00:29:59.208717+0000","last_peered":"2023-12-02T00:29:59.208717+0000","last_clean":"2023-12-02T00:29:59.208717+0000","last_became_active":"2023-12-02T00:29:57.925597+0000","last_became_peered":"2023-12-02T00:29:57.925597+0000","last_unstale":"2023-12-02T00:29:59.208717+0000","last_undegraded":"2023-12-02T00:29:59.208717+0000","last_fullsized":"2023-12-02T00:29:59.208717+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:55:55.421230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-02T00:29:57.916976+0000","last_change":"2023-12-02T00:29:56.915644+0000","last_active":"2023-12-02T00:29:57.916976+0000","last_peered":"2023-12-02T00:29:57.916976+0000","last_clean":"2023-12-02T00:29:57.916976+0000","last_became_active":"2023-12-02T00:29:56.915052+0000","last_became_peered":"2023-12-02T00:29:56.915052+0000","last_unstale":"2023-12-02T00:29:57.916976+0000","last_undegraded":"2023-12-02T00:29:57.916976+0000","last_fullsized":"2023-12-02T00:29:57.916976+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T02:14:08.136616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:59.731143+0000","last_change":"2023-12-02T00:29:57.914810+0000","last_active":"2023-12-02T00:29:59.731143+0000","last_peered":"2023-12-02T00:29:59.731143+0000","last_clean":"2023-12-02T00:29:59.731143+0000","last_became_active":"2023-12-02T00:29:57.913962+0000","last_became_peered":"2023-12-02T00:29:57.913962+0000","last_unstale":"2023-12-02T00:29:59.731143+0000","last_undegraded":"2023-12-02T00:29:59.731143+0000","last_fullsized":"2023-12-02T00:29:59.731143+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T02:59:31.950169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-02T00:29:57.912474+0000","last_change":"2023-12-02T00:29:56.919607+0000","last_active":"2023-12-02T00:29:57.912474+0000","last_peered":"2023-12-02T00:29:57.912474+0000","last_clean":"2023-12-02T00:29:57.912474+0000","last_became_active":"2023-12-02T00:29:56.919340+0000","last_became_peered":"2023-12-02T00:29:56.919340+0000","last_unstale":"2023-12-02T00:29:57.912474+0000","last_undegraded":"2023-12-02T00:29:57.912474+0000","last_fullsized":"2023-12-02T00:29:57.912474+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T09:33:06.728931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:59.208805+0000","last_change":"2023-12-02T00:29:57.926581+0000","last_active":"2023-12-02T00:29:59.208805+0000","last_peered":"2023-12-02T00:29:59.208805+0000","last_clean":"2023-12-02T00:29:59.208805+0000","last_became_active":"2023-12-02T00:29:57.926304+0000","last_became_peered":"2023-12-02T00:29:57.926304+0000","last_unstale":"2023-12-02T00:29:59.208805+0000","last_undegraded":"2023-12-02T00:29:59.208805+0000","last_fullsized":"2023-12-02T00:29:59.208805+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T01:35:44.664178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"34'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-02T00:29:57.939902+0000","last_change":"2023-12-02T00:29:56.923475+0000","last_active":"2023-12-02T00:29:57.939902+0000","last_peered":"2023-12-02T00:29:57.939902+0000","last_clean":"2023-12-02T00:29:57.939902+0000","last_became_active":"2023-12-02T00:29:56.923161+0000","last_became_peered":"2023-12-02T00:29:56.923161+0000","last_unstale":"2023-12-02T00:29:57.939902+0000","last_undegraded":"2023-12-02T00:29:57.939902+0000","last_fullsized":"2023-12-02T00:29:57.939902+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:25:26.355115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:58.927378+0000","last_change":"2023-12-02T00:29:57.931371+0000","last_active":"2023-12-02T00:29:58.927378+0000","last_peered":"2023-12-02T00:29:58.927378+0000","last_clean":"2023-12-02T00:29:58.927378+0000","last_became_active":"2023-12-02T00:29:57.931064+0000","last_became_peered":"2023-12-02T00:29:57.931064+0000","last_unstale":"2023-12-02T00:29:58.927378+0000","last_undegraded":"2023-12-02T00:29:58.927378+0000","last_fullsized":"2023-12-02T00:29:58.927378+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T02:53:46.303522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"35'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:59.618835+0000","last_change":"2023-12-02T00:29:56.915484+0000","last_active":"2023-12-02T00:29:59.618835+0000","last_peered":"2023-12-02T00:29:59.618835+0000","last_clean":"2023-12-02T00:29:59.618835+0000","last_became_active":"2023-12-02T00:29:56.915224+0000","last_became_peered":"2023-12-02T00:29:56.915224+0000","last_unstale":"2023-12-02T00:29:59.618835+0000","last_undegraded":"2023-12-02T00:29:59.618835+0000","last_fullsized":"2023-12-02T00:29:59.618835+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T10:41:58.073311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:59.617531+0000","last_change":"2023-12-02T00:29:57.932628+0000","last_active":"2023-12-02T00:29:59.617531+0000","last_peered":"2023-12-02T00:29:59.617531+0000","last_clean":"2023-12-02T00:29:59.617531+0000","last_became_active":"2023-12-02T00:29:57.932352+0000","last_became_peered":"2023-12-02T00:29:57.932352+0000","last_unstale":"2023-12-02T00:29:59.617531+0000","last_undegraded":"2023-12-02T00:29:59.617531+0000","last_fullsized":"2023-12-02T00:29:59.617531+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T01:36:06.225035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-02T00:29:57.929337+0000","last_change":"2023-12-02T00:29:56.914284+0000","last_active":"2023-12-02T00:29:57.929337+0000","last_peered":"2023-12-02T00:29:57.929337+0000","last_clean":"2023-12-02T00:29:57.929337+0000","last_became_active":"2023-12-02T00:29:56.913939+0000","last_became_peered":"2023-12-02T00:29:56.913939+0000","last_unstale":"2023-12-02T00:29:57.929337+0000","last_undegraded":"2023-12-02T00:29:57.929337+0000","last_fullsized":"2023-12-02T00:29:57.929337+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T01:36:05.220633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:58.926928+0000","last_change":"2023-12-02T00:29:57.919367+0000","last_active":"2023-12-02T00:29:58.926928+0000","last_peered":"2023-12-02T00:29:58.926928+0000","last_clean":"2023-12-02T00:29:58.926928+0000","last_became_active":"2023-12-02T00:29:57.919089+0000","last_became_peered":"2023-12-02T00:29:57.919089+0000","last_unstale":"2023-12-02T00:29:58.926928+0000","last_undegraded":"2023-12-02T00:29:58.926928+0000","last_fullsized":"2023-12-02T00:29:58.926928+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T02:14:09.141018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:59.617674+0000","last_change":"2023-12-02T00:29:56.915045+0000","last_active":"2023-12-02T00:29:59.617674+0000","last_peered":"2023-12-02T00:29:59.617674+0000","last_clean":"2023-12-02T00:29:59.617674+0000","last_became_active":"2023-12-02T00:29:56.914731+0000","last_became_peered":"2023-12-02T00:29:56.914731+0000","last_unstale":"2023-12-02T00:29:59.617674+0000","last_undegraded":"2023-12-02T00:29:59.617674+0000","last_fullsized":"2023-12-02T00:29:59.617674+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:44:41.457686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:59.617591+0000","last_change":"2023-12-02T00:29:57.919129+0000","last_active":"2023-12-02T00:29:59.617591+0000","last_peered":"2023-12-02T00:29:59.617591+0000","last_clean":"2023-12-02T00:29:59.617591+0000","last_became_active":"2023-12-02T00:29:57.918268+0000","last_became_peered":"2023-12-02T00:29:57.918268+0000","last_unstale":"2023-12-02T00:29:59.617591+0000","last_undegraded":"2023-12-02T00:29:59.617591+0000","last_fullsized":"2023-12-02T00:29:59.617591+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:19:20.317170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-02T00:29:57.929833+0000","last_change":"2023-12-02T00:29:56.923442+0000","last_active":"2023-12-02T00:29:57.929833+0000","last_peered":"2023-12-02T00:29:57.929833+0000","last_clean":"2023-12-02T00:29:57.929833+0000","last_became_active":"2023-12-02T00:29:56.923126+0000","last_became_peered":"2023-12-02T00:29:56.923126+0000","last_unstale":"2023-12-02T00:29:57.929833+0000","last_undegraded":"2023-12-02T00:29:57.929833+0000","last_fullsized":"2023-12-02T00:29:57.929833+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:19:19.312768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:59.730761+0000","last_change":"2023-12-02T00:29:57.914957+0000","last_active":"2023-12-02T00:29:59.730761+0000","last_peered":"2023-12-02T00:29:59.730761+0000","last_clean":"2023-12-02T00:29:59.730761+0000","last_became_active":"2023-12-02T00:29:57.914665+0000","last_became_peered":"2023-12-02T00:29:57.914665+0000","last_unstale":"2023-12-02T00:29:59.730761+0000","last_undegraded":"2023-12-02T00:29:59.730761+0000","last_fullsized":"2023-12-02T00:29:59.730761+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:57:01.750415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-02T00:29:57.917401+0000","last_change":"2023-12-02T00:29:56.918817+0000","last_active":"2023-12-02T00:29:57.917401+0000","last_peered":"2023-12-02T00:29:57.917401+0000","last_clean":"2023-12-02T00:29:57.917401+0000","last_became_active":"2023-12-02T00:29:56.918283+0000","last_became_peered":"2023-12-02T00:29:56.918283+0000","last_unstale":"2023-12-02T00:29:57.917401+0000","last_undegraded":"2023-12-02T00:29:57.917401+0000","last_fullsized":"2023-12-02T00:29:57.917401+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T09:28:12.352957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:58.927224+0000","last_change":"2023-12-02T00:29:57.925203+0000","last_active":"2023-12-02T00:29:58.927224+0000","last_peered":"2023-12-02T00:29:58.927224+0000","last_clean":"2023-12-02T00:29:58.927224+0000","last_became_active":"2023-12-02T00:29:57.924928+0000","last_became_peered":"2023-12-02T00:29:57.924928+0000","last_unstale":"2023-12-02T00:29:58.927224+0000","last_undegraded":"2023-12-02T00:29:58.927224+0000","last_fullsized":"2023-12-02T00:29:58.927224+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T09:33:07.733332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-02T00:29:57.906577+0000","last_change":"2023-12-02T00:29:56.920230+0000","last_active":"2023-12-02T00:29:57.906577+0000","last_peered":"2023-12-02T00:29:57.906577+0000","last_clean":"2023-12-02T00:29:57.906577+0000","last_became_active":"2023-12-02T00:29:56.919960+0000","last_became_peered":"2023-12-02T00:29:56.919960+0000","last_unstale":"2023-12-02T00:29:57.906577+0000","last_undegraded":"2023-12-02T00:29:57.906577+0000","last_fullsized":"2023-12-02T00:29:57.906577+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:25:26.355115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-02T00:29:59.731248+0000","last_change":"2023-12-02T00:29:57.917333+0000","last_active":"2023-12-02T00:29:59.731248+0000","last_peered":"2023-12-02T00:29:59.731248+0000","last_clean":"2023-12-02T00:29:59.731248+0000","last_became_active":"2023-12-02T00:29:57.917027+0000","last_became_peered":"2023-12-02T00:29:57.917027+0000","last_unstale":"2023-12-02T00:29:59.731248+0000","last_undegraded":"2023-12-02T00:29:59.731248+0000","last_fullsized":"2023-12-02T00:29:59.731248+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:56.896331+0000","last_clean_scrub_stamp":"2023-12-02T00:29:56.896331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T02:53:46.303522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-02T00:29:57.906827+0000","last_change":"2023-12-02T00:29:56.915904+0000","last_active":"2023-12-02T00:29:57.906827+0000","last_peered":"2023-12-02T00:29:57.906827+0000","last_clean":"2023-12-02T00:29:57.906827+0000","last_became_active":"2023-12-02T00:29:56.915619+0000","last_became_peered":"2023-12-02T00:29:56.915619+0000","last_unstale":"2023-12-02T00:29:57.906827+0000","last_undegraded":"2023-12-02T00:29:57.906827+0000","last_fullsized":"2023-12-02T00:29:57.906827+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:29:55.891930+0000","last_clean_scrub_stamp":"2023-12-02T00:29:55.891930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:12:09.768429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":178,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-02T00:29:57.909587+0000","last_change":"2023-12-02T00:28:39.379380+0000","last_active":"2023-12-02T00:29:57.909587+0000","last_peered":"2023-12-02T00:29:57.909587+0000","last_clean":"2023-12-02T00:29:57.909587+0000","last_became_active":"2023-12-02T00:28:39.378255+0000","last_became_peered":"2023-12-02T00:28:39.378255+0000","last_unstale":"2023-12-02T00:29:57.909587+0000","last_undegraded":"2023-12-02T00:29:57.909587+0000","last_fullsized":"2023-12-02T00:29:57.909587+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:38.177345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:38.177345+0000","last_clean_scrub_stamp":"2023-12-02T00:28:38.177345+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:21:08.582412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":995328,"data_stored":747190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.54600000000000004}]},{"osd":1,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.59499999999999997}]},{"osd":2,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.373},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.47499999999999998}]},{"osd":3,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.56000000000000005}]},{"osd":4,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.502},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.52500000000000002}]},{"osd":5,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.629}]},{"osd":6,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.44500000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":405504,"data_stored":157348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.51200000000000001}]},{"osd":1,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.432}]},{"osd":2,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.47899999999999998}]},{"osd":3,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.33700000000000002}]},{"osd":4,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.39100000000000001}]},{"osd":5,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.5}]},{"osd":7,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.56799999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":405504,"data_stored":157404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.66100000000000003}]},{"osd":1,"last update":"Sat Dec 2 00:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.64100000000000001}]},{"osd":2,"last update":"Sat Dec 2 00:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.624},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.50900000000000001}]},{"osd":3,"last update":"Sat Dec 2 00:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.69299999999999995}]},{"osd":4,"last update":"Sat Dec 2 00:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.373},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.47299999999999998}]},{"osd":6,"last update":"Sat Dec 2 00:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.45400000000000001}]},{"osd":7,"last update":"Sat Dec 2 00:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.441},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.58099999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":405504,"data_stored":156836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.47099999999999997}]},{"osd":1,"last update":"Sat Dec 2 00:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.70499999999999996}]},{"osd":2,"last update":"Sat Dec 2 00:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.879}]},{"osd":3,"last update":"Sat Dec 2 00:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.97799999999999998}]},{"osd":5,"last update":"Sat Dec 2 00:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.89600000000000002}]},{"osd":6,"last update":"Sat Dec 2 00:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.92300000000000004}]},{"osd":7,"last update":"Sat Dec 2 00:29:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":1.004}]}]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31588,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340252,"statfs":{"total":96636764160,"available":96604418048,"internally_reserved":0,"allocated":413696,"data_stored":158366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.217},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.27500000000000002}]},{"osd":1,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.25800000000000001}]},{"osd":2,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.32500000000000001}]},{"osd":4,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.47899999999999998}]},{"osd":5,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.39300000000000002}]},{"osd":6,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.48999999999999999}]},{"osd":7,"last update":"Sat Dec 2 00:29:36 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.441},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.50700000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31568,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340272,"statfs":{"total":96636764160,"available":96604438528,"internally_reserved":0,"allocated":397312,"data_stored":156183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.312}]},{"osd":1,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.503,"5min":0.503,"15min":0.503},"last":0.374}]},{"osd":3,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.40100000000000002}]},{"osd":4,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.57999999999999996}]},{"osd":5,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.56699999999999995}]},{"osd":6,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.61399999999999999}]},{"osd":7,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.52900000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":413696,"data_stored":158366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.45100000000000001}]},{"osd":2,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.33800000000000002}]},{"osd":3,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.35999999999999999}]},{"osd":4,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.66300000000000003}]},{"osd":5,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.64000000000000001}]},{"osd":6,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.746},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.72799999999999998}]},{"osd":7,"last update":"Sat Dec 2 00:29:40 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.80000000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339696,"statfs":{"total":96636764160,"available":96603848704,"internally_reserved":0,"allocated":987136,"data_stored":746481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.437}]},{"osd":2,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.32300000000000001}]},{"osd":3,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.375}]},{"osd":4,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.45700000000000002}]},{"osd":5,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.56399999999999995}]},{"osd":6,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.46500000000000002}]},{"osd":7,"last update":"Sat Dec 2 00:29:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.47699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-02T00:30:05.688 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-02T00:30:05.688 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-02T00:30:05.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.836+0000 7ff5c8e75640 1 Processor -- start 2023-12-02T00:30:05.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.836+0000 7ff5c8e75640 1 -- start start 2023-12-02T00:30:05.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.837+0000 7ff5c8e75640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff5c415d420 0x7ff5c40b2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:05.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.837+0000 7ff5c8e75640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff5c40b43a0 0x7ff5c40cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:05.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.837+0000 7ff5c8e75640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5c40d00f0 con 0x7ff5c40d3140 2023-12-02T00:30:05.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.837+0000 7ff5c8e75640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5c40cff00 con 0x7ff5c415d420 2023-12-02T00:30:05.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.837+0000 7ff5c8e75640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5c40d1420 con 0x7ff5c40b43a0 2023-12-02T00:30:05.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.837+0000 7ff5c259c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff5c415d420 0x7ff5c40b2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:05.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.837+0000 7ff5c259c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff5c415d420 0x7ff5c40b2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46132/0 (socket says 172.21.15.46:46132) 2023-12-02T00:30:05.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.837+0000 7ff5c2d9d640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff5c40d3140 0x7ff5c40d3550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36030/0 (socket says 172.21.15.46:36030) 2023-12-02T00:30:05.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.837+0000 7ff5c259c640 1 -- 172.21.15.46:0/3798994670 learned_addr learned my addr 172.21.15.46:0/3798994670 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:05.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.838+0000 7ff5c1d9b640 1 --2- 172.21.15.46:0/3798994670 >> v2:172.21.15.162:3300/0 conn(0x7ff5c40b43a0 0x7ff5c40cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:05.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.838+0000 7ff5c259c640 1 -- 172.21.15.46:0/3798994670 >> v2:172.21.15.162:3300/0 conn(0x7ff5c40b43a0 msgr2=0x7ff5c40cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:05.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.838+0000 7ff5c259c640 1 --2- 172.21.15.46:0/3798994670 >> v2:172.21.15.162:3300/0 conn(0x7ff5c40b43a0 0x7ff5c40cf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:05.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.838+0000 7ff5c259c640 1 -- 172.21.15.46:0/3798994670 >> v1:172.21.15.46:6789/0 conn(0x7ff5c40d3140 legacy=0x7ff5c40d3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:05.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.838+0000 7ff5c259c640 1 -- 172.21.15.46:0/3798994670 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c40d03d0 con 0x7ff5c415d420 2023-12-02T00:30:05.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.838+0000 7ff5c159a640 1 -- 172.21.15.46:0/3798994670 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1900722555 0 0) 0x7ff5c40d00f0 con 0x7ff5c40d3140 2023-12-02T00:30:05.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.838+0000 7ff5c259c640 1 --2- 172.21.15.46:0/3798994670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5c415d420 0x7ff5c40b2b50 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff5c408ccf0 tx=0x7ff5bc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3c8c439b4752c5f server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:05.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.839+0000 7ff5c159a640 1 -- 172.21.15.46:0/3798994670 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc030860 con 0x7ff5c415d420 2023-12-02T00:30:05.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.839+0000 7ff5c159a640 1 -- 172.21.15.46:0/3798994670 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5bc0309c0 con 0x7ff5c415d420 2023-12-02T00:30:05.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.839+0000 7ff5c159a640 1 -- 172.21.15.46:0/3798994670 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc030d50 con 0x7ff5c415d420 2023-12-02T00:30:05.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.839+0000 7ff5c8e75640 1 -- 172.21.15.46:0/3798994670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5c415d420 msgr2=0x7ff5c40b2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:05.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.839+0000 7ff5c8e75640 1 --2- 172.21.15.46:0/3798994670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5c415d420 0x7ff5c40b2b50 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff5c408ccf0 tx=0x7ff5bc02b9a0 comp rx=0 tx=0).stop 2023-12-02T00:30:05.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.839+0000 7ff5c8e75640 1 -- 172.21.15.46:0/3798994670 shutdown_connections 2023-12-02T00:30:05.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.839+0000 7ff5c8e75640 1 --2- 172.21.15.46:0/3798994670 >> v2:172.21.15.162:3300/0 conn(0x7ff5c40b43a0 0x7ff5c40cf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:05.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.839+0000 7ff5c8e75640 1 --2- 172.21.15.46:0/3798994670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5c415d420 0x7ff5c40b2b50 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:05.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.839+0000 7ff5c8e75640 1 -- 172.21.15.46:0/3798994670 >> 172.21.15.46:0/3798994670 conn(0x7ff5c408b000 msgr2=0x7ff5c409c580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:05.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.839+0000 7ff5c8e75640 1 -- 172.21.15.46:0/3798994670 shutdown_connections 2023-12-02T00:30:05.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.839+0000 7ff5c8e75640 1 -- 172.21.15.46:0/3798994670 wait complete. 2023-12-02T00:30:05.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.840+0000 7ff5c8e75640 1 Processor -- start 2023-12-02T00:30:05.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.841+0000 7ff5c8e75640 1 -- start start 2023-12-02T00:30:05.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.841+0000 7ff5c8e75640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5c40b43a0 0x7ff5c41f6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:05.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.841+0000 7ff5c8e75640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5c40d3140 0x7ff5c4157ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:05.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.841+0000 7ff5c8e75640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5c415d420 0x7ff5c40b1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:05.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.841+0000 7ff5c8e75640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c40c8150 con 0x7ff5c415d420 2023-12-02T00:30:05.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.841+0000 7ff5c8e75640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c41f70c0 con 0x7ff5c40b43a0 2023-12-02T00:30:05.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.841+0000 7ff5c8e75640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c40c74d0 con 0x7ff5c40d3140 2023-12-02T00:30:05.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.841+0000 7ff5c2d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5c415d420 0x7ff5c40b1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:05.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.841+0000 7ff5c2d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5c415d420 0x7ff5c40b1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46134/0 (socket says 172.21.15.46:46134) 2023-12-02T00:30:05.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.841+0000 7ff5c2d9d640 1 -- 172.21.15.46:0/1235447482 learned_addr learned my addr 172.21.15.46:0/1235447482 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:05.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.842+0000 7ff5c259c640 1 --2- 172.21.15.46:0/1235447482 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5c40b43a0 0x7ff5c41f6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:05.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.842+0000 7ff5c1d9b640 1 --2- 172.21.15.46:0/1235447482 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5c40d3140 0x7ff5c4157ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:05.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.842+0000 7ff5c2d9d640 1 -- 172.21.15.46:0/1235447482 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5c40d3140 msgr2=0x7ff5c4157ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:05.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.842+0000 7ff5c2d9d640 1 --2- 172.21.15.46:0/1235447482 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5c40d3140 0x7ff5c4157ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:05.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.842+0000 7ff5c2d9d640 1 -- 172.21.15.46:0/1235447482 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5c40b43a0 msgr2=0x7ff5c41f6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:05.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.842+0000 7ff5c2d9d640 1 --2- 172.21.15.46:0/1235447482 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5c40b43a0 0x7ff5c41f6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:05.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.842+0000 7ff5c2d9d640 1 -- 172.21.15.46:0/1235447482 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5bc0094d0 con 0x7ff5c415d420 2023-12-02T00:30:05.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.842+0000 7ff5c2d9d640 1 --2- 172.21.15.46:0/1235447482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5c415d420 0x7ff5c40b1060 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff5b400c7e0 tx=0x7ff5b400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:05.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.842+0000 7ff5baffd640 1 -- 172.21.15.46:0/1235447482 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b400a640 con 0x7ff5c415d420 2023-12-02T00:30:05.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.842+0000 7ff5baffd640 1 -- 172.21.15.46:0/1235447482 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b400a7c0 con 0x7ff5c415d420 2023-12-02T00:30:05.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.842+0000 7ff5c8e75640 1 -- 172.21.15.46:0/1235447482 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5c40c77b0 con 0x7ff5c415d420 2023-12-02T00:30:05.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.842+0000 7ff5baffd640 1 -- 172.21.15.46:0/1235447482 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b400ab70 con 0x7ff5c415d420 2023-12-02T00:30:05.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.842+0000 7ff5c8e75640 1 -- 172.21.15.46:0/1235447482 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5c40c7d70 con 0x7ff5c415d420 2023-12-02T00:30:05.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.843+0000 7ff5c8e75640 1 -- 172.21.15.46:0/1235447482 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5c4069400 con 0x7ff5c415d420 2023-12-02T00:30:05.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.844+0000 7ff5baffd640 1 -- 172.21.15.46:0/1235447482 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff5b400acf0 con 0x7ff5c415d420 2023-12-02T00:30:05.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.844+0000 7ff5baffd640 1 --2- 172.21.15.46:0/1235447482 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff5a406ae40 0x7ff5a406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:05.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.844+0000 7ff5baffd640 1 -- 172.21.15.46:0/1235447482 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff5b4088d10 con 0x7ff5c415d420 2023-12-02T00:30:05.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.844+0000 7ff5c259c640 1 --2- 172.21.15.46:0/1235447482 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff5a406ae40 0x7ff5a406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:05.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.845+0000 7ff5c259c640 1 --2- 172.21.15.46:0/1235447482 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff5a406ae40 0x7ff5a406d300 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc036000 tx=0x7ff5bc008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:05.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:05.848+0000 7ff5baffd640 1 -- 172.21.15.46:0/1235447482 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff5b4053f00 con 0x7ff5c415d420 2023-12-02T00:30:06.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.098+0000 7ff5c8e75640 1 -- 172.21.15.46:0/1235447482 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ff5c40d3fe0 con 0x7ff5c415d420 2023-12-02T00:30:06.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.100+0000 7ff5baffd640 1 -- 172.21.15.46:0/1235447482 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7ff5b4018020 con 0x7ff5c415d420 2023-12-02T00:30:06.101 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2023-12-02T00:30:06.101 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAJempl5Lx1DhAAjpK6ajoQ5jQiUr/EMpFeLg== 2023-12-02T00:30:06.101 INFO:teuthology.orchestra.run.smithi046.stdout: caps mds = "allow" 2023-12-02T00:30:06.102 INFO:teuthology.orchestra.run.smithi046.stdout: caps mon = "allow r" 2023-12-02T00:30:06.102 INFO:teuthology.orchestra.run.smithi046.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-12-02T00:30:06.102 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2023-12-02T00:30:06.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.102+0000 7ff5c8e75640 1 -- 172.21.15.46:0/1235447482 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff5a406ae40 msgr2=0x7ff5a406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:06.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.102+0000 7ff5c8e75640 1 --2- 172.21.15.46:0/1235447482 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff5a406ae40 0x7ff5a406d300 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc036000 tx=0x7ff5bc008030 comp rx=0 tx=0).stop 2023-12-02T00:30:06.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.102+0000 7ff5c8e75640 1 -- 172.21.15.46:0/1235447482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5c415d420 msgr2=0x7ff5c40b1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:06.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.102+0000 7ff5c8e75640 1 --2- 172.21.15.46:0/1235447482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5c415d420 0x7ff5c40b1060 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff5b400c7e0 tx=0x7ff5b400cca0 comp rx=0 tx=0).stop 2023-12-02T00:30:06.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.103+0000 7ff5c8e75640 1 -- 172.21.15.46:0/1235447482 shutdown_connections 2023-12-02T00:30:06.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.103+0000 7ff5c8e75640 1 --2- 172.21.15.46:0/1235447482 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff5a406ae40 0x7ff5a406d300 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:06.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.103+0000 7ff5c8e75640 1 --2- 172.21.15.46:0/1235447482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5c415d420 0x7ff5c40b1060 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:06.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.103+0000 7ff5c8e75640 1 --2- 172.21.15.46:0/1235447482 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5c40d3140 0x7ff5c4157ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:06.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.103+0000 7ff5c8e75640 1 --2- 172.21.15.46:0/1235447482 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5c40b43a0 0x7ff5c41f6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:06.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.103+0000 7ff5c8e75640 1 -- 172.21.15.46:0/1235447482 >> 172.21.15.46:0/1235447482 conn(0x7ff5c408b000 msgr2=0x7ff5c40d3520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:06.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.103+0000 7ff5c8e75640 1 -- 172.21.15.46:0/1235447482 shutdown_connections 2023-12-02T00:30:06.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.103+0000 7ff5c8e75640 1 -- 172.21.15.46:0/1235447482 wait complete. 2023-12-02T00:30:06.114 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-02T00:30:06.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.219+0000 7f1df870f640 1 Processor -- start 2023-12-02T00:30:06.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.220+0000 7f1df870f640 1 -- start start 2023-12-02T00:30:06.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.220+0000 7f1df870f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1df0140310 0x7f1df01618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:06.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.220+0000 7f1df870f640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1df006abe0 0x7f1df006efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:06.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.221+0000 7f1df870f640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1df01406f0 con 0x7f1df01409d0 2023-12-02T00:30:06.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.221+0000 7f1df870f640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1df0161e20 con 0x7f1df006abe0 2023-12-02T00:30:06.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.221+0000 7f1df870f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df0143960 con 0x7f1df0140310 2023-12-02T00:30:06.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.221+0000 7f1df6cac640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1df006abe0 0x7f1df006efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:06.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.221+0000 7f1df6cac640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1df006abe0 0x7f1df006efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46136/0 (socket says 172.21.15.46:46136) 2023-12-02T00:30:06.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.221+0000 7f1df6cac640 1 -- 172.21.15.46:0/3193254286 learned_addr learned my addr 172.21.15.46:0/3193254286 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:06.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.221+0000 7f1df64ab640 1 --2- 172.21.15.46:0/3193254286 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1df0140310 0x7f1df01618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:06.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.221+0000 7f1df6cac640 1 -- 172.21.15.46:0/3193254286 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1df0140310 msgr2=0x7f1df01618e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:06.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.221+0000 7f1df6cac640 1 --2- 172.21.15.46:0/3193254286 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1df0140310 0x7f1df01618e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:06.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.221+0000 7f1df6cac640 1 -- 172.21.15.46:0/3193254286 >> v1:172.21.15.162:6789/0 conn(0x7f1df01409d0 legacy=0x7f1df006a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:06.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.221+0000 7f1df6cac640 1 -- 172.21.15.46:0/3193254286 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df0142de0 con 0x7f1df006abe0 2023-12-02T00:30:06.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.222+0000 7f1df6cac640 1 --2- 172.21.15.46:0/3193254286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1df006abe0 0x7f1df006efb0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1df0161f60 tx=0x7f1de802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc3640a5aac776d4 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:06.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.222+0000 7f1df54a9640 1 -- 172.21.15.46:0/3193254286 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de8031640 con 0x7f1df006abe0 2023-12-02T00:30:06.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.222+0000 7f1df54a9640 1 -- 172.21.15.46:0/3193254286 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de80317a0 con 0x7f1df006abe0 2023-12-02T00:30:06.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.222+0000 7f1df54a9640 1 -- 172.21.15.46:0/3193254286 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de8031b30 con 0x7f1df006abe0 2023-12-02T00:30:06.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.222+0000 7f1df870f640 1 -- 172.21.15.46:0/3193254286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1df006abe0 msgr2=0x7f1df006efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:06.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.222+0000 7f1df870f640 1 --2- 172.21.15.46:0/3193254286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1df006abe0 0x7f1df006efb0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1df0161f60 tx=0x7f1de802b8f0 comp rx=0 tx=0).stop 2023-12-02T00:30:06.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.223+0000 7f1df870f640 1 -- 172.21.15.46:0/3193254286 shutdown_connections 2023-12-02T00:30:06.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.223+0000 7f1df870f640 1 --2- 172.21.15.46:0/3193254286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1df006abe0 0x7f1df006efb0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:06.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.223+0000 7f1df870f640 1 --2- 172.21.15.46:0/3193254286 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1df0140310 0x7f1df01618e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:06.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.223+0000 7f1df870f640 1 -- 172.21.15.46:0/3193254286 >> 172.21.15.46:0/3193254286 conn(0x7f1df00b1e50 msgr2=0x7f1df00b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:06.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.223+0000 7f1df870f640 1 -- 172.21.15.46:0/3193254286 shutdown_connections 2023-12-02T00:30:06.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.223+0000 7f1df870f640 1 -- 172.21.15.46:0/3193254286 wait complete. 2023-12-02T00:30:06.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.224+0000 7f1df870f640 1 Processor -- start 2023-12-02T00:30:06.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.224+0000 7f1df870f640 1 -- start start 2023-12-02T00:30:06.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.224+0000 7f1df870f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1df006abe0 0x7f1df00b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:06.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.224+0000 7f1df870f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1df0140310 0x7f1df01536a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:06.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.224+0000 7f1df870f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1df01409d0 0x7f1df00ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:06.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.224+0000 7f1df870f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df00b53b0 con 0x7f1df0140310 2023-12-02T00:30:06.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.224+0000 7f1df870f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df0071880 con 0x7f1df01409d0 2023-12-02T00:30:06.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.224+0000 7f1df870f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df0071b60 con 0x7f1df006abe0 2023-12-02T00:30:06.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.225+0000 7f1df5caa640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1df0140310 0x7f1df01536a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:06.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.225+0000 7f1df5caa640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1df0140310 0x7f1df01536a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46152/0 (socket says 172.21.15.46:46152) 2023-12-02T00:30:06.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.225+0000 7f1df5caa640 1 -- 172.21.15.46:0/3387479905 learned_addr learned my addr 172.21.15.46:0/3387479905 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:06.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.225+0000 7f1df6cac640 1 --2- 172.21.15.46:0/3387479905 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1df01409d0 0x7f1df00ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:06.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.225+0000 7f1df64ab640 1 --2- 172.21.15.46:0/3387479905 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1df006abe0 0x7f1df00b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:06.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.225+0000 7f1df5caa640 1 -- 172.21.15.46:0/3387479905 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1df006abe0 msgr2=0x7f1df00b4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:06.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.225+0000 7f1df5caa640 1 --2- 172.21.15.46:0/3387479905 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1df006abe0 0x7f1df00b4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:06.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.225+0000 7f1df5caa640 1 -- 172.21.15.46:0/3387479905 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1df01409d0 msgr2=0x7f1df00ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:06.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.225+0000 7f1df5caa640 1 --2- 172.21.15.46:0/3387479905 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1df01409d0 0x7f1df00ae6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:06.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.225+0000 7f1df5caa640 1 -- 172.21.15.46:0/3387479905 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1de80094d0 con 0x7f1df0140310 2023-12-02T00:30:06.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.225+0000 7f1df5caa640 1 --2- 172.21.15.46:0/3387479905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1df0140310 0x7f1df01536a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1de400c400 tx=0x7f1de400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:06.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.225+0000 7f1deeffd640 1 -- 172.21.15.46:0/3387479905 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de40052d0 con 0x7f1df0140310 2023-12-02T00:30:06.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.225+0000 7f1deeffd640 1 -- 172.21.15.46:0/3387479905 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de4005430 con 0x7f1df0140310 2023-12-02T00:30:06.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.225+0000 7f1df870f640 1 -- 172.21.15.46:0/3387479905 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1df0071e40 con 0x7f1df0140310 2023-12-02T00:30:06.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.225+0000 7f1deeffd640 1 -- 172.21.15.46:0/3387479905 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de4003830 con 0x7f1df0140310 2023-12-02T00:30:06.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.225+0000 7f1df870f640 1 -- 172.21.15.46:0/3387479905 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1df00b4ab0 con 0x7f1df0140310 2023-12-02T00:30:06.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.226+0000 7f1df870f640 1 -- 172.21.15.46:0/3387479905 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1df00cf210 con 0x7f1df0140310 2023-12-02T00:30:06.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.227+0000 7f1deeffd640 1 -- 172.21.15.46:0/3387479905 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1de4003990 con 0x7f1df0140310 2023-12-02T00:30:06.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.227+0000 7f1deeffd640 1 --2- 172.21.15.46:0/3387479905 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f1ddc06ae40 0x7f1ddc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:06.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.227+0000 7f1deeffd640 1 -- 172.21.15.46:0/3387479905 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1de4088c20 con 0x7f1df0140310 2023-12-02T00:30:06.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.227+0000 7f1df64ab640 1 --2- 172.21.15.46:0/3387479905 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f1ddc06ae40 0x7f1ddc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:06.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.228+0000 7f1df64ab640 1 --2- 172.21.15.46:0/3387479905 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f1ddc06ae40 0x7f1ddc06d300 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1de0003b80 tx=0x7f1de00098a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:06.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.231+0000 7f1deeffd640 1 -- 172.21.15.46:0/3387479905 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1de4053cf0 con 0x7f1df0140310 2023-12-02T00:30:06.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.443+0000 7f1df870f640 1 -- 172.21.15.46:0/3387479905 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1df0069400 con 0x7f1df0140310 2023-12-02T00:30:06.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.443+0000 7f1deeffd640 1 -- 172.21.15.46:0/3387479905 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+4917 (secure 0 0 0) 0x7f1de4017020 con 0x7f1df0140310 2023-12-02T00:30:06.445 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:30:06.445 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4811,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/3097254664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3097254664},{"type":"v1","addr":"172.21.15.162:6835","nonce":3097254664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":4826,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/3489476943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3489476943},{"type":"v1","addr":"172.21.15.162:6837","nonce":3489476943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5274,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1283185308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":1283185308},{"type":"v1","addr":"172.21.15.46:6835","nonce":1283185308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-02T00:29:57.279169+0000","modified":"2023-12-02T00:29:59.266062+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5277},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5277":{"gid":5277,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":5,"addr":"172.21.15.46:6834/2036053042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2036053042},{"type":"v1","addr":"172.21.15.46:6834","nonce":2036053042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-02T00:30:06.446 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 13 2023-12-02T00:30:06.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.445+0000 7f1df870f640 1 -- 172.21.15.46:0/3387479905 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f1ddc06ae40 msgr2=0x7f1ddc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:06.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.445+0000 7f1df870f640 1 --2- 172.21.15.46:0/3387479905 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f1ddc06ae40 0x7f1ddc06d300 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1de0003b80 tx=0x7f1de00098a0 comp rx=0 tx=0).stop 2023-12-02T00:30:06.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.445+0000 7f1df870f640 1 -- 172.21.15.46:0/3387479905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1df0140310 msgr2=0x7f1df01536a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:06.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.445+0000 7f1df870f640 1 --2- 172.21.15.46:0/3387479905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1df0140310 0x7f1df01536a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1de400c400 tx=0x7f1de400c8c0 comp rx=0 tx=0).stop 2023-12-02T00:30:06.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.445+0000 7f1df870f640 1 -- 172.21.15.46:0/3387479905 shutdown_connections 2023-12-02T00:30:06.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.446+0000 7f1df870f640 1 --2- 172.21.15.46:0/3387479905 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f1ddc06ae40 0x7f1ddc06d300 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:06.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.446+0000 7f1df870f640 1 --2- 172.21.15.46:0/3387479905 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1df01409d0 0x7f1df00ae6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:06.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.446+0000 7f1df870f640 1 --2- 172.21.15.46:0/3387479905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1df0140310 0x7f1df01536a0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:06.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.446+0000 7f1df870f640 1 --2- 172.21.15.46:0/3387479905 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1df006abe0 0x7f1df00b4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:06.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.446+0000 7f1df870f640 1 -- 172.21.15.46:0/3387479905 >> 172.21.15.46:0/3387479905 conn(0x7f1df00b1e50 msgr2=0x7f1df006ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:06.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.446+0000 7f1df870f640 1 -- 172.21.15.46:0/3387479905 shutdown_connections 2023-12-02T00:30:06.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.446+0000 7f1df870f640 1 -- 172.21.15.46:0/3387479905 wait complete. 2023-12-02T00:30:06.860 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 13, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-02T00:29:57.279169+0000', 'modified': '2023-12-02T00:29:59.266062+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5277}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5277': {'gid': 5277, 'name': 'a', 'rank': 0, 'incarnation': 11, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.46:6834/2036053042', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6832', 'nonce': 2036053042}, {'type': 'v1', 'addr': '172.21.15.46:6834', 'nonce': 2036053042}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-02T00:30:06.860 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-02T00:30:06.860 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-12-02T00:30:06.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.970+0000 7faf86b3c640 1 Processor -- start 2023-12-02T00:30:06.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.971+0000 7faf86b3c640 1 -- start start 2023-12-02T00:30:06.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.971+0000 7faf86b3c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faf8015d340 0x7faf8006b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:06.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.971+0000 7faf86b3c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7faf800aeb80 0x7faf8006ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:06.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.971+0000 7faf86b3c640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf8015d720 con 0x7faf800ad640 2023-12-02T00:30:06.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.971+0000 7faf86b3c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf800adae0 con 0x7faf800aeb80 2023-12-02T00:30:06.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.971+0000 7faf86b3c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf800addc0 con 0x7faf8015d340 2023-12-02T00:30:06.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.971+0000 7faf7ffff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7faf800ad640 0x7faf80160ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36044/0 (socket says 172.21.15.46:36044) 2023-12-02T00:30:06.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.972+0000 7faf7ffff640 1 -- 172.21.15.46:0/1158113695 learned_addr learned my addr 172.21.15.46:0/1158113695 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:30:06.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.972+0000 7faf850d9640 1 --2- 172.21.15.46:0/1158113695 >> v2:172.21.15.162:3300/0 conn(0x7faf800aeb80 0x7faf8006ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:06.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.972+0000 7faf848d8640 1 --2- 172.21.15.46:0/1158113695 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faf8015d340 0x7faf8006b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:06.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.972+0000 7faf7f7fe640 1 -- 172.21.15.46:0/1158113695 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3482688400 0 0) 0x7faf8015d720 con 0x7faf800ad640 2023-12-02T00:30:06.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.972+0000 7faf7f7fe640 1 -- 172.21.15.46:0/1158113695 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf68003520 con 0x7faf800ad640 2023-12-02T00:30:06.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.972+0000 7faf850d9640 1 -- 172.21.15.46:0/1158113695 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faf8015d340 msgr2=0x7faf8006b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:06.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.972+0000 7faf850d9640 1 --2- 172.21.15.46:0/1158113695 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faf8015d340 0x7faf8006b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:06.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.972+0000 7faf7f7fe640 1 -- 172.21.15.46:0/1158113695 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 218689212 0 0) 0x7faf68003520 con 0x7faf800ad640 2023-12-02T00:30:06.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.972+0000 7faf850d9640 1 -- 172.21.15.46:0/1158113695 >> v1:172.21.15.46:6789/0 conn(0x7faf800ad640 legacy=0x7faf80160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:06.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.972+0000 7faf850d9640 1 -- 172.21.15.46:0/1158113695 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf800ae0a0 con 0x7faf800aeb80 2023-12-02T00:30:06.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.973+0000 7faf850d9640 1 --2- 172.21.15.46:0/1158113695 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faf800aeb80 0x7faf8006ba20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7faf700097a0 tx=0x7faf7002b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37919874911a6b79 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:06.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.973+0000 7faf7f7fe640 1 -- 172.21.15.46:0/1158113695 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf74001670 con 0x7faf800aeb80 2023-12-02T00:30:06.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.973+0000 7faf7f7fe640 1 -- 172.21.15.46:0/1158113695 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf7002e050 con 0x7faf800aeb80 2023-12-02T00:30:06.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.973+0000 7faf7f7fe640 1 -- 172.21.15.46:0/1158113695 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf70030710 con 0x7faf800aeb80 2023-12-02T00:30:06.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.974+0000 7faf86b3c640 1 -- 172.21.15.46:0/1158113695 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faf800aeb80 msgr2=0x7faf8006ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:06.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.974+0000 7faf86b3c640 1 --2- 172.21.15.46:0/1158113695 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faf800aeb80 0x7faf8006ba20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7faf700097a0 tx=0x7faf7002b790 comp rx=0 tx=0).stop 2023-12-02T00:30:06.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.974+0000 7faf86b3c640 1 -- 172.21.15.46:0/1158113695 shutdown_connections 2023-12-02T00:30:06.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.974+0000 7faf86b3c640 1 --2- 172.21.15.46:0/1158113695 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faf800aeb80 0x7faf8006ba20 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:06.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.974+0000 7faf86b3c640 1 --2- 172.21.15.46:0/1158113695 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faf8015d340 0x7faf8006b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:06.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.974+0000 7faf86b3c640 1 -- 172.21.15.46:0/1158113695 >> 172.21.15.46:0/1158113695 conn(0x7faf80070e70 msgr2=0x7faf80071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:06.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.974+0000 7faf86b3c640 1 -- 172.21.15.46:0/1158113695 shutdown_connections 2023-12-02T00:30:06.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.974+0000 7faf86b3c640 1 -- 172.21.15.46:0/1158113695 wait complete. 2023-12-02T00:30:06.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.975+0000 7faf86b3c640 1 Processor -- start 2023-12-02T00:30:06.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.975+0000 7faf86b3c640 1 -- start start 2023-12-02T00:30:06.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.976+0000 7faf86b3c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faf800ad640 0x7faf80150770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:06.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.976+0000 7faf86b3c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faf800aeb80 0x7faf800cde10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:06.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.976+0000 7faf86b3c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf8015d340 0x7faf800ce350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:06.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.976+0000 7faf86b3c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf80157970 con 0x7faf8015d340 2023-12-02T00:30:06.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.976+0000 7faf86b3c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8015e7c0 con 0x7faf800aeb80 2023-12-02T00:30:06.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.976+0000 7faf86b3c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf8015eaa0 con 0x7faf800ad640 2023-12-02T00:30:06.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.976+0000 7faf850d9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf8015d340 0x7faf800ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:06.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.976+0000 7faf850d9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf8015d340 0x7faf800ce350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46168/0 (socket says 172.21.15.46:46168) 2023-12-02T00:30:06.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.976+0000 7faf850d9640 1 -- 172.21.15.46:0/635757869 learned_addr learned my addr 172.21.15.46:0/635757869 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:06.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.976+0000 7faf7ffff640 1 --2- 172.21.15.46:0/635757869 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faf800aeb80 0x7faf800cde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:06.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.976+0000 7faf848d8640 1 --2- 172.21.15.46:0/635757869 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faf800ad640 0x7faf80150770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:06.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.976+0000 7faf850d9640 1 -- 172.21.15.46:0/635757869 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faf800ad640 msgr2=0x7faf80150770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:06.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.976+0000 7faf850d9640 1 --2- 172.21.15.46:0/635757869 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faf800ad640 0x7faf80150770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:06.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.976+0000 7faf850d9640 1 -- 172.21.15.46:0/635757869 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faf800aeb80 msgr2=0x7faf800cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:06.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.976+0000 7faf850d9640 1 --2- 172.21.15.46:0/635757869 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faf800aeb80 0x7faf800cde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:06.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.976+0000 7faf850d9640 1 -- 172.21.15.46:0/635757869 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf70009400 con 0x7faf8015d340 2023-12-02T00:30:06.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.976+0000 7faf848d8640 1 --2- 172.21.15.46:0/635757869 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faf800ad640 0x7faf80150770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:30:06.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.976+0000 7faf7ffff640 1 --2- 172.21.15.46:0/635757869 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faf800aeb80 0x7faf800cde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:30:06.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.976+0000 7faf850d9640 1 --2- 172.21.15.46:0/635757869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf8015d340 0x7faf800ce350 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7faf7002b7d0 tx=0x7faf700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:06.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.976+0000 7faf7d7fa640 1 -- 172.21.15.46:0/635757869 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf70006e10 con 0x7faf8015d340 2023-12-02T00:30:06.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.976+0000 7faf7d7fa640 1 -- 172.21.15.46:0/635757869 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf7003c620 con 0x7faf8015d340 2023-12-02T00:30:06.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.977+0000 7faf86b3c640 1 -- 172.21.15.46:0/635757869 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf8015ed80 con 0x7faf8015d340 2023-12-02T00:30:06.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.977+0000 7faf7d7fa640 1 -- 172.21.15.46:0/635757869 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf7003c9d0 con 0x7faf8015d340 2023-12-02T00:30:06.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.977+0000 7faf86b3c640 1 -- 172.21.15.46:0/635757869 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf80150cb0 con 0x7faf8015d340 2023-12-02T00:30:06.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.978+0000 7faf7d7fa640 1 -- 172.21.15.46:0/635757869 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faf70039070 con 0x7faf8015d340 2023-12-02T00:30:06.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.979+0000 7faf7d7fa640 1 --2- 172.21.15.46:0/635757869 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7faf6c06ae40 0x7faf6c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:06.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.979+0000 7faf7d7fa640 1 -- 172.21.15.46:0/635757869 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faf700aacc0 con 0x7faf8015d340 2023-12-02T00:30:06.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.979+0000 7faf848d8640 1 --2- 172.21.15.46:0/635757869 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7faf6c06ae40 0x7faf6c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:06.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.980+0000 7faf897ab740 1 Processor -- start 2023-12-02T00:30:06.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.980+0000 7faf848d8640 1 --2- 172.21.15.46:0/635757869 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7faf6c06ae40 0x7faf6c06d300 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7faf78007440 tx=0x7faf7800b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:06.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.980+0000 7faf897ab740 1 -- start start 2023-12-02T00:30:06.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.980+0000 7faf897ab740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acf5a2ac50 0x55acf59fdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:06.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.980+0000 7faf897ab740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55acf5a2b030 0x55acf5a2d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:06.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.981+0000 7faf897ab740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55acf59fe130 0x55acf5a2df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:06.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.981+0000 7faf897ab740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf59be980 con 0x55acf5a2ac50 2023-12-02T00:30:06.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.981+0000 7faf897ab740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf5970b90 con 0x55acf5a2b030 2023-12-02T00:30:06.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.981+0000 7faf897ab740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55acf596e280 con 0x55acf59fe130 2023-12-02T00:30:06.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.981+0000 7faf848d8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acf5a2ac50 0x55acf59fdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:06.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.981+0000 7faf848d8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acf5a2ac50 0x55acf59fdbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46174/0 (socket says 172.21.15.46:46174) 2023-12-02T00:30:06.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.981+0000 7faf848d8640 1 -- 172.21.15.46:0/2009097710 learned_addr learned my addr 172.21.15.46:0/2009097710 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:06.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.981+0000 7faf7ffff640 1 --2- 172.21.15.46:0/2009097710 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55acf5a2b030 0x55acf5a2d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:06.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.981+0000 7faf850d9640 1 --2- 172.21.15.46:0/2009097710 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55acf59fe130 0x55acf5a2df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:06.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.981+0000 7faf848d8640 1 -- 172.21.15.46:0/2009097710 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55acf59fe130 msgr2=0x55acf5a2df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:06.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.981+0000 7faf848d8640 1 --2- 172.21.15.46:0/2009097710 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55acf59fe130 0x55acf5a2df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:06.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.981+0000 7faf848d8640 1 -- 172.21.15.46:0/2009097710 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55acf5a2b030 msgr2=0x55acf5a2d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:06.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.981+0000 7faf848d8640 1 --2- 172.21.15.46:0/2009097710 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55acf5a2b030 0x55acf5a2d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:06.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.981+0000 7faf848d8640 1 -- 172.21.15.46:0/2009097710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acf592f500 con 0x55acf5a2ac50 2023-12-02T00:30:06.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.981+0000 7faf848d8640 1 --2- 172.21.15.46:0/2009097710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acf5a2ac50 0x55acf59fdbf0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7faf78016710 tx=0x7faf78012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d19ea526c66219c8 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:06.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.981+0000 7faf667fc640 1 -- 172.21.15.46:0/2009097710 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf78014a20 con 0x55acf5a2ac50 2023-12-02T00:30:06.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.981+0000 7faf667fc640 1 -- 172.21.15.46:0/2009097710 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf78014ba0 con 0x55acf5a2ac50 2023-12-02T00:30:06.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.981+0000 7faf667fc640 1 -- 172.21.15.46:0/2009097710 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf78021400 con 0x55acf5a2ac50 2023-12-02T00:30:06.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.982+0000 7faf897ab740 1 -- 172.21.15.46:0/2009097710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acf5a2ac50 msgr2=0x55acf59fdbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:06.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.982+0000 7faf897ab740 1 --2- 172.21.15.46:0/2009097710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acf5a2ac50 0x55acf59fdbf0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7faf78016710 tx=0x7faf78012000 comp rx=0 tx=0).stop 2023-12-02T00:30:06.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.982+0000 7faf897ab740 1 -- 172.21.15.46:0/2009097710 shutdown_connections 2023-12-02T00:30:06.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.982+0000 7faf897ab740 1 --2- 172.21.15.46:0/2009097710 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55acf59fe130 0x55acf5a2df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:06.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.982+0000 7faf897ab740 1 --2- 172.21.15.46:0/2009097710 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55acf5a2b030 0x55acf5a2d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:06.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.982+0000 7faf897ab740 1 --2- 172.21.15.46:0/2009097710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acf5a2ac50 0x55acf59fdbf0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:06.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.982+0000 7faf897ab740 1 -- 172.21.15.46:0/2009097710 >> 172.21.15.46:0/2009097710 conn(0x55acf59ffe60 msgr2=0x55acf5a489e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:06.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.982+0000 7faf897ab740 1 -- 172.21.15.46:0/2009097710 shutdown_connections 2023-12-02T00:30:06.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.982+0000 7faf897ab740 1 -- 172.21.15.46:0/2009097710 wait complete. 2023-12-02T00:30:06.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.982+0000 7faf897ab740 1 -- start start 2023-12-02T00:30:06.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.983+0000 7faf897ab740 1 Processor -- start 2023-12-02T00:30:06.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.983+0000 7faf897ab740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55acf5a2ac50 0x55acf5a43010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:06.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.983+0000 7faf897ab740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acf5a2b030 0x55acf5a5bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:06.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.983+0000 7faf897ab740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55acf5a31160 0x55acf5a5c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:06.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.983+0000 7faf897ab740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf5969b70 con 0x55acf5a2b030 2023-12-02T00:30:06.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.983+0000 7faf897ab740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf5923e00 con 0x55acf5a31160 2023-12-02T00:30:06.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.983+0000 7faf897ab740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55acf5923aa0 con 0x55acf5a2ac50 2023-12-02T00:30:06.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.984+0000 7faf897ab740 10 client.0 _ll_register_callbacks cb 0x55acf567bac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-02T00:30:06.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.984+0000 7faf7ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acf5a2b030 0x55acf5a5bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:06.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.984+0000 7faf7ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acf5a2b030 0x55acf5a5bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46182/0 (socket says 172.21.15.46:46182) 2023-12-02T00:30:06.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.984+0000 7faf7ffff640 1 -- 172.21.15.46:0/2617863034 learned_addr learned my addr 172.21.15.46:0/2617863034 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:06.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.984+0000 7faf848d8640 1 --2- 172.21.15.46:0/2617863034 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55acf5a2ac50 0x55acf5a43010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:06.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.984+0000 7faf850d9640 1 --2- 172.21.15.46:0/2617863034 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55acf5a31160 0x55acf5a5c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:06.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.984+0000 7faf7ffff640 1 -- 172.21.15.46:0/2617863034 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55acf5a2ac50 msgr2=0x55acf5a43010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:06.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.984+0000 7faf7ffff640 1 --2- 172.21.15.46:0/2617863034 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55acf5a2ac50 0x55acf5a43010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:06.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.984+0000 7faf7ffff640 1 -- 172.21.15.46:0/2617863034 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55acf5a31160 msgr2=0x55acf5a5c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:06.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.984+0000 7faf7ffff640 1 --2- 172.21.15.46:0/2617863034 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55acf5a31160 0x55acf5a5c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:06.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.984+0000 7faf7ffff640 1 -- 172.21.15.46:0/2617863034 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf78016420 con 0x55acf5a2b030 2023-12-02T00:30:06.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.984+0000 7faf7ffff640 1 --2- 172.21.15.46:0/2617863034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acf5a2b030 0x55acf5a5bf90 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7faf7400f530 tx=0x7faf7400f9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec5a36ae7c14ebc server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:06.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.984+0000 7faf5f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2023-12-02T00:30:06.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.984+0000 7faf5f7fe640 1 -- 172.21.15.46:0/2617863034 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf74005ae0 con 0x55acf5a2b030 2023-12-02T00:30:06.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.984+0000 7faf5f7fe640 1 -- 172.21.15.46:0/2617863034 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faf74005c40 con 0x55acf5a2b030 2023-12-02T00:30:06.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.984+0000 7faf897ab740 1 -- 172.21.15.46:0/2617863034 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55acf5962b50 con 0x55acf5a2b030 2023-12-02T00:30:06.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.984+0000 7faf5f7fe640 1 -- 172.21.15.46:0/2617863034 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf74005e00 con 0x55acf5a2b030 2023-12-02T00:30:06.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.984+0000 7faf897ab740 10 client.5484 fetch_fsmap learned FSMap version 13 2023-12-02T00:30:06.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.984+0000 7faf897ab740 1 -- 172.21.15.46:0/2617863034 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x55acf59681f0 con 0x55acf5a2b030 2023-12-02T00:30:06.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.985+0000 7faf5f7fe640 1 -- 172.21.15.46:0/2617863034 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55acf5962b50 con 0x55acf5a2b030 2023-12-02T00:30:06.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.985+0000 7faf5f7fe640 1 -- 172.21.15.46:0/2617863034 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 13) v1 ==== 3294+0+0 (secure 0 0 0) 0x7faf74012070 con 0x55acf5a2b030 2023-12-02T00:30:06.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.985+0000 7faf897ab740 10 client.5484 fetch_fsmap finished waiting for FSMap version 13 2023-12-02T00:30:06.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.985+0000 7faf897ab740 10 client.5484 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-02T00:30:06.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.985+0000 7faf897ab740 20 client.5484 populate_metadata read hostname 'smithi046' 2023-12-02T00:30:06.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.985+0000 7faf897ab740 1 --2- 172.21.15.46:0/2617863034 >> [v2:172.21.15.46:6832/2036053042,v1:172.21.15.46:6834/2036053042] conn(0x55acf5a19120 0x55acf5a19500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:06.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.985+0000 7faf897ab740 4 client.5484 mds_command: new command op to 5277 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-02T00:30:06.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.985+0000 7faf897ab740 1 -- 172.21.15.46:0/2617863034 --> [v2:172.21.15.46:6832/2036053042,v1:172.21.15.46:6834/2036053042] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55acf5962870 con 0x55acf5a19120 2023-12-02T00:30:06.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.985+0000 7faf848d8640 1 --2- 172.21.15.46:0/2617863034 >> [v2:172.21.15.46:6832/2036053042,v1:172.21.15.46:6834/2036053042] conn(0x55acf5a19120 0x55acf5a19500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:06.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.985+0000 7faf5f7fe640 10 client.5484 ms_handle_connect on v2:172.21.15.46:6832/2036053042 2023-12-02T00:30:06.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.985+0000 7faf848d8640 1 --2- 172.21.15.46:0/2617863034 >> [v2:172.21.15.46:6832/2036053042,v1:172.21.15.46:6834/2036053042] conn(0x55acf5a19120 0x55acf5a19500 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e946942a8c4cbec server_cookie=8a246ca512222ff0 in_seq=0 out_seq=0 2023-12-02T00:30:06.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.986+0000 7faf5f7fe640 1 -- 172.21.15.46:0/2617863034 <== mds.0 v2:172.21.15.46:6832/2036053042 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55acf5962870 con 0x55acf5a19120 2023-12-02T00:30:06.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.986+0000 7faf5f7fe640 10 client.5484 handle_command_reply: tid=0 2023-12-02T00:30:06.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.986+0000 7faf897ab740 1 client.5484 shutdown 2023-12-02T00:30:06.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.986+0000 7faf897ab740 1 -- 172.21.15.46:0/2617863034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acf5a2b030 msgr2=0x55acf5a5bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:06.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.986+0000 7faf897ab740 1 --2- 172.21.15.46:0/2617863034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acf5a2b030 0x55acf5a5bf90 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7faf7400f530 tx=0x7faf7400f9f0 comp rx=0 tx=0).stop 2023-12-02T00:30:06.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.986+0000 7faf897ab740 1 -- 172.21.15.46:0/2617863034 shutdown_connections 2023-12-02T00:30:06.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.986+0000 7faf897ab740 1 --2- 172.21.15.46:0/2617863034 >> [v2:172.21.15.46:6832/2036053042,v1:172.21.15.46:6834/2036053042] conn(0x55acf5a19120 0x55acf5a19500 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:06.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.986+0000 7faf897ab740 1 --2- 172.21.15.46:0/2617863034 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55acf5a31160 0x55acf5a5c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:06.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.986+0000 7faf5f7fe640 0 client.5484 ms_handle_reset on v2:172.21.15.46:6832/2036053042 2023-12-02T00:30:06.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.986+0000 7faf897ab740 1 --2- 172.21.15.46:0/2617863034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acf5a2b030 0x55acf5a5bf90 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:06.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.986+0000 7faf897ab740 1 --2- 172.21.15.46:0/2617863034 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55acf5a2ac50 0x55acf5a43010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:06.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.986+0000 7faf897ab740 1 -- 172.21.15.46:0/2617863034 >> 172.21.15.46:0/2617863034 conn(0x55acf59fe130 msgr2=0x55acf5a01ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:06.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.986+0000 7faf897ab740 1 -- 172.21.15.46:0/2617863034 shutdown_connections 2023-12-02T00:30:06.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.986+0000 7faf897ab740 1 -- 172.21.15.46:0/2617863034 wait complete. 2023-12-02T00:30:06.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:06.986+0000 7faf897ab740 20 client.5484 trim_cache size 0 max 16384 2023-12-02T00:30:07.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.002+0000 7faf897ab740 1 Processor -- start 2023-12-02T00:30:07.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.002+0000 7faf897ab740 1 -- start start 2023-12-02T00:30:07.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.003+0000 7faf897ab740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55acf5a2ac50 0x55acf5a48fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:07.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.003+0000 7faf897ab740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55acf5a2b030 0x55acf5a29770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:07.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.003+0000 7faf897ab740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acf5a31160 0x55acf59fcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:07.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.003+0000 7faf897ab740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf58e9270 con 0x55acf5a31160 2023-12-02T00:30:07.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.003+0000 7faf897ab740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf5a04350 con 0x55acf5a2b030 2023-12-02T00:30:07.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.003+0000 7faf897ab740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55acf58f1f40 con 0x55acf5a2ac50 2023-12-02T00:30:07.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.003+0000 7faf850d9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acf5a31160 0x55acf59fcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:07.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.003+0000 7faf850d9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acf5a31160 0x55acf59fcbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46188/0 (socket says 172.21.15.46:46188) 2023-12-02T00:30:07.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.003+0000 7faf850d9640 1 -- 172.21.15.46:0/383008162 learned_addr learned my addr 172.21.15.46:0/383008162 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:07.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.003+0000 7faf7ffff640 1 --2- 172.21.15.46:0/383008162 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55acf5a2b030 0x55acf5a29770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:07.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.003+0000 7faf850d9640 1 -- 172.21.15.46:0/383008162 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55acf5a2ac50 msgr2=0x55acf5a48fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:07.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.003+0000 7faf848d8640 1 --2- 172.21.15.46:0/383008162 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55acf5a2ac50 0x55acf5a48fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:07.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.003+0000 7faf850d9640 1 --2- 172.21.15.46:0/383008162 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55acf5a2ac50 0x55acf5a48fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:07.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.003+0000 7faf850d9640 1 -- 172.21.15.46:0/383008162 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55acf5a2b030 msgr2=0x55acf5a29770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:07.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.003+0000 7faf850d9640 1 --2- 172.21.15.46:0/383008162 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55acf5a2b030 0x55acf5a29770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:07.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.003+0000 7faf850d9640 1 -- 172.21.15.46:0/383008162 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acf5905d00 con 0x55acf5a31160 2023-12-02T00:30:07.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.003+0000 7faf7ffff640 1 --2- 172.21.15.46:0/383008162 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55acf5a2b030 0x55acf5a29770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:30:07.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.003+0000 7faf848d8640 1 --2- 172.21.15.46:0/383008162 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55acf5a2ac50 0x55acf5a48fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:30:07.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.003+0000 7faf850d9640 1 --2- 172.21.15.46:0/383008162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acf5a31160 0x55acf59fcbf0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7faf7002bda0 tx=0x7faf70079930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93da2ad860ff8f49 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:07.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.003+0000 7faf66ffd640 1 -- 172.21.15.46:0/383008162 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf70091070 con 0x55acf5a31160 2023-12-02T00:30:07.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.003+0000 7faf66ffd640 1 -- 172.21.15.46:0/383008162 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf70077dd0 con 0x55acf5a31160 2023-12-02T00:30:07.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.003+0000 7faf66ffd640 1 -- 172.21.15.46:0/383008162 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf70076140 con 0x55acf5a31160 2023-12-02T00:30:07.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.004+0000 7faf897ab740 1 -- 172.21.15.46:0/383008162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acf5a31160 msgr2=0x55acf59fcbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:07.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.004+0000 7faf897ab740 1 --2- 172.21.15.46:0/383008162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acf5a31160 0x55acf59fcbf0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7faf7002bda0 tx=0x7faf70079930 comp rx=0 tx=0).stop 2023-12-02T00:30:07.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.004+0000 7faf897ab740 1 -- 172.21.15.46:0/383008162 shutdown_connections 2023-12-02T00:30:07.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.004+0000 7faf897ab740 1 --2- 172.21.15.46:0/383008162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acf5a31160 0x55acf59fcbf0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:07.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.004+0000 7faf897ab740 1 --2- 172.21.15.46:0/383008162 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55acf5a2b030 0x55acf5a29770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:07.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.004+0000 7faf897ab740 1 --2- 172.21.15.46:0/383008162 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55acf5a2ac50 0x55acf5a48fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:07.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.004+0000 7faf897ab740 1 -- 172.21.15.46:0/383008162 >> 172.21.15.46:0/383008162 conn(0x55acf59fe130 msgr2=0x55acf59f94b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:07.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.004+0000 7faf897ab740 1 -- 172.21.15.46:0/383008162 shutdown_connections 2023-12-02T00:30:07.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.004+0000 7faf897ab740 1 -- 172.21.15.46:0/383008162 wait complete. 2023-12-02T00:30:07.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.005+0000 7faf897ab740 1 -- start start 2023-12-02T00:30:07.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.005+0000 7faf897ab740 1 Processor -- start 2023-12-02T00:30:07.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.005+0000 7faf897ab740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55acf5a2ac50 0x55acf5ab2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:07.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.005+0000 7faf897ab740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acf5a2b030 0x55acf5a4bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:07.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.005+0000 7faf897ab740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55acf5a31160 0x55acf5a4c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:07.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.006+0000 7faf897ab740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf5ab3420 con 0x55acf5a2b030 2023-12-02T00:30:07.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.006+0000 7faf897ab740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf5a2dd40 con 0x55acf5a2ac50 2023-12-02T00:30:07.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.006+0000 7faf897ab740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55acf5a17ed0 con 0x55acf5a31160 2023-12-02T00:30:07.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.006+0000 7faf897ab740 10 client.0 _ll_register_callbacks cb 0x7faf74001cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-02T00:30:07.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.006+0000 7faf7ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acf5a2b030 0x55acf5a4bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:07.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.006+0000 7faf7ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acf5a2b030 0x55acf5a4bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46190/0 (socket says 172.21.15.46:46190) 2023-12-02T00:30:07.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.006+0000 7faf7ffff640 1 -- 172.21.15.46:0/2876938626 learned_addr learned my addr 172.21.15.46:0/2876938626 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:07.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.006+0000 7faf850d9640 1 --2- 172.21.15.46:0/2876938626 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55acf5a31160 0x55acf5a4c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:07.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.006+0000 7faf848d8640 1 --2- 172.21.15.46:0/2876938626 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55acf5a2ac50 0x55acf5ab2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:07.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.006+0000 7faf7ffff640 1 -- 172.21.15.46:0/2876938626 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55acf5a31160 msgr2=0x55acf5a4c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:07.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.006+0000 7faf7ffff640 1 --2- 172.21.15.46:0/2876938626 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55acf5a31160 0x55acf5a4c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:07.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.006+0000 7faf7ffff640 1 -- 172.21.15.46:0/2876938626 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55acf5a2ac50 msgr2=0x55acf5ab2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:07.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.006+0000 7faf7ffff640 1 --2- 172.21.15.46:0/2876938626 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55acf5a2ac50 0x55acf5ab2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:07.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.006+0000 7faf7ffff640 1 -- 172.21.15.46:0/2876938626 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf80150cb0 con 0x55acf5a2b030 2023-12-02T00:30:07.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.006+0000 7faf7ffff640 1 --2- 172.21.15.46:0/2876938626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acf5a2b030 0x55acf5a4bef0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7faf74007a70 tx=0x7faf74000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1523d39571ce233 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:07.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.006+0000 7faf5f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2023-12-02T00:30:07.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.006+0000 7faf5f7fe640 1 -- 172.21.15.46:0/2876938626 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf74007470 con 0x55acf5a2b030 2023-12-02T00:30:07.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.006+0000 7faf5f7fe640 1 -- 172.21.15.46:0/2876938626 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faf740075d0 con 0x55acf5a2b030 2023-12-02T00:30:07.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.006+0000 7faf897ab740 1 -- 172.21.15.46:0/2876938626 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faf78014ba0 con 0x55acf5a2b030 2023-12-02T00:30:07.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.006+0000 7faf5f7fe640 1 -- 172.21.15.46:0/2876938626 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf74007790 con 0x55acf5a2b030 2023-12-02T00:30:07.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.006+0000 7faf897ab740 10 client.5490 fetch_fsmap learned FSMap version 13 2023-12-02T00:30:07.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.006+0000 7faf897ab740 1 -- 172.21.15.46:0/2876938626 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x55acf592b4f0 con 0x55acf5a2b030 2023-12-02T00:30:07.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.007+0000 7faf5f7fe640 1 -- 172.21.15.46:0/2876938626 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf78014ba0 con 0x55acf5a2b030 2023-12-02T00:30:07.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.007+0000 7faf5f7fe640 1 -- 172.21.15.46:0/2876938626 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 13) v1 ==== 3294+0+0 (secure 0 0 0) 0x7faf7402c070 con 0x55acf5a2b030 2023-12-02T00:30:07.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.007+0000 7faf897ab740 10 client.5490 fetch_fsmap finished waiting for FSMap version 13 2023-12-02T00:30:07.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.007+0000 7faf897ab740 10 client.5490 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-02T00:30:07.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.007+0000 7faf897ab740 20 client.5490 populate_metadata read hostname 'smithi046' 2023-12-02T00:30:07.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.007+0000 7faf897ab740 1 --2- 172.21.15.46:0/2876938626 >> [v2:172.21.15.46:6832/2036053042,v1:172.21.15.46:6834/2036053042] conn(0x55acf5a57790 0x55acf5a57b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:07.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.007+0000 7faf897ab740 4 client.5490 mds_command: new command op to 5277 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-02T00:30:07.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.007+0000 7faf897ab740 1 -- 172.21.15.46:0/2876938626 --> [v2:172.21.15.46:6832/2036053042,v1:172.21.15.46:6834/2036053042] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55acf566a6c0 con 0x55acf5a57790 2023-12-02T00:30:07.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.007+0000 7faf848d8640 1 --2- 172.21.15.46:0/2876938626 >> [v2:172.21.15.46:6832/2036053042,v1:172.21.15.46:6834/2036053042] conn(0x55acf5a57790 0x55acf5a57b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:07.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.007+0000 7faf5f7fe640 10 client.5490 ms_handle_connect on v2:172.21.15.46:6832/2036053042 2023-12-02T00:30:07.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.007+0000 7faf848d8640 1 --2- 172.21.15.46:0/2876938626 >> [v2:172.21.15.46:6832/2036053042,v1:172.21.15.46:6834/2036053042] conn(0x55acf5a57790 0x55acf5a57b70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdded247e807775b server_cookie=880522c47248f15e in_seq=0 out_seq=0 2023-12-02T00:30:07.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.007+0000 7faf5f7fe640 1 -- 172.21.15.46:0/2876938626 <== mds.0 v2:172.21.15.46:6832/2036053042 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55acf566a6c0 con 0x55acf5a57790 2023-12-02T00:30:07.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.007+0000 7faf5f7fe640 10 client.5490 handle_command_reply: tid=0 2023-12-02T00:30:07.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.007+0000 7faf897ab740 1 client.5490 shutdown 2023-12-02T00:30:07.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.007+0000 7faf897ab740 1 -- 172.21.15.46:0/2876938626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acf5a2b030 msgr2=0x55acf5a4bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:07.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.007+0000 7faf897ab740 1 --2- 172.21.15.46:0/2876938626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acf5a2b030 0x55acf5a4bef0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7faf74007a70 tx=0x7faf74000ed0 comp rx=0 tx=0).stop 2023-12-02T00:30:07.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.008+0000 7faf897ab740 1 -- 172.21.15.46:0/2876938626 shutdown_connections 2023-12-02T00:30:07.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.008+0000 7faf897ab740 1 --2- 172.21.15.46:0/2876938626 >> [v2:172.21.15.46:6832/2036053042,v1:172.21.15.46:6834/2036053042] conn(0x55acf5a57790 0x55acf5a57b70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:07.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.008+0000 7faf897ab740 1 --2- 172.21.15.46:0/2876938626 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55acf5a31160 0x55acf5a4c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:07.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.008+0000 7faf5f7fe640 0 client.5490 ms_handle_reset on v2:172.21.15.46:6832/2036053042 2023-12-02T00:30:07.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.008+0000 7faf897ab740 1 --2- 172.21.15.46:0/2876938626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acf5a2b030 0x55acf5a4bef0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:07.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.008+0000 7faf897ab740 1 --2- 172.21.15.46:0/2876938626 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55acf5a2ac50 0x55acf5ab2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:07.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.008+0000 7faf897ab740 1 -- 172.21.15.46:0/2876938626 >> 172.21.15.46:0/2876938626 conn(0x55acf59fe130 msgr2=0x55acf5a29a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:07.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.008+0000 7faf897ab740 1 -- 172.21.15.46:0/2876938626 shutdown_connections 2023-12-02T00:30:07.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.008+0000 7faf897ab740 1 -- 172.21.15.46:0/2876938626 wait complete. 2023-12-02T00:30:07.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.008+0000 7faf897ab740 20 client.5490 trim_cache size 0 max 16384 2023-12-02T00:30:07.016 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-12-02T00:30:07.016 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "6d588070-5443-40f5-81f6-108a1e3f56c3", 2023-12-02T00:30:07.016 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2023-12-02T00:30:07.016 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 5277, 2023-12-02T00:30:07.016 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2023-12-02T00:30:07.016 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2023-12-02T00:30:07.016 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2023-12-02T00:30:07.016 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 9.1047441930000002, 2023-12-02T00:30:07.016 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 13, 2023-12-02T00:30:07.017 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 35, 2023-12-02T00:30:07.017 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 35, 2023-12-02T00:30:07.017 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 22.395233242 2023-12-02T00:30:07.017 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-12-02T00:30:07.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.008+0000 7faf86b3c640 1 -- 172.21.15.46:0/635757869 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7faf6c06ae40 msgr2=0x7faf6c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:07.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.008+0000 7faf86b3c640 1 --2- 172.21.15.46:0/635757869 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7faf6c06ae40 0x7faf6c06d300 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7faf78007440 tx=0x7faf7800b040 comp rx=0 tx=0).stop 2023-12-02T00:30:07.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.008+0000 7faf86b3c640 1 -- 172.21.15.46:0/635757869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf8015d340 msgr2=0x7faf800ce350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:07.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.008+0000 7faf86b3c640 1 --2- 172.21.15.46:0/635757869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf8015d340 0x7faf800ce350 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7faf7002b7d0 tx=0x7faf700046c0 comp rx=0 tx=0).stop 2023-12-02T00:30:07.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.008+0000 7faf86b3c640 1 -- 172.21.15.46:0/635757869 shutdown_connections 2023-12-02T00:30:07.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.008+0000 7faf86b3c640 1 --2- 172.21.15.46:0/635757869 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7faf6c06ae40 0x7faf6c06d300 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:07.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.008+0000 7faf86b3c640 1 --2- 172.21.15.46:0/635757869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf8015d340 0x7faf800ce350 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:07.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.008+0000 7faf86b3c640 1 --2- 172.21.15.46:0/635757869 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faf800aeb80 0x7faf800cde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:07.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.008+0000 7faf86b3c640 1 --2- 172.21.15.46:0/635757869 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faf800ad640 0x7faf80150770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:07.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.008+0000 7faf86b3c640 1 -- 172.21.15.46:0/635757869 >> 172.21.15.46:0/635757869 conn(0x7faf80070e70 msgr2=0x7faf80151510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:07.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.008+0000 7faf86b3c640 1 -- 172.21.15.46:0/635757869 shutdown_connections 2023-12-02T00:30:07.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.008+0000 7faf86b3c640 1 -- 172.21.15.46:0/635757869 wait complete. 2023-12-02T00:30:07.020 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-02T00:30:07.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.125+0000 7f4a2775a640 1 Processor -- start 2023-12-02T00:30:07.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.125+0000 7f4a2775a640 1 -- start start 2023-12-02T00:30:07.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.126+0000 7f4a2775a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a20144980 0x7f4a2015c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:07.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.126+0000 7f4a2775a640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4a200ceb70 0x7f4a200cc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:07.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.126+0000 7f4a2775a640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4a200b4370 0x7f4a200b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:07.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.126+0000 7f4a2775a640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a200b4d20 con 0x7f4a200ceb70 2023-12-02T00:30:07.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.126+0000 7f4a2775a640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a20071770 con 0x7f4a200b4370 2023-12-02T00:30:07.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.126+0000 7f4a2775a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a200ccdc0 con 0x7f4a20144980 2023-12-02T00:30:07.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.126+0000 7f4a24cf5640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4a200ceb70 0x7f4a200cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:07.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.126+0000 7f4a24cf5640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4a200ceb70 0x7f4a200cc450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46200/0 (socket says 172.21.15.46:46200) 2023-12-02T00:30:07.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.126+0000 7f4a24cf5640 1 -- 172.21.15.46:0/2405101120 learned_addr learned my addr 172.21.15.46:0/2405101120 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:07.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.126+0000 7f4a25cf7640 1 --2- 172.21.15.46:0/2405101120 >> v2:172.21.15.162:3300/0 conn(0x7f4a200b4370 0x7f4a200b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:07.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.126+0000 7f4a254f6640 1 --2- 172.21.15.46:0/2405101120 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a20144980 0x7f4a2015c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:07.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.127+0000 7f4a24cf5640 1 -- 172.21.15.46:0/2405101120 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a20144980 msgr2=0x7f4a2015c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:07.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.127+0000 7f4a24cf5640 1 --2- 172.21.15.46:0/2405101120 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a20144980 0x7f4a2015c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:07.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.127+0000 7f4a24cf5640 1 -- 172.21.15.46:0/2405101120 >> v2:172.21.15.162:3300/0 conn(0x7f4a200b4370 msgr2=0x7f4a200b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:07.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.127+0000 7f4a24cf5640 1 --2- 172.21.15.46:0/2405101120 >> v2:172.21.15.162:3300/0 conn(0x7f4a200b4370 0x7f4a200b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:07.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.127+0000 7f4a24cf5640 1 -- 172.21.15.46:0/2405101120 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a200cd0a0 con 0x7f4a200ceb70 2023-12-02T00:30:07.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.127+0000 7f4a254f6640 1 --2- 172.21.15.46:0/2405101120 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a20144980 0x7f4a2015c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:30:07.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.127+0000 7f4a25cf7640 1 --2- 172.21.15.46:0/2405101120 >> v2:172.21.15.162:3300/0 conn(0x7f4a200b4370 0x7f4a200b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:30:07.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.127+0000 7f4a24cf5640 1 --2- 172.21.15.46:0/2405101120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a200ceb70 0x7f4a200cc450 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f4a200b2110 tx=0x7f4a1402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c159d5b2dc6b401 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:07.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.127+0000 7f4a1ffff640 1 -- 172.21.15.46:0/2405101120 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a14031870 con 0x7f4a200ceb70 2023-12-02T00:30:07.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.128+0000 7f4a1ffff640 1 -- 172.21.15.46:0/2405101120 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a140319d0 con 0x7f4a200ceb70 2023-12-02T00:30:07.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.128+0000 7f4a1ffff640 1 -- 172.21.15.46:0/2405101120 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a14031d60 con 0x7f4a200ceb70 2023-12-02T00:30:07.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.128+0000 7f4a2775a640 1 -- 172.21.15.46:0/2405101120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a200ceb70 msgr2=0x7f4a200cc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:07.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.128+0000 7f4a2775a640 1 --2- 172.21.15.46:0/2405101120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a200ceb70 0x7f4a200cc450 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f4a200b2110 tx=0x7f4a1402b910 comp rx=0 tx=0).stop 2023-12-02T00:30:07.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.128+0000 7f4a2775a640 1 -- 172.21.15.46:0/2405101120 shutdown_connections 2023-12-02T00:30:07.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.128+0000 7f4a2775a640 1 --2- 172.21.15.46:0/2405101120 >> v2:172.21.15.162:3300/0 conn(0x7f4a200b4370 0x7f4a200b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:07.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.128+0000 7f4a2775a640 1 --2- 172.21.15.46:0/2405101120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a200ceb70 0x7f4a200cc450 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:07.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.128+0000 7f4a2775a640 1 --2- 172.21.15.46:0/2405101120 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a20144980 0x7f4a2015c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:07.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.128+0000 7f4a2775a640 1 -- 172.21.15.46:0/2405101120 >> 172.21.15.46:0/2405101120 conn(0x7f4a2006a4d0 msgr2=0x7f4a2006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:07.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.128+0000 7f4a2775a640 1 -- 172.21.15.46:0/2405101120 shutdown_connections 2023-12-02T00:30:07.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.128+0000 7f4a2775a640 1 -- 172.21.15.46:0/2405101120 wait complete. 2023-12-02T00:30:07.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.129+0000 7f4a2775a640 1 Processor -- start 2023-12-02T00:30:07.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.129+0000 7f4a2775a640 1 -- start start 2023-12-02T00:30:07.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.130+0000 7f4a2775a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a200b4370 0x7f4a20158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:07.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.130+0000 7f4a2775a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a200ceb70 0x7f4a2013db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:07.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.130+0000 7f4a2775a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a20144980 0x7f4a2013e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:07.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.130+0000 7f4a2775a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a200afb00 con 0x7f4a200b4370 2023-12-02T00:30:07.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.130+0000 7f4a2775a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a20131500 con 0x7f4a200ceb70 2023-12-02T00:30:07.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.130+0000 7f4a2775a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a20130740 con 0x7f4a20144980 2023-12-02T00:30:07.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.130+0000 7f4a254f6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a200b4370 0x7f4a20158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:07.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.130+0000 7f4a254f6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a200b4370 0x7f4a20158100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46214/0 (socket says 172.21.15.46:46214) 2023-12-02T00:30:07.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.130+0000 7f4a254f6640 1 -- 172.21.15.46:0/810839852 learned_addr learned my addr 172.21.15.46:0/810839852 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:07.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.130+0000 7f4a25cf7640 1 --2- 172.21.15.46:0/810839852 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a20144980 0x7f4a2013e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:07.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.130+0000 7f4a24cf5640 1 --2- 172.21.15.46:0/810839852 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a200ceb70 0x7f4a2013db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:07.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.130+0000 7f4a254f6640 1 -- 172.21.15.46:0/810839852 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a20144980 msgr2=0x7f4a2013e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:07.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.130+0000 7f4a254f6640 1 --2- 172.21.15.46:0/810839852 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a20144980 0x7f4a2013e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:07.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.130+0000 7f4a254f6640 1 -- 172.21.15.46:0/810839852 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a200ceb70 msgr2=0x7f4a2013db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:07.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.130+0000 7f4a254f6640 1 --2- 172.21.15.46:0/810839852 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a200ceb70 0x7f4a2013db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:07.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.130+0000 7f4a254f6640 1 -- 172.21.15.46:0/810839852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a140094d0 con 0x7f4a200b4370 2023-12-02T00:30:07.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.131+0000 7f4a254f6640 1 --2- 172.21.15.46:0/810839852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a200b4370 0x7f4a20158100 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4a18009fa0 tx=0x7f4a1800d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:07.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.131+0000 7f4a1dffb640 1 -- 172.21.15.46:0/810839852 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a18003b90 con 0x7f4a200b4370 2023-12-02T00:30:07.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.131+0000 7f4a1dffb640 1 -- 172.21.15.46:0/810839852 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a1800f040 con 0x7f4a200b4370 2023-12-02T00:30:07.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.131+0000 7f4a2775a640 1 -- 172.21.15.46:0/810839852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a20130a20 con 0x7f4a200b4370 2023-12-02T00:30:07.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.131+0000 7f4a1dffb640 1 -- 172.21.15.46:0/810839852 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a18004030 con 0x7f4a200b4370 2023-12-02T00:30:07.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.131+0000 7f4a2775a640 1 -- 172.21.15.46:0/810839852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a20130fe0 con 0x7f4a200b4370 2023-12-02T00:30:07.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.132+0000 7f4a2775a640 1 -- 172.21.15.46:0/810839852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a20069400 con 0x7f4a200b4370 2023-12-02T00:30:07.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.133+0000 7f4a1dffb640 1 -- 172.21.15.46:0/810839852 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4a18004190 con 0x7f4a200b4370 2023-12-02T00:30:07.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.133+0000 7f4a1dffb640 1 --2- 172.21.15.46:0/810839852 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f4a0c06ae40 0x7f4a0c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:07.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.133+0000 7f4a1dffb640 1 -- 172.21.15.46:0/810839852 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4a18089760 con 0x7f4a200b4370 2023-12-02T00:30:07.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.133+0000 7f4a24cf5640 1 --2- 172.21.15.46:0/810839852 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f4a0c06ae40 0x7f4a0c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:07.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.134+0000 7f4a24cf5640 1 --2- 172.21.15.46:0/810839852 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f4a0c06ae40 0x7f4a0c06d300 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4a14036000 tx=0x7f4a1403c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:07.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.137+0000 7f4a1dffb640 1 -- 172.21.15.46:0/810839852 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4a18054950 con 0x7f4a200b4370 2023-12-02T00:30:07.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.354+0000 7f4a2775a640 1 -- 172.21.15.46:0/810839852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4a20067670 con 0x7f4a200b4370 2023-12-02T00:30:07.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.355+0000 7f4a1dffb640 1 -- 172.21.15.46:0/810839852 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v13) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4a1801a060 con 0x7f4a200b4370 2023-12-02T00:30:07.356 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-02T00:30:07.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.357+0000 7f4a2775a640 1 -- 172.21.15.46:0/810839852 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f4a0c06ae40 msgr2=0x7f4a0c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:07.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.357+0000 7f4a2775a640 1 --2- 172.21.15.46:0/810839852 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f4a0c06ae40 0x7f4a0c06d300 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4a14036000 tx=0x7f4a1403c040 comp rx=0 tx=0).stop 2023-12-02T00:30:07.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.357+0000 7f4a2775a640 1 -- 172.21.15.46:0/810839852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a200b4370 msgr2=0x7f4a20158100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:07.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.357+0000 7f4a2775a640 1 --2- 172.21.15.46:0/810839852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a200b4370 0x7f4a20158100 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4a18009fa0 tx=0x7f4a1800d560 comp rx=0 tx=0).stop 2023-12-02T00:30:07.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.357+0000 7f4a2775a640 1 -- 172.21.15.46:0/810839852 shutdown_connections 2023-12-02T00:30:07.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.357+0000 7f4a2775a640 1 --2- 172.21.15.46:0/810839852 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f4a0c06ae40 0x7f4a0c06d300 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:07.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.357+0000 7f4a2775a640 1 --2- 172.21.15.46:0/810839852 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a20144980 0x7f4a2013e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:07.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.357+0000 7f4a2775a640 1 --2- 172.21.15.46:0/810839852 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a200ceb70 0x7f4a2013db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:07.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.357+0000 7f4a2775a640 1 --2- 172.21.15.46:0/810839852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a200b4370 0x7f4a20158100 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:07.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.357+0000 7f4a2775a640 1 -- 172.21.15.46:0/810839852 >> 172.21.15.46:0/810839852 conn(0x7f4a2006a4d0 msgr2=0x7f4a200b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:07.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.357+0000 7f4a2775a640 1 -- 172.21.15.46:0/810839852 shutdown_connections 2023-12-02T00:30:07.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:07.357+0000 7f4a2775a640 1 -- 172.21.15.46:0/810839852 wait complete. 2023-12-02T00:30:07.369 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-02T00:30:07.369 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-02T00:30:07.369 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2023-12-02T00:30:07.369 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-02T00:30:07.369 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-02T00:30:07.369 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-02T00:30:07.369 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-02T00:30:07.370 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-02T00:30:07.370 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-02T00:30:07.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:30:07.370 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2023-12-02T00:30:07.386 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-02T00:30:07.386 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-02T00:30:07.386 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2023-12-02T00:30:07.386 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:30:07.386 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2023-12-02T00:30:07.386 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:30:07.386 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-02T00:30:07.386 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2023-12-02T00:30:07.387 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2023-12-02T00:30:07.387 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-02T00:30:07.387 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2023-12-02T00:30:07.387 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2023-12-02T00:30:07.387 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-02T00:30:07.387 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2023-12-02T00:30:07.387 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2023-12-02T00:30:07.387 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-12-02T00:30:07.387 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42571sec preferred_lft 42571sec 2023-12-02T00:30:07.387 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link noprefixroute 2023-12-02T00:30:07.387 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:30:07.387 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-02T00:30:07.387 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2023-12-02T00:30:07.387 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2023-12-02T00:30:07.387 INFO:teuthology.orchestra.run.smithi046.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-02T00:30:07.388 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether b6:1a:8b:80:15:f1 brd ff:ff:ff:ff:ff:ff 2023-12-02T00:30:07.388 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-02T00:30:07.388 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:30:07.388 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::eca8:9ff:feb3:1726/64 scope link 2023-12-02T00:30:07.388 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:30:07.388 INFO:teuthology.orchestra.run.smithi046.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-02T00:30:07.388 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether b6:1a:8b:80:15:f1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-02T00:30:07.388 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::b41a:8bff:fe80:15f1/64 scope link 2023-12-02T00:30:07.389 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:30:07.389 INFO:teuthology.orchestra.run.smithi046.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-02T00:30:07.389 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether c6:93:a5:c0:e2:af brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-02T00:30:07.389 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c493:a5ff:fec0:e2af/64 scope link 2023-12-02T00:30:07.389 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:30:07.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:30:07.391 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-02T00:30:07.457 INFO:teuthology.orchestra.run.smithi046.stdout:1 2023-12-02T00:30:07.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:30:07.458 DEBUG:teuthology.orchestra.run.smithi046:> route 2023-12-02T00:30:07.514 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2023-12-02T00:30:07.514 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-02T00:30:07.514 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-12-02T00:30:07.514 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-12-02T00:30:07.514 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-02T00:30:07.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:30:07.515 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2023-12-02T00:30:07.515 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2023-12-02T00:30:07.515 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-02T00:30:07.515 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-02T00:30:07.515 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-02T00:30:07.515 DEBUG:teuthology.orchestra.run.smithi046:> ') 2023-12-02T00:30:07.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:30:07.621 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2023-12-02T00:30:07.636 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-02T00:30:07.636 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-02T00:30:07.636 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-02T00:30:07.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:30:07.637 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2023-12-02T00:30:07.707 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-12-02T00:30:07.707 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-02T00:30:07.707 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-02T00:30:07.762 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-02T00:30:07.762 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-02T00:30:07.762 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-02T00:30:07.818 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2023-12-02T00:30:07.889 DEBUG:teuthology.orchestra.run.smithi046:> cat /proc/self/mounts | awk '{print $2}' 2023-12-02T00:30:07.945 INFO:teuthology.orchestra.run.smithi046.stdout:/proc 2023-12-02T00:30:07.945 INFO:teuthology.orchestra.run.smithi046.stdout:/sys 2023-12-02T00:30:07.945 INFO:teuthology.orchestra.run.smithi046.stdout:/dev 2023-12-02T00:30:07.945 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/security 2023-12-02T00:30:07.945 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/shm 2023-12-02T00:30:07.945 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/pts 2023-12-02T00:30:07.946 INFO:teuthology.orchestra.run.smithi046.stdout:/run 2023-12-02T00:30:07.946 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/cgroup 2023-12-02T00:30:07.946 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/pstore 2023-12-02T00:30:07.946 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/bpf 2023-12-02T00:30:07.946 INFO:teuthology.orchestra.run.smithi046.stdout:/ 2023-12-02T00:30:07.946 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/selinux 2023-12-02T00:30:07.946 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2023-12-02T00:30:07.946 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/mqueue 2023-12-02T00:30:07.946 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/hugepages 2023-12-02T00:30:07.946 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/debug 2023-12-02T00:30:07.946 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/tracing 2023-12-02T00:30:07.946 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/fuse/connections 2023-12-02T00:30:07.946 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/config 2023-12-02T00:30:07.946 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-sysctl.service 2023-12-02T00:30:07.946 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-02T00:30:07.947 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-02T00:30:07.947 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/nfs/rpc_pipefs 2023-12-02T00:30:07.947 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/1000 2023-12-02T00:30:07.947 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph 2023-12-02T00:30:07.947 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-0 2023-12-02T00:30:07.947 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-1 2023-12-02T00:30:07.947 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-2 2023-12-02T00:30:07.947 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-3 2023-12-02T00:30:07.947 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns 2023-12-02T00:30:07.947 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-02T00:30:07.947 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-02T00:30:07.947 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-02T00:30:07.947 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-02T00:30:07.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:30:07.948 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2023-12-02T00:30:08.001 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-12-02T00:30:08.002 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-12-02T00:30:08.044 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2023-12-02T00:30:08.075 DEBUG:teuthology.orchestra.run.smithi046:> cat /proc/self/mounts | awk '{print $2}' 2023-12-02T00:30:08.131 INFO:teuthology.orchestra.run.smithi046.stdout:/proc 2023-12-02T00:30:08.131 INFO:teuthology.orchestra.run.smithi046.stdout:/sys 2023-12-02T00:30:08.131 INFO:teuthology.orchestra.run.smithi046.stdout:/dev 2023-12-02T00:30:08.131 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/security 2023-12-02T00:30:08.131 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/shm 2023-12-02T00:30:08.131 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/pts 2023-12-02T00:30:08.131 INFO:teuthology.orchestra.run.smithi046.stdout:/run 2023-12-02T00:30:08.131 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/cgroup 2023-12-02T00:30:08.131 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/pstore 2023-12-02T00:30:08.131 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/bpf 2023-12-02T00:30:08.131 INFO:teuthology.orchestra.run.smithi046.stdout:/ 2023-12-02T00:30:08.131 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/selinux 2023-12-02T00:30:08.132 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2023-12-02T00:30:08.132 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/mqueue 2023-12-02T00:30:08.132 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/hugepages 2023-12-02T00:30:08.132 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/debug 2023-12-02T00:30:08.132 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/tracing 2023-12-02T00:30:08.132 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/fuse/connections 2023-12-02T00:30:08.132 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/config 2023-12-02T00:30:08.132 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-sysctl.service 2023-12-02T00:30:08.132 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-02T00:30:08.132 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-02T00:30:08.132 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/nfs/rpc_pipefs 2023-12-02T00:30:08.132 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/1000 2023-12-02T00:30:08.132 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph 2023-12-02T00:30:08.132 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-0 2023-12-02T00:30:08.132 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-1 2023-12-02T00:30:08.133 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-2 2023-12-02T00:30:08.133 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-3 2023-12-02T00:30:08.133 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns 2023-12-02T00:30:08.133 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-02T00:30:08.133 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-02T00:30:08.133 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-02T00:30:08.133 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-02T00:30:08.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:30:08.134 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2023-12-02T00:30:08.138 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:2023-12-02T00:30:08.137+0000 7f859bb80480 -1 init, newargv = 0x559ae0109e50 newargc=15 2023-12-02T00:30:08.138 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[104429]: starting ceph client 2023-12-02T00:30:08.142 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[104429]: starting fuse 2023-12-02T00:30:08.153 INFO:teuthology.orchestra.run.smithi046.stdout:41 2023-12-02T00:30:08.153 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-12-02T00:30:08.154 DEBUG:teuthology.orchestra.run.smithi046:> sudo stdin-killer -- python3 -c ' 2023-12-02T00:30:08.154 DEBUG:teuthology.orchestra.run.smithi046:> import glob 2023-12-02T00:30:08.154 DEBUG:teuthology.orchestra.run.smithi046:> import re 2023-12-02T00:30:08.154 DEBUG:teuthology.orchestra.run.smithi046:> import os 2023-12-02T00:30:08.154 DEBUG:teuthology.orchestra.run.smithi046:> import subprocess 2023-12-02T00:30:08.154 DEBUG:teuthology.orchestra.run.smithi046:> 2023-12-02T00:30:08.154 DEBUG:teuthology.orchestra.run.smithi046:> def _find_admin_socket(client_name): 2023-12-02T00:30:08.154 DEBUG:teuthology.orchestra.run.smithi046:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-12-02T00:30:08.154 DEBUG:teuthology.orchestra.run.smithi046:> files = glob.glob(asok_path) 2023-12-02T00:30:08.154 DEBUG:teuthology.orchestra.run.smithi046:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-12-02T00:30:08.154 DEBUG:teuthology.orchestra.run.smithi046:> 2023-12-02T00:30:08.154 DEBUG:teuthology.orchestra.run.smithi046:> # Given a non-glob path, it better be there 2023-12-02T00:30:08.154 DEBUG:teuthology.orchestra.run.smithi046:> if "*" not in asok_path: 2023-12-02T00:30:08.155 DEBUG:teuthology.orchestra.run.smithi046:> assert(len(files) == 1) 2023-12-02T00:30:08.155 DEBUG:teuthology.orchestra.run.smithi046:> return files[0] 2023-12-02T00:30:08.155 DEBUG:teuthology.orchestra.run.smithi046:> 2023-12-02T00:30:08.155 DEBUG:teuthology.orchestra.run.smithi046:> for f in files: 2023-12-02T00:30:08.155 DEBUG:teuthology.orchestra.run.smithi046:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-02T00:30:08.155 DEBUG:teuthology.orchestra.run.smithi046:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-02T00:30:08.155 DEBUG:teuthology.orchestra.run.smithi046:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-02T00:30:08.155 DEBUG:teuthology.orchestra.run.smithi046:> contents = proc_f.read() 2023-12-02T00:30:08.155 DEBUG:teuthology.orchestra.run.smithi046:> if mountpoint in contents: 2023-12-02T00:30:08.155 DEBUG:teuthology.orchestra.run.smithi046:> return f 2023-12-02T00:30:08.155 DEBUG:teuthology.orchestra.run.smithi046:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-02T00:30:08.155 DEBUG:teuthology.orchestra.run.smithi046:> 2023-12-02T00:30:08.155 DEBUG:teuthology.orchestra.run.smithi046:> print(_find_admin_socket("client.0")) 2023-12-02T00:30:08.155 DEBUG:teuthology.orchestra.run.smithi046:> ' 2023-12-02T00:30:08.279 INFO:teuthology.orchestra.run.smithi046.stdout:/var/run/ceph/ceph-client.0.104429.asok 2023-12-02T00:30:08.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-12-02T00:30:08.289 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.104429.asok 2023-12-02T00:30:08.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:30:08.290 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.104429.asok status 2023-12-02T00:30:08.432 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-12-02T00:30:08.432 INFO:teuthology.orchestra.run.smithi046.stdout: "metadata": { 2023-12-02T00:30:08.432 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_sha1": "bed7681480ccb0192b91a35e691658867e1e96b7", 2023-12-02T00:30:08.432 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_version": "ceph version 18.0.0-7631-gbed76814 (bed7681480ccb0192b91a35e691658867e1e96b7) reef (dev)", 2023-12-02T00:30:08.432 INFO:teuthology.orchestra.run.smithi046.stdout: "entity_id": "0", 2023-12-02T00:30:08.432 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": "smithi046", 2023-12-02T00:30:08.432 INFO:teuthology.orchestra.run.smithi046.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-12-02T00:30:08.432 INFO:teuthology.orchestra.run.smithi046.stdout: "pid": "104429", 2023-12-02T00:30:08.433 INFO:teuthology.orchestra.run.smithi046.stdout: "root": "/" 2023-12-02T00:30:08.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:30:08.433 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_count": 0, 2023-12-02T00:30:08.433 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_pinned_count": 0, 2023-12-02T00:30:08.433 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 5502, 2023-12-02T00:30:08.433 INFO:teuthology.orchestra.run.smithi046.stdout: "inst": { 2023-12-02T00:30:08.433 INFO:teuthology.orchestra.run.smithi046.stdout: "name": { 2023-12-02T00:30:08.433 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "client", 2023-12-02T00:30:08.433 INFO:teuthology.orchestra.run.smithi046.stdout: "num": 5502 2023-12-02T00:30:08.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:30:08.433 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2023-12-02T00:30:08.433 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2023-12-02T00:30:08.433 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2023-12-02T00:30:08.433 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2044825109 2023-12-02T00:30:08.433 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:30:08.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:30:08.434 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2023-12-02T00:30:08.434 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2023-12-02T00:30:08.434 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2023-12-02T00:30:08.434 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2044825109 2023-12-02T00:30:08.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:30:08.434 INFO:teuthology.orchestra.run.smithi046.stdout: "inst_str": "client.5502 192.168.0.1:0/2044825109", 2023-12-02T00:30:08.434 INFO:teuthology.orchestra.run.smithi046.stdout: "addr_str": "192.168.0.1:0/2044825109", 2023-12-02T00:30:08.434 INFO:teuthology.orchestra.run.smithi046.stdout: "inode_count": 1, 2023-12-02T00:30:08.434 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_epoch": 13, 2023-12-02T00:30:08.434 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch": 35, 2023-12-02T00:30:08.434 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch_barrier": 0, 2023-12-02T00:30:08.434 INFO:teuthology.orchestra.run.smithi046.stdout: "blocklisted": false, 2023-12-02T00:30:08.434 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs" 2023-12-02T00:30:08.434 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-12-02T00:30:08.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:30:08.440 DEBUG:teuthology.orchestra.run.smithi046:> stat --file-system '--printf=%T 2023-12-02T00:30:08.440 DEBUG:teuthology.orchestra.run.smithi046:> ' -- /home/ubuntu/cephtest/mnt.0 2023-12-02T00:30:08.456 INFO:teuthology.orchestra.run.smithi046.stdout:fuseblk 2023-12-02T00:30:08.456 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-12-02T00:30:08.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:30:08.457 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-02T00:30:08.528 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create data_new --pg_num_min 64 2023-12-02T00:30:08.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.678+0000 7f3a19864640 1 Processor -- start 2023-12-02T00:30:08.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.678+0000 7f3a19864640 1 -- start start 2023-12-02T00:30:08.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.679+0000 7f3a19864640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a140709c0 0x7f3a140cd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:08.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.679+0000 7f3a19864640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3a140cdd10 0x7f3a140cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:08.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.679+0000 7f3a19864640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a1406d580 con 0x7f3a1415d270 2023-12-02T00:30:08.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.679+0000 7f3a19864640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a140ce1a0 con 0x7f3a140cdd10 2023-12-02T00:30:08.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.679+0000 7f3a19864640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a140d0c40 con 0x7f3a140709c0 2023-12-02T00:30:08.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.679+0000 7f3a137fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3a140cdd10 0x7f3a140cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:08.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.679+0000 7f3a137fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3a140cdd10 0x7f3a140cc6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46224/0 (socket says 172.21.15.46:46224) 2023-12-02T00:30:08.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.679+0000 7f3a137fe640 1 -- 172.21.15.46:0/1622038170 learned_addr learned my addr 172.21.15.46:0/1622038170 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:08.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.680+0000 7f3a127fc640 1 --2- 172.21.15.46:0/1622038170 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a140709c0 0x7f3a140cd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:08.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.680+0000 7f3a137fe640 1 -- 172.21.15.46:0/1622038170 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a140709c0 msgr2=0x7f3a140cd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:08.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.680+0000 7f3a137fe640 1 --2- 172.21.15.46:0/1622038170 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a140709c0 0x7f3a140cd7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:08.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.680+0000 7f3a137fe640 1 -- 172.21.15.46:0/1622038170 >> v1:172.21.15.162:6789/0 conn(0x7f3a1415d270 legacy=0x7f3a1406ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:08.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.680+0000 7f3a137fe640 1 -- 172.21.15.46:0/1622038170 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a140ced90 con 0x7f3a140cdd10 2023-12-02T00:30:08.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.680+0000 7f3a127fc640 1 --2- 172.21.15.46:0/1622038170 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a140709c0 0x7f3a140cd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:30:08.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.680+0000 7f3a137fe640 1 --2- 172.21.15.46:0/1622038170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a140cdd10 0x7f3a140cc6a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3a04009e50 tx=0x7f3a0402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2da5490d462976e0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:08.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.681+0000 7f3a11ffb640 1 -- 172.21.15.46:0/1622038170 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0402c9e0 con 0x7f3a140cdd10 2023-12-02T00:30:08.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.681+0000 7f3a11ffb640 1 -- 172.21.15.46:0/1622038170 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a0402cb40 con 0x7f3a140cdd10 2023-12-02T00:30:08.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.681+0000 7f3a11ffb640 1 -- 172.21.15.46:0/1622038170 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a040343f0 con 0x7f3a140cdd10 2023-12-02T00:30:08.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.681+0000 7f3a19864640 1 -- 172.21.15.46:0/1622038170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a140cdd10 msgr2=0x7f3a140cc6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:08.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.681+0000 7f3a19864640 1 --2- 172.21.15.46:0/1622038170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a140cdd10 0x7f3a140cc6a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3a04009e50 tx=0x7f3a0402be40 comp rx=0 tx=0).stop 2023-12-02T00:30:08.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.681+0000 7f3a19864640 1 -- 172.21.15.46:0/1622038170 shutdown_connections 2023-12-02T00:30:08.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.681+0000 7f3a19864640 1 --2- 172.21.15.46:0/1622038170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a140cdd10 0x7f3a140cc6a0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:08.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.681+0000 7f3a19864640 1 --2- 172.21.15.46:0/1622038170 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a140709c0 0x7f3a140cd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:08.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.681+0000 7f3a19864640 1 -- 172.21.15.46:0/1622038170 >> 172.21.15.46:0/1622038170 conn(0x7f3a140b2c10 msgr2=0x7f3a140b2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:08.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.681+0000 7f3a19864640 1 -- 172.21.15.46:0/1622038170 shutdown_connections 2023-12-02T00:30:08.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.681+0000 7f3a19864640 1 -- 172.21.15.46:0/1622038170 wait complete. 2023-12-02T00:30:08.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.682+0000 7f3a19864640 1 Processor -- start 2023-12-02T00:30:08.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.683+0000 7f3a19864640 1 -- start start 2023-12-02T00:30:08.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.683+0000 7f3a19864640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a140709c0 0x7f3a1415c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:08.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.683+0000 7f3a19864640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a140cdd10 0x7f3a140c72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:08.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.683+0000 7f3a19864640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a1415d270 0x7f3a140c5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:08.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.683+0000 7f3a19864640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a140c7d40 con 0x7f3a140709c0 2023-12-02T00:30:08.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.683+0000 7f3a19864640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a141f0d40 con 0x7f3a140cdd10 2023-12-02T00:30:08.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.683+0000 7f3a19864640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a141f1020 con 0x7f3a1415d270 2023-12-02T00:30:08.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.683+0000 7f3a12ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a140709c0 0x7f3a1415c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:08.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.683+0000 7f3a12ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a140709c0 0x7f3a1415c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46238/0 (socket says 172.21.15.46:46238) 2023-12-02T00:30:08.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.683+0000 7f3a12ffd640 1 -- 172.21.15.46:0/1597370366 learned_addr learned my addr 172.21.15.46:0/1597370366 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:08.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.683+0000 7f3a127fc640 1 --2- 172.21.15.46:0/1597370366 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a140cdd10 0x7f3a140c72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:08.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.683+0000 7f3a137fe640 1 --2- 172.21.15.46:0/1597370366 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a1415d270 0x7f3a140c5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:08.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.684+0000 7f3a12ffd640 1 -- 172.21.15.46:0/1597370366 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a1415d270 msgr2=0x7f3a140c5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:08.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.684+0000 7f3a12ffd640 1 --2- 172.21.15.46:0/1597370366 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a1415d270 0x7f3a140c5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:08.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.684+0000 7f3a12ffd640 1 -- 172.21.15.46:0/1597370366 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a140cdd10 msgr2=0x7f3a140c72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:08.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.684+0000 7f3a12ffd640 1 --2- 172.21.15.46:0/1597370366 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a140cdd10 0x7f3a140c72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:08.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.684+0000 7f3a12ffd640 1 -- 172.21.15.46:0/1597370366 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a04009b30 con 0x7f3a140709c0 2023-12-02T00:30:08.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.684+0000 7f3a12ffd640 1 --2- 172.21.15.46:0/1597370366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a140709c0 0x7f3a1415c3f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c00c430 tx=0x7f3a0c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:08.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.684+0000 7f3a0bfff640 1 -- 172.21.15.46:0/1597370366 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c011790 con 0x7f3a140709c0 2023-12-02T00:30:08.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.684+0000 7f3a0bfff640 1 -- 172.21.15.46:0/1597370366 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a0c0118f0 con 0x7f3a140709c0 2023-12-02T00:30:08.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.684+0000 7f3a19864640 1 -- 172.21.15.46:0/1597370366 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a141f1300 con 0x7f3a140709c0 2023-12-02T00:30:08.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.684+0000 7f3a0bfff640 1 -- 172.21.15.46:0/1597370366 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c011ce0 con 0x7f3a140709c0 2023-12-02T00:30:08.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.684+0000 7f3a19864640 1 -- 172.21.15.46:0/1597370366 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a1415c930 con 0x7f3a140709c0 2023-12-02T00:30:08.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.685+0000 7f3a19864640 1 -- 172.21.15.46:0/1597370366 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a14067fc0 con 0x7f3a140709c0 2023-12-02T00:30:08.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.686+0000 7f3a0bfff640 1 -- 172.21.15.46:0/1597370366 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3a0c01c460 con 0x7f3a140709c0 2023-12-02T00:30:08.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.686+0000 7f3a0bfff640 1 --2- 172.21.15.46:0/1597370366 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f39f406ae40 0x7f39f406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:08.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.686+0000 7f3a0bfff640 1 -- 172.21.15.46:0/1597370366 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3a0c089450 con 0x7f3a140709c0 2023-12-02T00:30:08.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.686+0000 7f3a127fc640 1 --2- 172.21.15.46:0/1597370366 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f39f406ae40 0x7f39f406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:08.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.687+0000 7f3a127fc640 1 --2- 172.21.15.46:0/1597370366 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f39f406ae40 0x7f39f406d300 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3a140c44b0 tx=0x7f39fc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:08.693 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.690+0000 7f3a0bfff640 1 -- 172.21.15.46:0/1597370366 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3a0c0545b0 con 0x7f3a140709c0 2023-12-02T00:30:08.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.866+0000 7f3a19864640 1 -- 172.21.15.46:0/1597370366 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "data_new", "pg_num_min": 64} v 0) v1 -- 0x7f3a141f1820 con 0x7f3a140709c0 2023-12-02T00:30:08.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.949+0000 7f3a0bfff640 1 -- 172.21.15.46:0/1597370366 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "data_new", "pg_num_min": 64}]=0 pool 'data_new' created v36) v1 ==== 124+0+0 (secure 0 0 0) 0x7f3a0c057bd0 con 0x7f3a140709c0 2023-12-02T00:30:08.951 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'data_new' created 2023-12-02T00:30:08.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.952+0000 7f3a19864640 1 -- 172.21.15.46:0/1597370366 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f39f406ae40 msgr2=0x7f39f406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:08.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.952+0000 7f3a19864640 1 --2- 172.21.15.46:0/1597370366 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f39f406ae40 0x7f39f406d300 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3a140c44b0 tx=0x7f39fc008040 comp rx=0 tx=0).stop 2023-12-02T00:30:08.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.952+0000 7f3a19864640 1 -- 172.21.15.46:0/1597370366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a140709c0 msgr2=0x7f3a1415c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:08.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.952+0000 7f3a19864640 1 --2- 172.21.15.46:0/1597370366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a140709c0 0x7f3a1415c3f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c00c430 tx=0x7f3a0c00c8f0 comp rx=0 tx=0).stop 2023-12-02T00:30:08.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.952+0000 7f3a19864640 1 -- 172.21.15.46:0/1597370366 shutdown_connections 2023-12-02T00:30:08.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.952+0000 7f3a19864640 1 --2- 172.21.15.46:0/1597370366 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f39f406ae40 0x7f39f406d300 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:08.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.952+0000 7f3a19864640 1 --2- 172.21.15.46:0/1597370366 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a1415d270 0x7f3a140c5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:08.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.952+0000 7f3a19864640 1 --2- 172.21.15.46:0/1597370366 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a140cdd10 0x7f3a140c72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:08.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.952+0000 7f3a19864640 1 --2- 172.21.15.46:0/1597370366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a140709c0 0x7f3a1415c3f0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:08.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.952+0000 7f3a19864640 1 -- 172.21.15.46:0/1597370366 >> 172.21.15.46:0/1597370366 conn(0x7f3a140b2c10 msgr2=0x7f3a140b0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:08.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.952+0000 7f3a19864640 1 -- 172.21.15.46:0/1597370366 shutdown_connections 2023-12-02T00:30:08.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:08.952+0000 7f3a19864640 1 -- 172.21.15.46:0/1597370366 wait complete. 2023-12-02T00:30:08.968 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs data_new 2023-12-02T00:30:09.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.075+0000 7fb3c4cd7640 1 Processor -- start 2023-12-02T00:30:09.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.075+0000 7fb3c4cd7640 1 -- start start 2023-12-02T00:30:09.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.075+0000 7fb3c4cd7640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb3c0160f70 0x7fb3c00cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:09.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.075+0000 7fb3c4cd7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3c00ce480 0x7fb3c00d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:09.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.075+0000 7fb3c4cd7640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3c01614e0 con 0x7fb3c015d2d0 2023-12-02T00:30:09.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.075+0000 7fb3c4cd7640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3c01617c0 con 0x7fb3c0160f70 2023-12-02T00:30:09.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.075+0000 7fb3c4cd7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c0161aa0 con 0x7fb3c00ce480 2023-12-02T00:30:09.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.076+0000 7fb3be59c640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb3c015d2d0 0x7fb3c00b8b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36048/0 (socket says 172.21.15.46:36048) 2023-12-02T00:30:09.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.076+0000 7fb3be59c640 1 -- 172.21.15.46:0/3404501886 learned_addr learned my addr 172.21.15.46:0/3404501886 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:30:09.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.076+0000 7fb3b7fff640 1 --2- 172.21.15.46:0/3404501886 >> v2:172.21.15.46:3300/0 conn(0x7fb3c0160f70 0x7fb3c00cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:09.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.076+0000 7fb3b7fff640 1 -- 172.21.15.46:0/3404501886 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3c00ce480 msgr2=0x7fb3c00d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:09.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.076+0000 7fb3b7fff640 1 --2- 172.21.15.46:0/3404501886 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3c00ce480 0x7fb3c00d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:09.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.076+0000 7fb3b7fff640 1 -- 172.21.15.46:0/3404501886 >> v1:172.21.15.46:6789/0 conn(0x7fb3c015d2d0 legacy=0x7fb3c00b8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:09.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.076+0000 7fb3b7fff640 1 -- 172.21.15.46:0/3404501886 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3c0161d80 con 0x7fb3c0160f70 2023-12-02T00:30:09.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.076+0000 7fb3bdd9b640 1 -- 172.21.15.46:0/3404501886 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3511228046 0 0) 0x7fb3c01614e0 con 0x7fb3c015d2d0 2023-12-02T00:30:09.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.077+0000 7fb3b7fff640 1 --2- 172.21.15.46:0/3404501886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3c0160f70 0x7fb3c00cdf40 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb3a80097a0 tx=0x7fb3a802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d88de45d34ed4a7 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:09.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.077+0000 7fb3bdd9b640 1 -- 172.21.15.46:0/3404501886 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a8031830 con 0x7fb3c0160f70 2023-12-02T00:30:09.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.077+0000 7fb3bdd9b640 1 -- 172.21.15.46:0/3404501886 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3a8031990 con 0x7fb3c0160f70 2023-12-02T00:30:09.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.077+0000 7fb3bdd9b640 1 -- 172.21.15.46:0/3404501886 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a8031d20 con 0x7fb3c0160f70 2023-12-02T00:30:09.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.077+0000 7fb3c4cd7640 1 -- 172.21.15.46:0/3404501886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3c0160f70 msgr2=0x7fb3c00cdf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:09.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.077+0000 7fb3c4cd7640 1 --2- 172.21.15.46:0/3404501886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3c0160f70 0x7fb3c00cdf40 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb3a80097a0 tx=0x7fb3a802b8d0 comp rx=0 tx=0).stop 2023-12-02T00:30:09.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.077+0000 7fb3c4cd7640 1 -- 172.21.15.46:0/3404501886 shutdown_connections 2023-12-02T00:30:09.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.077+0000 7fb3c4cd7640 1 --2- 172.21.15.46:0/3404501886 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3c00ce480 0x7fb3c00d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:09.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.078+0000 7fb3c4cd7640 1 --2- 172.21.15.46:0/3404501886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3c0160f70 0x7fb3c00cdf40 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:09.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.078+0000 7fb3c4cd7640 1 -- 172.21.15.46:0/3404501886 >> 172.21.15.46:0/3404501886 conn(0x7fb3c0072230 msgr2=0x7fb3c00cfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:09.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.078+0000 7fb3c4cd7640 1 -- 172.21.15.46:0/3404501886 shutdown_connections 2023-12-02T00:30:09.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.078+0000 7fb3c4cd7640 1 -- 172.21.15.46:0/3404501886 wait complete. 2023-12-02T00:30:09.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.079+0000 7fb3c4cd7640 1 Processor -- start 2023-12-02T00:30:09.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.079+0000 7fb3c4cd7640 1 -- start start 2023-12-02T00:30:09.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.080+0000 7fb3c4cd7640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3c00ce480 0x7fb3c0160510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:09.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.080+0000 7fb3c4cd7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3c015d2d0 0x7fb3c00b56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:09.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.080+0000 7fb3c4cd7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3c0160f70 0x7fb3c0144aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:09.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.080+0000 7fb3c4cd7640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c0144fe0 con 0x7fb3c015d2d0 2023-12-02T00:30:09.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.080+0000 7fb3c4cd7640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c00b43b0 con 0x7fb3c00ce480 2023-12-02T00:30:09.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.080+0000 7fb3c4cd7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c00b4650 con 0x7fb3c0160f70 2023-12-02T00:30:09.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.080+0000 7fb3b7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3c015d2d0 0x7fb3c00b56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:09.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.080+0000 7fb3b7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3c015d2d0 0x7fb3c00b56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46248/0 (socket says 172.21.15.46:46248) 2023-12-02T00:30:09.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.080+0000 7fb3b7fff640 1 -- 172.21.15.46:0/323300333 learned_addr learned my addr 172.21.15.46:0/323300333 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:09.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.080+0000 7fb3be59c640 1 --2- 172.21.15.46:0/323300333 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3c00ce480 0x7fb3c0160510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:09.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.080+0000 7fb3bed9d640 1 --2- 172.21.15.46:0/323300333 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3c0160f70 0x7fb3c0144aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:09.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.080+0000 7fb3b7fff640 1 -- 172.21.15.46:0/323300333 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3c0160f70 msgr2=0x7fb3c0144aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:09.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.080+0000 7fb3b7fff640 1 --2- 172.21.15.46:0/323300333 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3c0160f70 0x7fb3c0144aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:09.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.080+0000 7fb3b7fff640 1 -- 172.21.15.46:0/323300333 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3c00ce480 msgr2=0x7fb3c0160510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:09.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.080+0000 7fb3b7fff640 1 --2- 172.21.15.46:0/323300333 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3c00ce480 0x7fb3c0160510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:09.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.080+0000 7fb3b7fff640 1 -- 172.21.15.46:0/323300333 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a8009400 con 0x7fb3c015d2d0 2023-12-02T00:30:09.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.080+0000 7fb3be59c640 1 --2- 172.21.15.46:0/323300333 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3c00ce480 0x7fb3c0160510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:30:09.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.080+0000 7fb3b7fff640 1 --2- 172.21.15.46:0/323300333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3c015d2d0 0x7fb3c00b56f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb3a80097a0 tx=0x7fb3a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:09.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.080+0000 7fb3b6ffd640 1 -- 172.21.15.46:0/323300333 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a80317d0 con 0x7fb3c015d2d0 2023-12-02T00:30:09.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.080+0000 7fb3b6ffd640 1 -- 172.21.15.46:0/323300333 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3a8004030 con 0x7fb3c015d2d0 2023-12-02T00:30:09.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.080+0000 7fb3b6ffd640 1 -- 172.21.15.46:0/323300333 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a80043e0 con 0x7fb3c015d2d0 2023-12-02T00:30:09.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.080+0000 7fb3c4cd7640 1 -- 172.21.15.46:0/323300333 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3c00b4930 con 0x7fb3c015d2d0 2023-12-02T00:30:09.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.081+0000 7fb3c4cd7640 1 -- 172.21.15.46:0/323300333 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3c00b4ef0 con 0x7fb3c015d2d0 2023-12-02T00:30:09.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.081+0000 7fb3c4cd7640 1 -- 172.21.15.46:0/323300333 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3c0069400 con 0x7fb3c015d2d0 2023-12-02T00:30:09.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.082+0000 7fb3b6ffd640 1 -- 172.21.15.46:0/323300333 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb3a8043070 con 0x7fb3c015d2d0 2023-12-02T00:30:09.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.082+0000 7fb3b6ffd640 1 --2- 172.21.15.46:0/323300333 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb3a006ae40 0x7fb3a006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:09.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.083+0000 7fb3b6ffd640 1 -- 172.21.15.46:0/323300333 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6072+0+0 (secure 0 0 0) 0x7fb3a80ab550 con 0x7fb3c015d2d0 2023-12-02T00:30:09.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.083+0000 7fb3be59c640 1 --2- 172.21.15.46:0/323300333 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb3a006ae40 0x7fb3a006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:09.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.083+0000 7fb3be59c640 1 --2- 172.21.15.46:0/323300333 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb3a006ae40 0x7fb3a006d300 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0000f60 tx=0x7fb3b0007c10 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:09.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.087+0000 7fb3b6ffd640 1 -- 172.21.15.46:0/323300333 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb3a80765b0 con 0x7fb3c015d2d0 2023-12-02T00:30:09.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.300+0000 7fb3c4cd7640 1 -- 172.21.15.46:0/323300333 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "data_new"} v 0) v1 -- 0x7fb3c00d0aa0 con 0x7fb3c015d2d0 2023-12-02T00:30:09.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.954+0000 7fb3b6ffd640 1 -- 172.21.15.46:0/323300333 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "data_new"}]=0 added data pool 6 to fsmap v14) v1 ==== 131+0+0 (secure 0 0 0) 0x7fb3a8079bd0 con 0x7fb3c015d2d0 2023-12-02T00:30:09.956 INFO:teuthology.orchestra.run.smithi046.stderr:added data pool 6 to fsmap 2023-12-02T00:30:09.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.956+0000 7fb3c4cd7640 1 -- 172.21.15.46:0/323300333 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb3a006ae40 msgr2=0x7fb3a006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:09.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.956+0000 7fb3c4cd7640 1 --2- 172.21.15.46:0/323300333 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb3a006ae40 0x7fb3a006d300 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0000f60 tx=0x7fb3b0007c10 comp rx=0 tx=0).stop 2023-12-02T00:30:09.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.956+0000 7fb3c4cd7640 1 -- 172.21.15.46:0/323300333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3c015d2d0 msgr2=0x7fb3c00b56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:09.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.956+0000 7fb3c4cd7640 1 --2- 172.21.15.46:0/323300333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3c015d2d0 0x7fb3c00b56f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb3a80097a0 tx=0x7fb3a80046c0 comp rx=0 tx=0).stop 2023-12-02T00:30:09.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.956+0000 7fb3c4cd7640 1 -- 172.21.15.46:0/323300333 shutdown_connections 2023-12-02T00:30:09.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.956+0000 7fb3c4cd7640 1 --2- 172.21.15.46:0/323300333 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb3a006ae40 0x7fb3a006d300 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:09.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.956+0000 7fb3c4cd7640 1 --2- 172.21.15.46:0/323300333 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3c0160f70 0x7fb3c0144aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:09.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.956+0000 7fb3c4cd7640 1 --2- 172.21.15.46:0/323300333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3c015d2d0 0x7fb3c00b56f0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:09.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.956+0000 7fb3c4cd7640 1 --2- 172.21.15.46:0/323300333 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3c00ce480 0x7fb3c0160510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:09.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.956+0000 7fb3c4cd7640 1 -- 172.21.15.46:0/323300333 >> 172.21.15.46:0/323300333 conn(0x7fb3c0072230 msgr2=0x7fb3c0159720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:09.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.956+0000 7fb3c4cd7640 1 -- 172.21.15.46:0/323300333 shutdown_connections 2023-12-02T00:30:09.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:09.956+0000 7fb3c4cd7640 1 -- 172.21.15.46:0/323300333 wait complete. 2023-12-02T00:30:09.970 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-02T00:30:10.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.079+0000 7fddb8525640 1 Processor -- start 2023-12-02T00:30:10.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.079+0000 7fddb8525640 1 -- start start 2023-12-02T00:30:10.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.080+0000 7fddb8525640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fddb0144980 0x7fddb015c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:10.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.080+0000 7fddb8525640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fddb00b4370 0x7fddb00b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:10.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.080+0000 7fddb8525640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddb0071770 con 0x7fddb00ceb70 2023-12-02T00:30:10.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.080+0000 7fddb8525640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddb00ccb60 con 0x7fddb0144980 2023-12-02T00:30:10.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.080+0000 7fddb8525640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddb00b4ce0 con 0x7fddb00b4370 2023-12-02T00:30:10.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.080+0000 7fddb62c1640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fddb0144980 0x7fddb015c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:10.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.080+0000 7fddb5ac0640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fddb00ceb70 0x7fddb00cc450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36062/0 (socket says 172.21.15.46:36062) 2023-12-02T00:30:10.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.080+0000 7fddb5ac0640 1 -- 172.21.15.46:0/1125380879 learned_addr learned my addr 172.21.15.46:0/1125380879 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:30:10.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.080+0000 7fddb62c1640 1 -- 172.21.15.46:0/1125380879 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fddb00b4370 msgr2=0x7fddb00b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:10.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.080+0000 7fddb62c1640 1 --2- 172.21.15.46:0/1125380879 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fddb00b4370 0x7fddb00b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:10.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.080+0000 7fddb62c1640 1 -- 172.21.15.46:0/1125380879 >> v1:172.21.15.46:6789/0 conn(0x7fddb00ceb70 legacy=0x7fddb00cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:10.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.080+0000 7fddb62c1640 1 -- 172.21.15.46:0/1125380879 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb00ccdc0 con 0x7fddb0144980 2023-12-02T00:30:10.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.080+0000 7fddb52bf640 1 -- 172.21.15.46:0/1125380879 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4200702057 0 0) 0x7fddb0071770 con 0x7fddb00ceb70 2023-12-02T00:30:10.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.081+0000 7fddb62c1640 1 --2- 172.21.15.46:0/1125380879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddb0144980 0x7fddb015c7c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fdda00097a0 tx=0x7fdda002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57315299f1880e8d server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:10.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.081+0000 7fddb52bf640 1 -- 172.21.15.46:0/1125380879 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda0031830 con 0x7fddb0144980 2023-12-02T00:30:10.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.081+0000 7fddb52bf640 1 -- 172.21.15.46:0/1125380879 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdda0031990 con 0x7fddb0144980 2023-12-02T00:30:10.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.081+0000 7fddb52bf640 1 -- 172.21.15.46:0/1125380879 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda0031d20 con 0x7fddb0144980 2023-12-02T00:30:10.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.082+0000 7fddb8525640 1 -- 172.21.15.46:0/1125380879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddb0144980 msgr2=0x7fddb015c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:10.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.082+0000 7fddb8525640 1 --2- 172.21.15.46:0/1125380879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddb0144980 0x7fddb015c7c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fdda00097a0 tx=0x7fdda002b8d0 comp rx=0 tx=0).stop 2023-12-02T00:30:10.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.082+0000 7fddb8525640 1 -- 172.21.15.46:0/1125380879 shutdown_connections 2023-12-02T00:30:10.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.082+0000 7fddb8525640 1 --2- 172.21.15.46:0/1125380879 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fddb00b4370 0x7fddb00b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:10.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.082+0000 7fddb8525640 1 --2- 172.21.15.46:0/1125380879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddb0144980 0x7fddb015c7c0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:10.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.082+0000 7fddb8525640 1 -- 172.21.15.46:0/1125380879 >> 172.21.15.46:0/1125380879 conn(0x7fddb006a4d0 msgr2=0x7fddb006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:10.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.082+0000 7fddb8525640 1 -- 172.21.15.46:0/1125380879 shutdown_connections 2023-12-02T00:30:10.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.082+0000 7fddb8525640 1 -- 172.21.15.46:0/1125380879 wait complete. 2023-12-02T00:30:10.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.083+0000 7fddb8525640 1 Processor -- start 2023-12-02T00:30:10.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.083+0000 7fddb8525640 1 -- start start 2023-12-02T00:30:10.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.084+0000 7fddb8525640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fddb00b4370 0x7fddb01ede60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:10.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.084+0000 7fddb8525640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fddb00ceb70 0x7fddb01f1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:10.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.084+0000 7fddb8525640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddb0144980 0x7fddb00b12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:10.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.084+0000 7fddb8525640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb00b0ed0 con 0x7fddb0144980 2023-12-02T00:30:10.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.084+0000 7fddb8525640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb0141160 con 0x7fddb00ceb70 2023-12-02T00:30:10.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.084+0000 7fddb8525640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddb00b0250 con 0x7fddb00b4370 2023-12-02T00:30:10.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.084+0000 7fddb6ac2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddb0144980 0x7fddb00b12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:10.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.084+0000 7fddb6ac2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddb0144980 0x7fddb00b12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46264/0 (socket says 172.21.15.46:46264) 2023-12-02T00:30:10.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.084+0000 7fddb6ac2640 1 -- 172.21.15.46:0/767223239 learned_addr learned my addr 172.21.15.46:0/767223239 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:10.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.084+0000 7fddb5ac0640 1 --2- 172.21.15.46:0/767223239 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fddb00ceb70 0x7fddb01f1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:10.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.084+0000 7fddb62c1640 1 --2- 172.21.15.46:0/767223239 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fddb00b4370 0x7fddb01ede60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:10.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.084+0000 7fddb6ac2640 1 -- 172.21.15.46:0/767223239 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fddb00b4370 msgr2=0x7fddb01ede60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:10.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.084+0000 7fddb6ac2640 1 --2- 172.21.15.46:0/767223239 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fddb00b4370 0x7fddb01ede60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:10.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.084+0000 7fddb6ac2640 1 -- 172.21.15.46:0/767223239 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fddb00ceb70 msgr2=0x7fddb01f1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:10.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.084+0000 7fddb6ac2640 1 --2- 172.21.15.46:0/767223239 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fddb00ceb70 0x7fddb01f1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:10.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.084+0000 7fddb6ac2640 1 -- 172.21.15.46:0/767223239 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdda0009400 con 0x7fddb0144980 2023-12-02T00:30:10.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.084+0000 7fddb62c1640 1 --2- 172.21.15.46:0/767223239 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fddb00b4370 0x7fddb01ede60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:30:10.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.085+0000 7fddb6ac2640 1 --2- 172.21.15.46:0/767223239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddb0144980 0x7fddb00b12f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fdda800b970 tx=0x7fdda800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:10.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.085+0000 7fdda6ffd640 1 -- 172.21.15.46:0/767223239 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda800cae0 con 0x7fddb0144980 2023-12-02T00:30:10.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.085+0000 7fdda6ffd640 1 -- 172.21.15.46:0/767223239 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdda800cc60 con 0x7fddb0144980 2023-12-02T00:30:10.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.085+0000 7fdda6ffd640 1 -- 172.21.15.46:0/767223239 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda8014440 con 0x7fddb0144980 2023-12-02T00:30:10.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.085+0000 7fddb8525640 1 -- 172.21.15.46:0/767223239 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddb00b0530 con 0x7fddb0144980 2023-12-02T00:30:10.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.085+0000 7fddb8525640 1 -- 172.21.15.46:0/767223239 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddb00b0af0 con 0x7fddb0144980 2023-12-02T00:30:10.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.086+0000 7fddb8525640 1 -- 172.21.15.46:0/767223239 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddb0155b10 con 0x7fddb0144980 2023-12-02T00:30:10.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.086+0000 7fdda6ffd640 1 -- 172.21.15.46:0/767223239 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdda801d3b0 con 0x7fddb0144980 2023-12-02T00:30:10.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.087+0000 7fdda6ffd640 1 --2- 172.21.15.46:0/767223239 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fdd9406ae70 0x7fdd9406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:10.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.087+0000 7fdda6ffd640 1 -- 172.21.15.46:0/767223239 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fdda8089540 con 0x7fddb0144980 2023-12-02T00:30:10.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.087+0000 7fddb62c1640 1 --2- 172.21.15.46:0/767223239 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fdd9406ae70 0x7fdd9406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:10.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.088+0000 7fddb62c1640 1 --2- 172.21.15.46:0/767223239 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fdd9406ae70 0x7fdd9406d330 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdda0036000 tx=0x7fdda003c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:10.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.091+0000 7fdda6ffd640 1 -- 172.21.15.46:0/767223239 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdda80545a0 con 0x7fddb0144980 2023-12-02T00:30:10.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.303+0000 7fddb8525640 1 -- 172.21.15.46:0/767223239 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fddb00b1010 con 0x7fddb0144980 2023-12-02T00:30:10.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.304+0000 7fdda6ffd640 1 -- 172.21.15.46:0/767223239 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4919 (secure 0 0 0) 0x7fddb00b1010 con 0x7fddb0144980 2023-12-02T00:30:10.305 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:30:10.305 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4811,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/3097254664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3097254664},{"type":"v1","addr":"172.21.15.162:6835","nonce":3097254664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":4826,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/3489476943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3489476943},{"type":"v1","addr":"172.21.15.162:6837","nonce":3489476943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5274,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1283185308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":1283185308},{"type":"v1","addr":"172.21.15.46:6835","nonce":1283185308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-02T00:29:57.279169+0000","modified":"2023-12-02T00:30:09.302069+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5277},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5277":{"gid":5277,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":5,"addr":"172.21.15.46:6834/2036053042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2036053042},{"type":"v1","addr":"172.21.15.46:6834","nonce":2036053042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-02T00:30:10.306 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 14 2023-12-02T00:30:10.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.306+0000 7fddb8525640 1 -- 172.21.15.46:0/767223239 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fdd9406ae70 msgr2=0x7fdd9406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:10.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.306+0000 7fddb8525640 1 --2- 172.21.15.46:0/767223239 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fdd9406ae70 0x7fdd9406d330 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdda0036000 tx=0x7fdda003c040 comp rx=0 tx=0).stop 2023-12-02T00:30:10.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.306+0000 7fddb8525640 1 -- 172.21.15.46:0/767223239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddb0144980 msgr2=0x7fddb00b12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:10.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.306+0000 7fddb8525640 1 --2- 172.21.15.46:0/767223239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddb0144980 0x7fddb00b12f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fdda800b970 tx=0x7fdda800be30 comp rx=0 tx=0).stop 2023-12-02T00:30:10.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.306+0000 7fddb8525640 1 -- 172.21.15.46:0/767223239 shutdown_connections 2023-12-02T00:30:10.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.306+0000 7fddb8525640 1 --2- 172.21.15.46:0/767223239 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fdd9406ae70 0x7fdd9406d330 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:10.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.306+0000 7fddb8525640 1 --2- 172.21.15.46:0/767223239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddb0144980 0x7fddb00b12f0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:10.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.306+0000 7fddb8525640 1 --2- 172.21.15.46:0/767223239 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fddb00ceb70 0x7fddb01f1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:10.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.306+0000 7fddb8525640 1 --2- 172.21.15.46:0/767223239 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fddb00b4370 0x7fddb01ede60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:10.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.306+0000 7fddb8525640 1 -- 172.21.15.46:0/767223239 >> 172.21.15.46:0/767223239 conn(0x7fddb006a4d0 msgr2=0x7fddb00cbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:10.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.306+0000 7fddb8525640 1 -- 172.21.15.46:0/767223239 shutdown_connections 2023-12-02T00:30:10.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.306+0000 7fddb8525640 1 -- 172.21.15.46:0/767223239 wait complete. 2023-12-02T00:30:10.318 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:30:10.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.425+0000 7f7c08d7c640 1 Processor -- start 2023-12-02T00:30:10.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.425+0000 7f7c08d7c640 1 -- start start 2023-12-02T00:30:10.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.426+0000 7f7c08d7c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c040d2910 0x7f7c040d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:10.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.426+0000 7f7c08d7c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7c040d3730 0x7f7c040b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:10.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.426+0000 7f7c08d7c640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c040b2b10 con 0x7f7c0415d330 2023-12-02T00:30:10.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.426+0000 7f7c08d7c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c040d3b50 con 0x7f7c040d3730 2023-12-02T00:30:10.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.426+0000 7f7c08d7c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c040d1590 con 0x7f7c040d2910 2023-12-02T00:30:10.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.427+0000 7f7c02d9d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7c040d3730 0x7f7c040b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:10.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.427+0000 7f7c01d9b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c040d2910 0x7f7c040d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:10.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.427+0000 7f7c0259c640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f7c0415d330 0x7f7c040b8b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.46:46440/0 (socket says 172.21.15.46:46440) 2023-12-02T00:30:10.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.427+0000 7f7c0259c640 1 -- 172.21.15.46:0/1774406658 learned_addr learned my addr 172.21.15.46:0/1774406658 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:30:10.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.427+0000 7f7c0159a640 1 -- 172.21.15.46:0/1774406658 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3757499260 0 0) 0x7f7c040b2b10 con 0x7f7c0415d330 2023-12-02T00:30:10.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.427+0000 7f7c0159a640 1 -- 172.21.15.46:0/1774406658 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bf0003520 con 0x7f7c0415d330 2023-12-02T00:30:10.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.427+0000 7f7c02d9d640 1 -- 172.21.15.46:0/1774406658 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c040d2910 msgr2=0x7f7c040d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:10.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.427+0000 7f7c02d9d640 1 --2- 172.21.15.46:0/1774406658 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c040d2910 0x7f7c040d31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:10.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.427+0000 7f7c02d9d640 1 -- 172.21.15.46:0/1774406658 >> v1:172.21.15.162:6789/0 conn(0x7f7c0415d330 legacy=0x7f7c040b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:10.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.427+0000 7f7c02d9d640 1 -- 172.21.15.46:0/1774406658 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c040d1870 con 0x7f7c040d3730 2023-12-02T00:30:10.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.427+0000 7f7c01d9b640 1 --2- 172.21.15.46:0/1774406658 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c040d2910 0x7f7c040d31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:30:10.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.428+0000 7f7c02d9d640 1 --2- 172.21.15.46:0/1774406658 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c040d3730 0x7f7c040b6390 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7bf40097a0 tx=0x7f7bf402b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ba45e20f9b07352 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:10.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.429+0000 7f7c0159a640 1 -- 172.21.15.46:0/1774406658 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf402e070 con 0x7f7c040d3730 2023-12-02T00:30:10.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.429+0000 7f7c0159a640 1 -- 172.21.15.46:0/1774406658 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf40315a0 con 0x7f7c040d3730 2023-12-02T00:30:10.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.429+0000 7f7c0159a640 1 -- 172.21.15.46:0/1774406658 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf4031930 con 0x7f7c040d3730 2023-12-02T00:30:10.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.429+0000 7f7c08d7c640 1 -- 172.21.15.46:0/1774406658 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c040d3730 msgr2=0x7f7c040b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:10.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.429+0000 7f7c08d7c640 1 --2- 172.21.15.46:0/1774406658 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c040d3730 0x7f7c040b6390 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7bf40097a0 tx=0x7f7bf402b840 comp rx=0 tx=0).stop 2023-12-02T00:30:10.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.429+0000 7f7c08d7c640 1 -- 172.21.15.46:0/1774406658 shutdown_connections 2023-12-02T00:30:10.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.429+0000 7f7c08d7c640 1 --2- 172.21.15.46:0/1774406658 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c040d2910 0x7f7c040d31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:10.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.429+0000 7f7c08d7c640 1 --2- 172.21.15.46:0/1774406658 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c040d3730 0x7f7c040b6390 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:10.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.429+0000 7f7c08d7c640 1 -- 172.21.15.46:0/1774406658 >> 172.21.15.46:0/1774406658 conn(0x7f7c04072020 msgr2=0x7f7c04072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:10.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.429+0000 7f7c08d7c640 1 -- 172.21.15.46:0/1774406658 shutdown_connections 2023-12-02T00:30:10.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.429+0000 7f7c08d7c640 1 -- 172.21.15.46:0/1774406658 wait complete. 2023-12-02T00:30:10.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.431+0000 7f7c08d7c640 1 Processor -- start 2023-12-02T00:30:10.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.431+0000 7f7c08d7c640 1 -- start start 2023-12-02T00:30:10.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.431+0000 7f7c08d7c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c040d2910 0x7f7c0415d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:10.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.431+0000 7f7c08d7c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c040d3730 0x7f7c041f61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:10.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.431+0000 7f7c08d7c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c0415d330 0x7f7c041f66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:10.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.431+0000 7f7c08d7c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c041f5fd0 con 0x7f7c040d3730 2023-12-02T00:30:10.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.431+0000 7f7c08d7c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c040cb630 con 0x7f7c040d2910 2023-12-02T00:30:10.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.431+0000 7f7c08d7c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c04160370 con 0x7f7c0415d330 2023-12-02T00:30:10.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.431+0000 7f7c01d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c040d3730 0x7f7c041f61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:10.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.432+0000 7f7c01d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c040d3730 0x7f7c041f61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46278/0 (socket says 172.21.15.46:46278) 2023-12-02T00:30:10.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.432+0000 7f7c01d9b640 1 -- 172.21.15.46:0/1652871848 learned_addr learned my addr 172.21.15.46:0/1652871848 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:10.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.432+0000 7f7c0259c640 1 --2- 172.21.15.46:0/1652871848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c040d2910 0x7f7c0415d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:10.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.432+0000 7f7c02d9d640 1 --2- 172.21.15.46:0/1652871848 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c0415d330 0x7f7c041f66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:10.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.432+0000 7f7c01d9b640 1 -- 172.21.15.46:0/1652871848 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c0415d330 msgr2=0x7f7c041f66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:10.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.432+0000 7f7c01d9b640 1 --2- 172.21.15.46:0/1652871848 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c0415d330 0x7f7c041f66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:10.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.432+0000 7f7c01d9b640 1 -- 172.21.15.46:0/1652871848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c040d2910 msgr2=0x7f7c0415d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:10.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.432+0000 7f7c01d9b640 1 --2- 172.21.15.46:0/1652871848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c040d2910 0x7f7c0415d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:10.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.432+0000 7f7c01d9b640 1 -- 172.21.15.46:0/1652871848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bec009400 con 0x7f7c040d3730 2023-12-02T00:30:10.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.432+0000 7f7c01d9b640 1 --2- 172.21.15.46:0/1652871848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c040d3730 0x7f7c041f61a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f7bec00e960 tx=0x7f7bec00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:10.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.432+0000 7f7bfaffd640 1 -- 172.21.15.46:0/1652871848 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bec00ca00 con 0x7f7c040d3730 2023-12-02T00:30:10.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.432+0000 7f7c08d7c640 1 -- 172.21.15.46:0/1652871848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bf4009400 con 0x7f7c040d3730 2023-12-02T00:30:10.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.432+0000 7f7bfaffd640 1 -- 172.21.15.46:0/1652871848 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bec00cb60 con 0x7f7c040d3730 2023-12-02T00:30:10.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.432+0000 7f7bfaffd640 1 -- 172.21.15.46:0/1652871848 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bec013470 con 0x7f7c040d3730 2023-12-02T00:30:10.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.432+0000 7f7c08d7c640 1 -- 172.21.15.46:0/1652871848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c040cb8f0 con 0x7f7c040d3730 2023-12-02T00:30:10.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.433+0000 7f7c08d7c640 1 -- 172.21.15.46:0/1652871848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c04069400 con 0x7f7c040d3730 2023-12-02T00:30:10.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.434+0000 7f7bfaffd640 1 -- 172.21.15.46:0/1652871848 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7bec0135d0 con 0x7f7c040d3730 2023-12-02T00:30:10.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.434+0000 7f7bfaffd640 1 --2- 172.21.15.46:0/1652871848 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7be406ae40 0x7f7be406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:10.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.434+0000 7f7bfaffd640 1 -- 172.21.15.46:0/1652871848 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f7bec089970 con 0x7f7c040d3730 2023-12-02T00:30:10.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.435+0000 7f7c0259c640 1 --2- 172.21.15.46:0/1652871848 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7be406ae40 0x7f7be406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:10.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.435+0000 7f7c0259c640 1 --2- 172.21.15.46:0/1652871848 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7be406ae40 0x7f7be406d300 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc001560 tx=0x7f7bfc001a90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:10.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.439+0000 7f7bfaffd640 1 -- 172.21.15.46:0/1652871848 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7bec054950 con 0x7f7c040d3730 2023-12-02T00:30:10.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.616+0000 7f7c08d7c640 1 -- 172.21.15.46:0/1652871848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7c040b6390 con 0x7f7c040d3730 2023-12-02T00:30:10.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.618+0000 7f7bfaffd640 1 -- 172.21.15.46:0/1652871848 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18337 (secure 0 0 0) 0x7f7bec057f70 con 0x7f7c040d3730 2023-12-02T00:30:10.619 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:30:10.619 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":37,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:30:09.301846+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-02T00:29:55.676572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-02T00:29:56.244869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"data_new","create_time":"2023-12-02T00:30:08.897990+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:30:10.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.620+0000 7f7c08d7c640 1 -- 172.21.15.46:0/1652871848 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7be406ae40 msgr2=0x7f7be406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:10.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.620+0000 7f7c08d7c640 1 --2- 172.21.15.46:0/1652871848 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7be406ae40 0x7f7be406d300 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc001560 tx=0x7f7bfc001a90 comp rx=0 tx=0).stop 2023-12-02T00:30:10.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.620+0000 7f7c08d7c640 1 -- 172.21.15.46:0/1652871848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c040d3730 msgr2=0x7f7c041f61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:10.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.620+0000 7f7c08d7c640 1 --2- 172.21.15.46:0/1652871848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c040d3730 0x7f7c041f61a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f7bec00e960 tx=0x7f7bec00ee20 comp rx=0 tx=0).stop 2023-12-02T00:30:10.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.620+0000 7f7c08d7c640 1 -- 172.21.15.46:0/1652871848 shutdown_connections 2023-12-02T00:30:10.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.620+0000 7f7c08d7c640 1 --2- 172.21.15.46:0/1652871848 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7be406ae40 0x7f7be406d300 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:10.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.620+0000 7f7c08d7c640 1 --2- 172.21.15.46:0/1652871848 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c0415d330 0x7f7c041f66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:10.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.620+0000 7f7c08d7c640 1 --2- 172.21.15.46:0/1652871848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c040d3730 0x7f7c041f61a0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:10.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.620+0000 7f7c08d7c640 1 --2- 172.21.15.46:0/1652871848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c040d2910 0x7f7c0415d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:10.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.620+0000 7f7c08d7c640 1 -- 172.21.15.46:0/1652871848 >> 172.21.15.46:0/1652871848 conn(0x7f7c04072020 msgr2=0x7f7c040cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:10.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.620+0000 7f7c08d7c640 1 -- 172.21.15.46:0/1652871848 shutdown_connections 2023-12-02T00:30:10.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.620+0000 7f7c08d7c640 1 -- 172.21.15.46:0/1652871848 wait complete. 2023-12-02T00:30:10.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:30:10.632 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch layoutfile) 2023-12-02T00:30:10.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:30:10.652 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir layoutdir) 2023-12-02T00:30:10.709 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds 'allow rw' mon 'allow r' osd 'allow rw pool=cephfs_data,allow rw pool=data_new' 2023-12-02T00:30:10.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.862+0000 7fef6b5a5640 1 Processor -- start 2023-12-02T00:30:10.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.862+0000 7fef6b5a5640 1 -- start start 2023-12-02T00:30:10.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.863+0000 7fef6b5a5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef640d3850 0x7fef640b2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:10.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.863+0000 7fef6b5a5640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef640d00f0 con 0x7fef6415d420 2023-12-02T00:30:10.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.863+0000 7fef6b5a5640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef640d03d0 con 0x7fef640b43a0 2023-12-02T00:30:10.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.863+0000 7fef6b5a5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef640d1420 con 0x7fef640d3850 2023-12-02T00:30:10.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.863+0000 7fef69341640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fef6415d420 0x7fef640d3140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36074/0 (socket says 172.21.15.46:36074) 2023-12-02T00:30:10.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.863+0000 7fef69341640 1 -- 172.21.15.46:0/3279448317 learned_addr learned my addr 172.21.15.46:0/3279448317 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:30:10.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.863+0000 7fef57fff640 1 -- 172.21.15.46:0/3279448317 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2219904291 0 0) 0x7fef640d00f0 con 0x7fef6415d420 2023-12-02T00:30:10.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.863+0000 7fef68b40640 1 --2- 172.21.15.46:0/3279448317 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef640d3850 0x7fef640b2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:10.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.863+0000 7fef57fff640 1 -- 172.21.15.46:0/3279448317 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef50003520 con 0x7fef6415d420 2023-12-02T00:30:10.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.864+0000 7fef57fff640 1 -- 172.21.15.46:0/3279448317 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1281925378 0 0) 0x7fef50003520 con 0x7fef6415d420 2023-12-02T00:30:10.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.864+0000 7fef57fff640 1 -- 172.21.15.46:0/3279448317 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef640d3850 msgr2=0x7fef640b2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:10.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.864+0000 7fef57fff640 1 --2- 172.21.15.46:0/3279448317 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef640d3850 0x7fef640b2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:10.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.864+0000 7fef57fff640 1 -- 172.21.15.46:0/3279448317 >> v1:172.21.15.162:6789/0 conn(0x7fef640b43a0 legacy=0x7fef640ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:10.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.864+0000 7fef57fff640 1 -- 172.21.15.46:0/3279448317 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef640d06b0 con 0x7fef6415d420 2023-12-02T00:30:10.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.864+0000 7fef57fff640 1 -- 172.21.15.46:0/3279448317 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7fef60001670 con 0x7fef6415d420 2023-12-02T00:30:10.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.864+0000 7fef57fff640 1 -- 172.21.15.46:0/3279448317 >> v1:172.21.15.46:6789/0 conn(0x7fef6415d420 legacy=0x7fef640d3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:10.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.864+0000 7fef57fff640 1 --2- 172.21.15.46:0/3279448317 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef500039a0 0x7fef50003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:10.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.864+0000 7fef57fff640 1 --2- 172.21.15.46:0/3279448317 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef640d3850 0x7fef500047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:10.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.864+0000 7fef57fff640 1 --2- 172.21.15.46:0/3279448317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef50004d00 0x7fef50005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:10.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.864+0000 7fef57fff640 1 -- 172.21.15.46:0/3279448317 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef640d1420 con 0x7fef50004d00 2023-12-02T00:30:10.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.864+0000 7fef57fff640 1 -- 172.21.15.46:0/3279448317 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef5000ba60 con 0x7fef500039a0 2023-12-02T00:30:10.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.864+0000 7fef57fff640 1 -- 172.21.15.46:0/3279448317 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef5000bc20 con 0x7fef640d3850 2023-12-02T00:30:10.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.864+0000 7fef69b42640 1 --2- 172.21.15.46:0/3279448317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef50004d00 0x7fef50005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:10.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.864+0000 7fef69341640 1 --2- 172.21.15.46:0/3279448317 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef500039a0 0x7fef50003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:10.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.864+0000 7fef68b40640 1 --2- 172.21.15.46:0/3279448317 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef640d3850 0x7fef500047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:10.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.864+0000 7fef69b42640 1 -- 172.21.15.46:0/3279448317 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef640d3850 msgr2=0x7fef500047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:10.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.864+0000 7fef69b42640 1 --2- 172.21.15.46:0/3279448317 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef640d3850 0x7fef500047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:10.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.864+0000 7fef69b42640 1 -- 172.21.15.46:0/3279448317 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef500039a0 msgr2=0x7fef50003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:10.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.864+0000 7fef69b42640 1 --2- 172.21.15.46:0/3279448317 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef500039a0 0x7fef50003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:10.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.864+0000 7fef69b42640 1 -- 172.21.15.46:0/3279448317 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef50003520 con 0x7fef50004d00 2023-12-02T00:30:10.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.865+0000 7fef69b42640 1 --2- 172.21.15.46:0/3279448317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef50004d00 0x7fef50005180 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fef640b4850 tx=0x7fef500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9da70cea8de86a30 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:10.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.865+0000 7fef57fff640 1 -- 172.21.15.46:0/3279448317 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5c003b70 con 0x7fef50004d00 2023-12-02T00:30:10.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.866+0000 7fef57fff640 1 -- 172.21.15.46:0/3279448317 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef5c003cd0 con 0x7fef50004d00 2023-12-02T00:30:10.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.866+0000 7fef57fff640 1 -- 172.21.15.46:0/3279448317 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5c005310 con 0x7fef50004d00 2023-12-02T00:30:10.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.866+0000 7fef6b5a5640 1 -- 172.21.15.46:0/3279448317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef50004d00 msgr2=0x7fef50005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:10.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.866+0000 7fef6b5a5640 1 --2- 172.21.15.46:0/3279448317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef50004d00 0x7fef50005180 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fef640b4850 tx=0x7fef500057f0 comp rx=0 tx=0).stop 2023-12-02T00:30:10.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.866+0000 7fef69b42640 1 -- 172.21.15.46:0/3279448317 reap_dead start 2023-12-02T00:30:10.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.866+0000 7fef6b5a5640 1 -- 172.21.15.46:0/3279448317 shutdown_connections 2023-12-02T00:30:10.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.866+0000 7fef6b5a5640 1 -- 172.21.15.46:0/3279448317 >> 172.21.15.46:0/3279448317 conn(0x7fef6408b000 msgr2=0x7fef6409c580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:10.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.866+0000 7fef6b5a5640 1 -- 172.21.15.46:0/3279448317 shutdown_connections 2023-12-02T00:30:10.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.866+0000 7fef6b5a5640 1 -- 172.21.15.46:0/3279448317 wait complete. 2023-12-02T00:30:10.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.867+0000 7fef6b5a5640 1 Processor -- start 2023-12-02T00:30:10.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.867+0000 7fef6b5a5640 1 -- start start 2023-12-02T00:30:10.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.868+0000 7fef6b5a5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef64150000 0x7fef64157560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:10.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.868+0000 7fef6b5a5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef641503e0 0x7fef64157aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:10.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.868+0000 7fef6b5a5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef640c8c20 0x7fef640c90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:10.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.868+0000 7fef6b5a5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef640c95f0 con 0x7fef640c8c20 2023-12-02T00:30:10.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.868+0000 7fef6b5a5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef641509b0 con 0x7fef641503e0 2023-12-02T00:30:10.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.868+0000 7fef6b5a5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef64150c90 con 0x7fef64150000 2023-12-02T00:30:10.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.868+0000 7fef69b42640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef640c8c20 0x7fef640c90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:10.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.868+0000 7fef69b42640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef640c8c20 0x7fef640c90b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46294/0 (socket says 172.21.15.46:46294) 2023-12-02T00:30:10.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.868+0000 7fef69b42640 1 -- 172.21.15.46:0/2360211901 learned_addr learned my addr 172.21.15.46:0/2360211901 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:10.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.868+0000 7fef69341640 1 --2- 172.21.15.46:0/2360211901 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef64150000 0x7fef64157560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:10.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.868+0000 7fef68b40640 1 --2- 172.21.15.46:0/2360211901 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef641503e0 0x7fef64157aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:10.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.869+0000 7fef69b42640 1 -- 172.21.15.46:0/2360211901 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef64150000 msgr2=0x7fef64157560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:10.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.869+0000 7fef69b42640 1 --2- 172.21.15.46:0/2360211901 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef64150000 0x7fef64157560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:10.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.869+0000 7fef69b42640 1 -- 172.21.15.46:0/2360211901 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef641503e0 msgr2=0x7fef64157aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:10.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.869+0000 7fef69b42640 1 --2- 172.21.15.46:0/2360211901 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef641503e0 0x7fef64157aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:10.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.869+0000 7fef69b42640 1 -- 172.21.15.46:0/2360211901 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef64150f70 con 0x7fef640c8c20 2023-12-02T00:30:10.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.869+0000 7fef69b42640 1 --2- 172.21.15.46:0/2360211901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef640c8c20 0x7fef640c90b0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fef5c02c020 tx=0x7fef5c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:10.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.869+0000 7fef55ffb640 1 -- 172.21.15.46:0/2360211901 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5c006d50 con 0x7fef640c8c20 2023-12-02T00:30:10.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.869+0000 7fef55ffb640 1 -- 172.21.15.46:0/2360211901 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef5c007db0 con 0x7fef640c8c20 2023-12-02T00:30:10.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.869+0000 7fef6b5a5640 1 -- 172.21.15.46:0/2360211901 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef64151250 con 0x7fef640c8c20 2023-12-02T00:30:10.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.869+0000 7fef55ffb640 1 -- 172.21.15.46:0/2360211901 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5c007230 con 0x7fef640c8c20 2023-12-02T00:30:10.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.869+0000 7fef6b5a5640 1 -- 172.21.15.46:0/2360211901 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef641f6a20 con 0x7fef640c8c20 2023-12-02T00:30:10.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.871+0000 7fef6b5a5640 1 -- 172.21.15.46:0/2360211901 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef640c95f0 con 0x7fef640c8c20 2023-12-02T00:30:10.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.871+0000 7fef55ffb640 1 -- 172.21.15.46:0/2360211901 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fef5c007390 con 0x7fef640c8c20 2023-12-02T00:30:10.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.871+0000 7fef55ffb640 1 --2- 172.21.15.46:0/2360211901 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fef4806ae40 0x7fef4806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:10.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.871+0000 7fef55ffb640 1 -- 172.21.15.46:0/2360211901 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fef5c0ab010 con 0x7fef640c8c20 2023-12-02T00:30:10.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.872+0000 7fef69341640 1 --2- 172.21.15.46:0/2360211901 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fef4806ae40 0x7fef4806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:10.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.872+0000 7fef69341640 1 --2- 172.21.15.46:0/2360211901 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fef4806ae40 0x7fef4806d300 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fef641571c0 tx=0x7fef60000bc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:10.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:10.876+0000 7fef55ffb640 1 -- 172.21.15.46:0/2360211901 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fef5c076120 con 0x7fef640c8c20 2023-12-02T00:30:11.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.120+0000 7fef6b5a5640 1 -- 172.21.15.46:0/2360211901 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rw", "mon", "allow r", "osd", "allow rw pool=cephfs_data,allow rw pool=data_new"]} v 0) v1 -- 0x7fef64151770 con 0x7fef640c8c20 2023-12-02T00:30:11.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.122+0000 7fef55ffb640 1 -- 172.21.15.46:0/2360211901 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rw", "mon", "allow r", "osd", "allow rw pool=cephfs_data,allow rw pool=data_new"]}]=0 updated caps for client.0 v14) v1 ==== 210+0+167 (secure 0 0 0) 0x7fef64151770 con 0x7fef640c8c20 2023-12-02T00:30:11.123 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2023-12-02T00:30:11.123 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAJempl5Lx1DhAAjpK6ajoQ5jQiUr/EMpFeLg== 2023-12-02T00:30:11.123 INFO:teuthology.orchestra.run.smithi046.stdout: caps mds = "allow rw" 2023-12-02T00:30:11.123 INFO:teuthology.orchestra.run.smithi046.stdout: caps mon = "allow r" 2023-12-02T00:30:11.123 INFO:teuthology.orchestra.run.smithi046.stdout: caps osd = "allow rw pool=cephfs_data,allow rw pool=data_new" 2023-12-02T00:30:11.123 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2023-12-02T00:30:11.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.124+0000 7fef6b5a5640 1 -- 172.21.15.46:0/2360211901 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fef4806ae40 msgr2=0x7fef4806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:11.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.124+0000 7fef6b5a5640 1 --2- 172.21.15.46:0/2360211901 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fef4806ae40 0x7fef4806d300 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fef641571c0 tx=0x7fef60000bc0 comp rx=0 tx=0).stop 2023-12-02T00:30:11.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.124+0000 7fef6b5a5640 1 -- 172.21.15.46:0/2360211901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef640c8c20 msgr2=0x7fef640c90b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:11.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.124+0000 7fef6b5a5640 1 --2- 172.21.15.46:0/2360211901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef640c8c20 0x7fef640c90b0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fef5c02c020 tx=0x7fef5c0014a0 comp rx=0 tx=0).stop 2023-12-02T00:30:11.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.124+0000 7fef6b5a5640 1 -- 172.21.15.46:0/2360211901 shutdown_connections 2023-12-02T00:30:11.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.124+0000 7fef6b5a5640 1 --2- 172.21.15.46:0/2360211901 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fef4806ae40 0x7fef4806d300 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:11.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.124+0000 7fef6b5a5640 1 --2- 172.21.15.46:0/2360211901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef640c8c20 0x7fef640c90b0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:11.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.124+0000 7fef6b5a5640 1 --2- 172.21.15.46:0/2360211901 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef641503e0 0x7fef64157aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:11.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.124+0000 7fef6b5a5640 1 --2- 172.21.15.46:0/2360211901 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef64150000 0x7fef64157560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:11.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.124+0000 7fef6b5a5640 1 -- 172.21.15.46:0/2360211901 >> 172.21.15.46:0/2360211901 conn(0x7fef6408b000 msgr2=0x7fef640b0550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:11.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.124+0000 7fef6b5a5640 1 -- 172.21.15.46:0/2360211901 shutdown_connections 2023-12-02T00:30:11.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.124+0000 7fef6b5a5640 1 -- 172.21.15.46:0/2360211901 wait complete. 2023-12-02T00:30:11.135 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-02T00:30:11.135 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-02T00:30:11.151 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:30:11.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.298+0000 7f7a4d700640 1 Processor -- start 2023-12-02T00:30:11.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.298+0000 7f7a4d700640 1 -- start start 2023-12-02T00:30:11.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.299+0000 7f7a4d700640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7a4815b130 0x7f7a480d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:11.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.299+0000 7f7a4d700640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7a480cf320 0x7f7a480cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:11.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.299+0000 7f7a4d700640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a480b3f10 con 0x7f7a4815f760 2023-12-02T00:30:11.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.299+0000 7f7a4d700640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a4815fd20 con 0x7f7a4815b130 2023-12-02T00:30:11.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.299+0000 7f7a4d700640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a480d0730 con 0x7f7a480cf320 2023-12-02T00:30:11.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.299+0000 7f7a46ffd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7a4815b130 0x7f7a480d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:11.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.299+0000 7f7a467fc640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7a4815f760 0x7f7a480cec10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36082/0 (socket says 172.21.15.46:36082) 2023-12-02T00:30:11.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.299+0000 7f7a467fc640 1 -- 172.21.15.46:0/1584426163 learned_addr learned my addr 172.21.15.46:0/1584426163 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:30:11.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.299+0000 7f7a477fe640 1 --2- 172.21.15.46:0/1584426163 >> v2:172.21.15.162:3300/0 conn(0x7f7a480cf320 0x7f7a480cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:11.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.300+0000 7f7a46ffd640 1 -- 172.21.15.46:0/1584426163 >> v2:172.21.15.162:3300/0 conn(0x7f7a480cf320 msgr2=0x7f7a480cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:11.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.300+0000 7f7a46ffd640 1 --2- 172.21.15.46:0/1584426163 >> v2:172.21.15.162:3300/0 conn(0x7f7a480cf320 0x7f7a480cd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:11.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.300+0000 7f7a46ffd640 1 -- 172.21.15.46:0/1584426163 >> v1:172.21.15.46:6789/0 conn(0x7f7a4815f760 legacy=0x7f7a480cec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:11.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.300+0000 7f7a45ffb640 1 -- 172.21.15.46:0/1584426163 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 191878503 0 0) 0x7f7a480b3f10 con 0x7f7a4815f760 2023-12-02T00:30:11.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.300+0000 7f7a46ffd640 1 -- 172.21.15.46:0/1584426163 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a480d0a50 con 0x7f7a4815b130 2023-12-02T00:30:11.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.300+0000 7f7a46ffd640 1 --2- 172.21.15.46:0/1584426163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a4815b130 0x7f7a480d01f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7a480cb620 tx=0x7f7a3802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52feb3a0538f3bd server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:11.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.301+0000 7f7a45ffb640 1 -- 172.21.15.46:0/1584426163 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3802cbd0 con 0x7f7a4815b130 2023-12-02T00:30:11.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.301+0000 7f7a45ffb640 1 -- 172.21.15.46:0/1584426163 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a3802cd30 con 0x7f7a4815b130 2023-12-02T00:30:11.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.301+0000 7f7a45ffb640 1 -- 172.21.15.46:0/1584426163 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3802cea0 con 0x7f7a4815b130 2023-12-02T00:30:11.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.301+0000 7f7a4d700640 1 -- 172.21.15.46:0/1584426163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a4815b130 msgr2=0x7f7a480d01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:11.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.301+0000 7f7a4d700640 1 --2- 172.21.15.46:0/1584426163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a4815b130 0x7f7a480d01f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7a480cb620 tx=0x7f7a3802be20 comp rx=0 tx=0).stop 2023-12-02T00:30:11.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.301+0000 7f7a4d700640 1 -- 172.21.15.46:0/1584426163 shutdown_connections 2023-12-02T00:30:11.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.301+0000 7f7a4d700640 1 --2- 172.21.15.46:0/1584426163 >> v2:172.21.15.162:3300/0 conn(0x7f7a480cf320 0x7f7a480cd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:11.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.301+0000 7f7a4d700640 1 --2- 172.21.15.46:0/1584426163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a4815b130 0x7f7a480d01f0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:11.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.301+0000 7f7a4d700640 1 -- 172.21.15.46:0/1584426163 >> 172.21.15.46:0/1584426163 conn(0x7f7a480b2c10 msgr2=0x7f7a480b2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:11.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.301+0000 7f7a4d700640 1 -- 172.21.15.46:0/1584426163 shutdown_connections 2023-12-02T00:30:11.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.301+0000 7f7a4d700640 1 -- 172.21.15.46:0/1584426163 wait complete. 2023-12-02T00:30:11.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.303+0000 7f7a4d700640 1 Processor -- start 2023-12-02T00:30:11.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.303+0000 7f7a4d700640 1 -- start start 2023-12-02T00:30:11.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.303+0000 7f7a4d700640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a480cf320 0x7f7a480afec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:11.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.303+0000 7f7a4d700640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7a4815b130 0x7f7a4815be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:11.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.303+0000 7f7a4d700640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a4815f760 0x7f7a481f5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:11.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.303+0000 7f7a4d700640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4806b5e0 con 0x7f7a480cf320 2023-12-02T00:30:11.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.303+0000 7f7a4d700640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a481f68e0 con 0x7f7a4815f760 2023-12-02T00:30:11.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.303+0000 7f7a4d700640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a481f6bc0 con 0x7f7a4815b130 2023-12-02T00:30:11.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.303+0000 7f7a46ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a480cf320 0x7f7a480afec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:11.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.303+0000 7f7a46ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a480cf320 0x7f7a480afec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46310/0 (socket says 172.21.15.46:46310) 2023-12-02T00:30:11.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.303+0000 7f7a46ffd640 1 -- 172.21.15.46:0/1786281223 learned_addr learned my addr 172.21.15.46:0/1786281223 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:11.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.304+0000 7f7a477fe640 1 --2- 172.21.15.46:0/1786281223 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a4815f760 0x7f7a481f5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:11.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.304+0000 7f7a467fc640 1 --2- 172.21.15.46:0/1786281223 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7a4815b130 0x7f7a4815be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:11.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.304+0000 7f7a46ffd640 1 -- 172.21.15.46:0/1786281223 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7a4815b130 msgr2=0x7f7a4815be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:11.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.304+0000 7f7a46ffd640 1 --2- 172.21.15.46:0/1786281223 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7a4815b130 0x7f7a4815be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:11.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.304+0000 7f7a46ffd640 1 -- 172.21.15.46:0/1786281223 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a4815f760 msgr2=0x7f7a481f5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:11.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.304+0000 7f7a46ffd640 1 --2- 172.21.15.46:0/1786281223 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a4815f760 0x7f7a481f5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:11.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.304+0000 7f7a46ffd640 1 -- 172.21.15.46:0/1786281223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a38009a70 con 0x7f7a480cf320 2023-12-02T00:30:11.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.304+0000 7f7a477fe640 1 --2- 172.21.15.46:0/1786281223 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a4815f760 0x7f7a481f5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:30:11.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.304+0000 7f7a467fc640 1 --2- 172.21.15.46:0/1786281223 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7a4815b130 0x7f7a4815be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:30:11.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.304+0000 7f7a46ffd640 1 --2- 172.21.15.46:0/1786281223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a480cf320 0x7f7a480afec0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7a38009d90 tx=0x7f7a38004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:11.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.304+0000 7f7a3f7fe640 1 -- 172.21.15.46:0/1786281223 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3802cea0 con 0x7f7a480cf320 2023-12-02T00:30:11.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.304+0000 7f7a3f7fe640 1 -- 172.21.15.46:0/1786281223 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a3803e630 con 0x7f7a480cf320 2023-12-02T00:30:11.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.304+0000 7f7a4d700640 1 -- 172.21.15.46:0/1786281223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a4806b7f0 con 0x7f7a480cf320 2023-12-02T00:30:11.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.304+0000 7f7a3f7fe640 1 -- 172.21.15.46:0/1786281223 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3803e9e0 con 0x7f7a480cf320 2023-12-02T00:30:11.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.304+0000 7f7a4d700640 1 -- 172.21.15.46:0/1786281223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a4815c360 con 0x7f7a480cf320 2023-12-02T00:30:11.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.305+0000 7f7a4d700640 1 -- 172.21.15.46:0/1786281223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a48067fc0 con 0x7f7a480cf320 2023-12-02T00:30:11.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.306+0000 7f7a3f7fe640 1 -- 172.21.15.46:0/1786281223 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7a3803b050 con 0x7f7a480cf320 2023-12-02T00:30:11.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.306+0000 7f7a3f7fe640 1 --2- 172.21.15.46:0/1786281223 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7a2806ae40 0x7f7a2806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:11.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.306+0000 7f7a3f7fe640 1 -- 172.21.15.46:0/1786281223 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f7a380ab990 con 0x7f7a480cf320 2023-12-02T00:30:11.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.306+0000 7f7a467fc640 1 --2- 172.21.15.46:0/1786281223 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7a2806ae40 0x7f7a2806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:11.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.307+0000 7f7a467fc640 1 --2- 172.21.15.46:0/1786281223 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7a2806ae40 0x7f7a2806d300 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7a480b36d0 tx=0x7f7a30007ca0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:11.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.311+0000 7f7a3f7fe640 1 -- 172.21.15.46:0/1786281223 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7a380768b0 con 0x7f7a480cf320 2023-12-02T00:30:11.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.489+0000 7f7a4d700640 1 -- 172.21.15.46:0/1786281223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7a48067420 con 0x7f7a480cf320 2023-12-02T00:30:11.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.491+0000 7f7a3f7fe640 1 -- 172.21.15.46:0/1786281223 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+18324 (secure 0 0 0) 0x7f7a38039090 con 0x7f7a480cf320 2023-12-02T00:30:11.492 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:30:11.492 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":38,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:30:10.714057+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-02T00:29:55.676572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-02T00:29:56.244869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"data_new","create_time":"2023-12-02T00:30:08.897990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:30:11.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.493+0000 7f7a4d700640 1 -- 172.21.15.46:0/1786281223 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7a2806ae40 msgr2=0x7f7a2806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:11.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.493+0000 7f7a4d700640 1 --2- 172.21.15.46:0/1786281223 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7a2806ae40 0x7f7a2806d300 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7a480b36d0 tx=0x7f7a30007ca0 comp rx=0 tx=0).stop 2023-12-02T00:30:11.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.493+0000 7f7a4d700640 1 -- 172.21.15.46:0/1786281223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a480cf320 msgr2=0x7f7a480afec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:11.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.493+0000 7f7a4d700640 1 --2- 172.21.15.46:0/1786281223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a480cf320 0x7f7a480afec0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7a38009d90 tx=0x7f7a38004a40 comp rx=0 tx=0).stop 2023-12-02T00:30:11.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.493+0000 7f7a4d700640 1 -- 172.21.15.46:0/1786281223 shutdown_connections 2023-12-02T00:30:11.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.493+0000 7f7a4d700640 1 --2- 172.21.15.46:0/1786281223 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7a2806ae40 0x7f7a2806d300 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:11.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.493+0000 7f7a4d700640 1 --2- 172.21.15.46:0/1786281223 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a4815f760 0x7f7a481f5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:11.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.493+0000 7f7a4d700640 1 --2- 172.21.15.46:0/1786281223 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7a4815b130 0x7f7a4815be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:11.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.493+0000 7f7a4d700640 1 --2- 172.21.15.46:0/1786281223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a480cf320 0x7f7a480afec0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:11.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.493+0000 7f7a4d700640 1 -- 172.21.15.46:0/1786281223 >> 172.21.15.46:0/1786281223 conn(0x7f7a480b2c10 msgr2=0x7f7a48159a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:11.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.493+0000 7f7a4d700640 1 -- 172.21.15.46:0/1786281223 shutdown_connections 2023-12-02T00:30:11.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.493+0000 7f7a4d700640 1 -- 172.21.15.46:0/1786281223 wait complete. 2023-12-02T00:30:11.507 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-02T00:28:37.213645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-02T00:29:55.676572+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-12-02T00:29:56.244869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'data_new', 'create_time': '2023-12-02T00:30:08.897990+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-02T00:30:11.507 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-02T00:30:11.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.615+0000 7fb83ad05640 1 Processor -- start 2023-12-02T00:30:11.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.615+0000 7fb83ad05640 1 -- start start 2023-12-02T00:30:11.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.616+0000 7fb83ad05640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb834144980 0x7fb83415c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:11.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.616+0000 7fb83ad05640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb834071770 con 0x7fb8340ceb70 2023-12-02T00:30:11.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.616+0000 7fb83ad05640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8340ccb60 con 0x7fb8340b4370 2023-12-02T00:30:11.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.616+0000 7fb83ad05640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8340ccdc0 con 0x7fb834144980 2023-12-02T00:30:11.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.616+0000 7fb838aa1640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb834144980 0x7fb83415c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:11.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.616+0000 7fb833fff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb8340ceb70 0x7fb8340cc450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36092/0 (socket says 172.21.15.46:36092) 2023-12-02T00:30:11.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.616+0000 7fb833fff640 1 -- 172.21.15.46:0/2022929333 learned_addr learned my addr 172.21.15.46:0/2022929333 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:30:11.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.617+0000 7fb8337fe640 1 -- 172.21.15.46:0/2022929333 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 152484300 0 0) 0x7fb834071770 con 0x7fb8340ceb70 2023-12-02T00:30:11.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.617+0000 7fb8337fe640 1 -- 172.21.15.46:0/2022929333 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb820003520 con 0x7fb8340ceb70 2023-12-02T00:30:11.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.617+0000 7fb838aa1640 1 -- 172.21.15.46:0/2022929333 >> v1:172.21.15.162:6789/0 conn(0x7fb8340b4370 legacy=0x7fb834071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:11.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.617+0000 7fb838aa1640 1 -- 172.21.15.46:0/2022929333 >> v1:172.21.15.46:6789/0 conn(0x7fb8340ceb70 legacy=0x7fb8340cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:11.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.617+0000 7fb838aa1640 1 -- 172.21.15.46:0/2022929333 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8340cd0a0 con 0x7fb834144980 2023-12-02T00:30:11.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.617+0000 7fb838aa1640 1 --2- 172.21.15.46:0/2022929333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb834144980 0x7fb83415c7c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb82c009d90 tx=0x7fb82c02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a89e540dc2c9ee8d server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:11.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.618+0000 7fb8337fe640 1 -- 172.21.15.46:0/2022929333 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82c02cd00 con 0x7fb834144980 2023-12-02T00:30:11.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.618+0000 7fb8337fe640 1 -- 172.21.15.46:0/2022929333 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb82c02ce60 con 0x7fb834144980 2023-12-02T00:30:11.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.618+0000 7fb8337fe640 1 -- 172.21.15.46:0/2022929333 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82c034640 con 0x7fb834144980 2023-12-02T00:30:11.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.618+0000 7fb83ad05640 1 -- 172.21.15.46:0/2022929333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb834144980 msgr2=0x7fb83415c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:11.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.618+0000 7fb83ad05640 1 --2- 172.21.15.46:0/2022929333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb834144980 0x7fb83415c7c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb82c009d90 tx=0x7fb82c02bf50 comp rx=0 tx=0).stop 2023-12-02T00:30:11.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.618+0000 7fb83ad05640 1 -- 172.21.15.46:0/2022929333 shutdown_connections 2023-12-02T00:30:11.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.618+0000 7fb83ad05640 1 --2- 172.21.15.46:0/2022929333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb834144980 0x7fb83415c7c0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:11.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.618+0000 7fb83ad05640 1 -- 172.21.15.46:0/2022929333 >> 172.21.15.46:0/2022929333 conn(0x7fb83406a4d0 msgr2=0x7fb83406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:11.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.618+0000 7fb83ad05640 1 -- 172.21.15.46:0/2022929333 shutdown_connections 2023-12-02T00:30:11.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.618+0000 7fb83ad05640 1 -- 172.21.15.46:0/2022929333 wait complete. 2023-12-02T00:30:11.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.619+0000 7fb83ad05640 1 Processor -- start 2023-12-02T00:30:11.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.619+0000 7fb83ad05640 1 -- start start 2023-12-02T00:30:11.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.620+0000 7fb83ad05640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8340b4370 0x7fb8340b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:11.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.620+0000 7fb83ad05640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8340ceb70 0x7fb834154ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:11.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.620+0000 7fb83ad05640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb834144980 0x7fb8340a9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:11.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.620+0000 7fb83ad05640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8340af1e0 con 0x7fb8340b4370 2023-12-02T00:30:11.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.620+0000 7fb83ad05640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8340af4c0 con 0x7fb834144980 2023-12-02T00:30:11.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.620+0000 7fb83ad05640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8340af7a0 con 0x7fb8340ceb70 2023-12-02T00:30:11.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.620+0000 7fb838aa1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8340b4370 0x7fb8340b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:11.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.620+0000 7fb838aa1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8340b4370 0x7fb8340b0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46338/0 (socket says 172.21.15.46:46338) 2023-12-02T00:30:11.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.620+0000 7fb838aa1640 1 -- 172.21.15.46:0/1848498424 learned_addr learned my addr 172.21.15.46:0/1848498424 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:11.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.621+0000 7fb8392a2640 1 --2- 172.21.15.46:0/1848498424 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb834144980 0x7fb8340a9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:11.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.621+0000 7fb833fff640 1 --2- 172.21.15.46:0/1848498424 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8340ceb70 0x7fb834154ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:11.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.621+0000 7fb838aa1640 1 -- 172.21.15.46:0/1848498424 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8340ceb70 msgr2=0x7fb834154ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:11.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.621+0000 7fb838aa1640 1 --2- 172.21.15.46:0/1848498424 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8340ceb70 0x7fb834154ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:11.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.621+0000 7fb838aa1640 1 -- 172.21.15.46:0/1848498424 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb834144980 msgr2=0x7fb8340a9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:11.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.621+0000 7fb838aa1640 1 --2- 172.21.15.46:0/1848498424 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb834144980 0x7fb8340a9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:11.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.621+0000 7fb838aa1640 1 -- 172.21.15.46:0/1848498424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb82c009a70 con 0x7fb8340b4370 2023-12-02T00:30:11.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.621+0000 7fb838aa1640 1 --2- 172.21.15.46:0/1848498424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8340b4370 0x7fb8340b0280 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb82c009d90 tx=0x7fb82c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:11.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.621+0000 7fb8317fa640 1 -- 172.21.15.46:0/1848498424 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82c03e610 con 0x7fb8340b4370 2023-12-02T00:30:11.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.621+0000 7fb8317fa640 1 -- 172.21.15.46:0/1848498424 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb82c03e770 con 0x7fb8340b4370 2023-12-02T00:30:11.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.621+0000 7fb83ad05640 1 -- 172.21.15.46:0/1848498424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8340afa80 con 0x7fb8340b4370 2023-12-02T00:30:11.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.621+0000 7fb8317fa640 1 -- 172.21.15.46:0/1848498424 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82c03eb00 con 0x7fb8340b4370 2023-12-02T00:30:11.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.621+0000 7fb83ad05640 1 -- 172.21.15.46:0/1848498424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb834155220 con 0x7fb8340b4370 2023-12-02T00:30:11.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.622+0000 7fb83ad05640 1 -- 172.21.15.46:0/1848498424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8340affa0 con 0x7fb8340b4370 2023-12-02T00:30:11.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.623+0000 7fb8317fa640 1 -- 172.21.15.46:0/1848498424 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb82c03b050 con 0x7fb8340b4370 2023-12-02T00:30:11.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.623+0000 7fb8317fa640 1 --2- 172.21.15.46:0/1848498424 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb81406ae40 0x7fb81406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:11.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.623+0000 7fb8317fa640 1 -- 172.21.15.46:0/1848498424 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fb82c0ab9a0 con 0x7fb8340b4370 2023-12-02T00:30:11.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.623+0000 7fb833fff640 1 --2- 172.21.15.46:0/1848498424 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb81406ae40 0x7fb81406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:11.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.624+0000 7fb833fff640 1 --2- 172.21.15.46:0/1848498424 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb81406ae40 0x7fb81406d300 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb8340cc450 tx=0x7fb824009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:11.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.628+0000 7fb8317fa640 1 -- 172.21.15.46:0/1848498424 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb8340affa0 con 0x7fb8340b4370 2023-12-02T00:30:11.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.802+0000 7fb83ad05640 1 -- 172.21.15.46:0/1848498424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb834069400 con 0x7fb8340b4370 2023-12-02T00:30:11.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.802+0000 7fb8317fa640 1 -- 172.21.15.46:0/1848498424 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb82c04d0a0 con 0x7fb8340b4370 2023-12-02T00:30:11.804 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-12-02T00:30:11.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.804+0000 7fb83ad05640 1 -- 172.21.15.46:0/1848498424 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb81406ae40 msgr2=0x7fb81406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:11.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.804+0000 7fb83ad05640 1 --2- 172.21.15.46:0/1848498424 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb81406ae40 0x7fb81406d300 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb8340cc450 tx=0x7fb824009040 comp rx=0 tx=0).stop 2023-12-02T00:30:11.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.804+0000 7fb83ad05640 1 -- 172.21.15.46:0/1848498424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8340b4370 msgr2=0x7fb8340b0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:11.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.804+0000 7fb83ad05640 1 --2- 172.21.15.46:0/1848498424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8340b4370 0x7fb8340b0280 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb82c009d90 tx=0x7fb82c004a40 comp rx=0 tx=0).stop 2023-12-02T00:30:11.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.804+0000 7fb83ad05640 1 -- 172.21.15.46:0/1848498424 shutdown_connections 2023-12-02T00:30:11.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.804+0000 7fb83ad05640 1 --2- 172.21.15.46:0/1848498424 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb81406ae40 0x7fb81406d300 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:11.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.804+0000 7fb83ad05640 1 --2- 172.21.15.46:0/1848498424 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb834144980 0x7fb8340a9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:11.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.804+0000 7fb83ad05640 1 --2- 172.21.15.46:0/1848498424 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8340ceb70 0x7fb834154ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:11.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.804+0000 7fb83ad05640 1 --2- 172.21.15.46:0/1848498424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8340b4370 0x7fb8340b0280 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:11.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.804+0000 7fb83ad05640 1 -- 172.21.15.46:0/1848498424 >> 172.21.15.46:0/1848498424 conn(0x7fb83406a4d0 msgr2=0x7fb834071b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:11.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.804+0000 7fb83ad05640 1 -- 172.21.15.46:0/1848498424 shutdown_connections 2023-12-02T00:30:11.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.804+0000 7fb83ad05640 1 -- 172.21.15.46:0/1848498424 wait complete. 2023-12-02T00:30:11.816 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-02T00:30:11.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.924+0000 7f5683f9e640 1 Processor -- start 2023-12-02T00:30:11.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.924+0000 7f5683f9e640 1 -- start start 2023-12-02T00:30:11.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.925+0000 7f5683f9e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f567c143490 0x7f567c0b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:11.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.925+0000 7f5683f9e640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f567c15e2a0 con 0x7f567c15e730 2023-12-02T00:30:11.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.925+0000 7f5683f9e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f567c15b240 con 0x7f567c15abf0 2023-12-02T00:30:11.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.925+0000 7f5683f9e640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f567c15b520 con 0x7f567c143490 2023-12-02T00:30:11.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.925+0000 7f5681d3a640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f567c143490 0x7f567c0b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:11.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.925+0000 7f5681539640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f567c15e730 0x7f567c15a4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36094/0 (socket says 172.21.15.46:36094) 2023-12-02T00:30:11.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.925+0000 7f5681539640 1 -- 172.21.15.46:0/2058495612 learned_addr learned my addr 172.21.15.46:0/2058495612 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:30:11.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.926+0000 7f5681d3a640 1 -- 172.21.15.46:0/2058495612 >> v1:172.21.15.162:6789/0 conn(0x7f567c15abf0 legacy=0x7f567c0b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:11.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.926+0000 7f5681d3a640 1 -- 172.21.15.46:0/2058495612 >> v1:172.21.15.46:6789/0 conn(0x7f567c15e730 legacy=0x7f567c15a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:11.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.926+0000 7f5681d3a640 1 -- 172.21.15.46:0/2058495612 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f567c15dc50 con 0x7f567c143490 2023-12-02T00:30:11.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.926+0000 7f5680d38640 1 -- 172.21.15.46:0/2058495612 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4129065564 0 0) 0x7f567c15e2a0 con 0x7f567c15e730 2023-12-02T00:30:11.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.926+0000 7f5681d3a640 1 --2- 172.21.15.46:0/2058495612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f567c143490 0x7f567c0b0cf0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f56740097a0 tx=0x7f567402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60c0f16ab350f3c6 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:11.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.926+0000 7f5680d38640 1 -- 172.21.15.46:0/2058495612 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5674030610 con 0x7f567c143490 2023-12-02T00:30:11.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.927+0000 7f5680d38640 1 -- 172.21.15.46:0/2058495612 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5674030770 con 0x7f567c143490 2023-12-02T00:30:11.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.927+0000 7f5680d38640 1 -- 172.21.15.46:0/2058495612 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5674030b00 con 0x7f567c143490 2023-12-02T00:30:11.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.927+0000 7f5683f9e640 1 -- 172.21.15.46:0/2058495612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f567c143490 msgr2=0x7f567c0b0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:11.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.927+0000 7f5683f9e640 1 --2- 172.21.15.46:0/2058495612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f567c143490 0x7f567c0b0cf0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f56740097a0 tx=0x7f567402b960 comp rx=0 tx=0).stop 2023-12-02T00:30:11.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.927+0000 7f5683f9e640 1 -- 172.21.15.46:0/2058495612 shutdown_connections 2023-12-02T00:30:11.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.927+0000 7f5683f9e640 1 --2- 172.21.15.46:0/2058495612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f567c143490 0x7f567c0b0cf0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:11.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.927+0000 7f5683f9e640 1 -- 172.21.15.46:0/2058495612 >> 172.21.15.46:0/2058495612 conn(0x7f567c06a4d0 msgr2=0x7f567c06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:11.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.927+0000 7f5683f9e640 1 -- 172.21.15.46:0/2058495612 shutdown_connections 2023-12-02T00:30:11.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.927+0000 7f5683f9e640 1 -- 172.21.15.46:0/2058495612 wait complete. 2023-12-02T00:30:11.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.928+0000 7f5683f9e640 1 Processor -- start 2023-12-02T00:30:11.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.928+0000 7f5683f9e640 1 -- start start 2023-12-02T00:30:11.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.929+0000 7f5683f9e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f567c143490 0x7f567c159960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:11.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.929+0000 7f5683f9e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f567c15abf0 0x7f567c130290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:11.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.929+0000 7f5683f9e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f567c15e730 0x7f567c1307d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:11.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.929+0000 7f5683f9e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f567c131bd0 con 0x7f567c15e730 2023-12-02T00:30:11.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.929+0000 7f5683f9e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f567c130d10 con 0x7f567c15abf0 2023-12-02T00:30:11.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.929+0000 7f5683f9e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f567c130fc0 con 0x7f567c143490 2023-12-02T00:30:11.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.929+0000 7f568253b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f567c15e730 0x7f567c1307d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:11.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.929+0000 7f568253b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f567c15e730 0x7f567c1307d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46360/0 (socket says 172.21.15.46:46360) 2023-12-02T00:30:11.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.929+0000 7f568253b640 1 -- 172.21.15.46:0/3089808096 learned_addr learned my addr 172.21.15.46:0/3089808096 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:11.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.929+0000 7f5681d3a640 1 --2- 172.21.15.46:0/3089808096 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f567c143490 0x7f567c159960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:11.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.929+0000 7f5681539640 1 --2- 172.21.15.46:0/3089808096 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f567c15abf0 0x7f567c130290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:11.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.930+0000 7f568253b640 1 -- 172.21.15.46:0/3089808096 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f567c143490 msgr2=0x7f567c159960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:11.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.930+0000 7f568253b640 1 --2- 172.21.15.46:0/3089808096 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f567c143490 0x7f567c159960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:11.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.930+0000 7f568253b640 1 -- 172.21.15.46:0/3089808096 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f567c15abf0 msgr2=0x7f567c130290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:11.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.930+0000 7f568253b640 1 --2- 172.21.15.46:0/3089808096 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f567c15abf0 0x7f567c130290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:11.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.930+0000 7f568253b640 1 -- 172.21.15.46:0/3089808096 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5674009400 con 0x7f567c15e730 2023-12-02T00:30:11.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.930+0000 7f568253b640 1 --2- 172.21.15.46:0/3089808096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f567c15e730 0x7f567c1307d0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f566c00c7a0 tx=0x7f566c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:11.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.930+0000 7f567a7fc640 1 -- 172.21.15.46:0/3089808096 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566c003b70 con 0x7f567c15e730 2023-12-02T00:30:11.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.930+0000 7f567a7fc640 1 -- 172.21.15.46:0/3089808096 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f566c003cf0 con 0x7f567c15e730 2023-12-02T00:30:11.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.930+0000 7f5683f9e640 1 -- 172.21.15.46:0/3089808096 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f567c1312a0 con 0x7f567c15e730 2023-12-02T00:30:11.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.930+0000 7f567a7fc640 1 -- 172.21.15.46:0/3089808096 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566c003ea0 con 0x7f567c15e730 2023-12-02T00:30:11.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.930+0000 7f5683f9e640 1 -- 172.21.15.46:0/3089808096 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f567c131860 con 0x7f567c15e730 2023-12-02T00:30:11.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.931+0000 7f5683f9e640 1 -- 172.21.15.46:0/3089808096 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f567c131d40 con 0x7f567c15e730 2023-12-02T00:30:11.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.932+0000 7f567a7fc640 1 -- 172.21.15.46:0/3089808096 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f566c00aa90 con 0x7f567c15e730 2023-12-02T00:30:11.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.932+0000 7f567a7fc640 1 --2- 172.21.15.46:0/3089808096 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f566806ae70 0x7f566806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:11.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.932+0000 7f567a7fc640 1 -- 172.21.15.46:0/3089808096 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f566c089440 con 0x7f567c15e730 2023-12-02T00:30:11.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.932+0000 7f5681d3a640 1 --2- 172.21.15.46:0/3089808096 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f566806ae70 0x7f566806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:11.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.933+0000 7f5681d3a640 1 --2- 172.21.15.46:0/3089808096 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f566806ae70 0x7f566806d330 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5674036000 tx=0x7f5674008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:11.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:11.936+0000 7f567a7fc640 1 -- 172.21.15.46:0/3089808096 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f566c054410 con 0x7f567c15e730 2023-12-02T00:30:12.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.114+0000 7f5683f9e640 1 -- 172.21.15.46:0/3089808096 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f567c069400 con 0x7f567c15e730 2023-12-02T00:30:12.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.115+0000 7f567a7fc640 1 -- 172.21.15.46:0/3089808096 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v38) v1 ==== 104+0+11 (secure 0 0 0) 0x7f566c057a30 con 0x7f567c15e730 2023-12-02T00:30:12.118 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-12-02T00:30:12.118 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.117+0000 7f5683f9e640 1 -- 172.21.15.46:0/3089808096 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f566806ae70 msgr2=0x7f566806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:12.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.117+0000 7f5683f9e640 1 --2- 172.21.15.46:0/3089808096 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f566806ae70 0x7f566806d330 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5674036000 tx=0x7f5674008030 comp rx=0 tx=0).stop 2023-12-02T00:30:12.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.117+0000 7f5683f9e640 1 -- 172.21.15.46:0/3089808096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f567c15e730 msgr2=0x7f567c1307d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:12.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.117+0000 7f5683f9e640 1 --2- 172.21.15.46:0/3089808096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f567c15e730 0x7f567c1307d0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f566c00c7a0 tx=0x7f566c00cc60 comp rx=0 tx=0).stop 2023-12-02T00:30:12.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.117+0000 7f5683f9e640 1 -- 172.21.15.46:0/3089808096 shutdown_connections 2023-12-02T00:30:12.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.117+0000 7f5683f9e640 1 --2- 172.21.15.46:0/3089808096 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f566806ae70 0x7f566806d330 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:12.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.117+0000 7f5683f9e640 1 --2- 172.21.15.46:0/3089808096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f567c15e730 0x7f567c1307d0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:12.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.117+0000 7f5683f9e640 1 --2- 172.21.15.46:0/3089808096 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f567c15abf0 0x7f567c130290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:12.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.117+0000 7f5683f9e640 1 --2- 172.21.15.46:0/3089808096 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f567c143490 0x7f567c159960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:12.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.117+0000 7f5683f9e640 1 -- 172.21.15.46:0/3089808096 >> 172.21.15.46:0/3089808096 conn(0x7f567c06a4d0 msgr2=0x7f567c070390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:12.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.117+0000 7f5683f9e640 1 -- 172.21.15.46:0/3089808096 shutdown_connections 2023-12-02T00:30:12.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.117+0000 7f5683f9e640 1 -- 172.21.15.46:0/3089808096 wait complete. 2023-12-02T00:30:12.129 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-02T00:30:12.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.238+0000 7fdd790ee640 1 Processor -- start 2023-12-02T00:30:12.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.238+0000 7fdd790ee640 1 -- start start 2023-12-02T00:30:12.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.239+0000 7fdd790ee640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd74143490 0x7fdd740b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:12.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.239+0000 7fdd790ee640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fdd7415e730 0x7fdd740b1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:12.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.239+0000 7fdd790ee640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdd7415a4e0 0x7fdd7415a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:12.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.239+0000 7fdd790ee640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd7415b520 con 0x7fdd7415a4e0 2023-12-02T00:30:12.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.239+0000 7fdd790ee640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd74143870 con 0x7fdd7415e730 2023-12-02T00:30:12.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.239+0000 7fdd790ee640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7415e2a0 con 0x7fdd74143490 2023-12-02T00:30:12.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.239+0000 7fdd7359e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdd7415a4e0 0x7fdd7415a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:12.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.239+0000 7fdd7359e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdd7415a4e0 0x7fdd7415a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46366/0 (socket says 172.21.15.46:46366) 2023-12-02T00:30:12.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.239+0000 7fdd7359e640 1 -- 172.21.15.46:0/3242373863 learned_addr learned my addr 172.21.15.46:0/3242373863 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:12.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.239+0000 7fdd72d9d640 1 --2- 172.21.15.46:0/3242373863 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd74143490 0x7fdd740b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:12.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.239+0000 7fdd6a59c640 1 --2- 172.21.15.46:0/3242373863 >> v2:172.21.15.162:3300/0 conn(0x7fdd7415e730 0x7fdd740b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:12.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.239+0000 7fdd7359e640 1 -- 172.21.15.46:0/3242373863 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd74143490 msgr2=0x7fdd740b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:12.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.239+0000 7fdd7359e640 1 --2- 172.21.15.46:0/3242373863 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd74143490 0x7fdd740b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:12.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.239+0000 7fdd7359e640 1 -- 172.21.15.46:0/3242373863 >> v2:172.21.15.162:3300/0 conn(0x7fdd7415e730 msgr2=0x7fdd740b1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:12.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.239+0000 7fdd7359e640 1 --2- 172.21.15.46:0/3242373863 >> v2:172.21.15.162:3300/0 conn(0x7fdd7415e730 0x7fdd740b1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:12.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.239+0000 7fdd7359e640 1 -- 172.21.15.46:0/3242373863 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd7415b240 con 0x7fdd7415a4e0 2023-12-02T00:30:12.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.240+0000 7fdd7359e640 1 --2- 172.21.15.46:0/3242373863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd7415a4e0 0x7fdd7415a970 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fdd7406e680 tx=0x7fdd6c02e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc3e674cf92523b server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:12.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.240+0000 7fdd7259c640 1 -- 172.21.15.46:0/3242373863 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c02ec80 con 0x7fdd7415a4e0 2023-12-02T00:30:12.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.240+0000 7fdd7259c640 1 -- 172.21.15.46:0/3242373863 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd6c02ede0 con 0x7fdd7415a4e0 2023-12-02T00:30:12.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.240+0000 7fdd7259c640 1 -- 172.21.15.46:0/3242373863 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c036610 con 0x7fdd7415a4e0 2023-12-02T00:30:12.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.241+0000 7fdd790ee640 1 -- 172.21.15.46:0/3242373863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd7415a4e0 msgr2=0x7fdd7415a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:12.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.241+0000 7fdd790ee640 1 --2- 172.21.15.46:0/3242373863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd7415a4e0 0x7fdd7415a970 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fdd7406e680 tx=0x7fdd6c02e0e0 comp rx=0 tx=0).stop 2023-12-02T00:30:12.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.241+0000 7fdd790ee640 1 -- 172.21.15.46:0/3242373863 shutdown_connections 2023-12-02T00:30:12.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.241+0000 7fdd790ee640 1 --2- 172.21.15.46:0/3242373863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd7415a4e0 0x7fdd7415a970 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:12.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.241+0000 7fdd790ee640 1 --2- 172.21.15.46:0/3242373863 >> v2:172.21.15.162:3300/0 conn(0x7fdd7415e730 0x7fdd740b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:12.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.241+0000 7fdd790ee640 1 --2- 172.21.15.46:0/3242373863 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd74143490 0x7fdd740b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:12.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.241+0000 7fdd790ee640 1 -- 172.21.15.46:0/3242373863 >> 172.21.15.46:0/3242373863 conn(0x7fdd7406a4d0 msgr2=0x7fdd7406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:12.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.241+0000 7fdd790ee640 1 -- 172.21.15.46:0/3242373863 shutdown_connections 2023-12-02T00:30:12.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.241+0000 7fdd790ee640 1 -- 172.21.15.46:0/3242373863 wait complete. 2023-12-02T00:30:12.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.242+0000 7fdd790ee640 1 Processor -- start 2023-12-02T00:30:12.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.242+0000 7fdd790ee640 1 -- start start 2023-12-02T00:30:12.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.243+0000 7fdd790ee640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd74143490 0x7fdd740aff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:12.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.243+0000 7fdd790ee640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd7415a4e0 0x7fdd740b04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:12.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.243+0000 7fdd790ee640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd7415e730 0x7fdd74140800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:12.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.243+0000 7fdd790ee640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7413ce20 con 0x7fdd7415a4e0 2023-12-02T00:30:12.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.243+0000 7fdd790ee640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7413c230 con 0x7fdd74143490 2023-12-02T00:30:12.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.243+0000 7fdd790ee640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7413c480 con 0x7fdd7415e730 2023-12-02T00:30:12.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.243+0000 7fdd6a59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd7415a4e0 0x7fdd740b04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:12.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.243+0000 7fdd6a59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd7415a4e0 0x7fdd740b04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46378/0 (socket says 172.21.15.46:46378) 2023-12-02T00:30:12.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.243+0000 7fdd6a59c640 1 -- 172.21.15.46:0/2399192324 learned_addr learned my addr 172.21.15.46:0/2399192324 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:12.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.243+0000 7fdd72d9d640 1 --2- 172.21.15.46:0/2399192324 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd74143490 0x7fdd740aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:12.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.243+0000 7fdd7359e640 1 --2- 172.21.15.46:0/2399192324 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd7415e730 0x7fdd74140800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:12.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.243+0000 7fdd6a59c640 1 -- 172.21.15.46:0/2399192324 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd7415e730 msgr2=0x7fdd74140800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:12.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.243+0000 7fdd6a59c640 1 --2- 172.21.15.46:0/2399192324 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd7415e730 0x7fdd74140800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:12.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.243+0000 7fdd6a59c640 1 -- 172.21.15.46:0/2399192324 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd74143490 msgr2=0x7fdd740aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:12.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.243+0000 7fdd6a59c640 1 --2- 172.21.15.46:0/2399192324 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd74143490 0x7fdd740aff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:12.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.243+0000 7fdd6a59c640 1 -- 172.21.15.46:0/2399192324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd6c009b30 con 0x7fdd7415a4e0 2023-12-02T00:30:12.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.243+0000 7fdd7359e640 1 --2- 172.21.15.46:0/2399192324 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd7415e730 0x7fdd74140800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:30:12.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.243+0000 7fdd72d9d640 1 --2- 172.21.15.46:0/2399192324 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd74143490 0x7fdd740aff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:30:12.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.244+0000 7fdd6a59c640 1 --2- 172.21.15.46:0/2399192324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd7415a4e0 0x7fdd740b04a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c00d5f0 tx=0x7fdd5c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:12.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.244+0000 7fdd6bfff640 1 -- 172.21.15.46:0/2399192324 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c0125f0 con 0x7fdd7415a4e0 2023-12-02T00:30:12.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.244+0000 7fdd6bfff640 1 -- 172.21.15.46:0/2399192324 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd5c012750 con 0x7fdd7415a4e0 2023-12-02T00:30:12.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.244+0000 7fdd790ee640 1 -- 172.21.15.46:0/2399192324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd7413c760 con 0x7fdd7415a4e0 2023-12-02T00:30:12.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.244+0000 7fdd6bfff640 1 -- 172.21.15.46:0/2399192324 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c012b00 con 0x7fdd7415a4e0 2023-12-02T00:30:12.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.244+0000 7fdd790ee640 1 -- 172.21.15.46:0/2399192324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd741eed20 con 0x7fdd7415a4e0 2023-12-02T00:30:12.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.245+0000 7fdd790ee640 1 -- 172.21.15.46:0/2399192324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd7413cf60 con 0x7fdd7415a4e0 2023-12-02T00:30:12.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.245+0000 7fdd6bfff640 1 -- 172.21.15.46:0/2399192324 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdd5c012c60 con 0x7fdd7415a4e0 2023-12-02T00:30:12.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.246+0000 7fdd6bfff640 1 --2- 172.21.15.46:0/2399192324 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fdd5406ae40 0x7fdd5406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:12.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.246+0000 7fdd6bfff640 1 -- 172.21.15.46:0/2399192324 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fdd5c089c80 con 0x7fdd7415a4e0 2023-12-02T00:30:12.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.246+0000 7fdd72d9d640 1 --2- 172.21.15.46:0/2399192324 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fdd5406ae40 0x7fdd5406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:12.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.247+0000 7fdd72d9d640 1 --2- 172.21.15.46:0/2399192324 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fdd5406ae40 0x7fdd5406d300 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fdd64006fd0 tx=0x7fdd64008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:12.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.250+0000 7fdd6bfff640 1 -- 172.21.15.46:0/2399192324 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdd7413cf60 con 0x7fdd7415a4e0 2023-12-02T00:30:12.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.427+0000 7fdd790ee640 1 -- 172.21.15.46:0/2399192324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdd7413cf60 con 0x7fdd7415a4e0 2023-12-02T00:30:12.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.428+0000 7fdd6bfff640 1 -- 172.21.15.46:0/2399192324 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v38) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdd7413cf60 con 0x7fdd7415a4e0 2023-12-02T00:30:12.429 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-12-02T00:30:12.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.430+0000 7fdd790ee640 1 -- 172.21.15.46:0/2399192324 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fdd5406ae40 msgr2=0x7fdd5406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:12.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.430+0000 7fdd790ee640 1 --2- 172.21.15.46:0/2399192324 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fdd5406ae40 0x7fdd5406d300 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fdd64006fd0 tx=0x7fdd64008040 comp rx=0 tx=0).stop 2023-12-02T00:30:12.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.430+0000 7fdd790ee640 1 -- 172.21.15.46:0/2399192324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd7415a4e0 msgr2=0x7fdd740b04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:12.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.430+0000 7fdd790ee640 1 --2- 172.21.15.46:0/2399192324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd7415a4e0 0x7fdd740b04a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c00d5f0 tx=0x7fdd5c00dab0 comp rx=0 tx=0).stop 2023-12-02T00:30:12.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.430+0000 7fdd790ee640 1 -- 172.21.15.46:0/2399192324 shutdown_connections 2023-12-02T00:30:12.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.430+0000 7fdd790ee640 1 --2- 172.21.15.46:0/2399192324 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fdd5406ae40 0x7fdd5406d300 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:12.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.430+0000 7fdd790ee640 1 --2- 172.21.15.46:0/2399192324 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd7415e730 0x7fdd74140800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:12.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.430+0000 7fdd790ee640 1 --2- 172.21.15.46:0/2399192324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd7415a4e0 0x7fdd740b04a0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:12.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.430+0000 7fdd790ee640 1 --2- 172.21.15.46:0/2399192324 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd74143490 0x7fdd740aff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:12.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.430+0000 7fdd790ee640 1 -- 172.21.15.46:0/2399192324 >> 172.21.15.46:0/2399192324 conn(0x7fdd7406a4d0 msgr2=0x7fdd740b32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:12.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.430+0000 7fdd790ee640 1 -- 172.21.15.46:0/2399192324 shutdown_connections 2023-12-02T00:30:12.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.430+0000 7fdd790ee640 1 -- 172.21.15.46:0/2399192324 wait complete. 2023-12-02T00:30:12.443 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2023-12-02T00:30:12.552 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.550+0000 7f07a956e640 1 Processor -- start 2023-12-02T00:30:12.552 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.550+0000 7f07a956e640 1 -- start start 2023-12-02T00:30:12.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.551+0000 7f07a956e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07a406ea50 0x7f07a406ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:12.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.551+0000 7f07a956e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f07a4160fb0 0x7f07a4161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:12.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.551+0000 7f07a956e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f07a40b3ff0 0x7f07a40b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:12.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.551+0000 7f07a956e640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07a4161d20 con 0x7f07a4160fb0 2023-12-02T00:30:12.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.551+0000 7f07a956e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07a406f370 con 0x7f07a40b3ff0 2023-12-02T00:30:12.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.551+0000 7f07a956e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07a415e760 con 0x7f07a406ea50 2023-12-02T00:30:12.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.551+0000 7f07a27fc640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f07a4160fb0 0x7f07a4161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:12.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.551+0000 7f07a27fc640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f07a4160fb0 0x7f07a4161410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46388/0 (socket says 172.21.15.46:46388) 2023-12-02T00:30:12.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.551+0000 7f07a27fc640 1 -- 172.21.15.46:0/2049766937 learned_addr learned my addr 172.21.15.46:0/2049766937 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:12.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.552+0000 7f07a37fe640 1 --2- 172.21.15.46:0/2049766937 >> v2:172.21.15.162:3300/0 conn(0x7f07a40b3ff0 0x7f07a40b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:12.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.552+0000 7f07a2ffd640 1 --2- 172.21.15.46:0/2049766937 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07a406ea50 0x7f07a406ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:12.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.552+0000 7f07a27fc640 1 -- 172.21.15.46:0/2049766937 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07a406ea50 msgr2=0x7f07a406ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:12.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.552+0000 7f07a27fc640 1 --2- 172.21.15.46:0/2049766937 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07a406ea50 0x7f07a406ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:12.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.552+0000 7f07a27fc640 1 -- 172.21.15.46:0/2049766937 >> v2:172.21.15.162:3300/0 conn(0x7f07a40b3ff0 msgr2=0x7f07a40b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:12.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.552+0000 7f07a27fc640 1 --2- 172.21.15.46:0/2049766937 >> v2:172.21.15.162:3300/0 conn(0x7f07a40b3ff0 0x7f07a40b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:12.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.552+0000 7f07a27fc640 1 -- 172.21.15.46:0/2049766937 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a415d9a0 con 0x7f07a4160fb0 2023-12-02T00:30:12.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.552+0000 7f07a27fc640 1 --2- 172.21.15.46:0/2049766937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07a4160fb0 0x7f07a4161410 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f07a40b66c0 tx=0x7f078c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7b10dc9933fd88a server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:12.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.553+0000 7f07a1ffb640 1 -- 172.21.15.46:0/2049766937 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c0317e0 con 0x7f07a4160fb0 2023-12-02T00:30:12.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.553+0000 7f07a1ffb640 1 -- 172.21.15.46:0/2049766937 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f078c031940 con 0x7f07a4160fb0 2023-12-02T00:30:12.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.553+0000 7f07a1ffb640 1 -- 172.21.15.46:0/2049766937 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c031cd0 con 0x7f07a4160fb0 2023-12-02T00:30:12.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.553+0000 7f07a956e640 1 -- 172.21.15.46:0/2049766937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07a4160fb0 msgr2=0x7f07a4161410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:12.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.553+0000 7f07a956e640 1 --2- 172.21.15.46:0/2049766937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07a4160fb0 0x7f07a4161410 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f07a40b66c0 tx=0x7f078c02b880 comp rx=0 tx=0).stop 2023-12-02T00:30:12.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.553+0000 7f07a956e640 1 -- 172.21.15.46:0/2049766937 shutdown_connections 2023-12-02T00:30:12.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.553+0000 7f07a956e640 1 --2- 172.21.15.46:0/2049766937 >> v2:172.21.15.162:3300/0 conn(0x7f07a40b3ff0 0x7f07a40b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:12.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.553+0000 7f07a956e640 1 --2- 172.21.15.46:0/2049766937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07a4160fb0 0x7f07a4161410 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:12.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.553+0000 7f07a956e640 1 --2- 172.21.15.46:0/2049766937 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07a406ea50 0x7f07a406ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:12.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.553+0000 7f07a956e640 1 -- 172.21.15.46:0/2049766937 >> 172.21.15.46:0/2049766937 conn(0x7f07a406a6b0 msgr2=0x7f07a406aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:12.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.553+0000 7f07a956e640 1 -- 172.21.15.46:0/2049766937 shutdown_connections 2023-12-02T00:30:12.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.553+0000 7f07a956e640 1 -- 172.21.15.46:0/2049766937 wait complete. 2023-12-02T00:30:12.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.554+0000 7f07a956e640 1 Processor -- start 2023-12-02T00:30:12.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.555+0000 7f07a956e640 1 -- start start 2023-12-02T00:30:12.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.555+0000 7f07a956e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07a406ea50 0x7f07a4157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:12.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.555+0000 7f07a956e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07a40b3ff0 0x7f07a4158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:12.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.555+0000 7f07a956e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07a4160fb0 0x7f07a415fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:12.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.555+0000 7f07a956e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a4159fc0 con 0x7f07a40b3ff0 2023-12-02T00:30:12.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.555+0000 7f07a956e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a4158a30 con 0x7f07a4160fb0 2023-12-02T00:30:12.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.555+0000 7f07a956e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07a4158d10 con 0x7f07a406ea50 2023-12-02T00:30:12.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.555+0000 7f07a27fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07a40b3ff0 0x7f07a4158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:12.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.555+0000 7f07a27fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07a40b3ff0 0x7f07a4158400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46398/0 (socket says 172.21.15.46:46398) 2023-12-02T00:30:12.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.555+0000 7f07a27fc640 1 -- 172.21.15.46:0/3510970946 learned_addr learned my addr 172.21.15.46:0/3510970946 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:12.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.556+0000 7f07a2ffd640 1 --2- 172.21.15.46:0/3510970946 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07a406ea50 0x7f07a4157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:12.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.556+0000 7f07a37fe640 1 --2- 172.21.15.46:0/3510970946 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07a4160fb0 0x7f07a415fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:12.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.556+0000 7f07a27fc640 1 -- 172.21.15.46:0/3510970946 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07a406ea50 msgr2=0x7f07a4157ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:12.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.556+0000 7f07a27fc640 1 --2- 172.21.15.46:0/3510970946 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07a406ea50 0x7f07a4157ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:12.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.556+0000 7f07a27fc640 1 -- 172.21.15.46:0/3510970946 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07a4160fb0 msgr2=0x7f07a415fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:12.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.556+0000 7f07a27fc640 1 --2- 172.21.15.46:0/3510970946 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07a4160fb0 0x7f07a415fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:12.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.556+0000 7f07a27fc640 1 -- 172.21.15.46:0/3510970946 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f078c0094d0 con 0x7f07a40b3ff0 2023-12-02T00:30:12.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.556+0000 7f07a37fe640 1 --2- 172.21.15.46:0/3510970946 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07a4160fb0 0x7f07a415fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:30:12.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.556+0000 7f07a27fc640 1 --2- 172.21.15.46:0/3510970946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07a40b3ff0 0x7f07a4158400 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f078c0090b0 tx=0x7f078c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:12.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.556+0000 7f079b7fe640 1 -- 172.21.15.46:0/3510970946 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c032880 con 0x7f07a40b3ff0 2023-12-02T00:30:12.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.556+0000 7f079b7fe640 1 -- 172.21.15.46:0/3510970946 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f078c0329e0 con 0x7f07a40b3ff0 2023-12-02T00:30:12.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.556+0000 7f07a956e640 1 -- 172.21.15.46:0/3510970946 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07a4158ff0 con 0x7f07a40b3ff0 2023-12-02T00:30:12.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.556+0000 7f079b7fe640 1 -- 172.21.15.46:0/3510970946 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c032d90 con 0x7f07a40b3ff0 2023-12-02T00:30:12.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.556+0000 7f07a956e640 1 -- 172.21.15.46:0/3510970946 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07a41f34d0 con 0x7f07a40b3ff0 2023-12-02T00:30:12.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.557+0000 7f07a956e640 1 -- 172.21.15.46:0/3510970946 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07a4070d30 con 0x7f07a40b3ff0 2023-12-02T00:30:12.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.557+0000 7f079b7fe640 1 -- 172.21.15.46:0/3510970946 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f078c04d3f0 con 0x7f07a40b3ff0 2023-12-02T00:30:12.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.558+0000 7f079b7fe640 1 --2- 172.21.15.46:0/3510970946 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f078406ae40 0x7f078406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:12.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.558+0000 7f079b7fe640 1 -- 172.21.15.46:0/3510970946 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f078c0ab3d0 con 0x7f07a40b3ff0 2023-12-02T00:30:12.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.558+0000 7f07a2ffd640 1 --2- 172.21.15.46:0/3510970946 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f078406ae40 0x7f078406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:12.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.559+0000 7f07a2ffd640 1 --2- 172.21.15.46:0/3510970946 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f078406ae40 0x7f078406d300 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0794009e90 tx=0x7f079400b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:12.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.562+0000 7f079b7fe640 1 -- 172.21.15.46:0/3510970946 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f078c076430 con 0x7f07a40b3ff0 2023-12-02T00:30:12.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.737+0000 7f07a956e640 1 -- 172.21.15.46:0/3510970946 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7f07a40693e0 con 0x7f07a40b3ff0 2023-12-02T00:30:12.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.737+0000 7f079b7fe640 1 -- 172.21.15.46:0/3510970946 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v38) v1 ==== 97+0+11 (secure 0 0 0) 0x7f078c079a50 con 0x7f07a40b3ff0 2023-12-02T00:30:12.738 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-12-02T00:30:12.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.739+0000 7f07a956e640 1 -- 172.21.15.46:0/3510970946 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f078406ae40 msgr2=0x7f078406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:12.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.739+0000 7f07a956e640 1 --2- 172.21.15.46:0/3510970946 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f078406ae40 0x7f078406d300 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0794009e90 tx=0x7f079400b040 comp rx=0 tx=0).stop 2023-12-02T00:30:12.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.739+0000 7f07a956e640 1 -- 172.21.15.46:0/3510970946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07a40b3ff0 msgr2=0x7f07a4158400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:12.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.739+0000 7f07a956e640 1 --2- 172.21.15.46:0/3510970946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07a40b3ff0 0x7f07a4158400 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f078c0090b0 tx=0x7f078c0046c0 comp rx=0 tx=0).stop 2023-12-02T00:30:12.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.739+0000 7f07a956e640 1 -- 172.21.15.46:0/3510970946 shutdown_connections 2023-12-02T00:30:12.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.739+0000 7f07a956e640 1 --2- 172.21.15.46:0/3510970946 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f078406ae40 0x7f078406d300 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:12.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.739+0000 7f07a956e640 1 --2- 172.21.15.46:0/3510970946 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07a4160fb0 0x7f07a415fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:12.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.739+0000 7f07a956e640 1 --2- 172.21.15.46:0/3510970946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07a40b3ff0 0x7f07a4158400 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:12.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.739+0000 7f07a956e640 1 --2- 172.21.15.46:0/3510970946 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07a406ea50 0x7f07a4157ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:12.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.739+0000 7f07a956e640 1 -- 172.21.15.46:0/3510970946 >> 172.21.15.46:0/3510970946 conn(0x7f07a406a6b0 msgr2=0x7f07a4140850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:12.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.739+0000 7f07a956e640 1 -- 172.21.15.46:0/3510970946 shutdown_connections 2023-12-02T00:30:12.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.739+0000 7f07a956e640 1 -- 172.21.15.46:0/3510970946 wait complete. 2023-12-02T00:30:12.751 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-02T00:30:12.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.857+0000 7f33faaee640 1 Processor -- start 2023-12-02T00:30:12.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.857+0000 7f33faaee640 1 -- start start 2023-12-02T00:30:12.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.858+0000 7f33faaee640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f33f415d330 0x7f33f40b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:12.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.858+0000 7f33faaee640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f33f40d2d90 0x7f33f40b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:12.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.858+0000 7f33faaee640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33f40b2b10 con 0x7f33f40d2910 2023-12-02T00:30:12.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.858+0000 7f33faaee640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33f40d1590 con 0x7f33f40d2d90 2023-12-02T00:30:12.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.858+0000 7f33faaee640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33f40d1870 con 0x7f33f415d330 2023-12-02T00:30:12.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.859+0000 7f33f908b640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f33f40d2d90 0x7f33f40b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:12.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.859+0000 7f33f888a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f33f415d330 0x7f33f40b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:12.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.859+0000 7f33f3fff640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f33f40d2910 0x7f33f40d31f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.46:56746/0 (socket says 172.21.15.46:56746) 2023-12-02T00:30:12.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.859+0000 7f33f3fff640 1 -- 172.21.15.46:0/989754631 learned_addr learned my addr 172.21.15.46:0/989754631 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:30:12.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.859+0000 7f33f888a640 1 -- 172.21.15.46:0/989754631 >> v2:172.21.15.162:3300/0 conn(0x7f33f40d2d90 msgr2=0x7f33f40b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:12.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.859+0000 7f33f888a640 1 --2- 172.21.15.46:0/989754631 >> v2:172.21.15.162:3300/0 conn(0x7f33f40d2d90 0x7f33f40b6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:12.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.859+0000 7f33f888a640 1 -- 172.21.15.46:0/989754631 >> v1:172.21.15.162:6789/0 conn(0x7f33f40d2910 legacy=0x7f33f40d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:12.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.859+0000 7f33f888a640 1 -- 172.21.15.46:0/989754631 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f40d1b50 con 0x7f33f415d330 2023-12-02T00:30:12.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.859+0000 7f33f908b640 1 --2- 172.21.15.46:0/989754631 >> v2:172.21.15.162:3300/0 conn(0x7f33f40d2d90 0x7f33f40b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:30:12.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.860+0000 7f33f888a640 1 --2- 172.21.15.46:0/989754631 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f33f415d330 0x7f33f40b8b80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f33ec009d90 tx=0x7f33ec02be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1832d7df0366a4a server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:12.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.860+0000 7f33f37fe640 1 -- 172.21.15.46:0/989754631 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ec02cbc0 con 0x7f33f415d330 2023-12-02T00:30:12.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.860+0000 7f33f37fe640 1 -- 172.21.15.46:0/989754631 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33ec02cd20 con 0x7f33f415d330 2023-12-02T00:30:12.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.860+0000 7f33f37fe640 1 -- 172.21.15.46:0/989754631 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ec034660 con 0x7f33f415d330 2023-12-02T00:30:12.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.860+0000 7f33faaee640 1 -- 172.21.15.46:0/989754631 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f33f415d330 msgr2=0x7f33f40b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:12.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.860+0000 7f33faaee640 1 --2- 172.21.15.46:0/989754631 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f33f415d330 0x7f33f40b8b80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f33ec009d90 tx=0x7f33ec02be10 comp rx=0 tx=0).stop 2023-12-02T00:30:12.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.861+0000 7f33faaee640 1 -- 172.21.15.46:0/989754631 shutdown_connections 2023-12-02T00:30:12.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.861+0000 7f33faaee640 1 --2- 172.21.15.46:0/989754631 >> v2:172.21.15.162:3300/0 conn(0x7f33f40d2d90 0x7f33f40b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:12.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.861+0000 7f33faaee640 1 --2- 172.21.15.46:0/989754631 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f33f415d330 0x7f33f40b8b80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:12.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.861+0000 7f33faaee640 1 -- 172.21.15.46:0/989754631 >> 172.21.15.46:0/989754631 conn(0x7f33f4072020 msgr2=0x7f33f4072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:12.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.861+0000 7f33faaee640 1 -- 172.21.15.46:0/989754631 shutdown_connections 2023-12-02T00:30:12.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.861+0000 7f33faaee640 1 -- 172.21.15.46:0/989754631 wait complete. 2023-12-02T00:30:12.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.862+0000 7f33faaee640 1 Processor -- start 2023-12-02T00:30:12.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.862+0000 7f33faaee640 1 -- start start 2023-12-02T00:30:12.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.863+0000 7f33faaee640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f33f40d2910 0x7f33f41522a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:12.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.863+0000 7f33faaee640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33f40d2d90 0x7f33f40ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:12.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.863+0000 7f33faaee640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33f415d330 0x7f33f40c8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:12.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.863+0000 7f33faaee640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f40caa20 con 0x7f33f415d330 2023-12-02T00:30:12.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.863+0000 7f33faaee640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f40cb2c0 con 0x7f33f40d2d90 2023-12-02T00:30:12.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.863+0000 7f33faaee640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33f4153240 con 0x7f33f40d2910 2023-12-02T00:30:12.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.863+0000 7f33f908b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33f415d330 0x7f33f40c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:12.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.863+0000 7f33f908b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33f415d330 0x7f33f40c8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56856/0 (socket says 172.21.15.46:56856) 2023-12-02T00:30:12.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.863+0000 7f33f908b640 1 -- 172.21.15.46:0/1880055865 learned_addr learned my addr 172.21.15.46:0/1880055865 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:12.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.863+0000 7f33f888a640 1 --2- 172.21.15.46:0/1880055865 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f33f40d2910 0x7f33f41522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:12.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.863+0000 7f33f3fff640 1 --2- 172.21.15.46:0/1880055865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33f40d2d90 0x7f33f40ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:12.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.863+0000 7f33f908b640 1 -- 172.21.15.46:0/1880055865 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f33f40d2910 msgr2=0x7f33f41522a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:12.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.863+0000 7f33f908b640 1 --2- 172.21.15.46:0/1880055865 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f33f40d2910 0x7f33f41522a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:12.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.863+0000 7f33f908b640 1 -- 172.21.15.46:0/1880055865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33f40d2d90 msgr2=0x7f33f40ccfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:12.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.863+0000 7f33f908b640 1 --2- 172.21.15.46:0/1880055865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33f40d2d90 0x7f33f40ccfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:12.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.863+0000 7f33f908b640 1 -- 172.21.15.46:0/1880055865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e4009400 con 0x7f33f415d330 2023-12-02T00:30:12.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.863+0000 7f33f908b640 1 --2- 172.21.15.46:0/1880055865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33f415d330 0x7f33f40c8350 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f33e400e930 tx=0x7f33e400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:12.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.863+0000 7f33f17fa640 1 -- 172.21.15.46:0/1880055865 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e40096e0 con 0x7f33f415d330 2023-12-02T00:30:12.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.864+0000 7f33f17fa640 1 -- 172.21.15.46:0/1880055865 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e400ca00 con 0x7f33f415d330 2023-12-02T00:30:12.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.864+0000 7f33faaee640 1 -- 172.21.15.46:0/1880055865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33ec009a70 con 0x7f33f415d330 2023-12-02T00:30:12.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.864+0000 7f33f17fa640 1 -- 172.21.15.46:0/1880055865 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e400cd90 con 0x7f33f415d330 2023-12-02T00:30:12.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.864+0000 7f33faaee640 1 -- 172.21.15.46:0/1880055865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33f415e810 con 0x7f33f415d330 2023-12-02T00:30:12.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.865+0000 7f33faaee640 1 -- 172.21.15.46:0/1880055865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33f4069400 con 0x7f33f415d330 2023-12-02T00:30:12.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.865+0000 7f33f17fa640 1 -- 172.21.15.46:0/1880055865 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f33e4013470 con 0x7f33f415d330 2023-12-02T00:30:12.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.865+0000 7f33f17fa640 1 --2- 172.21.15.46:0/1880055865 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f33e006ae40 0x7f33e006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:12.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.866+0000 7f33f17fa640 1 -- 172.21.15.46:0/1880055865 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f33e4089a90 con 0x7f33f415d330 2023-12-02T00:30:12.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.866+0000 7f33f888a640 1 --2- 172.21.15.46:0/1880055865 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f33e006ae40 0x7f33e006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:12.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.866+0000 7f33f888a640 1 --2- 172.21.15.46:0/1880055865 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f33e006ae40 0x7f33e006d300 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f33ec036000 tx=0x7f33ec030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:12.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:12.870+0000 7f33f17fa640 1 -- 172.21.15.46:0/1880055865 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f33e40549b0 con 0x7f33f415d330 2023-12-02T00:30:13.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.062+0000 7f33faaee640 1 -- 172.21.15.46:0/1880055865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f33f40b2050 con 0x7f33f415d330 2023-12-02T00:30:13.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.062+0000 7f33f17fa640 1 -- 172.21.15.46:0/1880055865 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v38) v1 ==== 80+0+0 (secure 0 0 0) 0x7f33e4017080 con 0x7f33f415d330 2023-12-02T00:30:13.063 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2023-12-02T00:30:13.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.064+0000 7f33faaee640 1 -- 172.21.15.46:0/1880055865 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f33e006ae40 msgr2=0x7f33e006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:13.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.064+0000 7f33faaee640 1 --2- 172.21.15.46:0/1880055865 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f33e006ae40 0x7f33e006d300 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f33ec036000 tx=0x7f33ec030000 comp rx=0 tx=0).stop 2023-12-02T00:30:13.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.064+0000 7f33faaee640 1 -- 172.21.15.46:0/1880055865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33f415d330 msgr2=0x7f33f40c8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:13.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.064+0000 7f33faaee640 1 --2- 172.21.15.46:0/1880055865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33f415d330 0x7f33f40c8350 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f33e400e930 tx=0x7f33e400edf0 comp rx=0 tx=0).stop 2023-12-02T00:30:13.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.064+0000 7f33faaee640 1 -- 172.21.15.46:0/1880055865 shutdown_connections 2023-12-02T00:30:13.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.064+0000 7f33faaee640 1 --2- 172.21.15.46:0/1880055865 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f33e006ae40 0x7f33e006d300 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:13.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.064+0000 7f33faaee640 1 --2- 172.21.15.46:0/1880055865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33f415d330 0x7f33f40c8350 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:13.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.064+0000 7f33faaee640 1 --2- 172.21.15.46:0/1880055865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33f40d2d90 0x7f33f40ccfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:13.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.064+0000 7f33faaee640 1 --2- 172.21.15.46:0/1880055865 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f33f40d2910 0x7f33f41522a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:13.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.064+0000 7f33faaee640 1 -- 172.21.15.46:0/1880055865 >> 172.21.15.46:0/1880055865 conn(0x7f33f4072020 msgr2=0x7f33f40b6350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:13.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.064+0000 7f33faaee640 1 -- 172.21.15.46:0/1880055865 shutdown_connections 2023-12-02T00:30:13.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.064+0000 7f33faaee640 1 -- 172.21.15.46:0/1880055865 wait complete. 2023-12-02T00:30:13.076 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-02T00:30:13.076 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-02T00:30:13.092 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:30:13.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.237+0000 7f216991d640 1 Processor -- start 2023-12-02T00:30:13.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.237+0000 7f216991d640 1 -- start start 2023-12-02T00:30:13.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.238+0000 7f216991d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2164160f70 0x7f21640cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:13.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.238+0000 7f216991d640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21641614e0 con 0x7f216415d2d0 2023-12-02T00:30:13.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.238+0000 7f216991d640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21641617c0 con 0x7f21640ce480 2023-12-02T00:30:13.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.238+0000 7f216991d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2164161aa0 con 0x7f2164160f70 2023-12-02T00:30:13.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.238+0000 7f2162ffd640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f216415d2d0 0x7f21640b8b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59114/0 (socket says 172.21.15.46:59114) 2023-12-02T00:30:13.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.238+0000 7f2162ffd640 1 -- 172.21.15.46:0/3780149204 learned_addr learned my addr 172.21.15.46:0/3780149204 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:30:13.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.238+0000 7f21627fc640 1 --2- 172.21.15.46:0/3780149204 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2164160f70 0x7f21640cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:13.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.238+0000 7f2161ffb640 1 -- 172.21.15.46:0/3780149204 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1084375506 0 0) 0x7f21641614e0 con 0x7f216415d2d0 2023-12-02T00:30:13.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.238+0000 7f2161ffb640 1 -- 172.21.15.46:0/3780149204 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2150003520 con 0x7f216415d2d0 2023-12-02T00:30:13.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.239+0000 7f2161ffb640 1 -- 172.21.15.46:0/3780149204 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 492043941 0 0) 0x7f21641617c0 con 0x7f21640ce480 2023-12-02T00:30:13.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.239+0000 7f2161ffb640 1 -- 172.21.15.46:0/3780149204 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21641614e0 con 0x7f21640ce480 2023-12-02T00:30:13.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.239+0000 7f2161ffb640 1 -- 172.21.15.46:0/3780149204 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1493845781 0 0) 0x7f2150003520 con 0x7f216415d2d0 2023-12-02T00:30:13.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.239+0000 7f21627fc640 1 -- 172.21.15.46:0/3780149204 >> v1:172.21.15.162:6789/0 conn(0x7f21640ce480 legacy=0x7f21640d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:13.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.239+0000 7f21627fc640 1 -- 172.21.15.46:0/3780149204 >> v1:172.21.15.46:6789/0 conn(0x7f216415d2d0 legacy=0x7f21640b8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:13.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.239+0000 7f21627fc640 1 -- 172.21.15.46:0/3780149204 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2164161d80 con 0x7f2164160f70 2023-12-02T00:30:13.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.240+0000 7f21627fc640 1 --2- 172.21.15.46:0/3780149204 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2164160f70 0x7f21640cdf40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f214c0097d0 tx=0x7f214c02b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72e1b1ed334b7397 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:13.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.240+0000 7f2161ffb640 1 -- 172.21.15.46:0/3780149204 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21540019f0 con 0x7f2164160f70 2023-12-02T00:30:13.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.240+0000 7f2161ffb640 1 -- 172.21.15.46:0/3780149204 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f214c02e050 con 0x7f2164160f70 2023-12-02T00:30:13.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.240+0000 7f2161ffb640 1 -- 172.21.15.46:0/3780149204 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214c030710 con 0x7f2164160f70 2023-12-02T00:30:13.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.240+0000 7f216991d640 1 -- 172.21.15.46:0/3780149204 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2164160f70 msgr2=0x7f21640cdf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:13.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.240+0000 7f216991d640 1 --2- 172.21.15.46:0/3780149204 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2164160f70 0x7f21640cdf40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f214c0097d0 tx=0x7f214c02b800 comp rx=0 tx=0).stop 2023-12-02T00:30:13.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.240+0000 7f216991d640 1 -- 172.21.15.46:0/3780149204 shutdown_connections 2023-12-02T00:30:13.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.240+0000 7f216991d640 1 --2- 172.21.15.46:0/3780149204 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2164160f70 0x7f21640cdf40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:13.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.240+0000 7f216991d640 1 -- 172.21.15.46:0/3780149204 >> 172.21.15.46:0/3780149204 conn(0x7f2164072230 msgr2=0x7f21640cfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:13.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.240+0000 7f216991d640 1 -- 172.21.15.46:0/3780149204 shutdown_connections 2023-12-02T00:30:13.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.240+0000 7f216991d640 1 -- 172.21.15.46:0/3780149204 wait complete. 2023-12-02T00:30:13.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.241+0000 7f216991d640 1 Processor -- start 2023-12-02T00:30:13.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.242+0000 7f216991d640 1 -- start start 2023-12-02T00:30:13.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.242+0000 7f216991d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21640ce480 0x7f216415fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:13.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.242+0000 7f216991d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f216415d2d0 0x7f2164160180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:13.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.242+0000 7f216991d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2164160f70 0x7f21641f28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:13.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.242+0000 7f216991d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21641f45e0 con 0x7f2164160f70 2023-12-02T00:30:13.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.242+0000 7f216991d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21641f38e0 con 0x7f216415d2d0 2023-12-02T00:30:13.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.242+0000 7f216991d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21641f3f20 con 0x7f21640ce480 2023-12-02T00:30:13.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.242+0000 7f21637fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2164160f70 0x7f21641f28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:13.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.243+0000 7f21637fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2164160f70 0x7f21641f28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56872/0 (socket says 172.21.15.46:56872) 2023-12-02T00:30:13.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.243+0000 7f21637fe640 1 -- 172.21.15.46:0/1203559122 learned_addr learned my addr 172.21.15.46:0/1203559122 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:13.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.243+0000 7f21627fc640 1 --2- 172.21.15.46:0/1203559122 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f216415d2d0 0x7f2164160180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:13.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.243+0000 7f2162ffd640 1 --2- 172.21.15.46:0/1203559122 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21640ce480 0x7f216415fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:13.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.243+0000 7f21637fe640 1 -- 172.21.15.46:0/1203559122 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21640ce480 msgr2=0x7f216415fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:13.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.243+0000 7f21637fe640 1 --2- 172.21.15.46:0/1203559122 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21640ce480 0x7f216415fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:13.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.243+0000 7f21637fe640 1 -- 172.21.15.46:0/1203559122 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f216415d2d0 msgr2=0x7f2164160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:13.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.243+0000 7f21637fe640 1 --2- 172.21.15.46:0/1203559122 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f216415d2d0 0x7f2164160180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:13.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.243+0000 7f21637fe640 1 -- 172.21.15.46:0/1203559122 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f214c009400 con 0x7f2164160f70 2023-12-02T00:30:13.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.243+0000 7f2162ffd640 1 --2- 172.21.15.46:0/1203559122 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21640ce480 0x7f216415fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:30:13.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.243+0000 7f21637fe640 1 --2- 172.21.15.46:0/1203559122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2164160f70 0x7f21641f28a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f215c00c400 tx=0x7f215c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:13.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.243+0000 7f215bfff640 1 -- 172.21.15.46:0/1203559122 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215c0052d0 con 0x7f2164160f70 2023-12-02T00:30:13.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.243+0000 7f216991d640 1 -- 172.21.15.46:0/1203559122 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21641f4200 con 0x7f2164160f70 2023-12-02T00:30:13.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.243+0000 7f215bfff640 1 -- 172.21.15.46:0/1203559122 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f215c005430 con 0x7f2164160f70 2023-12-02T00:30:13.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.243+0000 7f215bfff640 1 -- 172.21.15.46:0/1203559122 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215c0037f0 con 0x7f2164160f70 2023-12-02T00:30:13.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.243+0000 7f216991d640 1 -- 172.21.15.46:0/1203559122 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21641f3bc0 con 0x7f2164160f70 2023-12-02T00:30:13.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.244+0000 7f216991d640 1 -- 172.21.15.46:0/1203559122 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21641f4720 con 0x7f2164160f70 2023-12-02T00:30:13.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.245+0000 7f215bfff640 1 -- 172.21.15.46:0/1203559122 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f215c003950 con 0x7f2164160f70 2023-12-02T00:30:13.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.245+0000 7f215bfff640 1 --2- 172.21.15.46:0/1203559122 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f214406ae70 0x7f214406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:13.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.245+0000 7f215bfff640 1 -- 172.21.15.46:0/1203559122 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f215c088ce0 con 0x7f2164160f70 2023-12-02T00:30:13.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.245+0000 7f2162ffd640 1 --2- 172.21.15.46:0/1203559122 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f214406ae70 0x7f214406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:13.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.246+0000 7f2162ffd640 1 --2- 172.21.15.46:0/1203559122 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f214406ae70 0x7f214406d330 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2154001560 tx=0x7f2154009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:13.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.249+0000 7f215bfff640 1 -- 172.21.15.46:0/1203559122 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f21641f4720 con 0x7f2164160f70 2023-12-02T00:30:13.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.425+0000 7f216991d640 1 -- 172.21.15.46:0/1203559122 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f21641f4720 con 0x7f2164160f70 2023-12-02T00:30:13.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.426+0000 7f215bfff640 1 -- 172.21.15.46:0/1203559122 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+18324 (secure 0 0 0) 0x7f21641f4720 con 0x7f2164160f70 2023-12-02T00:30:13.428 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:30:13.428 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":38,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:30:10.714057+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-02T00:29:55.676572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-02T00:29:56.244869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"data_new","create_time":"2023-12-02T00:30:08.897990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:30:13.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.428+0000 7f216991d640 1 -- 172.21.15.46:0/1203559122 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f214406ae70 msgr2=0x7f214406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:13.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.428+0000 7f216991d640 1 --2- 172.21.15.46:0/1203559122 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f214406ae70 0x7f214406d330 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2154001560 tx=0x7f2154009040 comp rx=0 tx=0).stop 2023-12-02T00:30:13.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.428+0000 7f216991d640 1 -- 172.21.15.46:0/1203559122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2164160f70 msgr2=0x7f21641f28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:13.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.428+0000 7f216991d640 1 --2- 172.21.15.46:0/1203559122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2164160f70 0x7f21641f28a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f215c00c400 tx=0x7f215c00c8c0 comp rx=0 tx=0).stop 2023-12-02T00:30:13.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.428+0000 7f216991d640 1 -- 172.21.15.46:0/1203559122 shutdown_connections 2023-12-02T00:30:13.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.428+0000 7f216991d640 1 --2- 172.21.15.46:0/1203559122 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f214406ae70 0x7f214406d330 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:13.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.428+0000 7f216991d640 1 --2- 172.21.15.46:0/1203559122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2164160f70 0x7f21641f28a0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:13.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.428+0000 7f216991d640 1 --2- 172.21.15.46:0/1203559122 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f216415d2d0 0x7f2164160180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:13.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.428+0000 7f216991d640 1 --2- 172.21.15.46:0/1203559122 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21640ce480 0x7f216415fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:13.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.428+0000 7f216991d640 1 -- 172.21.15.46:0/1203559122 >> 172.21.15.46:0/1203559122 conn(0x7f2164072230 msgr2=0x7f21640b74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:13.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.429+0000 7f216991d640 1 -- 172.21.15.46:0/1203559122 shutdown_connections 2023-12-02T00:30:13.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.429+0000 7f216991d640 1 -- 172.21.15.46:0/1203559122 wait complete. 2023-12-02T00:30:13.441 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-02T00:28:37.213645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-02T00:29:55.676572+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-12-02T00:29:56.244869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'data_new', 'create_time': '2023-12-02T00:30:08.897990+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-02T00:30:13.441 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-02T00:30:13.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.549+0000 7f285a7de640 1 Processor -- start 2023-12-02T00:30:13.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.549+0000 7f285a7de640 1 -- start start 2023-12-02T00:30:13.556 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.550+0000 7f285a7de640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f28540d2910 0x7f28540d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:13.556 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.550+0000 7f285a7de640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28540b2b10 con 0x7f28540d3730 2023-12-02T00:30:13.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.550+0000 7f285a7de640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28540d1590 con 0x7f285415d330 2023-12-02T00:30:13.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.550+0000 7f285a7de640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28540d3bc0 con 0x7f28540d2910 2023-12-02T00:30:13.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.550+0000 7f28537fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f28540d2910 0x7f28540d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:13.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.550+0000 7f28537fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f28540d2910 0x7f28540d31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56888/0 (socket says 172.21.15.46:56888) 2023-12-02T00:30:13.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.550+0000 7f2858d7b640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f28540d3730 0x7f28540b6390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59118/0 (socket says 172.21.15.46:59118) 2023-12-02T00:30:13.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.550+0000 7f28537fe640 1 -- 172.21.15.46:0/2249361133 learned_addr learned my addr 172.21.15.46:0/2249361133 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:13.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.551+0000 7f28537fe640 1 -- 172.21.15.46:0/2249361133 >> v1:172.21.15.162:6789/0 conn(0x7f285415d330 legacy=0x7f28540b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:13.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.551+0000 7f28537fe640 1 -- 172.21.15.46:0/2249361133 >> v1:172.21.15.46:6789/0 conn(0x7f28540d3730 legacy=0x7f28540b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:13.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.551+0000 7f2852ffd640 1 -- 172.21.15.46:0/2249361133 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1177066741 0 0) 0x7f28540b2b10 con 0x7f28540d3730 2023-12-02T00:30:13.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.551+0000 7f28537fe640 1 -- 172.21.15.46:0/2249361133 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28540d1870 con 0x7f28540d2910 2023-12-02T00:30:13.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.551+0000 7f28537fe640 1 --2- 172.21.15.46:0/2249361133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28540d2910 0x7f28540d31f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f2848009870 tx=0x7f284802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5262a7644c2e2ba8 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:13.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.552+0000 7f2852ffd640 1 -- 172.21.15.46:0/2249361133 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2848030860 con 0x7f28540d2910 2023-12-02T00:30:13.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.552+0000 7f2852ffd640 1 -- 172.21.15.46:0/2249361133 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28480309c0 con 0x7f28540d2910 2023-12-02T00:30:13.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.552+0000 7f2852ffd640 1 -- 172.21.15.46:0/2249361133 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2848030d50 con 0x7f28540d2910 2023-12-02T00:30:13.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.552+0000 7f285a7de640 1 -- 172.21.15.46:0/2249361133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28540d2910 msgr2=0x7f28540d31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:13.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.552+0000 7f285a7de640 1 --2- 172.21.15.46:0/2249361133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28540d2910 0x7f28540d31f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f2848009870 tx=0x7f284802b9a0 comp rx=0 tx=0).stop 2023-12-02T00:30:13.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.552+0000 7f285a7de640 1 -- 172.21.15.46:0/2249361133 shutdown_connections 2023-12-02T00:30:13.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.552+0000 7f285a7de640 1 --2- 172.21.15.46:0/2249361133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28540d2910 0x7f28540d31f0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:13.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.552+0000 7f285a7de640 1 -- 172.21.15.46:0/2249361133 >> 172.21.15.46:0/2249361133 conn(0x7f2854072020 msgr2=0x7f2854072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:13.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.552+0000 7f285a7de640 1 -- 172.21.15.46:0/2249361133 shutdown_connections 2023-12-02T00:30:13.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.552+0000 7f285a7de640 1 -- 172.21.15.46:0/2249361133 wait complete. 2023-12-02T00:30:13.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.553+0000 7f285a7de640 1 Processor -- start 2023-12-02T00:30:13.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.554+0000 7f285a7de640 1 -- start start 2023-12-02T00:30:13.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.554+0000 7f285a7de640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28540d2910 0x7f285415ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:13.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.554+0000 7f285a7de640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f28540d3730 0x7f285415ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:13.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.554+0000 7f285a7de640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f285415d330 0x7f285415f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:13.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.554+0000 7f285a7de640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f285415bdb0 con 0x7f28540d2910 2023-12-02T00:30:13.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.554+0000 7f285a7de640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28540cb4e0 con 0x7f285415d330 2023-12-02T00:30:13.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.554+0000 7f285a7de640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28540cb7c0 con 0x7f28540d3730 2023-12-02T00:30:13.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.554+0000 7f2853fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28540d2910 0x7f285415ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:13.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.554+0000 7f2853fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28540d2910 0x7f285415ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56896/0 (socket says 172.21.15.46:56896) 2023-12-02T00:30:13.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.554+0000 7f2853fff640 1 -- 172.21.15.46:0/166208300 learned_addr learned my addr 172.21.15.46:0/166208300 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:13.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.555+0000 7f2858d7b640 1 --2- 172.21.15.46:0/166208300 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f285415d330 0x7f285415f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:13.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.555+0000 7f28537fe640 1 --2- 172.21.15.46:0/166208300 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f28540d3730 0x7f285415ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:13.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.555+0000 7f2853fff640 1 -- 172.21.15.46:0/166208300 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f28540d3730 msgr2=0x7f285415ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:13.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.555+0000 7f2853fff640 1 --2- 172.21.15.46:0/166208300 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f28540d3730 0x7f285415ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:13.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.555+0000 7f2853fff640 1 -- 172.21.15.46:0/166208300 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f285415d330 msgr2=0x7f285415f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:13.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.555+0000 7f2853fff640 1 --2- 172.21.15.46:0/166208300 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f285415d330 0x7f285415f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:13.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.555+0000 7f2853fff640 1 -- 172.21.15.46:0/166208300 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28480094d0 con 0x7f28540d2910 2023-12-02T00:30:13.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.555+0000 7f2858d7b640 1 --2- 172.21.15.46:0/166208300 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f285415d330 0x7f285415f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:30:13.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.555+0000 7f28537fe640 1 --2- 172.21.15.46:0/166208300 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f28540d3730 0x7f285415ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:30:13.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.555+0000 7f2853fff640 1 --2- 172.21.15.46:0/166208300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28540d2910 0x7f285415ad50 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f284400c400 tx=0x7f284400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:13.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.555+0000 7f2850ff9640 1 -- 172.21.15.46:0/166208300 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28440052d0 con 0x7f28540d2910 2023-12-02T00:30:13.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.555+0000 7f2850ff9640 1 -- 172.21.15.46:0/166208300 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2844005430 con 0x7f28540d2910 2023-12-02T00:30:13.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.555+0000 7f285a7de640 1 -- 172.21.15.46:0/166208300 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28540cbaa0 con 0x7f28540d2910 2023-12-02T00:30:13.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.555+0000 7f2850ff9640 1 -- 172.21.15.46:0/166208300 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2844003830 con 0x7f28540d2910 2023-12-02T00:30:13.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.555+0000 7f285a7de640 1 -- 172.21.15.46:0/166208300 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28540cc060 con 0x7f28540d2910 2023-12-02T00:30:13.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.556+0000 7f285a7de640 1 -- 172.21.15.46:0/166208300 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2854069400 con 0x7f28540d2910 2023-12-02T00:30:13.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.557+0000 7f2850ff9640 1 -- 172.21.15.46:0/166208300 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2844003990 con 0x7f28540d2910 2023-12-02T00:30:13.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.557+0000 7f2850ff9640 1 --2- 172.21.15.46:0/166208300 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f284006ae40 0x7f284006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:13.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.557+0000 7f2850ff9640 1 -- 172.21.15.46:0/166208300 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f2844088d20 con 0x7f28540d2910 2023-12-02T00:30:13.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.557+0000 7f28537fe640 1 --2- 172.21.15.46:0/166208300 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f284006ae40 0x7f284006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:13.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.558+0000 7f28537fe640 1 --2- 172.21.15.46:0/166208300 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f284006ae40 0x7f284006d300 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2848036000 tx=0x7f2848008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:13.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.561+0000 7f2850ff9640 1 -- 172.21.15.46:0/166208300 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2844053c40 con 0x7f28540d2910 2023-12-02T00:30:13.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.738+0000 7f285a7de640 1 -- 172.21.15.46:0/166208300 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2854072650 con 0x7f28540d2910 2023-12-02T00:30:13.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.738+0000 7f2850ff9640 1 -- 172.21.15.46:0/166208300 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) v1 ==== 93+0+10 (secure 0 0 0) 0x7f28440170a0 con 0x7f28540d2910 2023-12-02T00:30:13.740 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-12-02T00:30:13.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.740+0000 7f285a7de640 1 -- 172.21.15.46:0/166208300 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f284006ae40 msgr2=0x7f284006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:13.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.740+0000 7f285a7de640 1 --2- 172.21.15.46:0/166208300 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f284006ae40 0x7f284006d300 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2848036000 tx=0x7f2848008030 comp rx=0 tx=0).stop 2023-12-02T00:30:13.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.740+0000 7f285a7de640 1 -- 172.21.15.46:0/166208300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28540d2910 msgr2=0x7f285415ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:13.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.740+0000 7f285a7de640 1 --2- 172.21.15.46:0/166208300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28540d2910 0x7f285415ad50 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f284400c400 tx=0x7f284400c8c0 comp rx=0 tx=0).stop 2023-12-02T00:30:13.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.740+0000 7f285a7de640 1 -- 172.21.15.46:0/166208300 shutdown_connections 2023-12-02T00:30:13.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.740+0000 7f285a7de640 1 --2- 172.21.15.46:0/166208300 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f284006ae40 0x7f284006d300 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:13.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.740+0000 7f285a7de640 1 --2- 172.21.15.46:0/166208300 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f285415d330 0x7f285415f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:13.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.740+0000 7f285a7de640 1 --2- 172.21.15.46:0/166208300 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f28540d3730 0x7f285415ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:13.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.740+0000 7f285a7de640 1 --2- 172.21.15.46:0/166208300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28540d2910 0x7f285415ad50 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:13.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.740+0000 7f285a7de640 1 -- 172.21.15.46:0/166208300 >> 172.21.15.46:0/166208300 conn(0x7f2854072020 msgr2=0x7f28540b7bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:13.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.741+0000 7f285a7de640 1 -- 172.21.15.46:0/166208300 shutdown_connections 2023-12-02T00:30:13.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.741+0000 7f285a7de640 1 -- 172.21.15.46:0/166208300 wait complete. 2023-12-02T00:30:13.752 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-02T00:30:13.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.861+0000 7ff82afd0640 1 Processor -- start 2023-12-02T00:30:13.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.861+0000 7ff82afd0640 1 -- start start 2023-12-02T00:30:13.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.862+0000 7ff82afd0640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff8240b4370 0x7ff8240b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:13.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.862+0000 7ff82afd0640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff824071770 con 0x7ff8240ceb70 2023-12-02T00:30:13.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.862+0000 7ff82afd0640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8240ccb60 con 0x7ff824144980 2023-12-02T00:30:13.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.862+0000 7ff82afd0640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff82415ced0 con 0x7ff8240b4370 2023-12-02T00:30:13.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.862+0000 7ff823fff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff8240ceb70 0x7ff8240cc450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59124/0 (socket says 172.21.15.46:59124) 2023-12-02T00:30:13.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.862+0000 7ff823fff640 1 -- 172.21.15.46:0/4150539996 learned_addr learned my addr 172.21.15.46:0/4150539996 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:30:13.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.862+0000 7ff82956d640 1 --2- 172.21.15.46:0/4150539996 >> v2:172.21.15.46:3300/0 conn(0x7ff8240b4370 0x7ff8240b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:13.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.862+0000 7ff82956d640 1 -- 172.21.15.46:0/4150539996 >> v1:172.21.15.162:6789/0 conn(0x7ff824144980 legacy=0x7ff82415c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:13.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.862+0000 7ff82956d640 1 -- 172.21.15.46:0/4150539996 >> v1:172.21.15.46:6789/0 conn(0x7ff8240ceb70 legacy=0x7ff8240cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:13.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.862+0000 7ff82956d640 1 -- 172.21.15.46:0/4150539996 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8240b4ce0 con 0x7ff8240b4370 2023-12-02T00:30:13.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.863+0000 7ff82956d640 1 --2- 172.21.15.46:0/4150539996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8240b4370 0x7ff8240b47a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff8240b4e70 tx=0x7ff81c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87082099310a0d47 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:13.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.863+0000 7ff8237fe640 1 -- 172.21.15.46:0/4150539996 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81c02cc50 con 0x7ff8240b4370 2023-12-02T00:30:13.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.863+0000 7ff8237fe640 1 -- 172.21.15.46:0/4150539996 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff81c02cdb0 con 0x7ff8240b4370 2023-12-02T00:30:13.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.863+0000 7ff8237fe640 1 -- 172.21.15.46:0/4150539996 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81c034660 con 0x7ff8240b4370 2023-12-02T00:30:13.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.864+0000 7ff82afd0640 1 -- 172.21.15.46:0/4150539996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8240b4370 msgr2=0x7ff8240b47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:13.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.864+0000 7ff82afd0640 1 --2- 172.21.15.46:0/4150539996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8240b4370 0x7ff8240b47a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff8240b4e70 tx=0x7ff81c02bea0 comp rx=0 tx=0).stop 2023-12-02T00:30:13.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.864+0000 7ff82afd0640 1 -- 172.21.15.46:0/4150539996 shutdown_connections 2023-12-02T00:30:13.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.864+0000 7ff82afd0640 1 --2- 172.21.15.46:0/4150539996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8240b4370 0x7ff8240b47a0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:13.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.864+0000 7ff82afd0640 1 -- 172.21.15.46:0/4150539996 >> 172.21.15.46:0/4150539996 conn(0x7ff82406a4d0 msgr2=0x7ff82406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:13.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.864+0000 7ff82afd0640 1 -- 172.21.15.46:0/4150539996 shutdown_connections 2023-12-02T00:30:13.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.864+0000 7ff82afd0640 1 -- 172.21.15.46:0/4150539996 wait complete. 2023-12-02T00:30:13.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.865+0000 7ff82afd0640 1 Processor -- start 2023-12-02T00:30:13.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.865+0000 7ff82afd0640 1 -- start start 2023-12-02T00:30:13.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.866+0000 7ff82afd0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8240b4370 0x7ff8240c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:13.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.866+0000 7ff82afd0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8240ceb70 0x7ff8241f65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:13.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.866+0000 7ff82afd0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff824144980 0x7ff8241544d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:13.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.866+0000 7ff82afd0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8241e4230 con 0x7ff8240ceb70 2023-12-02T00:30:13.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.866+0000 7ff82afd0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8241e5370 con 0x7ff8240b4370 2023-12-02T00:30:13.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.866+0000 7ff82afd0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8241e4410 con 0x7ff824144980 2023-12-02T00:30:13.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.866+0000 7ff823fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8240ceb70 0x7ff8241f65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:13.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.866+0000 7ff823fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8240ceb70 0x7ff8241f65c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56918/0 (socket says 172.21.15.46:56918) 2023-12-02T00:30:13.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.866+0000 7ff823fff640 1 -- 172.21.15.46:0/1967329839 learned_addr learned my addr 172.21.15.46:0/1967329839 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:13.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.866+0000 7ff828d6c640 1 --2- 172.21.15.46:0/1967329839 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8240b4370 0x7ff8240c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:13.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.866+0000 7ff82956d640 1 --2- 172.21.15.46:0/1967329839 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff824144980 0x7ff8241544d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:13.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.866+0000 7ff823fff640 1 -- 172.21.15.46:0/1967329839 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff824144980 msgr2=0x7ff8241544d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:13.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.866+0000 7ff823fff640 1 --2- 172.21.15.46:0/1967329839 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff824144980 0x7ff8241544d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:13.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.866+0000 7ff823fff640 1 -- 172.21.15.46:0/1967329839 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8240b4370 msgr2=0x7ff8240c8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:13.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.866+0000 7ff823fff640 1 --2- 172.21.15.46:0/1967329839 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8240b4370 0x7ff8240c8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:13.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.866+0000 7ff823fff640 1 -- 172.21.15.46:0/1967329839 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff81c009a70 con 0x7ff8240ceb70 2023-12-02T00:30:13.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.866+0000 7ff828d6c640 1 --2- 172.21.15.46:0/1967329839 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8240b4370 0x7ff8240c8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:30:13.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.866+0000 7ff823fff640 1 --2- 172.21.15.46:0/1967329839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8240ceb70 0x7ff8241f65c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ff81000c4a0 tx=0x7ff81000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:13.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.867+0000 7ff8217fa640 1 -- 172.21.15.46:0/1967329839 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff810005ce0 con 0x7ff8240ceb70 2023-12-02T00:30:13.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.867+0000 7ff8217fa640 1 -- 172.21.15.46:0/1967329839 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff810005e40 con 0x7ff8240ceb70 2023-12-02T00:30:13.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.867+0000 7ff82afd0640 1 -- 172.21.15.46:0/1967329839 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8241e4d20 con 0x7ff8240ceb70 2023-12-02T00:30:13.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.867+0000 7ff8217fa640 1 -- 172.21.15.46:0/1967329839 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff810007270 con 0x7ff8240ceb70 2023-12-02T00:30:13.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.867+0000 7ff82afd0640 1 -- 172.21.15.46:0/1967329839 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8241e46f0 con 0x7ff8240ceb70 2023-12-02T00:30:13.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.868+0000 7ff82afd0640 1 -- 172.21.15.46:0/1967329839 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8241556d0 con 0x7ff8240ceb70 2023-12-02T00:30:13.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.868+0000 7ff8217fa640 1 -- 172.21.15.46:0/1967329839 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff8100073d0 con 0x7ff8240ceb70 2023-12-02T00:30:13.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.868+0000 7ff8217fa640 1 --2- 172.21.15.46:0/1967329839 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff80806ae40 0x7ff80806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:13.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.868+0000 7ff8217fa640 1 -- 172.21.15.46:0/1967329839 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7ff810088be0 con 0x7ff8240ceb70 2023-12-02T00:30:13.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.869+0000 7ff828d6c640 1 --2- 172.21.15.46:0/1967329839 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff80806ae40 0x7ff80806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:13.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.869+0000 7ff828d6c640 1 --2- 172.21.15.46:0/1967329839 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff80806ae40 0x7ff80806d300 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff818006fd0 tx=0x7ff818008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:13.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:13.872+0000 7ff8217fa640 1 -- 172.21.15.46:0/1967329839 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff810053bb0 con 0x7ff8240ceb70 2023-12-02T00:30:14.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.051+0000 7ff82afd0640 1 -- 172.21.15.46:0/1967329839 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff824069400 con 0x7ff8240ceb70 2023-12-02T00:30:14.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.051+0000 7ff8217fa640 1 -- 172.21.15.46:0/1967329839 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v38) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff8100571d0 con 0x7ff8240ceb70 2023-12-02T00:30:14.053 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-12-02T00:30:14.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.053+0000 7ff82afd0640 1 -- 172.21.15.46:0/1967329839 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff80806ae40 msgr2=0x7ff80806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:14.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.053+0000 7ff82afd0640 1 --2- 172.21.15.46:0/1967329839 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff80806ae40 0x7ff80806d300 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff818006fd0 tx=0x7ff818008040 comp rx=0 tx=0).stop 2023-12-02T00:30:14.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.053+0000 7ff82afd0640 1 -- 172.21.15.46:0/1967329839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8240ceb70 msgr2=0x7ff8241f65c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:14.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.053+0000 7ff82afd0640 1 --2- 172.21.15.46:0/1967329839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8240ceb70 0x7ff8241f65c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ff81000c4a0 tx=0x7ff81000c960 comp rx=0 tx=0).stop 2023-12-02T00:30:14.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.054+0000 7ff82afd0640 1 -- 172.21.15.46:0/1967329839 shutdown_connections 2023-12-02T00:30:14.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.054+0000 7ff82afd0640 1 --2- 172.21.15.46:0/1967329839 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff80806ae40 0x7ff80806d300 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:14.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.054+0000 7ff82afd0640 1 --2- 172.21.15.46:0/1967329839 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff824144980 0x7ff8241544d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:14.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.054+0000 7ff82afd0640 1 --2- 172.21.15.46:0/1967329839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8240ceb70 0x7ff8241f65c0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:14.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.054+0000 7ff82afd0640 1 --2- 172.21.15.46:0/1967329839 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8240b4370 0x7ff8240c8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:14.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.054+0000 7ff82afd0640 1 -- 172.21.15.46:0/1967329839 >> 172.21.15.46:0/1967329839 conn(0x7ff82406a4d0 msgr2=0x7ff82415b670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:14.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.054+0000 7ff82afd0640 1 -- 172.21.15.46:0/1967329839 shutdown_connections 2023-12-02T00:30:14.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.054+0000 7ff82afd0640 1 -- 172.21.15.46:0/1967329839 wait complete. 2023-12-02T00:30:14.065 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-02T00:30:14.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.174+0000 7f6fe27b0640 1 Processor -- start 2023-12-02T00:30:14.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.174+0000 7f6fe27b0640 1 -- start start 2023-12-02T00:30:14.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.175+0000 7f6fe27b0640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6fdc15e730 0x7f6fdc15abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:14.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.175+0000 7f6fe27b0640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fdc15e2a0 con 0x7f6fdc143490 2023-12-02T00:30:14.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.175+0000 7f6fe27b0640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fdc15b240 con 0x7f6fdc0b1400 2023-12-02T00:30:14.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.175+0000 7f6fe27b0640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fdc15b520 con 0x7f6fdc15e730 2023-12-02T00:30:14.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.175+0000 7f6fe0d4d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6fdc15e730 0x7f6fdc15abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:14.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.175+0000 7f6fdbfff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6fdc143490 0x7f6fdc0b0cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59134/0 (socket says 172.21.15.46:59134) 2023-12-02T00:30:14.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.175+0000 7f6fdbfff640 1 -- 172.21.15.46:0/2291359878 learned_addr learned my addr 172.21.15.46:0/2291359878 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:30:14.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.175+0000 7f6fe0d4d640 1 -- 172.21.15.46:0/2291359878 >> v1:172.21.15.162:6789/0 conn(0x7f6fdc0b1400 legacy=0x7f6fdc15a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:14.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.175+0000 7f6fe0d4d640 1 -- 172.21.15.46:0/2291359878 >> v1:172.21.15.46:6789/0 conn(0x7f6fdc143490 legacy=0x7f6fdc0b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:14.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.175+0000 7f6fdaffd640 1 -- 172.21.15.46:0/2291359878 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4201537912 0 0) 0x7f6fdc15e2a0 con 0x7f6fdc143490 2023-12-02T00:30:14.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.175+0000 7f6fe0d4d640 1 -- 172.21.15.46:0/2291359878 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fdc15dc50 con 0x7f6fdc15e730 2023-12-02T00:30:14.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.176+0000 7f6fe0d4d640 1 --2- 172.21.15.46:0/2291359878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fdc15e730 0x7f6fdc15abf0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4009d90 tx=0x7f6fd402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41ac0c1285ea3ecf server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:14.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.176+0000 7f6fdaffd640 1 -- 172.21.15.46:0/2291359878 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd402cc60 con 0x7f6fdc15e730 2023-12-02T00:30:14.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.176+0000 7f6fdaffd640 1 -- 172.21.15.46:0/2291359878 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fd402cdc0 con 0x7f6fdc15e730 2023-12-02T00:30:14.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.176+0000 7f6fdaffd640 1 -- 172.21.15.46:0/2291359878 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd4034660 con 0x7f6fdc15e730 2023-12-02T00:30:14.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.176+0000 7f6fe27b0640 1 -- 172.21.15.46:0/2291359878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fdc15e730 msgr2=0x7f6fdc15abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:14.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.176+0000 7f6fe27b0640 1 --2- 172.21.15.46:0/2291359878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fdc15e730 0x7f6fdc15abf0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4009d90 tx=0x7f6fd402beb0 comp rx=0 tx=0).stop 2023-12-02T00:30:14.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.177+0000 7f6fe27b0640 1 -- 172.21.15.46:0/2291359878 shutdown_connections 2023-12-02T00:30:14.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.177+0000 7f6fe27b0640 1 --2- 172.21.15.46:0/2291359878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fdc15e730 0x7f6fdc15abf0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:14.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.177+0000 7f6fe27b0640 1 -- 172.21.15.46:0/2291359878 >> 172.21.15.46:0/2291359878 conn(0x7f6fdc06a4d0 msgr2=0x7f6fdc06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:14.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.177+0000 7f6fe27b0640 1 -- 172.21.15.46:0/2291359878 shutdown_connections 2023-12-02T00:30:14.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.177+0000 7f6fe27b0640 1 -- 172.21.15.46:0/2291359878 wait complete. 2023-12-02T00:30:14.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.178+0000 7f6fe27b0640 1 Processor -- start 2023-12-02T00:30:14.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.178+0000 7f6fe27b0640 1 -- start start 2023-12-02T00:30:14.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.179+0000 7f6fe27b0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6fdc0b1400 0x7f6fdc0b0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:14.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.179+0000 7f6fe27b0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6fdc143490 0x7f6fdc154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:14.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.179+0000 7f6fe27b0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fdc15e730 0x7f6fdc0ce5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:14.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.179+0000 7f6fe27b0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fdc0b0e70 con 0x7f6fdc15e730 2023-12-02T00:30:14.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.179+0000 7f6fe27b0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fdc153c30 con 0x7f6fdc0b1400 2023-12-02T00:30:14.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.179+0000 7f6fe27b0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fdc0b0c40 con 0x7f6fdc143490 2023-12-02T00:30:14.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.179+0000 7f6fe0d4d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fdc15e730 0x7f6fdc0ce5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:14.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.179+0000 7f6fe0d4d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fdc15e730 0x7f6fdc0ce5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56942/0 (socket says 172.21.15.46:56942) 2023-12-02T00:30:14.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.179+0000 7f6fe0d4d640 1 -- 172.21.15.46:0/697736352 learned_addr learned my addr 172.21.15.46:0/697736352 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:14.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.179+0000 7f6fdbfff640 1 --2- 172.21.15.46:0/697736352 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6fdc0b1400 0x7f6fdc0b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:14.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.179+0000 7f6fe0d4d640 1 -- 172.21.15.46:0/697736352 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6fdc143490 msgr2=0x7f6fdc154050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:14.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.179+0000 7f6fdb7fe640 1 --2- 172.21.15.46:0/697736352 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6fdc143490 0x7f6fdc154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:14.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.179+0000 7f6fe0d4d640 1 --2- 172.21.15.46:0/697736352 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6fdc143490 0x7f6fdc154050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:14.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.179+0000 7f6fe0d4d640 1 -- 172.21.15.46:0/697736352 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6fdc0b1400 msgr2=0x7f6fdc0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:14.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.179+0000 7f6fe0d4d640 1 --2- 172.21.15.46:0/697736352 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6fdc0b1400 0x7f6fdc0b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:14.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.179+0000 7f6fe0d4d640 1 -- 172.21.15.46:0/697736352 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd4009a70 con 0x7f6fdc15e730 2023-12-02T00:30:14.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.179+0000 7f6fdbfff640 1 --2- 172.21.15.46:0/697736352 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6fdc0b1400 0x7f6fdc0b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:30:14.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.179+0000 7f6fe0d4d640 1 --2- 172.21.15.46:0/697736352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fdc15e730 0x7f6fdc0ce5d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4009eb0 tx=0x7f6fd4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:14.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.179+0000 7f6fd8ff9640 1 -- 172.21.15.46:0/697736352 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd403e610 con 0x7f6fdc15e730 2023-12-02T00:30:14.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.180+0000 7f6fd8ff9640 1 -- 172.21.15.46:0/697736352 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fd403e790 con 0x7f6fdc15e730 2023-12-02T00:30:14.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.180+0000 7f6fe27b0640 1 -- 172.21.15.46:0/697736352 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fdc13bb70 con 0x7f6fdc15e730 2023-12-02T00:30:14.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.180+0000 7f6fd8ff9640 1 -- 172.21.15.46:0/697736352 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd403eb40 con 0x7f6fdc15e730 2023-12-02T00:30:14.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.180+0000 7f6fe27b0640 1 -- 172.21.15.46:0/697736352 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fdc13b890 con 0x7f6fdc15e730 2023-12-02T00:30:14.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.180+0000 7f6fe27b0640 1 -- 172.21.15.46:0/697736352 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fdc153d70 con 0x7f6fdc15e730 2023-12-02T00:30:14.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.181+0000 7f6fd8ff9640 1 -- 172.21.15.46:0/697736352 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6fd403b050 con 0x7f6fdc15e730 2023-12-02T00:30:14.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.181+0000 7f6fd8ff9640 1 --2- 172.21.15.46:0/697736352 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f6fbc06ae40 0x7f6fbc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:14.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.181+0000 7f6fd8ff9640 1 -- 172.21.15.46:0/697736352 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f6fd40ac1a0 con 0x7f6fdc15e730 2023-12-02T00:30:14.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.182+0000 7f6fdbfff640 1 --2- 172.21.15.46:0/697736352 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f6fbc06ae40 0x7f6fbc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:14.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.182+0000 7f6fdbfff640 1 --2- 172.21.15.46:0/697736352 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f6fbc06ae40 0x7f6fbc06d300 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4000f60 tx=0x7f6fc4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:14.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.185+0000 7f6fd8ff9640 1 -- 172.21.15.46:0/697736352 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6fdc153d70 con 0x7f6fdc15e730 2023-12-02T00:30:14.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.360+0000 7f6fe27b0640 1 -- 172.21.15.46:0/697736352 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6fdc153d70 con 0x7f6fdc15e730 2023-12-02T00:30:14.362 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.360+0000 7f6fd8ff9640 1 -- 172.21.15.46:0/697736352 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v38) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6fdc153d70 con 0x7f6fdc15e730 2023-12-02T00:30:14.362 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-12-02T00:30:14.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.362+0000 7f6fe27b0640 1 -- 172.21.15.46:0/697736352 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f6fbc06ae40 msgr2=0x7f6fbc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:14.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.362+0000 7f6fe27b0640 1 --2- 172.21.15.46:0/697736352 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f6fbc06ae40 0x7f6fbc06d300 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4000f60 tx=0x7f6fc4009040 comp rx=0 tx=0).stop 2023-12-02T00:30:14.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.362+0000 7f6fe27b0640 1 -- 172.21.15.46:0/697736352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fdc15e730 msgr2=0x7f6fdc0ce5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:14.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.362+0000 7f6fe27b0640 1 --2- 172.21.15.46:0/697736352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fdc15e730 0x7f6fdc0ce5d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4009eb0 tx=0x7f6fd4004a40 comp rx=0 tx=0).stop 2023-12-02T00:30:14.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.362+0000 7f6fe27b0640 1 -- 172.21.15.46:0/697736352 shutdown_connections 2023-12-02T00:30:14.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.362+0000 7f6fe27b0640 1 --2- 172.21.15.46:0/697736352 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f6fbc06ae40 0x7f6fbc06d300 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:14.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.363+0000 7f6fe27b0640 1 --2- 172.21.15.46:0/697736352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fdc15e730 0x7f6fdc0ce5d0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:14.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.363+0000 7f6fe27b0640 1 --2- 172.21.15.46:0/697736352 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6fdc143490 0x7f6fdc154050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:14.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.363+0000 7f6fe27b0640 1 --2- 172.21.15.46:0/697736352 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6fdc0b1400 0x7f6fdc0b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:14.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.363+0000 7f6fe27b0640 1 -- 172.21.15.46:0/697736352 >> 172.21.15.46:0/697736352 conn(0x7f6fdc06a4d0 msgr2=0x7f6fdc06c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:14.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.363+0000 7f6fe27b0640 1 -- 172.21.15.46:0/697736352 shutdown_connections 2023-12-02T00:30:14.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.363+0000 7f6fe27b0640 1 -- 172.21.15.46:0/697736352 wait complete. 2023-12-02T00:30:14.375 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2023-12-02T00:30:14.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.484+0000 7f865ea10640 1 Processor -- start 2023-12-02T00:30:14.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.484+0000 7f865ea10640 1 -- start start 2023-12-02T00:30:14.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.484+0000 7f865ea10640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f865806ea50 0x7f865806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:14.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.484+0000 7f865ea10640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8658160fb0 0x7f8658161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:14.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.484+0000 7f865ea10640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8658161d20 con 0x7f86580b3ff0 2023-12-02T00:30:14.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.484+0000 7f865ea10640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86580d1ee0 con 0x7f8658160fb0 2023-12-02T00:30:14.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.484+0000 7f865ea10640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f865806f370 con 0x7f865806ea50 2023-12-02T00:30:14.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.485+0000 7f86577fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8658160fb0 0x7f8658161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:14.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.485+0000 7f865cfad640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f86580b3ff0 0x7f86580b4460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59140/0 (socket says 172.21.15.46:59140) 2023-12-02T00:30:14.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.485+0000 7f86577fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8658160fb0 0x7f8658161410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56958/0 (socket says 172.21.15.46:56958) 2023-12-02T00:30:14.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.485+0000 7f865cfad640 1 -- 172.21.15.46:0/2781964238 learned_addr learned my addr 172.21.15.46:0/2781964238 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:30:14.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.485+0000 7f8657fff640 1 --2- 172.21.15.46:0/2781964238 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f865806ea50 0x7f865806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:14.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.485+0000 7f86577fe640 1 -- 172.21.15.46:0/2781964238 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f865806ea50 msgr2=0x7f865806ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:14.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.485+0000 7f86577fe640 1 --2- 172.21.15.46:0/2781964238 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f865806ea50 0x7f865806ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:14.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.485+0000 7f86577fe640 1 -- 172.21.15.46:0/2781964238 >> v1:172.21.15.46:6789/0 conn(0x7f86580b3ff0 legacy=0x7f86580b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:14.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.485+0000 7f86577fe640 1 -- 172.21.15.46:0/2781964238 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f865815e760 con 0x7f8658160fb0 2023-12-02T00:30:14.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.485+0000 7f8656ffd640 1 -- 172.21.15.46:0/2781964238 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2871924799 0 0) 0x7f8658161d20 con 0x7f86580b3ff0 2023-12-02T00:30:14.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.485+0000 7f8657fff640 1 --2- 172.21.15.46:0/2781964238 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f865806ea50 0x7f865806ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:30:14.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.486+0000 7f86577fe640 1 --2- 172.21.15.46:0/2781964238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8658160fb0 0x7f8658161410 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f864c0097a0 tx=0x7f864c02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ea1a4bcd207192d server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:14.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.486+0000 7f8656ffd640 1 -- 172.21.15.46:0/2781964238 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864c031780 con 0x7f8658160fb0 2023-12-02T00:30:14.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.486+0000 7f8656ffd640 1 -- 172.21.15.46:0/2781964238 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f864c0318e0 con 0x7f8658160fb0 2023-12-02T00:30:14.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.486+0000 7f8656ffd640 1 -- 172.21.15.46:0/2781964238 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864c031c70 con 0x7f8658160fb0 2023-12-02T00:30:14.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.486+0000 7f865ea10640 1 -- 172.21.15.46:0/2781964238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8658160fb0 msgr2=0x7f8658161410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:14.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.486+0000 7f865ea10640 1 --2- 172.21.15.46:0/2781964238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8658160fb0 0x7f8658161410 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f864c0097a0 tx=0x7f864c02b820 comp rx=0 tx=0).stop 2023-12-02T00:30:14.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.486+0000 7f865ea10640 1 -- 172.21.15.46:0/2781964238 shutdown_connections 2023-12-02T00:30:14.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.486+0000 7f865ea10640 1 --2- 172.21.15.46:0/2781964238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8658160fb0 0x7f8658161410 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:14.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.486+0000 7f865ea10640 1 --2- 172.21.15.46:0/2781964238 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f865806ea50 0x7f865806ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:14.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.486+0000 7f865ea10640 1 -- 172.21.15.46:0/2781964238 >> 172.21.15.46:0/2781964238 conn(0x7f865806a6b0 msgr2=0x7f865806aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:14.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.486+0000 7f865ea10640 1 -- 172.21.15.46:0/2781964238 shutdown_connections 2023-12-02T00:30:14.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.486+0000 7f865ea10640 1 -- 172.21.15.46:0/2781964238 wait complete. 2023-12-02T00:30:14.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.488+0000 7f865ea10640 1 Processor -- start 2023-12-02T00:30:14.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.488+0000 7f865ea10640 1 -- start start 2023-12-02T00:30:14.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.488+0000 7f865ea10640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f865806ea50 0x7f8658157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:14.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.488+0000 7f865ea10640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86580b3ff0 0x7f8658140a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:14.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.488+0000 7f865ea10640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8658160fb0 0x7f8658140f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:14.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.488+0000 7f865ea10640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865815a620 con 0x7f86580b3ff0 2023-12-02T00:30:14.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.488+0000 7f865ea10640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865815a870 con 0x7f8658160fb0 2023-12-02T00:30:14.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.488+0000 7f865ea10640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f865815ab50 con 0x7f865806ea50 2023-12-02T00:30:14.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.489+0000 7f86577fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86580b3ff0 0x7f8658140a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:14.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.489+0000 7f86577fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86580b3ff0 0x7f8658140a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56970/0 (socket says 172.21.15.46:56970) 2023-12-02T00:30:14.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.489+0000 7f86577fe640 1 -- 172.21.15.46:0/2361830108 learned_addr learned my addr 172.21.15.46:0/2361830108 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:14.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.489+0000 7f8657fff640 1 --2- 172.21.15.46:0/2361830108 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f865806ea50 0x7f8658157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:14.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.489+0000 7f865cfad640 1 --2- 172.21.15.46:0/2361830108 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8658160fb0 0x7f8658140f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:14.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.489+0000 7f86577fe640 1 -- 172.21.15.46:0/2361830108 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f865806ea50 msgr2=0x7f8658157ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:14.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.489+0000 7f86577fe640 1 --2- 172.21.15.46:0/2361830108 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f865806ea50 0x7f8658157ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:14.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.489+0000 7f86577fe640 1 -- 172.21.15.46:0/2361830108 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8658160fb0 msgr2=0x7f8658140f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:14.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.489+0000 7f86577fe640 1 --2- 172.21.15.46:0/2361830108 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8658160fb0 0x7f8658140f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:14.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.489+0000 7f86577fe640 1 -- 172.21.15.46:0/2361830108 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f864c009400 con 0x7f86580b3ff0 2023-12-02T00:30:14.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.489+0000 7f865cfad640 1 --2- 172.21.15.46:0/2361830108 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8658160fb0 0x7f8658140f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:30:14.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.489+0000 7f8657fff640 1 --2- 172.21.15.46:0/2361830108 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f865806ea50 0x7f8658157ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:30:14.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.489+0000 7f86577fe640 1 --2- 172.21.15.46:0/2361830108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86580b3ff0 0x7f8658140a50 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f864c0098c0 tx=0x7f864c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:14.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.489+0000 7f8654ff9640 1 -- 172.21.15.46:0/2361830108 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864c032820 con 0x7f86580b3ff0 2023-12-02T00:30:14.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.489+0000 7f8654ff9640 1 -- 172.21.15.46:0/2361830108 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f864c0329a0 con 0x7f86580b3ff0 2023-12-02T00:30:14.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.489+0000 7f865ea10640 1 -- 172.21.15.46:0/2361830108 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f865815ae30 con 0x7f86580b3ff0 2023-12-02T00:30:14.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.489+0000 7f8654ff9640 1 -- 172.21.15.46:0/2361830108 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864c032d50 con 0x7f86580b3ff0 2023-12-02T00:30:14.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.489+0000 7f865ea10640 1 -- 172.21.15.46:0/2361830108 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f865813de00 con 0x7f86580b3ff0 2023-12-02T00:30:14.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.490+0000 7f865ea10640 1 -- 172.21.15.46:0/2361830108 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86581f3570 con 0x7f86580b3ff0 2023-12-02T00:30:14.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.491+0000 7f8654ff9640 1 -- 172.21.15.46:0/2361830108 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f864c04d3f0 con 0x7f86580b3ff0 2023-12-02T00:30:14.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.491+0000 7f8654ff9640 1 --2- 172.21.15.46:0/2361830108 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f864406ae70 0x7f864406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:14.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.491+0000 7f8654ff9640 1 -- 172.21.15.46:0/2361830108 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f864c0ab400 con 0x7f86580b3ff0 2023-12-02T00:30:14.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.491+0000 7f8657fff640 1 --2- 172.21.15.46:0/2361830108 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f864406ae70 0x7f864406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:14.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.492+0000 7f8657fff640 1 --2- 172.21.15.46:0/2361830108 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f864406ae70 0x7f864406d330 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8648006fd0 tx=0x7f8648008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:14.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.495+0000 7f8654ff9640 1 -- 172.21.15.46:0/2361830108 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f86581f3570 con 0x7f86580b3ff0 2023-12-02T00:30:14.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.675+0000 7f865ea10640 1 -- 172.21.15.46:0/2361830108 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7f86581f3570 con 0x7f86580b3ff0 2023-12-02T00:30:14.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.675+0000 7f8654ff9640 1 -- 172.21.15.46:0/2361830108 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v38) v1 ==== 97+0+11 (secure 0 0 0) 0x7f86581f3570 con 0x7f86580b3ff0 2023-12-02T00:30:14.676 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-12-02T00:30:14.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.677+0000 7f865ea10640 1 -- 172.21.15.46:0/2361830108 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f864406ae70 msgr2=0x7f864406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:14.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.677+0000 7f865ea10640 1 --2- 172.21.15.46:0/2361830108 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f864406ae70 0x7f864406d330 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8648006fd0 tx=0x7f8648008040 comp rx=0 tx=0).stop 2023-12-02T00:30:14.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.677+0000 7f865ea10640 1 -- 172.21.15.46:0/2361830108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86580b3ff0 msgr2=0x7f8658140a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:14.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.677+0000 7f865ea10640 1 --2- 172.21.15.46:0/2361830108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86580b3ff0 0x7f8658140a50 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f864c0098c0 tx=0x7f864c0046c0 comp rx=0 tx=0).stop 2023-12-02T00:30:14.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.677+0000 7f865ea10640 1 -- 172.21.15.46:0/2361830108 shutdown_connections 2023-12-02T00:30:14.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.677+0000 7f865ea10640 1 --2- 172.21.15.46:0/2361830108 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f864406ae70 0x7f864406d330 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:14.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.677+0000 7f865ea10640 1 --2- 172.21.15.46:0/2361830108 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8658160fb0 0x7f8658140f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:14.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.677+0000 7f865ea10640 1 --2- 172.21.15.46:0/2361830108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86580b3ff0 0x7f8658140a50 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:14.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.677+0000 7f865ea10640 1 --2- 172.21.15.46:0/2361830108 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f865806ea50 0x7f8658157ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:14.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.677+0000 7f865ea10640 1 -- 172.21.15.46:0/2361830108 >> 172.21.15.46:0/2361830108 conn(0x7f865806a6b0 msgr2=0x7f865815b490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:14.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.677+0000 7f865ea10640 1 -- 172.21.15.46:0/2361830108 shutdown_connections 2023-12-02T00:30:14.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.677+0000 7f865ea10640 1 -- 172.21.15.46:0/2361830108 wait complete. 2023-12-02T00:30:14.689 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-02T00:30:14.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.799+0000 7f7c7b9d0640 1 Processor -- start 2023-12-02T00:30:14.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.799+0000 7f7c7b9d0640 1 -- start start 2023-12-02T00:30:14.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.800+0000 7f7c7b9d0640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7c740b4370 0x7f7c740b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:14.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.800+0000 7f7c7b9d0640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c74071770 con 0x7f7c740ceb70 2023-12-02T00:30:14.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.800+0000 7f7c7b9d0640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c740ccb60 con 0x7f7c74144980 2023-12-02T00:30:14.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.800+0000 7f7c7b9d0640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c7415ced0 con 0x7f7c740b4370 2023-12-02T00:30:14.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.800+0000 7f7c78f6b640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7c740ceb70 0x7f7c740cc450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59144/0 (socket says 172.21.15.46:59144) 2023-12-02T00:30:14.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.800+0000 7f7c78f6b640 1 -- 172.21.15.46:0/2343963966 learned_addr learned my addr 172.21.15.46:0/2343963966 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:30:14.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.800+0000 7f7c79f6d640 1 --2- 172.21.15.46:0/2343963966 >> v2:172.21.15.162:3300/0 conn(0x7f7c740b4370 0x7f7c740b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:14.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.800+0000 7f7c73fff640 1 -- 172.21.15.46:0/2343963966 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1657670250 0 0) 0x7f7c74071770 con 0x7f7c740ceb70 2023-12-02T00:30:14.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.800+0000 7f7c73fff640 1 -- 172.21.15.46:0/2343963966 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c5c003520 con 0x7f7c740ceb70 2023-12-02T00:30:14.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.801+0000 7f7c73fff640 1 -- 172.21.15.46:0/2343963966 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2995298527 0 0) 0x7f7c5c003520 con 0x7f7c740ceb70 2023-12-02T00:30:14.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.801+0000 7f7c73fff640 1 -- 172.21.15.46:0/2343963966 >> v2:172.21.15.162:3300/0 conn(0x7f7c740b4370 msgr2=0x7f7c740b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:14.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.801+0000 7f7c73fff640 1 --2- 172.21.15.46:0/2343963966 >> v2:172.21.15.162:3300/0 conn(0x7f7c740b4370 0x7f7c740b47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:14.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.801+0000 7f7c73fff640 1 -- 172.21.15.46:0/2343963966 >> v1:172.21.15.162:6789/0 conn(0x7f7c74144980 legacy=0x7f7c7415c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:14.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.801+0000 7f7c73fff640 1 -- 172.21.15.46:0/2343963966 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c740b4ce0 con 0x7f7c740ceb70 2023-12-02T00:30:14.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.801+0000 7f7c73fff640 1 -- 172.21.15.46:0/2343963966 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7f7c68001670 con 0x7f7c740ceb70 2023-12-02T00:30:14.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.801+0000 7f7c73fff640 1 -- 172.21.15.46:0/2343963966 >> v1:172.21.15.46:6789/0 conn(0x7f7c740ceb70 legacy=0x7f7c740cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:14.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.801+0000 7f7c73fff640 1 --2- 172.21.15.46:0/2343963966 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c5c003970 0x7f7c5c003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:14.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.801+0000 7f7c73fff640 1 --2- 172.21.15.46:0/2343963966 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c5c0047c0 0x7f7c5c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:14.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.801+0000 7f7c73fff640 1 --2- 172.21.15.46:0/2343963966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c5c005180 0x7f7c5c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:14.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.801+0000 7f7c73fff640 1 -- 172.21.15.46:0/2343963966 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7415ced0 con 0x7f7c5c005180 2023-12-02T00:30:14.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.801+0000 7f7c73fff640 1 -- 172.21.15.46:0/2343963966 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5c00c050 con 0x7f7c5c0047c0 2023-12-02T00:30:14.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.801+0000 7f7c73fff640 1 -- 172.21.15.46:0/2343963966 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5c00c210 con 0x7f7c5c003970 2023-12-02T00:30:14.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.801+0000 7f7c79f6d640 1 --2- 172.21.15.46:0/2343963966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c5c005180 0x7f7c5c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:14.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.801+0000 7f7c78f6b640 1 --2- 172.21.15.46:0/2343963966 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c5c0047c0 0x7f7c5c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:14.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.801+0000 7f7c7976c640 1 --2- 172.21.15.46:0/2343963966 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c5c003970 0x7f7c5c003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:14.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.801+0000 7f7c79f6d640 1 -- 172.21.15.46:0/2343963966 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c5c003970 msgr2=0x7f7c5c003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:14.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.801+0000 7f7c79f6d640 1 --2- 172.21.15.46:0/2343963966 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c5c003970 0x7f7c5c003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:14.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.801+0000 7f7c79f6d640 1 -- 172.21.15.46:0/2343963966 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c5c0047c0 msgr2=0x7f7c5c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:14.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.801+0000 7f7c79f6d640 1 --2- 172.21.15.46:0/2343963966 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c5c0047c0 0x7f7c5c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:14.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.801+0000 7f7c79f6d640 1 -- 172.21.15.46:0/2343963966 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c5c003520 con 0x7f7c5c005180 2023-12-02T00:30:14.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.801+0000 7f7c7976c640 1 --2- 172.21.15.46:0/2343963966 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c5c003970 0x7f7c5c003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:30:14.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.802+0000 7f7c79f6d640 1 -- 172.21.15.46:0/2343963966 reap_dead start 2023-12-02T00:30:14.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.802+0000 7f7c79f6d640 1 --2- 172.21.15.46:0/2343963966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c5c005180 0x7f7c5c00b980 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7c64009fc0 tx=0x7f7c5c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bbf2712f640b4aa server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:14.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.802+0000 7f7c73fff640 1 -- 172.21.15.46:0/2343963966 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6402cc60 con 0x7f7c5c005180 2023-12-02T00:30:14.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.802+0000 7f7c73fff640 1 -- 172.21.15.46:0/2343963966 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c6400f040 con 0x7f7c5c005180 2023-12-02T00:30:14.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.802+0000 7f7c73fff640 1 -- 172.21.15.46:0/2343963966 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c64003990 con 0x7f7c5c005180 2023-12-02T00:30:14.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.802+0000 7f7c7b9d0640 1 -- 172.21.15.46:0/2343963966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c5c005180 msgr2=0x7f7c5c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:14.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.802+0000 7f7c7b9d0640 1 --2- 172.21.15.46:0/2343963966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c5c005180 0x7f7c5c00b980 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7c64009fc0 tx=0x7f7c5c0057f0 comp rx=0 tx=0).stop 2023-12-02T00:30:14.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.803+0000 7f7c7b9d0640 1 -- 172.21.15.46:0/2343963966 shutdown_connections 2023-12-02T00:30:14.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.803+0000 7f7c7b9d0640 1 --2- 172.21.15.46:0/2343963966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c5c005180 0x7f7c5c00b980 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:14.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.803+0000 7f7c7b9d0640 1 -- 172.21.15.46:0/2343963966 >> 172.21.15.46:0/2343963966 conn(0x7f7c7406a4d0 msgr2=0x7f7c7406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:14.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.803+0000 7f7c7b9d0640 1 -- 172.21.15.46:0/2343963966 shutdown_connections 2023-12-02T00:30:14.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.803+0000 7f7c7b9d0640 1 -- 172.21.15.46:0/2343963966 wait complete. 2023-12-02T00:30:14.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.804+0000 7f7c7b9d0640 1 Processor -- start 2023-12-02T00:30:14.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.804+0000 7f7c7b9d0640 1 -- start start 2023-12-02T00:30:14.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.805+0000 7f7c7b9d0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c5c005180 0x7f7c7413dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:14.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.805+0000 7f7c7b9d0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c7413e510 0x7f7c741e3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:14.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.805+0000 7f7c7b9d0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c741e5280 0x7f7c741f0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:14.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.805+0000 7f7c7b9d0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7413e1e0 con 0x7f7c741e5280 2023-12-02T00:30:14.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.805+0000 7f7c7b9d0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c741e41e0 con 0x7f7c5c005180 2023-12-02T00:30:14.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.805+0000 7f7c7b9d0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c741e44c0 con 0x7f7c7413e510 2023-12-02T00:30:14.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.805+0000 7f7c79f6d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c741e5280 0x7f7c741f0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:14.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.805+0000 7f7c79f6d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c741e5280 0x7f7c741f0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56994/0 (socket says 172.21.15.46:56994) 2023-12-02T00:30:14.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.805+0000 7f7c79f6d640 1 -- 172.21.15.46:0/449567060 learned_addr learned my addr 172.21.15.46:0/449567060 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:14.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.805+0000 7f7c7976c640 1 --2- 172.21.15.46:0/449567060 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c5c005180 0x7f7c7413dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:14.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.805+0000 7f7c78f6b640 1 --2- 172.21.15.46:0/449567060 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c7413e510 0x7f7c741e3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:14.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.805+0000 7f7c79f6d640 1 -- 172.21.15.46:0/449567060 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c7413e510 msgr2=0x7f7c741e3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:14.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.805+0000 7f7c79f6d640 1 --2- 172.21.15.46:0/449567060 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c7413e510 0x7f7c741e3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:14.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.805+0000 7f7c79f6d640 1 -- 172.21.15.46:0/449567060 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c5c005180 msgr2=0x7f7c7413dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:14.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.805+0000 7f7c79f6d640 1 --2- 172.21.15.46:0/449567060 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c5c005180 0x7f7c7413dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:14.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.805+0000 7f7c79f6d640 1 -- 172.21.15.46:0/449567060 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c741e47a0 con 0x7f7c741e5280 2023-12-02T00:30:14.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.805+0000 7f7c78f6b640 1 --2- 172.21.15.46:0/449567060 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c7413e510 0x7f7c741e3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:30:14.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.805+0000 7f7c79f6d640 1 --2- 172.21.15.46:0/449567060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c741e5280 0x7f7c741f0b40 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7c6402c6d0 tx=0x7f7c64009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:14.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.805+0000 7f7c71ffb640 1 -- 172.21.15.46:0/449567060 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c64035dc0 con 0x7f7c741e5280 2023-12-02T00:30:14.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.805+0000 7f7c71ffb640 1 -- 172.21.15.46:0/449567060 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c64003990 con 0x7f7c741e5280 2023-12-02T00:30:14.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.805+0000 7f7c7b9d0640 1 -- 172.21.15.46:0/449567060 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c741e4a80 con 0x7f7c741e5280 2023-12-02T00:30:14.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.805+0000 7f7c71ffb640 1 -- 172.21.15.46:0/449567060 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c64003d80 con 0x7f7c741e5280 2023-12-02T00:30:14.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.806+0000 7f7c7b9d0640 1 -- 172.21.15.46:0/449567060 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c741e3dd0 con 0x7f7c741e5280 2023-12-02T00:30:14.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.806+0000 7f7c7b9d0640 1 -- 172.21.15.46:0/449567060 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c741e4fa0 con 0x7f7c741e5280 2023-12-02T00:30:14.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.807+0000 7f7c71ffb640 1 -- 172.21.15.46:0/449567060 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7c64004030 con 0x7f7c741e5280 2023-12-02T00:30:14.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.807+0000 7f7c71ffb640 1 --2- 172.21.15.46:0/449567060 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7c6006ae40 0x7f7c6006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:14.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.808+0000 7f7c71ffb640 1 -- 172.21.15.46:0/449567060 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f7c640ac770 con 0x7f7c741e5280 2023-12-02T00:30:14.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.808+0000 7f7c7976c640 1 --2- 172.21.15.46:0/449567060 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7c6006ae40 0x7f7c6006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:14.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.808+0000 7f7c7976c640 1 --2- 172.21.15.46:0/449567060 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7c6006ae40 0x7f7c6006d300 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c000c00 tx=0x7f7c6c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:14.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:14.812+0000 7f7c71ffb640 1 -- 172.21.15.46:0/449567060 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7c741e4fa0 con 0x7f7c741e5280 2023-12-02T00:30:15.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.003+0000 7f7c7b9d0640 1 -- 172.21.15.46:0/449567060 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7c741e4fa0 con 0x7f7c741e5280 2023-12-02T00:30:15.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.004+0000 7f7c71ffb640 1 -- 172.21.15.46:0/449567060 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v38) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7c741e4fa0 con 0x7f7c741e5280 2023-12-02T00:30:15.005 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2023-12-02T00:30:15.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.005+0000 7f7c7b9d0640 1 -- 172.21.15.46:0/449567060 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7c6006ae40 msgr2=0x7f7c6006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:15.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.005+0000 7f7c7b9d0640 1 --2- 172.21.15.46:0/449567060 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7c6006ae40 0x7f7c6006d300 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c000c00 tx=0x7f7c6c009040 comp rx=0 tx=0).stop 2023-12-02T00:30:15.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.005+0000 7f7c7b9d0640 1 -- 172.21.15.46:0/449567060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c741e5280 msgr2=0x7f7c741f0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:15.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.005+0000 7f7c7b9d0640 1 --2- 172.21.15.46:0/449567060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c741e5280 0x7f7c741f0b40 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7c6402c6d0 tx=0x7f7c64009750 comp rx=0 tx=0).stop 2023-12-02T00:30:15.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.006+0000 7f7c7b9d0640 1 -- 172.21.15.46:0/449567060 shutdown_connections 2023-12-02T00:30:15.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.006+0000 7f7c7b9d0640 1 --2- 172.21.15.46:0/449567060 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7c6006ae40 0x7f7c6006d300 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:15.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.006+0000 7f7c7b9d0640 1 --2- 172.21.15.46:0/449567060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c741e5280 0x7f7c741f0b40 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:15.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.006+0000 7f7c7b9d0640 1 --2- 172.21.15.46:0/449567060 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c7413e510 0x7f7c741e3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:15.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.006+0000 7f7c7b9d0640 1 --2- 172.21.15.46:0/449567060 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c5c005180 0x7f7c7413dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:15.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.006+0000 7f7c7b9d0640 1 -- 172.21.15.46:0/449567060 >> 172.21.15.46:0/449567060 conn(0x7f7c7406a4d0 msgr2=0x7f7c7415a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:15.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.006+0000 7f7c7b9d0640 1 -- 172.21.15.46:0/449567060 shutdown_connections 2023-12-02T00:30:15.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.006+0000 7f7c7b9d0640 1 -- 172.21.15.46:0/449567060 wait complete. 2023-12-02T00:30:15.017 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi046.front.sepia.ceph.com... 2023-12-02T00:30:15.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:30:15.017 DEBUG:teuthology.orchestra.run.smithi046:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-12-02T00:30:15.046 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[104429]: fuse finished with error 0 and tester_r 0 2023-12-02T00:30:15.048 INFO:teuthology.orchestra.run:waiting for 300 2023-12-02T00:30:15.135 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2023-12-02T00:30:15.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:30:15.135 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-02T00:30:15.151 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-02T00:30:15.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.297+0000 7f08e3ae5640 1 Processor -- start 2023-12-02T00:30:15.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.297+0000 7f08e3ae5640 1 -- start start 2023-12-02T00:30:15.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.298+0000 7f08e3ae5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08dc143490 0x7f08dc0b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:15.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.298+0000 7f08e3ae5640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08dc15e2a0 con 0x7f08dc15abf0 2023-12-02T00:30:15.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.298+0000 7f08e3ae5640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08dc15b240 con 0x7f08dc15e730 2023-12-02T00:30:15.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.298+0000 7f08e3ae5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc15b520 con 0x7f08dc143490 2023-12-02T00:30:15.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.298+0000 7f08e2082640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f08dc15abf0 0x7f08dc0b2f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59148/0 (socket says 172.21.15.46:59148) 2023-12-02T00:30:15.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.298+0000 7f08e2082640 1 -- 172.21.15.46:0/3994306426 learned_addr learned my addr 172.21.15.46:0/3994306426 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:30:15.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.298+0000 7f08e1881640 1 --2- 172.21.15.46:0/3994306426 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08dc143490 0x7f08dc0b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:15.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.298+0000 7f08e087f640 1 -- 172.21.15.46:0/3994306426 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2669407125 0 0) 0x7f08dc15e2a0 con 0x7f08dc15abf0 2023-12-02T00:30:15.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.298+0000 7f08e087f640 1 -- 172.21.15.46:0/3994306426 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08c8005730 con 0x7f08dc15abf0 2023-12-02T00:30:15.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.298+0000 7f08e087f640 1 -- 172.21.15.46:0/3994306426 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1424202082 0 0) 0x7f08dc15b240 con 0x7f08dc15e730 2023-12-02T00:30:15.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.298+0000 7f08e087f640 1 -- 172.21.15.46:0/3994306426 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08dc15e2a0 con 0x7f08dc15e730 2023-12-02T00:30:15.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.298+0000 7f08e087f640 1 -- 172.21.15.46:0/3994306426 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3951452093 0 0) 0x7f08c8005730 con 0x7f08dc15abf0 2023-12-02T00:30:15.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.298+0000 7f08e087f640 1 -- 172.21.15.46:0/3994306426 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08dc143490 msgr2=0x7f08dc0b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:15.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.298+0000 7f08e087f640 1 --2- 172.21.15.46:0/3994306426 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08dc143490 0x7f08dc0b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:15.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.298+0000 7f08e087f640 1 -- 172.21.15.46:0/3994306426 >> v1:172.21.15.162:6789/0 conn(0x7f08dc15e730 legacy=0x7f08dc15a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:15.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.298+0000 7f08e087f640 1 -- 172.21.15.46:0/3994306426 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08dc15dc50 con 0x7f08dc15abf0 2023-12-02T00:30:15.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.298+0000 7f08e087f640 1 -- 172.21.15.46:0/3994306426 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7f08d4001970 con 0x7f08dc15abf0 2023-12-02T00:30:15.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.299+0000 7f08e087f640 1 -- 172.21.15.46:0/3994306426 >> v1:172.21.15.46:6789/0 conn(0x7f08dc15abf0 legacy=0x7f08dc0b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:15.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.299+0000 7f08e087f640 1 --2- 172.21.15.46:0/3994306426 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08c8005d40 0x7f08c8006140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:15.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.299+0000 7f08e087f640 1 --2- 172.21.15.46:0/3994306426 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08dc143490 0x7f08c80069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:15.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.299+0000 7f08e087f640 1 --2- 172.21.15.46:0/3994306426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08c8006f10 0x7f08c8007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:15.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.299+0000 7f08e087f640 1 -- 172.21.15.46:0/3994306426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc15b520 con 0x7f08c8006f10 2023-12-02T00:30:15.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.299+0000 7f08e087f640 1 -- 172.21.15.46:0/3994306426 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c800dc40 con 0x7f08c8005d40 2023-12-02T00:30:15.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.299+0000 7f08e087f640 1 -- 172.21.15.46:0/3994306426 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08c800de00 con 0x7f08dc143490 2023-12-02T00:30:15.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.299+0000 7f08e2082640 1 --2- 172.21.15.46:0/3994306426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08c8006f10 0x7f08c8007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:15.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.299+0000 7f08e1881640 1 --2- 172.21.15.46:0/3994306426 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08c8005d40 0x7f08c8006140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:15.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.299+0000 7f08e1080640 1 --2- 172.21.15.46:0/3994306426 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08dc143490 0x7f08c80069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:15.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.299+0000 7f08e2082640 1 -- 172.21.15.46:0/3994306426 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08dc143490 msgr2=0x7f08c80069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:15.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.299+0000 7f08e2082640 1 --2- 172.21.15.46:0/3994306426 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08dc143490 0x7f08c80069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:15.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.299+0000 7f08e2082640 1 -- 172.21.15.46:0/3994306426 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08c8005d40 msgr2=0x7f08c8006140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:15.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.299+0000 7f08e2082640 1 --2- 172.21.15.46:0/3994306426 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08c8005d40 0x7f08c8006140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:15.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.299+0000 7f08e2082640 1 -- 172.21.15.46:0/3994306426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c8005730 con 0x7f08c8006f10 2023-12-02T00:30:15.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.300+0000 7f08e2082640 1 --2- 172.21.15.46:0/3994306426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08c8006f10 0x7f08c8007390 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f08d400a680 tx=0x7f08c8007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=898a6262125aeef4 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:15.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.300+0000 7f08e087f640 1 -- 172.21.15.46:0/3994306426 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d40072d0 con 0x7f08c8006f10 2023-12-02T00:30:15.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.300+0000 7f08e087f640 1 -- 172.21.15.46:0/3994306426 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08d402f070 con 0x7f08c8006f10 2023-12-02T00:30:15.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.300+0000 7f08e087f640 1 -- 172.21.15.46:0/3994306426 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d4007670 con 0x7f08c8006f10 2023-12-02T00:30:15.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.300+0000 7f08e3ae5640 1 -- 172.21.15.46:0/3994306426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08c8006f10 msgr2=0x7f08c8007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:15.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.300+0000 7f08e3ae5640 1 --2- 172.21.15.46:0/3994306426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08c8006f10 0x7f08c8007390 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f08d400a680 tx=0x7f08c8007a00 comp rx=0 tx=0).stop 2023-12-02T00:30:15.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.300+0000 7f08e2082640 1 -- 172.21.15.46:0/3994306426 reap_dead start 2023-12-02T00:30:15.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.300+0000 7f08e3ae5640 1 -- 172.21.15.46:0/3994306426 shutdown_connections 2023-12-02T00:30:15.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.300+0000 7f08e3ae5640 1 -- 172.21.15.46:0/3994306426 >> 172.21.15.46:0/3994306426 conn(0x7f08dc06a4d0 msgr2=0x7f08dc06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:15.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.301+0000 7f08e3ae5640 1 -- 172.21.15.46:0/3994306426 shutdown_connections 2023-12-02T00:30:15.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.301+0000 7f08e3ae5640 1 -- 172.21.15.46:0/3994306426 wait complete. 2023-12-02T00:30:15.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.302+0000 7f08e3ae5640 1 Processor -- start 2023-12-02T00:30:15.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.302+0000 7f08e3ae5640 1 -- start start 2023-12-02T00:30:15.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.302+0000 7f08e3ae5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08c8006f10 0x7f08dc0c8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:15.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.302+0000 7f08e3ae5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08dc159a60 0x7f08dc12ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:15.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.302+0000 7f08e3ae5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08dc0c5b90 0x7f08dc0ce130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:15.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.302+0000 7f08e3ae5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc159ee0 con 0x7f08c8006f10 2023-12-02T00:30:15.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.302+0000 7f08e3ae5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc12f4a0 con 0x7f08dc159a60 2023-12-02T00:30:15.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.302+0000 7f08e3ae5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc12e510 con 0x7f08dc0c5b90 2023-12-02T00:30:15.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.303+0000 7f08e1881640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08c8006f10 0x7f08dc0c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:15.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.303+0000 7f08e1881640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08c8006f10 0x7f08dc0c8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57020/0 (socket says 172.21.15.46:57020) 2023-12-02T00:30:15.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.303+0000 7f08e1881640 1 -- 172.21.15.46:0/1972633535 learned_addr learned my addr 172.21.15.46:0/1972633535 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:15.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.303+0000 7f08e2082640 1 --2- 172.21.15.46:0/1972633535 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08dc0c5b90 0x7f08dc0ce130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:15.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.303+0000 7f08e1080640 1 --2- 172.21.15.46:0/1972633535 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08dc159a60 0x7f08dc12ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:15.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.303+0000 7f08e1881640 1 -- 172.21.15.46:0/1972633535 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08dc0c5b90 msgr2=0x7f08dc0ce130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:15.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.303+0000 7f08e1881640 1 --2- 172.21.15.46:0/1972633535 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08dc0c5b90 0x7f08dc0ce130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:15.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.303+0000 7f08e1881640 1 -- 172.21.15.46:0/1972633535 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08dc159a60 msgr2=0x7f08dc12ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:15.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.303+0000 7f08e1881640 1 --2- 172.21.15.46:0/1972633535 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08dc159a60 0x7f08dc12ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:15.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.303+0000 7f08e1881640 1 -- 172.21.15.46:0/1972633535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08dc0c8b90 con 0x7f08c8006f10 2023-12-02T00:30:15.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.303+0000 7f08e1881640 1 --2- 172.21.15.46:0/1972633535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08c8006f10 0x7f08dc0c8510 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f08d80119d0 tx=0x7f08d8011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:15.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.303+0000 7f08d27fc640 1 -- 172.21.15.46:0/1972633535 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d800faf0 con 0x7f08c8006f10 2023-12-02T00:30:15.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.303+0000 7f08d27fc640 1 -- 172.21.15.46:0/1972633535 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08d800fc50 con 0x7f08c8006f10 2023-12-02T00:30:15.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.303+0000 7f08e3ae5640 1 -- 172.21.15.46:0/1972633535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08dc0c8e70 con 0x7f08c8006f10 2023-12-02T00:30:15.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.303+0000 7f08d27fc640 1 -- 172.21.15.46:0/1972633535 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d8014440 con 0x7f08c8006f10 2023-12-02T00:30:15.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.303+0000 7f08e3ae5640 1 -- 172.21.15.46:0/1972633535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08dc12ea40 con 0x7f08c8006f10 2023-12-02T00:30:15.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.304+0000 7f08e3ae5640 1 -- 172.21.15.46:0/1972633535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08dc069400 con 0x7f08c8006f10 2023-12-02T00:30:15.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.305+0000 7f08d27fc640 1 -- 172.21.15.46:0/1972633535 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f08d80145a0 con 0x7f08c8006f10 2023-12-02T00:30:15.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.305+0000 7f08d27fc640 1 --2- 172.21.15.46:0/1972633535 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f08c006ae70 0x7f08c006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:15.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.305+0000 7f08d27fc640 1 -- 172.21.15.46:0/1972633535 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f08d8089ea0 con 0x7f08c8006f10 2023-12-02T00:30:15.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.305+0000 7f08e1080640 1 --2- 172.21.15.46:0/1972633535 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f08c006ae70 0x7f08c006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:15.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.306+0000 7f08e1080640 1 --2- 172.21.15.46:0/1972633535 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f08c006ae70 0x7f08c006d330 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f08dc0c53b0 tx=0x7f08cc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:15.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.309+0000 7f08d27fc640 1 -- 172.21.15.46:0/1972633535 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f08d8054e50 con 0x7f08c8006f10 2023-12-02T00:30:15.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.521+0000 7f08e3ae5640 1 -- 172.21.15.46:0/1972633535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f08dc0b3f90 con 0x7f08c8006f10 2023-12-02T00:30:15.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.521+0000 7f08d27fc640 1 -- 172.21.15.46:0/1972633535 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) v1 ==== 53+0+82 (secure 0 0 0) 0x7f08d8058470 con 0x7f08c8006f10 2023-12-02T00:30:15.522 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data data_new ] 2023-12-02T00:30:15.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.523+0000 7f08e3ae5640 1 -- 172.21.15.46:0/1972633535 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f08c006ae70 msgr2=0x7f08c006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:15.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.523+0000 7f08e3ae5640 1 --2- 172.21.15.46:0/1972633535 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f08c006ae70 0x7f08c006d330 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f08dc0c53b0 tx=0x7f08cc009040 comp rx=0 tx=0).stop 2023-12-02T00:30:15.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.523+0000 7f08e3ae5640 1 -- 172.21.15.46:0/1972633535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08c8006f10 msgr2=0x7f08dc0c8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:15.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.523+0000 7f08e3ae5640 1 --2- 172.21.15.46:0/1972633535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08c8006f10 0x7f08dc0c8510 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f08d80119d0 tx=0x7f08d8011e90 comp rx=0 tx=0).stop 2023-12-02T00:30:15.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.523+0000 7f08e3ae5640 1 -- 172.21.15.46:0/1972633535 shutdown_connections 2023-12-02T00:30:15.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.523+0000 7f08e3ae5640 1 --2- 172.21.15.46:0/1972633535 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f08c006ae70 0x7f08c006d330 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:15.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.523+0000 7f08e3ae5640 1 --2- 172.21.15.46:0/1972633535 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08dc0c5b90 0x7f08dc0ce130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:15.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.523+0000 7f08e3ae5640 1 --2- 172.21.15.46:0/1972633535 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08dc159a60 0x7f08dc12ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:15.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.523+0000 7f08e3ae5640 1 --2- 172.21.15.46:0/1972633535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08c8006f10 0x7f08dc0c8510 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:15.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.523+0000 7f08e3ae5640 1 -- 172.21.15.46:0/1972633535 >> 172.21.15.46:0/1972633535 conn(0x7f08dc06a4d0 msgr2=0x7f08dc0cd800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:15.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.523+0000 7f08e3ae5640 1 -- 172.21.15.46:0/1972633535 shutdown_connections 2023-12-02T00:30:15.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:15.523+0000 7f08e3ae5640 1 -- 172.21.15.46:0/1972633535 wait complete. 2023-12-02T00:30:15.535 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-02T00:30:15.535 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-02T00:30:15.535 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2023-12-02T00:30:15.535 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-02T00:30:15.535 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-02T00:30:15.535 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-02T00:30:15.535 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-02T00:30:15.535 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-02T00:30:15.536 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-02T00:30:15.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:30:15.536 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2023-12-02T00:30:15.551 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-02T00:30:15.552 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-02T00:30:15.552 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2023-12-02T00:30:15.552 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:30:15.552 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2023-12-02T00:30:15.552 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:30:15.552 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-02T00:30:15.552 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2023-12-02T00:30:15.552 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2023-12-02T00:30:15.552 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-02T00:30:15.552 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2023-12-02T00:30:15.552 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2023-12-02T00:30:15.552 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-02T00:30:15.552 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2023-12-02T00:30:15.552 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2023-12-02T00:30:15.553 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-12-02T00:30:15.553 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42563sec preferred_lft 42563sec 2023-12-02T00:30:15.553 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link noprefixroute 2023-12-02T00:30:15.553 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:30:15.553 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-02T00:30:15.553 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2023-12-02T00:30:15.553 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2023-12-02T00:30:15.553 INFO:teuthology.orchestra.run.smithi046.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-02T00:30:15.553 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether b6:1a:8b:80:15:f1 brd ff:ff:ff:ff:ff:ff 2023-12-02T00:30:15.553 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-02T00:30:15.553 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:30:15.553 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::eca8:9ff:feb3:1726/64 scope link 2023-12-02T00:30:15.553 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:30:15.554 INFO:teuthology.orchestra.run.smithi046.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-02T00:30:15.554 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether b6:1a:8b:80:15:f1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-02T00:30:15.554 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::b41a:8bff:fe80:15f1/64 scope link 2023-12-02T00:30:15.554 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:30:15.554 INFO:teuthology.orchestra.run.smithi046.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-02T00:30:15.554 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether c6:93:a5:c0:e2:af brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-02T00:30:15.554 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c493:a5ff:fec0:e2af/64 scope link 2023-12-02T00:30:15.554 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:30:15.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:30:15.555 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-02T00:30:15.621 INFO:teuthology.orchestra.run.smithi046.stdout:1 2023-12-02T00:30:15.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:30:15.623 DEBUG:teuthology.orchestra.run.smithi046:> route 2023-12-02T00:30:15.679 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2023-12-02T00:30:15.679 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-02T00:30:15.679 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-12-02T00:30:15.679 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-12-02T00:30:15.679 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-02T00:30:15.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:30:15.680 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2023-12-02T00:30:15.680 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2023-12-02T00:30:15.680 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-02T00:30:15.680 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-02T00:30:15.680 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-02T00:30:15.680 DEBUG:teuthology.orchestra.run.smithi046:> ') 2023-12-02T00:30:15.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:30:15.787 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2023-12-02T00:30:15.802 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-02T00:30:15.802 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-02T00:30:15.802 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-02T00:30:15.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:30:15.802 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2023-12-02T00:30:15.872 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-12-02T00:30:15.873 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-02T00:30:15.873 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-02T00:30:15.929 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-02T00:30:15.929 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-02T00:30:15.929 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-02T00:30:15.984 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2023-12-02T00:30:16.055 DEBUG:teuthology.orchestra.run.smithi046:> cat /proc/self/mounts | awk '{print $2}' 2023-12-02T00:30:16.112 INFO:teuthology.orchestra.run.smithi046.stdout:/proc 2023-12-02T00:30:16.112 INFO:teuthology.orchestra.run.smithi046.stdout:/sys 2023-12-02T00:30:16.112 INFO:teuthology.orchestra.run.smithi046.stdout:/dev 2023-12-02T00:30:16.112 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/security 2023-12-02T00:30:16.112 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/shm 2023-12-02T00:30:16.112 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/pts 2023-12-02T00:30:16.112 INFO:teuthology.orchestra.run.smithi046.stdout:/run 2023-12-02T00:30:16.112 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/cgroup 2023-12-02T00:30:16.113 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/pstore 2023-12-02T00:30:16.113 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/bpf 2023-12-02T00:30:16.113 INFO:teuthology.orchestra.run.smithi046.stdout:/ 2023-12-02T00:30:16.113 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/selinux 2023-12-02T00:30:16.113 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2023-12-02T00:30:16.113 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/mqueue 2023-12-02T00:30:16.113 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/hugepages 2023-12-02T00:30:16.113 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/debug 2023-12-02T00:30:16.113 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/tracing 2023-12-02T00:30:16.113 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/fuse/connections 2023-12-02T00:30:16.113 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/config 2023-12-02T00:30:16.113 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-sysctl.service 2023-12-02T00:30:16.113 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-02T00:30:16.113 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-02T00:30:16.113 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/nfs/rpc_pipefs 2023-12-02T00:30:16.114 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/1000 2023-12-02T00:30:16.114 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph 2023-12-02T00:30:16.114 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-0 2023-12-02T00:30:16.114 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-1 2023-12-02T00:30:16.114 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-2 2023-12-02T00:30:16.114 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-3 2023-12-02T00:30:16.114 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns 2023-12-02T00:30:16.114 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-02T00:30:16.114 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-02T00:30:16.114 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-02T00:30:16.114 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-02T00:30:16.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:30:16.115 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2023-12-02T00:30:16.167 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-12-02T00:30:16.167 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-12-02T00:30:16.209 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2023-12-02T00:30:16.240 DEBUG:teuthology.orchestra.run.smithi046:> cat /proc/self/mounts | awk '{print $2}' 2023-12-02T00:30:16.297 INFO:teuthology.orchestra.run.smithi046.stdout:/proc 2023-12-02T00:30:16.297 INFO:teuthology.orchestra.run.smithi046.stdout:/sys 2023-12-02T00:30:16.297 INFO:teuthology.orchestra.run.smithi046.stdout:/dev 2023-12-02T00:30:16.297 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/security 2023-12-02T00:30:16.297 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/shm 2023-12-02T00:30:16.297 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/pts 2023-12-02T00:30:16.297 INFO:teuthology.orchestra.run.smithi046.stdout:/run 2023-12-02T00:30:16.297 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/cgroup 2023-12-02T00:30:16.297 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/pstore 2023-12-02T00:30:16.297 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/bpf 2023-12-02T00:30:16.297 INFO:teuthology.orchestra.run.smithi046.stdout:/ 2023-12-02T00:30:16.297 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/selinux 2023-12-02T00:30:16.298 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2023-12-02T00:30:16.298 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/mqueue 2023-12-02T00:30:16.298 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/hugepages 2023-12-02T00:30:16.298 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/debug 2023-12-02T00:30:16.298 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/tracing 2023-12-02T00:30:16.298 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/fuse/connections 2023-12-02T00:30:16.298 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/config 2023-12-02T00:30:16.298 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-sysctl.service 2023-12-02T00:30:16.298 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-02T00:30:16.298 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-02T00:30:16.298 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/nfs/rpc_pipefs 2023-12-02T00:30:16.298 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/1000 2023-12-02T00:30:16.298 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph 2023-12-02T00:30:16.298 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-0 2023-12-02T00:30:16.299 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-1 2023-12-02T00:30:16.299 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-2 2023-12-02T00:30:16.299 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-3 2023-12-02T00:30:16.299 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns 2023-12-02T00:30:16.299 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-02T00:30:16.299 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-02T00:30:16.299 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-02T00:30:16.299 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-02T00:30:16.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:30:16.299 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2023-12-02T00:30:16.304 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:2023-12-02T00:30:16.303+0000 7ff6f4a39480 -1 init, newargv = 0x5589c6c382e0 newargc=15 2023-12-02T00:30:16.304 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[105675]: starting ceph client 2023-12-02T00:30:16.308 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[105675]: starting fuse 2023-12-02T00:30:16.320 INFO:teuthology.orchestra.run.smithi046.stdout:41 2023-12-02T00:30:16.320 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-12-02T00:30:16.320 DEBUG:teuthology.orchestra.run.smithi046:> sudo stdin-killer -- python3 -c ' 2023-12-02T00:30:16.321 DEBUG:teuthology.orchestra.run.smithi046:> import glob 2023-12-02T00:30:16.321 DEBUG:teuthology.orchestra.run.smithi046:> import re 2023-12-02T00:30:16.321 DEBUG:teuthology.orchestra.run.smithi046:> import os 2023-12-02T00:30:16.321 DEBUG:teuthology.orchestra.run.smithi046:> import subprocess 2023-12-02T00:30:16.321 DEBUG:teuthology.orchestra.run.smithi046:> 2023-12-02T00:30:16.321 DEBUG:teuthology.orchestra.run.smithi046:> def _find_admin_socket(client_name): 2023-12-02T00:30:16.321 DEBUG:teuthology.orchestra.run.smithi046:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-12-02T00:30:16.321 DEBUG:teuthology.orchestra.run.smithi046:> files = glob.glob(asok_path) 2023-12-02T00:30:16.321 DEBUG:teuthology.orchestra.run.smithi046:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-12-02T00:30:16.321 DEBUG:teuthology.orchestra.run.smithi046:> 2023-12-02T00:30:16.321 DEBUG:teuthology.orchestra.run.smithi046:> # Given a non-glob path, it better be there 2023-12-02T00:30:16.322 DEBUG:teuthology.orchestra.run.smithi046:> if "*" not in asok_path: 2023-12-02T00:30:16.322 DEBUG:teuthology.orchestra.run.smithi046:> assert(len(files) == 1) 2023-12-02T00:30:16.322 DEBUG:teuthology.orchestra.run.smithi046:> return files[0] 2023-12-02T00:30:16.322 DEBUG:teuthology.orchestra.run.smithi046:> 2023-12-02T00:30:16.322 DEBUG:teuthology.orchestra.run.smithi046:> for f in files: 2023-12-02T00:30:16.322 DEBUG:teuthology.orchestra.run.smithi046:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-02T00:30:16.322 DEBUG:teuthology.orchestra.run.smithi046:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-02T00:30:16.322 DEBUG:teuthology.orchestra.run.smithi046:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-02T00:30:16.322 DEBUG:teuthology.orchestra.run.smithi046:> contents = proc_f.read() 2023-12-02T00:30:16.322 DEBUG:teuthology.orchestra.run.smithi046:> if mountpoint in contents: 2023-12-02T00:30:16.322 DEBUG:teuthology.orchestra.run.smithi046:> return f 2023-12-02T00:30:16.322 DEBUG:teuthology.orchestra.run.smithi046:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-02T00:30:16.322 DEBUG:teuthology.orchestra.run.smithi046:> 2023-12-02T00:30:16.322 DEBUG:teuthology.orchestra.run.smithi046:> print(_find_admin_socket("client.0")) 2023-12-02T00:30:16.322 DEBUG:teuthology.orchestra.run.smithi046:> ' 2023-12-02T00:30:16.446 INFO:teuthology.orchestra.run.smithi046.stdout:/var/run/ceph/ceph-client.0.105675.asok 2023-12-02T00:30:16.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-12-02T00:30:16.456 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.105675.asok 2023-12-02T00:30:16.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:30:16.456 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.105675.asok status 2023-12-02T00:30:16.598 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-12-02T00:30:16.598 INFO:teuthology.orchestra.run.smithi046.stdout: "metadata": { 2023-12-02T00:30:16.598 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_sha1": "bed7681480ccb0192b91a35e691658867e1e96b7", 2023-12-02T00:30:16.598 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_version": "ceph version 18.0.0-7631-gbed76814 (bed7681480ccb0192b91a35e691658867e1e96b7) reef (dev)", 2023-12-02T00:30:16.599 INFO:teuthology.orchestra.run.smithi046.stdout: "entity_id": "0", 2023-12-02T00:30:16.599 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": "smithi046", 2023-12-02T00:30:16.599 INFO:teuthology.orchestra.run.smithi046.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-12-02T00:30:16.599 INFO:teuthology.orchestra.run.smithi046.stdout: "pid": "105675", 2023-12-02T00:30:16.599 INFO:teuthology.orchestra.run.smithi046.stdout: "root": "/" 2023-12-02T00:30:16.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:30:16.599 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_count": 0, 2023-12-02T00:30:16.599 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_pinned_count": 0, 2023-12-02T00:30:16.599 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 5637, 2023-12-02T00:30:16.599 INFO:teuthology.orchestra.run.smithi046.stdout: "inst": { 2023-12-02T00:30:16.599 INFO:teuthology.orchestra.run.smithi046.stdout: "name": { 2023-12-02T00:30:16.599 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "client", 2023-12-02T00:30:16.599 INFO:teuthology.orchestra.run.smithi046.stdout: "num": 5637 2023-12-02T00:30:16.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:30:16.600 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2023-12-02T00:30:16.600 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2023-12-02T00:30:16.600 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2023-12-02T00:30:16.600 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3864846440 2023-12-02T00:30:16.600 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:30:16.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:30:16.600 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2023-12-02T00:30:16.600 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2023-12-02T00:30:16.600 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2023-12-02T00:30:16.600 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3864846440 2023-12-02T00:30:16.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:30:16.600 INFO:teuthology.orchestra.run.smithi046.stdout: "inst_str": "client.5637 192.168.0.1:0/3864846440", 2023-12-02T00:30:16.601 INFO:teuthology.orchestra.run.smithi046.stdout: "addr_str": "192.168.0.1:0/3864846440", 2023-12-02T00:30:16.601 INFO:teuthology.orchestra.run.smithi046.stdout: "inode_count": 1, 2023-12-02T00:30:16.601 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_epoch": 14, 2023-12-02T00:30:16.601 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch": 38, 2023-12-02T00:30:16.601 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch_barrier": 0, 2023-12-02T00:30:16.601 INFO:teuthology.orchestra.run.smithi046.stdout: "blocklisted": false, 2023-12-02T00:30:16.601 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs" 2023-12-02T00:30:16.601 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-12-02T00:30:16.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:30:16.607 DEBUG:teuthology.orchestra.run.smithi046:> stat --file-system '--printf=%T 2023-12-02T00:30:16.607 DEBUG:teuthology.orchestra.run.smithi046:> ' -- /home/ubuntu/cephtest/mnt.0 2023-12-02T00:30:16.622 INFO:teuthology.orchestra.run.smithi046.stdout:fuseblk 2023-12-02T00:30:16.623 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-12-02T00:30:16.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:30:16.623 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-02T00:30:16.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:30:16.694 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.file.layout.pool -v data_new layoutfile) 2023-12-02T00:30:16.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-02T00:30:16.761 INFO:teuthology.orchestra.run.smithi046.stderr:setfattr: layoutfile: Permission denied 2023-12-02T00:30:16.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:30:16.762 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.layout.pool -v data_new layoutdir) 2023-12-02T00:30:16.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-02T00:30:16.819 INFO:teuthology.orchestra.run.smithi046.stderr:setfattr: layoutdir: Permission denied 2023-12-02T00:30:16.819 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-02T00:30:16.819 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-02T00:30:16.874 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:30:17.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.021+0000 7fa9f9953640 1 Processor -- start 2023-12-02T00:30:17.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.021+0000 7fa9f9953640 1 -- start start 2023-12-02T00:30:17.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.022+0000 7fa9f9953640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa9f4143490 0x7fa9f40b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:17.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.022+0000 7fa9f9953640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa9f415e730 0x7fa9f40b1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:17.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.022+0000 7fa9f9953640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9f415e2a0 con 0x7fa9f415a4e0 2023-12-02T00:30:17.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.022+0000 7fa9f9953640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9f415b520 con 0x7fa9f415e730 2023-12-02T00:30:17.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.022+0000 7fa9f9953640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9f4143870 con 0x7fa9f4143490 2023-12-02T00:30:17.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.022+0000 7fa9f27fc640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa9f415e730 0x7fa9f40b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:17.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.022+0000 7fa9f27fc640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa9f415e730 0x7fa9f40b1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57022/0 (socket says 172.21.15.46:57022) 2023-12-02T00:30:17.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.022+0000 7fa9f27fc640 1 -- 172.21.15.46:0/2060856640 learned_addr learned my addr 172.21.15.46:0/2060856640 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:17.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.022+0000 7fa9f2ffd640 1 --2- 172.21.15.46:0/2060856640 >> v2:172.21.15.162:3300/0 conn(0x7fa9f4143490 0x7fa9f40b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:17.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.023+0000 7fa9f27fc640 1 -- 172.21.15.46:0/2060856640 >> v2:172.21.15.162:3300/0 conn(0x7fa9f4143490 msgr2=0x7fa9f40b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:17.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.023+0000 7fa9f27fc640 1 --2- 172.21.15.46:0/2060856640 >> v2:172.21.15.162:3300/0 conn(0x7fa9f4143490 0x7fa9f40b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:17.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.023+0000 7fa9f27fc640 1 -- 172.21.15.46:0/2060856640 >> v1:172.21.15.162:6789/0 conn(0x7fa9f415a4e0 legacy=0x7fa9f415a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:17.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.023+0000 7fa9f27fc640 1 -- 172.21.15.46:0/2060856640 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f415b240 con 0x7fa9f415e730 2023-12-02T00:30:17.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.023+0000 7fa9f27fc640 1 --2- 172.21.15.46:0/2060856640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9f415e730 0x7fa9f40b1230 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa9f406e620 tx=0x7fa9dc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a63b04e2c09208ac server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:17.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.023+0000 7fa9f1ffb640 1 -- 172.21.15.46:0/2060856640 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc031870 con 0x7fa9f415e730 2023-12-02T00:30:17.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.023+0000 7fa9f1ffb640 1 -- 172.21.15.46:0/2060856640 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9dc0319d0 con 0x7fa9f415e730 2023-12-02T00:30:17.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.023+0000 7fa9f1ffb640 1 -- 172.21.15.46:0/2060856640 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc031d60 con 0x7fa9f415e730 2023-12-02T00:30:17.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.024+0000 7fa9f9953640 1 -- 172.21.15.46:0/2060856640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9f415e730 msgr2=0x7fa9f40b1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:17.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.024+0000 7fa9f9953640 1 --2- 172.21.15.46:0/2060856640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9f415e730 0x7fa9f40b1230 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa9f406e620 tx=0x7fa9dc02b910 comp rx=0 tx=0).stop 2023-12-02T00:30:17.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.024+0000 7fa9f9953640 1 -- 172.21.15.46:0/2060856640 shutdown_connections 2023-12-02T00:30:17.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.024+0000 7fa9f9953640 1 --2- 172.21.15.46:0/2060856640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9f415e730 0x7fa9f40b1230 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:17.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.024+0000 7fa9f9953640 1 --2- 172.21.15.46:0/2060856640 >> v2:172.21.15.162:3300/0 conn(0x7fa9f4143490 0x7fa9f40b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:17.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.024+0000 7fa9f9953640 1 -- 172.21.15.46:0/2060856640 >> 172.21.15.46:0/2060856640 conn(0x7fa9f406a4d0 msgr2=0x7fa9f406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:17.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.024+0000 7fa9f9953640 1 -- 172.21.15.46:0/2060856640 shutdown_connections 2023-12-02T00:30:17.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.024+0000 7fa9f9953640 1 -- 172.21.15.46:0/2060856640 wait complete. 2023-12-02T00:30:17.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.025+0000 7fa9f9953640 1 Processor -- start 2023-12-02T00:30:17.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.025+0000 7fa9f9953640 1 -- start start 2023-12-02T00:30:17.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.026+0000 7fa9f9953640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9f4143490 0x7fa9f4134660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:17.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.026+0000 7fa9f9953640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9f415a4e0 0x7fa9f40b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:17.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.026+0000 7fa9f9953640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9f415e730 0x7fa9f40b1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:17.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.026+0000 7fa9f9953640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f415b800 con 0x7fa9f415a4e0 2023-12-02T00:30:17.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.026+0000 7fa9f9953640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f415bae0 con 0x7fa9f4143490 2023-12-02T00:30:17.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.026+0000 7fa9f9953640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f415bdc0 con 0x7fa9f415e730 2023-12-02T00:30:17.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.026+0000 7fa9f27fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9f415a4e0 0x7fa9f40b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:17.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.026+0000 7fa9f27fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9f415a4e0 0x7fa9f40b0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57036/0 (socket says 172.21.15.46:57036) 2023-12-02T00:30:17.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.026+0000 7fa9f27fc640 1 -- 172.21.15.46:0/2000041012 learned_addr learned my addr 172.21.15.46:0/2000041012 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:17.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.026+0000 7fa9f27fc640 1 -- 172.21.15.46:0/2000041012 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9f415e730 msgr2=0x7fa9f40b1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:17.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.026+0000 7fa9f27fc640 1 --2- 172.21.15.46:0/2000041012 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9f415e730 0x7fa9f40b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:17.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.026+0000 7fa9f27fc640 1 -- 172.21.15.46:0/2000041012 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9f4143490 msgr2=0x7fa9f4134660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:17.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.026+0000 7fa9f2ffd640 1 --2- 172.21.15.46:0/2000041012 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9f4143490 0x7fa9f4134660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:17.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.026+0000 7fa9f27fc640 1 --2- 172.21.15.46:0/2000041012 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9f4143490 0x7fa9f4134660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:17.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.026+0000 7fa9f27fc640 1 -- 172.21.15.46:0/2000041012 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9dc0094d0 con 0x7fa9f415a4e0 2023-12-02T00:30:17.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.027+0000 7fa9f27fc640 1 --2- 172.21.15.46:0/2000041012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9f415a4e0 0x7fa9f40b0cf0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc000c00 tx=0x7fa9dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:17.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.027+0000 7fa9ebfff640 1 -- 172.21.15.46:0/2000041012 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc03abe0 con 0x7fa9f415a4e0 2023-12-02T00:30:17.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.027+0000 7fa9ebfff640 1 -- 172.21.15.46:0/2000041012 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9dc03ad40 con 0x7fa9f415a4e0 2023-12-02T00:30:17.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.027+0000 7fa9f9953640 1 -- 172.21.15.46:0/2000041012 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9f415c0a0 con 0x7fa9f415a4e0 2023-12-02T00:30:17.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.027+0000 7fa9ebfff640 1 -- 172.21.15.46:0/2000041012 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc044610 con 0x7fa9f415a4e0 2023-12-02T00:30:17.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.027+0000 7fa9f9953640 1 -- 172.21.15.46:0/2000041012 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9f4134ba0 con 0x7fa9f415a4e0 2023-12-02T00:30:17.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.028+0000 7fa9f9953640 1 -- 172.21.15.46:0/2000041012 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9f415c5c0 con 0x7fa9f415a4e0 2023-12-02T00:30:17.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.028+0000 7fa9ebfff640 1 -- 172.21.15.46:0/2000041012 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa9dc04d450 con 0x7fa9f415a4e0 2023-12-02T00:30:17.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.029+0000 7fa9ebfff640 1 --2- 172.21.15.46:0/2000041012 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa9d406ae40 0x7fa9d406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:17.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.029+0000 7fa9ebfff640 1 -- 172.21.15.46:0/2000041012 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fa9dc02d040 con 0x7fa9f415a4e0 2023-12-02T00:30:17.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.029+0000 7fa9f2ffd640 1 --2- 172.21.15.46:0/2000041012 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa9d406ae40 0x7fa9d406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:17.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.029+0000 7fa9f2ffd640 1 --2- 172.21.15.46:0/2000041012 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa9d406ae40 0x7fa9d406d300 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec006fd0 tx=0x7fa9ec008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:17.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.033+0000 7fa9ebfff640 1 -- 172.21.15.46:0/2000041012 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa9f415c5c0 con 0x7fa9f415a4e0 2023-12-02T00:30:17.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.210+0000 7fa9f9953640 1 -- 172.21.15.46:0/2000041012 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9f415c5c0 con 0x7fa9f415a4e0 2023-12-02T00:30:17.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.212+0000 7fa9ebfff640 1 -- 172.21.15.46:0/2000041012 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+18324 (secure 0 0 0) 0x7fa9f415c5c0 con 0x7fa9f415a4e0 2023-12-02T00:30:17.213 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:30:17.213 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":38,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:30:10.714057+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-02T00:29:55.676572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-02T00:29:56.244869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"data_new","create_time":"2023-12-02T00:30:08.897990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:30:17.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.213+0000 7fa9f9953640 1 -- 172.21.15.46:0/2000041012 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa9d406ae40 msgr2=0x7fa9d406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:17.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.213+0000 7fa9f9953640 1 --2- 172.21.15.46:0/2000041012 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa9d406ae40 0x7fa9d406d300 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec006fd0 tx=0x7fa9ec008040 comp rx=0 tx=0).stop 2023-12-02T00:30:17.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.213+0000 7fa9f9953640 1 -- 172.21.15.46:0/2000041012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9f415a4e0 msgr2=0x7fa9f40b0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:17.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.213+0000 7fa9f9953640 1 --2- 172.21.15.46:0/2000041012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9f415a4e0 0x7fa9f40b0cf0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc000c00 tx=0x7fa9dc0046c0 comp rx=0 tx=0).stop 2023-12-02T00:30:17.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.213+0000 7fa9f9953640 1 -- 172.21.15.46:0/2000041012 shutdown_connections 2023-12-02T00:30:17.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.213+0000 7fa9f9953640 1 --2- 172.21.15.46:0/2000041012 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa9d406ae40 0x7fa9d406d300 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:17.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.213+0000 7fa9f9953640 1 --2- 172.21.15.46:0/2000041012 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9f415e730 0x7fa9f40b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:17.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.213+0000 7fa9f9953640 1 --2- 172.21.15.46:0/2000041012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9f415a4e0 0x7fa9f40b0cf0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:17.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.213+0000 7fa9f9953640 1 --2- 172.21.15.46:0/2000041012 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9f4143490 0x7fa9f4134660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:17.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.213+0000 7fa9f9953640 1 -- 172.21.15.46:0/2000041012 >> 172.21.15.46:0/2000041012 conn(0x7fa9f406a4d0 msgr2=0x7fa9f4070390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:17.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.214+0000 7fa9f9953640 1 -- 172.21.15.46:0/2000041012 shutdown_connections 2023-12-02T00:30:17.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.214+0000 7fa9f9953640 1 -- 172.21.15.46:0/2000041012 wait complete. 2023-12-02T00:30:17.225 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-02T00:28:37.213645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-02T00:29:55.676572+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-12-02T00:29:56.244869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'data_new', 'create_time': '2023-12-02T00:30:08.897990+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-02T00:30:17.226 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-02T00:30:17.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.335+0000 7f9f4c609640 1 Processor -- start 2023-12-02T00:30:17.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.335+0000 7f9f4c609640 1 -- start start 2023-12-02T00:30:17.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.336+0000 7f9f4c609640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9f4406bbf0 0x7f9f4406d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:17.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.336+0000 7f9f4c609640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f440adae0 con 0x7f9f440ad640 2023-12-02T00:30:17.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.336+0000 7f9f4c609640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f440addc0 con 0x7f9f4415d1d0 2023-12-02T00:30:17.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.336+0000 7f9f4c609640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f440ae0a0 con 0x7f9f4406bbf0 2023-12-02T00:30:17.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.336+0000 7f9f4aba6640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9f4406bbf0 0x7f9f4406d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:17.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.336+0000 7f9f49ba4640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9f440ad640 0x7f9f4406d0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59154/0 (socket says 172.21.15.46:59154) 2023-12-02T00:30:17.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.336+0000 7f9f49ba4640 1 -- 172.21.15.46:0/3604712908 learned_addr learned my addr 172.21.15.46:0/3604712908 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:30:17.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.336+0000 7f9f4aba6640 1 -- 172.21.15.46:0/3604712908 >> v1:172.21.15.162:6789/0 conn(0x7f9f4415d1d0 legacy=0x7f9f4406b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:17.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.336+0000 7f9f4aba6640 1 -- 172.21.15.46:0/3604712908 >> v1:172.21.15.46:6789/0 conn(0x7f9f440ad640 legacy=0x7f9f4406d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:17.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.336+0000 7f9f4aba6640 1 -- 172.21.15.46:0/3604712908 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f440ae380 con 0x7f9f4406bbf0 2023-12-02T00:30:17.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.336+0000 7f9f493a3640 1 -- 172.21.15.46:0/3604712908 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2928804193 0 0) 0x7f9f440adae0 con 0x7f9f440ad640 2023-12-02T00:30:17.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.337+0000 7f9f4aba6640 1 --2- 172.21.15.46:0/3604712908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f4406bbf0 0x7f9f4406d7c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c009d90 tx=0x7f9f3c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e89dfa4a94c239c7 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:17.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.337+0000 7f9f493a3640 1 -- 172.21.15.46:0/3604712908 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c02cc60 con 0x7f9f4406bbf0 2023-12-02T00:30:17.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.337+0000 7f9f493a3640 1 -- 172.21.15.46:0/3604712908 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f3c02cdc0 con 0x7f9f4406bbf0 2023-12-02T00:30:17.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.337+0000 7f9f493a3640 1 -- 172.21.15.46:0/3604712908 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c034660 con 0x7f9f4406bbf0 2023-12-02T00:30:17.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.338+0000 7f9f4c609640 1 -- 172.21.15.46:0/3604712908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f4406bbf0 msgr2=0x7f9f4406d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:17.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.338+0000 7f9f4c609640 1 --2- 172.21.15.46:0/3604712908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f4406bbf0 0x7f9f4406d7c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c009d90 tx=0x7f9f3c02beb0 comp rx=0 tx=0).stop 2023-12-02T00:30:17.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.338+0000 7f9f4c609640 1 -- 172.21.15.46:0/3604712908 shutdown_connections 2023-12-02T00:30:17.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.338+0000 7f9f4c609640 1 --2- 172.21.15.46:0/3604712908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f4406bbf0 0x7f9f4406d7c0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:17.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.338+0000 7f9f4c609640 1 -- 172.21.15.46:0/3604712908 >> 172.21.15.46:0/3604712908 conn(0x7f9f44070e70 msgr2=0x7f9f44071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:17.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.338+0000 7f9f4c609640 1 -- 172.21.15.46:0/3604712908 shutdown_connections 2023-12-02T00:30:17.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.338+0000 7f9f4c609640 1 -- 172.21.15.46:0/3604712908 wait complete. 2023-12-02T00:30:17.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.339+0000 7f9f4c609640 1 Processor -- start 2023-12-02T00:30:17.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.339+0000 7f9f4c609640 1 -- start start 2023-12-02T00:30:17.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.340+0000 7f9f4c609640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9f4406bbf0 0x7f9f44143c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:17.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.340+0000 7f9f4c609640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f440ad640 0x7f9f44157240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:17.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.340+0000 7f9f4c609640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f4415d1d0 0x7f9f44157780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:17.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.340+0000 7f9f4c609640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f44144160 con 0x7f9f440ad640 2023-12-02T00:30:17.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.340+0000 7f9f4c609640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f44159d70 con 0x7f9f4415d1d0 2023-12-02T00:30:17.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.340+0000 7f9f4c609640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f44144380 con 0x7f9f4406bbf0 2023-12-02T00:30:17.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.340+0000 7f9f49ba4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f440ad640 0x7f9f44157240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:17.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.340+0000 7f9f49ba4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f440ad640 0x7f9f44157240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57050/0 (socket says 172.21.15.46:57050) 2023-12-02T00:30:17.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.340+0000 7f9f49ba4640 1 -- 172.21.15.46:0/3000054006 learned_addr learned my addr 172.21.15.46:0/3000054006 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:17.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.340+0000 7f9f4aba6640 1 --2- 172.21.15.46:0/3000054006 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f4415d1d0 0x7f9f44157780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:17.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.340+0000 7f9f49ba4640 1 -- 172.21.15.46:0/3000054006 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9f4406bbf0 msgr2=0x7f9f44143c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:17.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.340+0000 7f9f4a3a5640 1 --2- 172.21.15.46:0/3000054006 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9f4406bbf0 0x7f9f44143c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:17.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.340+0000 7f9f49ba4640 1 --2- 172.21.15.46:0/3000054006 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9f4406bbf0 0x7f9f44143c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:17.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.340+0000 7f9f49ba4640 1 -- 172.21.15.46:0/3000054006 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f4415d1d0 msgr2=0x7f9f44157780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:17.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.340+0000 7f9f49ba4640 1 --2- 172.21.15.46:0/3000054006 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f4415d1d0 0x7f9f44157780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:17.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.340+0000 7f9f49ba4640 1 -- 172.21.15.46:0/3000054006 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f3c009a70 con 0x7f9f440ad640 2023-12-02T00:30:17.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.340+0000 7f9f4aba6640 1 --2- 172.21.15.46:0/3000054006 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f4415d1d0 0x7f9f44157780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:30:17.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.340+0000 7f9f4a3a5640 1 --2- 172.21.15.46:0/3000054006 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9f4406bbf0 0x7f9f44143c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:30:17.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.340+0000 7f9f49ba4640 1 --2- 172.21.15.46:0/3000054006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f440ad640 0x7f9f44157240 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f9f3400c4d0 tx=0x7f9f3400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:17.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.340+0000 7f9f3affd640 1 -- 172.21.15.46:0/3000054006 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f340074e0 con 0x7f9f440ad640 2023-12-02T00:30:17.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.340+0000 7f9f3affd640 1 -- 172.21.15.46:0/3000054006 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f34007660 con 0x7f9f440ad640 2023-12-02T00:30:17.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.340+0000 7f9f4c609640 1 -- 172.21.15.46:0/3000054006 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f44159440 con 0x7f9f440ad640 2023-12-02T00:30:17.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.341+0000 7f9f3affd640 1 -- 172.21.15.46:0/3000054006 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f34004a20 con 0x7f9f440ad640 2023-12-02T00:30:17.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.341+0000 7f9f4c609640 1 -- 172.21.15.46:0/3000054006 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f44159a00 con 0x7f9f440ad640 2023-12-02T00:30:17.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.342+0000 7f9f4c609640 1 -- 172.21.15.46:0/3000054006 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f44069400 con 0x7f9f440ad640 2023-12-02T00:30:17.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.342+0000 7f9f3affd640 1 -- 172.21.15.46:0/3000054006 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9f34004b80 con 0x7f9f440ad640 2023-12-02T00:30:17.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.343+0000 7f9f3affd640 1 --2- 172.21.15.46:0/3000054006 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9f2806ae40 0x7f9f2806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:17.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.343+0000 7f9f3affd640 1 -- 172.21.15.46:0/3000054006 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f9f34088dd0 con 0x7f9f440ad640 2023-12-02T00:30:17.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.343+0000 7f9f4a3a5640 1 --2- 172.21.15.46:0/3000054006 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9f2806ae40 0x7f9f2806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:17.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.343+0000 7f9f4a3a5640 1 --2- 172.21.15.46:0/3000054006 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9f2806ae40 0x7f9f2806d300 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9f4000a830 tx=0x7f9f40008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:17.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.347+0000 7f9f3affd640 1 -- 172.21.15.46:0/3000054006 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9f34053c50 con 0x7f9f440ad640 2023-12-02T00:30:17.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.522+0000 7f9f4c609640 1 -- 172.21.15.46:0/3000054006 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9f4406e0c0 con 0x7f9f440ad640 2023-12-02T00:30:17.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.522+0000 7f9f3affd640 1 -- 172.21.15.46:0/3000054006 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9f34018020 con 0x7f9f440ad640 2023-12-02T00:30:17.524 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-12-02T00:30:17.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.524+0000 7f9f4c609640 1 -- 172.21.15.46:0/3000054006 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9f2806ae40 msgr2=0x7f9f2806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:17.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.524+0000 7f9f4c609640 1 --2- 172.21.15.46:0/3000054006 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9f2806ae40 0x7f9f2806d300 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9f4000a830 tx=0x7f9f40008040 comp rx=0 tx=0).stop 2023-12-02T00:30:17.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.524+0000 7f9f4c609640 1 -- 172.21.15.46:0/3000054006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f440ad640 msgr2=0x7f9f44157240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:17.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.524+0000 7f9f4c609640 1 --2- 172.21.15.46:0/3000054006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f440ad640 0x7f9f44157240 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f9f3400c4d0 tx=0x7f9f3400c990 comp rx=0 tx=0).stop 2023-12-02T00:30:17.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.524+0000 7f9f4c609640 1 -- 172.21.15.46:0/3000054006 shutdown_connections 2023-12-02T00:30:17.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.524+0000 7f9f4c609640 1 --2- 172.21.15.46:0/3000054006 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9f2806ae40 0x7f9f2806d300 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:17.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.524+0000 7f9f4c609640 1 --2- 172.21.15.46:0/3000054006 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f4415d1d0 0x7f9f44157780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:17.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.524+0000 7f9f4c609640 1 --2- 172.21.15.46:0/3000054006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f440ad640 0x7f9f44157240 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:17.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.524+0000 7f9f4c609640 1 --2- 172.21.15.46:0/3000054006 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9f4406bbf0 0x7f9f44143c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:17.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.524+0000 7f9f4c609640 1 -- 172.21.15.46:0/3000054006 >> 172.21.15.46:0/3000054006 conn(0x7f9f44070e70 msgr2=0x7f9f4406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:17.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.524+0000 7f9f4c609640 1 -- 172.21.15.46:0/3000054006 shutdown_connections 2023-12-02T00:30:17.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.524+0000 7f9f4c609640 1 -- 172.21.15.46:0/3000054006 wait complete. 2023-12-02T00:30:17.535 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-02T00:30:17.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.644+0000 7efee8390640 1 Processor -- start 2023-12-02T00:30:17.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.644+0000 7efee8390640 1 -- start start 2023-12-02T00:30:17.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.645+0000 7efee8390640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7efee0140310 0x7efee01618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:17.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.645+0000 7efee8390640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7efee006abe0 0x7efee006efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:17.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.645+0000 7efee8390640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efee01406f0 con 0x7efee01409d0 2023-12-02T00:30:17.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.645+0000 7efee8390640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efee0161e20 con 0x7efee0140310 2023-12-02T00:30:17.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.645+0000 7efee8390640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efee0143960 con 0x7efee006abe0 2023-12-02T00:30:17.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.645+0000 7efee612c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7efee0140310 0x7efee01618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:17.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.645+0000 7efee612c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7efee0140310 0x7efee01618e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57060/0 (socket says 172.21.15.46:57060) 2023-12-02T00:30:17.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.645+0000 7efee612c640 1 -- 172.21.15.46:0/3393781077 learned_addr learned my addr 172.21.15.46:0/3393781077 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:17.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.645+0000 7efee612c640 1 -- 172.21.15.46:0/3393781077 >> v2:172.21.15.162:3300/0 conn(0x7efee006abe0 msgr2=0x7efee006efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:17.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.645+0000 7efee612c640 1 --2- 172.21.15.46:0/3393781077 >> v2:172.21.15.162:3300/0 conn(0x7efee006abe0 0x7efee006efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:17.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.645+0000 7efee612c640 1 -- 172.21.15.46:0/3393781077 >> v1:172.21.15.162:6789/0 conn(0x7efee01409d0 legacy=0x7efee006a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:17.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.645+0000 7efee612c640 1 -- 172.21.15.46:0/3393781077 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee0142de0 con 0x7efee0140310 2023-12-02T00:30:17.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.646+0000 7efee612c640 1 --2- 172.21.15.46:0/3393781077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efee0140310 0x7efee01618e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7efee0154930 tx=0x7efedc02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1181ed3c0bf49af4 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:17.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.646+0000 7efee512a640 1 -- 172.21.15.46:0/3393781077 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efedc0317e0 con 0x7efee0140310 2023-12-02T00:30:17.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.646+0000 7efee512a640 1 -- 172.21.15.46:0/3393781077 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efedc031940 con 0x7efee0140310 2023-12-02T00:30:17.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.646+0000 7efee512a640 1 -- 172.21.15.46:0/3393781077 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efedc031cd0 con 0x7efee0140310 2023-12-02T00:30:17.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.647+0000 7efee8390640 1 -- 172.21.15.46:0/3393781077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efee0140310 msgr2=0x7efee01618e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:17.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.647+0000 7efee8390640 1 --2- 172.21.15.46:0/3393781077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efee0140310 0x7efee01618e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7efee0154930 tx=0x7efedc02b880 comp rx=0 tx=0).stop 2023-12-02T00:30:17.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.647+0000 7efee8390640 1 -- 172.21.15.46:0/3393781077 shutdown_connections 2023-12-02T00:30:17.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.647+0000 7efee8390640 1 --2- 172.21.15.46:0/3393781077 >> v2:172.21.15.162:3300/0 conn(0x7efee006abe0 0x7efee006efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:17.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.647+0000 7efee8390640 1 --2- 172.21.15.46:0/3393781077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efee0140310 0x7efee01618e0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:17.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.647+0000 7efee8390640 1 -- 172.21.15.46:0/3393781077 >> 172.21.15.46:0/3393781077 conn(0x7efee00b1e50 msgr2=0x7efee00b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:17.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.647+0000 7efee8390640 1 -- 172.21.15.46:0/3393781077 shutdown_connections 2023-12-02T00:30:17.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.647+0000 7efee8390640 1 -- 172.21.15.46:0/3393781077 wait complete. 2023-12-02T00:30:17.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.648+0000 7efee8390640 1 Processor -- start 2023-12-02T00:30:17.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.648+0000 7efee8390640 1 -- start start 2023-12-02T00:30:17.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.649+0000 7efee8390640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efee006abe0 0x7efee00af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:17.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.649+0000 7efee8390640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efee0140310 0x7efee0145af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:17.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.649+0000 7efee8390640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efee01409d0 0x7efee0146030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:17.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.649+0000 7efee8390640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee0146570 con 0x7efee006abe0 2023-12-02T00:30:17.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.649+0000 7efee8390640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee00ae330 con 0x7efee01409d0 2023-12-02T00:30:17.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.649+0000 7efee8390640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efee00ae610 con 0x7efee0140310 2023-12-02T00:30:17.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.649+0000 7efee612c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efee006abe0 0x7efee00af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:17.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.649+0000 7efee612c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efee006abe0 0x7efee00af420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57072/0 (socket says 172.21.15.46:57072) 2023-12-02T00:30:17.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.649+0000 7efee612c640 1 -- 172.21.15.46:0/679131025 learned_addr learned my addr 172.21.15.46:0/679131025 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:17.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.649+0000 7efee592b640 1 --2- 172.21.15.46:0/679131025 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efee0140310 0x7efee0145af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:17.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.649+0000 7efee692d640 1 --2- 172.21.15.46:0/679131025 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efee01409d0 0x7efee0146030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:17.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.649+0000 7efee612c640 1 -- 172.21.15.46:0/679131025 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efee0140310 msgr2=0x7efee0145af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:17.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.649+0000 7efee612c640 1 --2- 172.21.15.46:0/679131025 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efee0140310 0x7efee0145af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:17.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.649+0000 7efee612c640 1 -- 172.21.15.46:0/679131025 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efee01409d0 msgr2=0x7efee0146030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:17.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.649+0000 7efee612c640 1 --2- 172.21.15.46:0/679131025 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efee01409d0 0x7efee0146030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:17.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.649+0000 7efee612c640 1 -- 172.21.15.46:0/679131025 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efedc0094d0 con 0x7efee006abe0 2023-12-02T00:30:17.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.649+0000 7efee612c640 1 --2- 172.21.15.46:0/679131025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efee006abe0 0x7efee00af420 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7efedc02b8c0 tx=0x7efedc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:17.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.650+0000 7efed6ffd640 1 -- 172.21.15.46:0/679131025 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efedc032940 con 0x7efee006abe0 2023-12-02T00:30:17.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.650+0000 7efed6ffd640 1 -- 172.21.15.46:0/679131025 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efedc032aa0 con 0x7efee006abe0 2023-12-02T00:30:17.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.650+0000 7efee8390640 1 -- 172.21.15.46:0/679131025 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efee00ae8f0 con 0x7efee006abe0 2023-12-02T00:30:17.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.650+0000 7efed6ffd640 1 -- 172.21.15.46:0/679131025 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efedc032e50 con 0x7efee006abe0 2023-12-02T00:30:17.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.650+0000 7efee8390640 1 -- 172.21.15.46:0/679131025 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efee00aeeb0 con 0x7efee006abe0 2023-12-02T00:30:17.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.651+0000 7efee8390640 1 -- 172.21.15.46:0/679131025 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efee0073c60 con 0x7efee006abe0 2023-12-02T00:30:17.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.651+0000 7efed6ffd640 1 -- 172.21.15.46:0/679131025 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efedc04e450 con 0x7efee006abe0 2023-12-02T00:30:17.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.651+0000 7efed6ffd640 1 --2- 172.21.15.46:0/679131025 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7efec406ae40 0x7efec406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:17.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.652+0000 7efed6ffd640 1 -- 172.21.15.46:0/679131025 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7efedc0aba10 con 0x7efee006abe0 2023-12-02T00:30:17.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.652+0000 7efee592b640 1 --2- 172.21.15.46:0/679131025 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7efec406ae40 0x7efec406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:17.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.652+0000 7efee592b640 1 --2- 172.21.15.46:0/679131025 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7efec406ae40 0x7efec406d300 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7efed0007fd0 tx=0x7efed0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:17.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.656+0000 7efed6ffd640 1 -- 172.21.15.46:0/679131025 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efee0073c60 con 0x7efee006abe0 2023-12-02T00:30:17.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.835+0000 7efee8390640 1 -- 172.21.15.46:0/679131025 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efee0143b30 con 0x7efee006abe0 2023-12-02T00:30:17.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.835+0000 7efed6ffd640 1 -- 172.21.15.46:0/679131025 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v38) v1 ==== 104+0+11 (secure 0 0 0) 0x7efee0143b30 con 0x7efee006abe0 2023-12-02T00:30:17.837 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-12-02T00:30:17.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.837+0000 7efee8390640 1 -- 172.21.15.46:0/679131025 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7efec406ae40 msgr2=0x7efec406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:17.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.837+0000 7efee8390640 1 --2- 172.21.15.46:0/679131025 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7efec406ae40 0x7efec406d300 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7efed0007fd0 tx=0x7efed0009040 comp rx=0 tx=0).stop 2023-12-02T00:30:17.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.837+0000 7efee8390640 1 -- 172.21.15.46:0/679131025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efee006abe0 msgr2=0x7efee00af420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:17.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.837+0000 7efee8390640 1 --2- 172.21.15.46:0/679131025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efee006abe0 0x7efee00af420 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7efedc02b8c0 tx=0x7efedc0046c0 comp rx=0 tx=0).stop 2023-12-02T00:30:17.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.837+0000 7efee8390640 1 -- 172.21.15.46:0/679131025 shutdown_connections 2023-12-02T00:30:17.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.837+0000 7efee8390640 1 --2- 172.21.15.46:0/679131025 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7efec406ae40 0x7efec406d300 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:17.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.837+0000 7efee8390640 1 --2- 172.21.15.46:0/679131025 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efee01409d0 0x7efee0146030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:17.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.837+0000 7efee8390640 1 --2- 172.21.15.46:0/679131025 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efee0140310 0x7efee0145af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:17.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.837+0000 7efee8390640 1 --2- 172.21.15.46:0/679131025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efee006abe0 0x7efee00af420 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:17.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.837+0000 7efee8390640 1 -- 172.21.15.46:0/679131025 >> 172.21.15.46:0/679131025 conn(0x7efee00b1e50 msgr2=0x7efee006f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:17.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.838+0000 7efee8390640 1 -- 172.21.15.46:0/679131025 shutdown_connections 2023-12-02T00:30:17.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.838+0000 7efee8390640 1 -- 172.21.15.46:0/679131025 wait complete. 2023-12-02T00:30:17.849 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-02T00:30:17.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.957+0000 7f532c2d4640 1 Processor -- start 2023-12-02T00:30:17.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.957+0000 7f532c2d4640 1 -- start start 2023-12-02T00:30:17.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.958+0000 7f532c2d4640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f532406ea50 0x7f532406ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:17.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.958+0000 7f532c2d4640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5324161d20 con 0x7f5324160fb0 2023-12-02T00:30:17.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.958+0000 7f532c2d4640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f532406f370 con 0x7f53240d0ed0 2023-12-02T00:30:17.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.958+0000 7f532c2d4640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f532415e760 con 0x7f532406ea50 2023-12-02T00:30:17.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.958+0000 7f532986f640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5324160fb0 0x7f5324161440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59166/0 (socket says 172.21.15.46:59166) 2023-12-02T00:30:17.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.958+0000 7f532986f640 1 -- 172.21.15.46:0/489264201 learned_addr learned my addr 172.21.15.46:0/489264201 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:30:17.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.959+0000 7f532906e640 1 -- 172.21.15.46:0/489264201 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3433328040 0 0) 0x7f5324161d20 con 0x7f5324160fb0 2023-12-02T00:30:17.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.959+0000 7f532a070640 1 --2- 172.21.15.46:0/489264201 >> v2:172.21.15.162:3300/0 conn(0x7f532406ea50 0x7f532406ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:17.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.959+0000 7f532906e640 1 -- 172.21.15.46:0/489264201 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f530c003520 con 0x7f5324160fb0 2023-12-02T00:30:17.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.959+0000 7f532906e640 1 -- 172.21.15.46:0/489264201 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4086255667 0 0) 0x7f530c003520 con 0x7f5324160fb0 2023-12-02T00:30:17.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.959+0000 7f532906e640 1 -- 172.21.15.46:0/489264201 >> v2:172.21.15.162:3300/0 conn(0x7f532406ea50 msgr2=0x7f532406ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:17.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.959+0000 7f532906e640 1 --2- 172.21.15.46:0/489264201 >> v2:172.21.15.162:3300/0 conn(0x7f532406ea50 0x7f532406ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:17.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.959+0000 7f532906e640 1 -- 172.21.15.46:0/489264201 >> v1:172.21.15.162:6789/0 conn(0x7f53240d0ed0 legacy=0x7f53240d1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:17.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.959+0000 7f532906e640 1 -- 172.21.15.46:0/489264201 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f532415d9a0 con 0x7f5324160fb0 2023-12-02T00:30:17.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.959+0000 7f532906e640 1 -- 172.21.15.46:0/489264201 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7f5318001670 con 0x7f5324160fb0 2023-12-02T00:30:17.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.959+0000 7f532906e640 1 -- 172.21.15.46:0/489264201 >> v1:172.21.15.46:6789/0 conn(0x7f5324160fb0 legacy=0x7f5324161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:17.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.959+0000 7f532906e640 1 --2- 172.21.15.46:0/489264201 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f530c003970 0x7f530c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:17.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.959+0000 7f532906e640 1 --2- 172.21.15.46:0/489264201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f530c0047c0 0x7f530c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:17.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.959+0000 7f532906e640 1 --2- 172.21.15.46:0/489264201 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f530c005180 0x7f530c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:17.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.959+0000 7f532906e640 1 -- 172.21.15.46:0/489264201 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532415e760 con 0x7f530c0047c0 2023-12-02T00:30:17.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.959+0000 7f532906e640 1 -- 172.21.15.46:0/489264201 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530c00c120 con 0x7f530c003970 2023-12-02T00:30:17.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.959+0000 7f532906e640 1 -- 172.21.15.46:0/489264201 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f530c00c2e0 con 0x7f530c005180 2023-12-02T00:30:17.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.959+0000 7f532986f640 1 --2- 172.21.15.46:0/489264201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f530c0047c0 0x7f530c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:17.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.959+0000 7f532a070640 1 --2- 172.21.15.46:0/489264201 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f530c003970 0x7f530c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:17.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.959+0000 7f532a871640 1 --2- 172.21.15.46:0/489264201 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f530c005180 0x7f530c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:17.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.959+0000 7f532986f640 1 -- 172.21.15.46:0/489264201 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f530c005180 msgr2=0x7f530c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:17.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.959+0000 7f532986f640 1 --2- 172.21.15.46:0/489264201 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f530c005180 0x7f530c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:17.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.959+0000 7f532986f640 1 -- 172.21.15.46:0/489264201 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f530c003970 msgr2=0x7f530c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:17.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.959+0000 7f532986f640 1 --2- 172.21.15.46:0/489264201 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f530c003970 0x7f530c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:17.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.959+0000 7f532986f640 1 -- 172.21.15.46:0/489264201 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f530c003520 con 0x7f530c0047c0 2023-12-02T00:30:17.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.959+0000 7f532a871640 1 --2- 172.21.15.46:0/489264201 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f530c005180 0x7f530c00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:30:17.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.959+0000 7f532a871640 1 -- 172.21.15.46:0/489264201 reap_dead start 2023-12-02T00:30:17.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.960+0000 7f532986f640 1 --2- 172.21.15.46:0/489264201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f530c0047c0 0x7f530c004c40 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f531800c240 tx=0x7f530c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=598709a3faf917d6 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:17.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.960+0000 7f532906e640 1 -- 172.21.15.46:0/489264201 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5318005ad0 con 0x7f530c0047c0 2023-12-02T00:30:17.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.960+0000 7f532906e640 1 -- 172.21.15.46:0/489264201 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5318005c50 con 0x7f530c0047c0 2023-12-02T00:30:17.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.960+0000 7f532906e640 1 -- 172.21.15.46:0/489264201 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531802c8b0 con 0x7f530c0047c0 2023-12-02T00:30:17.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.961+0000 7f532c2d4640 1 -- 172.21.15.46:0/489264201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f530c0047c0 msgr2=0x7f530c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:17.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.961+0000 7f532c2d4640 1 --2- 172.21.15.46:0/489264201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f530c0047c0 0x7f530c004c40 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f531800c240 tx=0x7f530c0057f0 comp rx=0 tx=0).stop 2023-12-02T00:30:17.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.961+0000 7f532c2d4640 1 -- 172.21.15.46:0/489264201 shutdown_connections 2023-12-02T00:30:17.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.961+0000 7f532c2d4640 1 --2- 172.21.15.46:0/489264201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f530c0047c0 0x7f530c004c40 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:17.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.961+0000 7f532c2d4640 1 -- 172.21.15.46:0/489264201 >> 172.21.15.46:0/489264201 conn(0x7f532406a6b0 msgr2=0x7f532406aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:17.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.961+0000 7f532c2d4640 1 -- 172.21.15.46:0/489264201 shutdown_connections 2023-12-02T00:30:17.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.961+0000 7f532c2d4640 1 -- 172.21.15.46:0/489264201 wait complete. 2023-12-02T00:30:17.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.962+0000 7f532c2d4640 1 Processor -- start 2023-12-02T00:30:17.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.962+0000 7f532c2d4640 1 -- start start 2023-12-02T00:30:17.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.963+0000 7f532c2d4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f530c0047c0 0x7f532413cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:17.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.963+0000 7f532c2d4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f532413da90 0x7f53241f2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:17.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.963+0000 7f532c2d4640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f53241f30e0 0x7f53241f3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:17.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.963+0000 7f532c2d4640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53241f3aa0 con 0x7f532413da90 2023-12-02T00:30:17.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.963+0000 7f532c2d4640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532413ee80 con 0x7f53241f30e0 2023-12-02T00:30:17.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.963+0000 7f532c2d4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f532413e270 con 0x7f530c0047c0 2023-12-02T00:30:17.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.963+0000 7f532986f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f532413da90 0x7f53241f2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:17.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.963+0000 7f532986f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f532413da90 0x7f53241f2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57076/0 (socket says 172.21.15.46:57076) 2023-12-02T00:30:17.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.963+0000 7f532986f640 1 -- 172.21.15.46:0/2733104496 learned_addr learned my addr 172.21.15.46:0/2733104496 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:17.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.963+0000 7f532a070640 1 --2- 172.21.15.46:0/2733104496 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f530c0047c0 0x7f532413cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:17.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.963+0000 7f532a871640 1 --2- 172.21.15.46:0/2733104496 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f53241f30e0 0x7f53241f3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:17.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.963+0000 7f532986f640 1 -- 172.21.15.46:0/2733104496 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f530c0047c0 msgr2=0x7f532413cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:17.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.963+0000 7f532986f640 1 --2- 172.21.15.46:0/2733104496 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f530c0047c0 0x7f532413cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:17.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.963+0000 7f532986f640 1 -- 172.21.15.46:0/2733104496 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f53241f30e0 msgr2=0x7f53241f3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:17.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.963+0000 7f532986f640 1 --2- 172.21.15.46:0/2733104496 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f53241f30e0 0x7f53241f3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:17.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.963+0000 7f532986f640 1 -- 172.21.15.46:0/2733104496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f532413e550 con 0x7f532413da90 2023-12-02T00:30:17.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.963+0000 7f532a871640 1 --2- 172.21.15.46:0/2733104496 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f53241f30e0 0x7f53241f3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:30:17.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.963+0000 7f532986f640 1 --2- 172.21.15.46:0/2733104496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f532413da90 0x7f53241f2ba0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f531800c240 tx=0x7f531802e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:17.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.964+0000 7f5322ffd640 1 -- 172.21.15.46:0/2733104496 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531803c040 con 0x7f532413da90 2023-12-02T00:30:17.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.964+0000 7f5322ffd640 1 -- 172.21.15.46:0/2733104496 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f531802ccc0 con 0x7f532413da90 2023-12-02T00:30:17.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.964+0000 7f532c2d4640 1 -- 172.21.15.46:0/2733104496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f532413e830 con 0x7f532413da90 2023-12-02T00:30:17.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.964+0000 7f5322ffd640 1 -- 172.21.15.46:0/2733104496 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5318007290 con 0x7f532413da90 2023-12-02T00:30:17.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.964+0000 7f532c2d4640 1 -- 172.21.15.46:0/2733104496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f532413df10 con 0x7f532413da90 2023-12-02T00:30:17.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.965+0000 7f532c2d4640 1 -- 172.21.15.46:0/2733104496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f532413f010 con 0x7f532413da90 2023-12-02T00:30:17.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.965+0000 7f5322ffd640 1 -- 172.21.15.46:0/2733104496 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5318032070 con 0x7f532413da90 2023-12-02T00:30:17.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.965+0000 7f5322ffd640 1 --2- 172.21.15.46:0/2733104496 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f531006ae70 0x7f531006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:17.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.965+0000 7f5322ffd640 1 -- 172.21.15.46:0/2733104496 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f53180aae60 con 0x7f532413da90 2023-12-02T00:30:17.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.966+0000 7f532a070640 1 --2- 172.21.15.46:0/2733104496 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f531006ae70 0x7f531006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:17.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.966+0000 7f532a070640 1 --2- 172.21.15.46:0/2733104496 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f531006ae70 0x7f531006d330 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f531c009a60 tx=0x7f531c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:17.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:17.970+0000 7f5322ffd640 1 -- 172.21.15.46:0/2733104496 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5318075e00 con 0x7f532413da90 2023-12-02T00:30:18.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.146+0000 7f532c2d4640 1 -- 172.21.15.46:0/2733104496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5324069400 con 0x7f532413da90 2023-12-02T00:30:18.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.146+0000 7f5322ffd640 1 -- 172.21.15.46:0/2733104496 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v38) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5318079420 con 0x7f532413da90 2023-12-02T00:30:18.147 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-12-02T00:30:18.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.148+0000 7f532c2d4640 1 -- 172.21.15.46:0/2733104496 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f531006ae70 msgr2=0x7f531006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:18.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.148+0000 7f532c2d4640 1 --2- 172.21.15.46:0/2733104496 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f531006ae70 0x7f531006d330 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f531c009a60 tx=0x7f531c008040 comp rx=0 tx=0).stop 2023-12-02T00:30:18.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.148+0000 7f532c2d4640 1 -- 172.21.15.46:0/2733104496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f532413da90 msgr2=0x7f53241f2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:18.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.148+0000 7f532c2d4640 1 --2- 172.21.15.46:0/2733104496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f532413da90 0x7f53241f2ba0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f531800c240 tx=0x7f531802e070 comp rx=0 tx=0).stop 2023-12-02T00:30:18.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.148+0000 7f532c2d4640 1 -- 172.21.15.46:0/2733104496 shutdown_connections 2023-12-02T00:30:18.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.148+0000 7f532c2d4640 1 --2- 172.21.15.46:0/2733104496 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f531006ae70 0x7f531006d330 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:18.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.148+0000 7f532c2d4640 1 --2- 172.21.15.46:0/2733104496 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f53241f30e0 0x7f53241f3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:18.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.148+0000 7f532c2d4640 1 --2- 172.21.15.46:0/2733104496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f532413da90 0x7f53241f2ba0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:18.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.148+0000 7f532c2d4640 1 --2- 172.21.15.46:0/2733104496 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f530c0047c0 0x7f532413cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:18.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.148+0000 7f532c2d4640 1 -- 172.21.15.46:0/2733104496 >> 172.21.15.46:0/2733104496 conn(0x7f532406a6b0 msgr2=0x7f532406aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:18.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.148+0000 7f532c2d4640 1 -- 172.21.15.46:0/2733104496 shutdown_connections 2023-12-02T00:30:18.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.148+0000 7f532c2d4640 1 -- 172.21.15.46:0/2733104496 wait complete. 2023-12-02T00:30:18.159 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2023-12-02T00:30:18.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.268+0000 7f00c754c640 1 Processor -- start 2023-12-02T00:30:18.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.268+0000 7f00c754c640 1 -- start start 2023-12-02T00:30:18.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.268+0000 7f00c754c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f00c0143490 0x7f00c00b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:18.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.269+0000 7f00c754c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f00c015e730 0x7f00c00b1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:18.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.269+0000 7f00c754c640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00c015e2a0 con 0x7f00c015a4e0 2023-12-02T00:30:18.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.269+0000 7f00c754c640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00c015b520 con 0x7f00c015e730 2023-12-02T00:30:18.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.269+0000 7f00c754c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00c0143870 con 0x7f00c0143490 2023-12-02T00:30:18.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.269+0000 7f00c4ae7640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f00c015e730 0x7f00c00b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:18.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.269+0000 7f00c4ae7640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f00c015e730 0x7f00c00b1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57090/0 (socket says 172.21.15.46:57090) 2023-12-02T00:30:18.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.269+0000 7f00c4ae7640 1 -- 172.21.15.46:0/2840394697 learned_addr learned my addr 172.21.15.46:0/2840394697 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:18.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.269+0000 7f00c52e8640 1 --2- 172.21.15.46:0/2840394697 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f00c0143490 0x7f00c00b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:18.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.269+0000 7f00c4ae7640 1 -- 172.21.15.46:0/2840394697 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f00c0143490 msgr2=0x7f00c00b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:18.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.269+0000 7f00c4ae7640 1 --2- 172.21.15.46:0/2840394697 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f00c0143490 0x7f00c00b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:18.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.269+0000 7f00c4ae7640 1 -- 172.21.15.46:0/2840394697 >> v1:172.21.15.162:6789/0 conn(0x7f00c015a4e0 legacy=0x7f00c015a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:18.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.269+0000 7f00c4ae7640 1 -- 172.21.15.46:0/2840394697 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c015b240 con 0x7f00c015e730 2023-12-02T00:30:18.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.269+0000 7f00c52e8640 1 --2- 172.21.15.46:0/2840394697 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f00c0143490 0x7f00c00b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:30:18.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.270+0000 7f00c4ae7640 1 --2- 172.21.15.46:0/2840394697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00c015e730 0x7f00c00b1230 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f00b4009870 tx=0x7f00b402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=699ee69d57f2b4f7 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:18.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.270+0000 7f00bffff640 1 -- 172.21.15.46:0/2840394697 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b4031640 con 0x7f00c015e730 2023-12-02T00:30:18.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.270+0000 7f00bffff640 1 -- 172.21.15.46:0/2840394697 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00b40317a0 con 0x7f00c015e730 2023-12-02T00:30:18.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.270+0000 7f00bffff640 1 -- 172.21.15.46:0/2840394697 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b4031b30 con 0x7f00c015e730 2023-12-02T00:30:18.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.270+0000 7f00c754c640 1 -- 172.21.15.46:0/2840394697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00c015e730 msgr2=0x7f00c00b1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:18.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.270+0000 7f00c754c640 1 --2- 172.21.15.46:0/2840394697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00c015e730 0x7f00c00b1230 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f00b4009870 tx=0x7f00b402b8f0 comp rx=0 tx=0).stop 2023-12-02T00:30:18.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.270+0000 7f00c754c640 1 -- 172.21.15.46:0/2840394697 shutdown_connections 2023-12-02T00:30:18.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.270+0000 7f00c754c640 1 --2- 172.21.15.46:0/2840394697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00c015e730 0x7f00c00b1230 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:18.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.270+0000 7f00c754c640 1 --2- 172.21.15.46:0/2840394697 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f00c0143490 0x7f00c00b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:18.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.270+0000 7f00c754c640 1 -- 172.21.15.46:0/2840394697 >> 172.21.15.46:0/2840394697 conn(0x7f00c006a4d0 msgr2=0x7f00c006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:18.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.271+0000 7f00c754c640 1 -- 172.21.15.46:0/2840394697 shutdown_connections 2023-12-02T00:30:18.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.271+0000 7f00c754c640 1 -- 172.21.15.46:0/2840394697 wait complete. 2023-12-02T00:30:18.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.272+0000 7f00c754c640 1 Processor -- start 2023-12-02T00:30:18.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.272+0000 7f00c754c640 1 -- start start 2023-12-02T00:30:18.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.272+0000 7f00c754c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00c0143490 0x7f00c013e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:18.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.272+0000 7f00c754c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f00c015a4e0 0x7f00c0070d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:18.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.272+0000 7f00c754c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00c015e730 0x7f00c00a9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:18.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.272+0000 7f00c754c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c00aa3a0 con 0x7f00c0143490 2023-12-02T00:30:18.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.272+0000 7f00c754c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c00aabe0 con 0x7f00c015e730 2023-12-02T00:30:18.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.273+0000 7f00c754c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00c00aa620 con 0x7f00c015a4e0 2023-12-02T00:30:18.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.273+0000 7f00c52e8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00c0143490 0x7f00c013e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:18.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.273+0000 7f00c52e8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00c0143490 0x7f00c013e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57106/0 (socket says 172.21.15.46:57106) 2023-12-02T00:30:18.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.273+0000 7f00c52e8640 1 -- 172.21.15.46:0/1750909958 learned_addr learned my addr 172.21.15.46:0/1750909958 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:18.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.273+0000 7f00c4ae7640 1 --2- 172.21.15.46:0/1750909958 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f00c015a4e0 0x7f00c0070d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:18.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.273+0000 7f00c5ae9640 1 --2- 172.21.15.46:0/1750909958 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00c015e730 0x7f00c00a9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:18.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.273+0000 7f00c52e8640 1 -- 172.21.15.46:0/1750909958 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f00c015a4e0 msgr2=0x7f00c0070d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:18.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.273+0000 7f00c52e8640 1 --2- 172.21.15.46:0/1750909958 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f00c015a4e0 0x7f00c0070d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:18.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.273+0000 7f00c52e8640 1 -- 172.21.15.46:0/1750909958 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00c015e730 msgr2=0x7f00c00a9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:18.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.273+0000 7f00c52e8640 1 --2- 172.21.15.46:0/1750909958 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00c015e730 0x7f00c00a9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:18.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.273+0000 7f00c52e8640 1 -- 172.21.15.46:0/1750909958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00b40094d0 con 0x7f00c0143490 2023-12-02T00:30:18.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.273+0000 7f00c52e8640 1 --2- 172.21.15.46:0/1750909958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00c0143490 0x7f00c013e490 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f00b800c7f0 tx=0x7f00b800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:18.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.273+0000 7f00bdffb640 1 -- 172.21.15.46:0/1750909958 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b8007a30 con 0x7f00c0143490 2023-12-02T00:30:18.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.273+0000 7f00bdffb640 1 -- 172.21.15.46:0/1750909958 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00b8007b90 con 0x7f00c0143490 2023-12-02T00:30:18.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.273+0000 7f00c754c640 1 -- 172.21.15.46:0/1750909958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00c00aa900 con 0x7f00c0143490 2023-12-02T00:30:18.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.273+0000 7f00bdffb640 1 -- 172.21.15.46:0/1750909958 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b800ce40 con 0x7f00c0143490 2023-12-02T00:30:18.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.274+0000 7f00c754c640 1 -- 172.21.15.46:0/1750909958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00c0070810 con 0x7f00c0143490 2023-12-02T00:30:18.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.275+0000 7f00c754c640 1 -- 172.21.15.46:0/1750909958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00c00b4380 con 0x7f00c0143490 2023-12-02T00:30:18.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.275+0000 7f00bdffb640 1 -- 172.21.15.46:0/1750909958 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f00b80113f0 con 0x7f00c0143490 2023-12-02T00:30:18.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.275+0000 7f00bdffb640 1 --2- 172.21.15.46:0/1750909958 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f00ac06ae40 0x7f00ac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:18.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.275+0000 7f00bdffb640 1 -- 172.21.15.46:0/1750909958 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f00b80898e0 con 0x7f00c0143490 2023-12-02T00:30:18.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.275+0000 7f00c4ae7640 1 --2- 172.21.15.46:0/1750909958 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f00ac06ae40 0x7f00ac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:18.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.276+0000 7f00c4ae7640 1 --2- 172.21.15.46:0/1750909958 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f00ac06ae40 0x7f00ac06d300 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f00b4036000 tx=0x7f00b403c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:18.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.279+0000 7f00bdffb640 1 -- 172.21.15.46:0/1750909958 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f00b80548b0 con 0x7f00c0143490 2023-12-02T00:30:18.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.455+0000 7f00c754c640 1 -- 172.21.15.46:0/1750909958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7f00c0070390 con 0x7f00c0143490 2023-12-02T00:30:18.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.455+0000 7f00bdffb640 1 -- 172.21.15.46:0/1750909958 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v38) v1 ==== 97+0+11 (secure 0 0 0) 0x7f00c0070390 con 0x7f00c0143490 2023-12-02T00:30:18.456 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-12-02T00:30:18.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.457+0000 7f00c754c640 1 -- 172.21.15.46:0/1750909958 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f00ac06ae40 msgr2=0x7f00ac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:18.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.457+0000 7f00c754c640 1 --2- 172.21.15.46:0/1750909958 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f00ac06ae40 0x7f00ac06d300 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f00b4036000 tx=0x7f00b403c040 comp rx=0 tx=0).stop 2023-12-02T00:30:18.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.457+0000 7f00c754c640 1 -- 172.21.15.46:0/1750909958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00c0143490 msgr2=0x7f00c013e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:18.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.457+0000 7f00c754c640 1 --2- 172.21.15.46:0/1750909958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00c0143490 0x7f00c013e490 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f00b800c7f0 tx=0x7f00b800cbf0 comp rx=0 tx=0).stop 2023-12-02T00:30:18.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.457+0000 7f00c754c640 1 -- 172.21.15.46:0/1750909958 shutdown_connections 2023-12-02T00:30:18.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.457+0000 7f00c754c640 1 --2- 172.21.15.46:0/1750909958 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f00ac06ae40 0x7f00ac06d300 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:18.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.457+0000 7f00c754c640 1 --2- 172.21.15.46:0/1750909958 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00c015e730 0x7f00c00a9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:18.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.457+0000 7f00c754c640 1 --2- 172.21.15.46:0/1750909958 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f00c015a4e0 0x7f00c0070d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:18.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.457+0000 7f00c754c640 1 --2- 172.21.15.46:0/1750909958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00c0143490 0x7f00c013e490 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:18.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.457+0000 7f00c754c640 1 -- 172.21.15.46:0/1750909958 >> 172.21.15.46:0/1750909958 conn(0x7f00c006a4d0 msgr2=0x7f00c006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:18.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.457+0000 7f00c754c640 1 -- 172.21.15.46:0/1750909958 shutdown_connections 2023-12-02T00:30:18.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.457+0000 7f00c754c640 1 -- 172.21.15.46:0/1750909958 wait complete. 2023-12-02T00:30:18.469 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-02T00:30:18.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.575+0000 7fd65b368640 1 Processor -- start 2023-12-02T00:30:18.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.575+0000 7fd65b368640 1 -- start start 2023-12-02T00:30:18.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.576+0000 7fd65b368640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd6540ad640 0x7fd65406d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:18.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.576+0000 7fd65b368640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd65406bbf0 0x7fd65406d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:18.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.576+0000 7fd65b368640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6540adae0 con 0x7fd65415d1d0 2023-12-02T00:30:18.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.576+0000 7fd65b368640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6540addc0 con 0x7fd65406bbf0 2023-12-02T00:30:18.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.576+0000 7fd65b368640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6540ae0a0 con 0x7fd6540ad640 2023-12-02T00:30:18.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.576+0000 7fd659905640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd65406bbf0 0x7fd65406d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:18.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.576+0000 7fd659905640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd65406bbf0 0x7fd65406d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57112/0 (socket says 172.21.15.46:57112) 2023-12-02T00:30:18.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.576+0000 7fd659905640 1 -- 172.21.15.46:0/479890361 learned_addr learned my addr 172.21.15.46:0/479890361 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:18.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.576+0000 7fd658903640 1 --2- 172.21.15.46:0/479890361 >> v2:172.21.15.162:3300/0 conn(0x7fd6540ad640 0x7fd65406d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:18.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.576+0000 7fd659905640 1 -- 172.21.15.46:0/479890361 >> v2:172.21.15.162:3300/0 conn(0x7fd6540ad640 msgr2=0x7fd65406d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:18.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.576+0000 7fd659905640 1 --2- 172.21.15.46:0/479890361 >> v2:172.21.15.162:3300/0 conn(0x7fd6540ad640 0x7fd65406d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:18.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.576+0000 7fd659905640 1 -- 172.21.15.46:0/479890361 >> v1:172.21.15.162:6789/0 conn(0x7fd65415d1d0 legacy=0x7fd65406b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:18.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.576+0000 7fd659905640 1 -- 172.21.15.46:0/479890361 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6540ae380 con 0x7fd65406bbf0 2023-12-02T00:30:18.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.577+0000 7fd659905640 1 --2- 172.21.15.46:0/479890361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd65406bbf0 0x7fd65406d5f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd6540cc440 tx=0x7fd64c02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b13dc77fd93dc3 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:18.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.577+0000 7fd64bfff640 1 -- 172.21.15.46:0/479890361 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64c02cb20 con 0x7fd65406bbf0 2023-12-02T00:30:18.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.577+0000 7fd64bfff640 1 -- 172.21.15.46:0/479890361 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd64c02cc80 con 0x7fd65406bbf0 2023-12-02T00:30:18.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.577+0000 7fd64bfff640 1 -- 172.21.15.46:0/479890361 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64c0343f0 con 0x7fd65406bbf0 2023-12-02T00:30:18.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.578+0000 7fd65b368640 1 -- 172.21.15.46:0/479890361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd65406bbf0 msgr2=0x7fd65406d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:18.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.578+0000 7fd65b368640 1 --2- 172.21.15.46:0/479890361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd65406bbf0 0x7fd65406d5f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd6540cc440 tx=0x7fd64c02bf80 comp rx=0 tx=0).stop 2023-12-02T00:30:18.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.578+0000 7fd65b368640 1 -- 172.21.15.46:0/479890361 shutdown_connections 2023-12-02T00:30:18.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.578+0000 7fd65b368640 1 --2- 172.21.15.46:0/479890361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd65406bbf0 0x7fd65406d5f0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:18.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.578+0000 7fd65b368640 1 --2- 172.21.15.46:0/479890361 >> v2:172.21.15.162:3300/0 conn(0x7fd6540ad640 0x7fd65406d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:18.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.578+0000 7fd65b368640 1 -- 172.21.15.46:0/479890361 >> 172.21.15.46:0/479890361 conn(0x7fd654070e70 msgr2=0x7fd654071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:18.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.578+0000 7fd65b368640 1 -- 172.21.15.46:0/479890361 shutdown_connections 2023-12-02T00:30:18.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.578+0000 7fd65b368640 1 -- 172.21.15.46:0/479890361 wait complete. 2023-12-02T00:30:18.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.579+0000 7fd65b368640 1 Processor -- start 2023-12-02T00:30:18.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.579+0000 7fd65b368640 1 -- start start 2023-12-02T00:30:18.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.580+0000 7fd65b368640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd65406bbf0 0x7fd65406dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:18.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.580+0000 7fd65b368640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6540ad640 0x7fd6540cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:18.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.580+0000 7fd65b368640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd65415d1d0 0x7fd65414c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:18.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.580+0000 7fd65b368640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6540cd7e0 con 0x7fd6540ad640 2023-12-02T00:30:18.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.580+0000 7fd65b368640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6540ccf40 con 0x7fd65406bbf0 2023-12-02T00:30:18.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.580+0000 7fd65b368640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6540cd220 con 0x7fd65415d1d0 2023-12-02T00:30:18.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.580+0000 7fd658903640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6540ad640 0x7fd6540cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:18.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.580+0000 7fd658903640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6540ad640 0x7fd6540cc4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57126/0 (socket says 172.21.15.46:57126) 2023-12-02T00:30:18.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.580+0000 7fd658903640 1 -- 172.21.15.46:0/1186395710 learned_addr learned my addr 172.21.15.46:0/1186395710 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:18.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.580+0000 7fd659905640 1 --2- 172.21.15.46:0/1186395710 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd65415d1d0 0x7fd65414c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:18.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.580+0000 7fd659104640 1 --2- 172.21.15.46:0/1186395710 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd65406bbf0 0x7fd65406dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:18.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.580+0000 7fd658903640 1 -- 172.21.15.46:0/1186395710 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd65415d1d0 msgr2=0x7fd65414c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:18.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.580+0000 7fd658903640 1 --2- 172.21.15.46:0/1186395710 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd65415d1d0 0x7fd65414c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:18.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.580+0000 7fd658903640 1 -- 172.21.15.46:0/1186395710 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd65406bbf0 msgr2=0x7fd65406dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:18.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.580+0000 7fd658903640 1 --2- 172.21.15.46:0/1186395710 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd65406bbf0 0x7fd65406dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:18.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.580+0000 7fd658903640 1 -- 172.21.15.46:0/1186395710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd64c009b30 con 0x7fd6540ad640 2023-12-02T00:30:18.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.580+0000 7fd659104640 1 --2- 172.21.15.46:0/1186395710 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd65406bbf0 0x7fd65406dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:30:18.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.580+0000 7fd659905640 1 --2- 172.21.15.46:0/1186395710 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd65415d1d0 0x7fd65414c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:30:18.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.580+0000 7fd658903640 1 --2- 172.21.15.46:0/1186395710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6540ad640 0x7fd6540cc4f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd64400d800 tx=0x7fd64400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:18.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.580+0000 7fd649ffb640 1 -- 172.21.15.46:0/1186395710 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64400b6c0 con 0x7fd6540ad640 2023-12-02T00:30:18.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.580+0000 7fd649ffb640 1 -- 172.21.15.46:0/1186395710 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd64400b820 con 0x7fd6540ad640 2023-12-02T00:30:18.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.581+0000 7fd649ffb640 1 -- 172.21.15.46:0/1186395710 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64400bbb0 con 0x7fd6540ad640 2023-12-02T00:30:18.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.581+0000 7fd65b368640 1 -- 172.21.15.46:0/1186395710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6540cd500 con 0x7fd6540ad640 2023-12-02T00:30:18.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.581+0000 7fd65b368640 1 -- 172.21.15.46:0/1186395710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6541602e0 con 0x7fd6540ad640 2023-12-02T00:30:18.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.582+0000 7fd65b368640 1 -- 172.21.15.46:0/1186395710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd654069400 con 0x7fd6540ad640 2023-12-02T00:30:18.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.582+0000 7fd649ffb640 1 -- 172.21.15.46:0/1186395710 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd64400bd10 con 0x7fd6540ad640 2023-12-02T00:30:18.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.582+0000 7fd649ffb640 1 --2- 172.21.15.46:0/1186395710 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd63806ae40 0x7fd63806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:18.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.582+0000 7fd649ffb640 1 -- 172.21.15.46:0/1186395710 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fd644089d50 con 0x7fd6540ad640 2023-12-02T00:30:18.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.583+0000 7fd659104640 1 --2- 172.21.15.46:0/1186395710 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd63806ae40 0x7fd63806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:18.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.583+0000 7fd659104640 1 --2- 172.21.15.46:0/1186395710 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd63806ae40 0x7fd63806d300 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd650000f60 tx=0x7fd650009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:18.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.587+0000 7fd649ffb640 1 -- 172.21.15.46:0/1186395710 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd644054c60 con 0x7fd6540ad640 2023-12-02T00:30:18.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.776+0000 7fd65b368640 1 -- 172.21.15.46:0/1186395710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd6540b09e0 con 0x7fd6540ad640 2023-12-02T00:30:18.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.776+0000 7fd649ffb640 1 -- 172.21.15.46:0/1186395710 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v38) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd644058280 con 0x7fd6540ad640 2023-12-02T00:30:18.777 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2023-12-02T00:30:18.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.778+0000 7fd65b368640 1 -- 172.21.15.46:0/1186395710 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd63806ae40 msgr2=0x7fd63806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:18.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.778+0000 7fd65b368640 1 --2- 172.21.15.46:0/1186395710 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd63806ae40 0x7fd63806d300 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd650000f60 tx=0x7fd650009040 comp rx=0 tx=0).stop 2023-12-02T00:30:18.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.778+0000 7fd65b368640 1 -- 172.21.15.46:0/1186395710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6540ad640 msgr2=0x7fd6540cc4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:18.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.778+0000 7fd65b368640 1 --2- 172.21.15.46:0/1186395710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6540ad640 0x7fd6540cc4f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd64400d800 tx=0x7fd64400dcc0 comp rx=0 tx=0).stop 2023-12-02T00:30:18.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.778+0000 7fd65b368640 1 -- 172.21.15.46:0/1186395710 shutdown_connections 2023-12-02T00:30:18.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.778+0000 7fd65b368640 1 --2- 172.21.15.46:0/1186395710 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd63806ae40 0x7fd63806d300 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:18.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.778+0000 7fd65b368640 1 --2- 172.21.15.46:0/1186395710 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd65415d1d0 0x7fd65414c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:18.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.778+0000 7fd65b368640 1 --2- 172.21.15.46:0/1186395710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6540ad640 0x7fd6540cc4f0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:18.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.778+0000 7fd65b368640 1 --2- 172.21.15.46:0/1186395710 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd65406bbf0 0x7fd65406dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:18.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.778+0000 7fd65b368640 1 -- 172.21.15.46:0/1186395710 >> 172.21.15.46:0/1186395710 conn(0x7fd654070e70 msgr2=0x7fd6540b0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:18.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.778+0000 7fd65b368640 1 -- 172.21.15.46:0/1186395710 shutdown_connections 2023-12-02T00:30:18.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.778+0000 7fd65b368640 1 -- 172.21.15.46:0/1186395710 wait complete. 2023-12-02T00:30:18.790 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-02T00:30:18.790 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-02T00:30:18.806 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:30:18.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.951+0000 7fa8119a5640 1 Processor -- start 2023-12-02T00:30:18.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.951+0000 7fa8119a5640 1 -- start start 2023-12-02T00:30:18.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.952+0000 7fa8119a5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa80c0709c0 0x7fa80c0cd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:18.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.952+0000 7fa8119a5640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa80c0cdd10 0x7fa80c0cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:18.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.952+0000 7fa8119a5640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa80c06d580 con 0x7fa80c15d270 2023-12-02T00:30:18.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.952+0000 7fa8119a5640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa80c0ce1a0 con 0x7fa80c0cdd10 2023-12-02T00:30:18.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.952+0000 7fa8119a5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa80c0d0c40 con 0x7fa80c0709c0 2023-12-02T00:30:18.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.952+0000 7fa80b7fe640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa80c0cdd10 0x7fa80c0cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:18.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.952+0000 7fa80b7fe640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa80c0cdd10 0x7fa80c0cc6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.46:55162/0 (socket says 172.21.15.46:55162) 2023-12-02T00:30:18.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.952+0000 7fa80a7fc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa80c0709c0 0x7fa80c0cd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:18.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.952+0000 7fa80affd640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fa80c15d270 0x7fa80c06ce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.46:56858/0 (socket says 172.21.15.46:56858) 2023-12-02T00:30:18.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.952+0000 7fa80b7fe640 1 -- 172.21.15.46:0/2140290925 learned_addr learned my addr 172.21.15.46:0/2140290925 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:18.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.953+0000 7fa80b7fe640 1 -- 172.21.15.46:0/2140290925 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa80c0709c0 msgr2=0x7fa80c0cd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:18.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.953+0000 7fa80b7fe640 1 --2- 172.21.15.46:0/2140290925 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa80c0709c0 0x7fa80c0cd7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:18.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.953+0000 7fa80b7fe640 1 -- 172.21.15.46:0/2140290925 >> v1:172.21.15.162:6789/0 conn(0x7fa80c15d270 legacy=0x7fa80c06ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:18.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.953+0000 7fa809ffb640 1 -- 172.21.15.46:0/2140290925 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2214140969 0 0) 0x7fa80c06d580 con 0x7fa80c15d270 2023-12-02T00:30:18.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.953+0000 7fa80b7fe640 1 -- 172.21.15.46:0/2140290925 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa80c0ced90 con 0x7fa80c0cdd10 2023-12-02T00:30:18.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.953+0000 7fa80a7fc640 1 --2- 172.21.15.46:0/2140290925 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa80c0709c0 0x7fa80c0cd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:30:18.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.954+0000 7fa80b7fe640 1 --2- 172.21.15.46:0/2140290925 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa80c0cdd10 0x7fa80c0cc6a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc009e50 tx=0x7fa7fc02bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff2a6cda77fb1e88 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:18.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.954+0000 7fa809ffb640 1 -- 172.21.15.46:0/2140290925 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc02ca70 con 0x7fa80c0cdd10 2023-12-02T00:30:18.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.954+0000 7fa809ffb640 1 -- 172.21.15.46:0/2140290925 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7fc02cbd0 con 0x7fa80c0cdd10 2023-12-02T00:30:18.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.954+0000 7fa809ffb640 1 -- 172.21.15.46:0/2140290925 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc0343f0 con 0x7fa80c0cdd10 2023-12-02T00:30:18.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.954+0000 7fa8119a5640 1 -- 172.21.15.46:0/2140290925 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa80c0cdd10 msgr2=0x7fa80c0cc6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:18.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.954+0000 7fa8119a5640 1 --2- 172.21.15.46:0/2140290925 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa80c0cdd10 0x7fa80c0cc6a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc009e50 tx=0x7fa7fc02bed0 comp rx=0 tx=0).stop 2023-12-02T00:30:18.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.954+0000 7fa8119a5640 1 -- 172.21.15.46:0/2140290925 shutdown_connections 2023-12-02T00:30:18.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.954+0000 7fa8119a5640 1 --2- 172.21.15.46:0/2140290925 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa80c0709c0 0x7fa80c0cd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:18.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.954+0000 7fa8119a5640 1 --2- 172.21.15.46:0/2140290925 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa80c0cdd10 0x7fa80c0cc6a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:18.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.954+0000 7fa8119a5640 1 -- 172.21.15.46:0/2140290925 >> 172.21.15.46:0/2140290925 conn(0x7fa80c0b2c10 msgr2=0x7fa80c0b2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:18.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.954+0000 7fa8119a5640 1 -- 172.21.15.46:0/2140290925 shutdown_connections 2023-12-02T00:30:18.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.954+0000 7fa8119a5640 1 -- 172.21.15.46:0/2140290925 wait complete. 2023-12-02T00:30:18.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.955+0000 7fa8119a5640 1 Processor -- start 2023-12-02T00:30:18.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.956+0000 7fa8119a5640 1 -- start start 2023-12-02T00:30:18.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.956+0000 7fa8119a5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa80c0709c0 0x7fa80c0cabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:18.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.956+0000 7fa8119a5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa80c0cdd10 0x7fa80c15e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:18.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.956+0000 7fa8119a5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa80c15d270 0x7fa80c1e87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:18.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.956+0000 7fa8119a5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa80c0c9b50 con 0x7fa80c15d270 2023-12-02T00:30:18.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.956+0000 7fa8119a5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa80c0c9e30 con 0x7fa80c0cdd10 2023-12-02T00:30:18.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.956+0000 7fa8119a5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa80c0ca110 con 0x7fa80c0709c0 2023-12-02T00:30:18.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.957+0000 7fa80b7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa80c15d270 0x7fa80c1e87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:18.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.957+0000 7fa80b7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa80c15d270 0x7fa80c1e87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57136/0 (socket says 172.21.15.46:57136) 2023-12-02T00:30:18.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.957+0000 7fa80affd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa80c0709c0 0x7fa80c0cabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:18.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.957+0000 7fa80a7fc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa80c0cdd10 0x7fa80c15e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:18.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.957+0000 7fa80b7fe640 1 -- 172.21.15.46:0/3573567748 learned_addr learned my addr 172.21.15.46:0/3573567748 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:18.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.957+0000 7fa80b7fe640 1 -- 172.21.15.46:0/3573567748 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa80c0709c0 msgr2=0x7fa80c0cabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:18.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.957+0000 7fa80b7fe640 1 --2- 172.21.15.46:0/3573567748 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa80c0709c0 0x7fa80c0cabf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:18.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.957+0000 7fa80b7fe640 1 -- 172.21.15.46:0/3573567748 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa80c0cdd10 msgr2=0x7fa80c15e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:18.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.957+0000 7fa80b7fe640 1 --2- 172.21.15.46:0/3573567748 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa80c0cdd10 0x7fa80c15e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:18.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.957+0000 7fa80b7fe640 1 -- 172.21.15.46:0/3573567748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa800009400 con 0x7fa80c15d270 2023-12-02T00:30:18.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.957+0000 7fa80affd640 1 --2- 172.21.15.46:0/3573567748 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa80c0709c0 0x7fa80c0cabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:30:18.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.957+0000 7fa80a7fc640 1 --2- 172.21.15.46:0/3573567748 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa80c0cdd10 0x7fa80c15e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:30:18.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.957+0000 7fa80b7fe640 1 --2- 172.21.15.46:0/3573567748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa80c15d270 0x7fa80c1e87b0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc036040 tx=0x7fa7fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:18.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.957+0000 7fa7f3fff640 1 -- 172.21.15.46:0/3573567748 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc009510 con 0x7fa80c15d270 2023-12-02T00:30:18.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.957+0000 7fa7f3fff640 1 -- 172.21.15.46:0/3573567748 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7fc03e450 con 0x7fa80c15d270 2023-12-02T00:30:18.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.957+0000 7fa8119a5640 1 -- 172.21.15.46:0/3573567748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7fc009b30 con 0x7fa80c15d270 2023-12-02T00:30:18.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.957+0000 7fa7f3fff640 1 -- 172.21.15.46:0/3573567748 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc03e5e0 con 0x7fa80c15d270 2023-12-02T00:30:18.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.957+0000 7fa8119a5640 1 -- 172.21.15.46:0/3573567748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa80c0ca6d0 con 0x7fa80c15d270 2023-12-02T00:30:18.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.958+0000 7fa8119a5640 1 -- 172.21.15.46:0/3573567748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa80c067fc0 con 0x7fa80c15d270 2023-12-02T00:30:18.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.959+0000 7fa7f3fff640 1 -- 172.21.15.46:0/3573567748 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa7fc03e740 con 0x7fa80c15d270 2023-12-02T00:30:18.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.959+0000 7fa7f3fff640 1 --2- 172.21.15.46:0/3573567748 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa7f806ae40 0x7fa7f806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:18.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.959+0000 7fa7f3fff640 1 -- 172.21.15.46:0/3573567748 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fa7fc0ac0a0 con 0x7fa80c15d270 2023-12-02T00:30:18.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.959+0000 7fa80affd640 1 --2- 172.21.15.46:0/3573567748 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa7f806ae40 0x7fa7f806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:18.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.960+0000 7fa80affd640 1 --2- 172.21.15.46:0/3573567748 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa7f806ae40 0x7fa7f806d300 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa804000f60 tx=0x7fa804007d50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:18.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:18.963+0000 7fa7f3fff640 1 -- 172.21.15.46:0/3573567748 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa7fc076f50 con 0x7fa80c15d270 2023-12-02T00:30:19.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.142+0000 7fa8119a5640 1 -- 172.21.15.46:0/3573567748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa80c06f500 con 0x7fa80c15d270 2023-12-02T00:30:19.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.143+0000 7fa7f3fff640 1 -- 172.21.15.46:0/3573567748 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+18324 (secure 0 0 0) 0x7fa7fc07a570 con 0x7fa80c15d270 2023-12-02T00:30:19.145 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:30:19.145 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":38,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:30:10.714057+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-02T00:29:55.676572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-02T00:29:56.244869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"data_new","create_time":"2023-12-02T00:30:08.897990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:30:19.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.145+0000 7fa8119a5640 1 -- 172.21.15.46:0/3573567748 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa7f806ae40 msgr2=0x7fa7f806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:19.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.145+0000 7fa8119a5640 1 --2- 172.21.15.46:0/3573567748 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa7f806ae40 0x7fa7f806d300 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa804000f60 tx=0x7fa804007d50 comp rx=0 tx=0).stop 2023-12-02T00:30:19.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.145+0000 7fa8119a5640 1 -- 172.21.15.46:0/3573567748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa80c15d270 msgr2=0x7fa80c1e87b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:19.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.145+0000 7fa8119a5640 1 --2- 172.21.15.46:0/3573567748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa80c15d270 0x7fa80c1e87b0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc036040 tx=0x7fa7fc004a40 comp rx=0 tx=0).stop 2023-12-02T00:30:19.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.145+0000 7fa8119a5640 1 -- 172.21.15.46:0/3573567748 shutdown_connections 2023-12-02T00:30:19.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.145+0000 7fa8119a5640 1 --2- 172.21.15.46:0/3573567748 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa7f806ae40 0x7fa7f806d300 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:19.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.145+0000 7fa8119a5640 1 --2- 172.21.15.46:0/3573567748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa80c15d270 0x7fa80c1e87b0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:19.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.145+0000 7fa8119a5640 1 --2- 172.21.15.46:0/3573567748 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa80c0cdd10 0x7fa80c15e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:19.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.145+0000 7fa8119a5640 1 --2- 172.21.15.46:0/3573567748 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa80c0709c0 0x7fa80c0cabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:19.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.145+0000 7fa8119a5640 1 -- 172.21.15.46:0/3573567748 >> 172.21.15.46:0/3573567748 conn(0x7fa80c0b2c10 msgr2=0x7fa80c0b2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:19.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.145+0000 7fa8119a5640 1 -- 172.21.15.46:0/3573567748 shutdown_connections 2023-12-02T00:30:19.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.145+0000 7fa8119a5640 1 -- 172.21.15.46:0/3573567748 wait complete. 2023-12-02T00:30:19.157 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-02T00:28:37.213645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-02T00:29:55.676572+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-12-02T00:29:56.244869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'data_new', 'create_time': '2023-12-02T00:30:08.897990+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-02T00:30:19.158 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-02T00:30:19.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.266+0000 7f984703d640 1 Processor -- start 2023-12-02T00:30:19.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.266+0000 7f984703d640 1 -- start start 2023-12-02T00:30:19.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.267+0000 7f984703d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f98400ceb70 0x7f98400cc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:19.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.267+0000 7f984703d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f98400b4370 0x7f98400b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:19.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.267+0000 7f984703d640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9840071770 con 0x7f9840144980 2023-12-02T00:30:19.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.267+0000 7f984703d640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f984015ced0 con 0x7f98400b4370 2023-12-02T00:30:19.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.267+0000 7f984703d640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98400b4ce0 con 0x7f98400ceb70 2023-12-02T00:30:19.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.267+0000 7f98455da640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f98400b4370 0x7f98400b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:19.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.267+0000 7f9844dd9640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9840144980 0x7f984015c7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59170/0 (socket says 172.21.15.46:59170) 2023-12-02T00:30:19.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.267+0000 7f9844dd9640 1 -- 172.21.15.46:0/189980915 learned_addr learned my addr 172.21.15.46:0/189980915 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:30:19.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.267+0000 7f983ffff640 1 --2- 172.21.15.46:0/189980915 >> v2:172.21.15.162:3300/0 conn(0x7f98400ceb70 0x7f98400cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:19.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.267+0000 7f98455da640 1 -- 172.21.15.46:0/189980915 >> v2:172.21.15.162:3300/0 conn(0x7f98400ceb70 msgr2=0x7f98400cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:19.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.267+0000 7f98455da640 1 --2- 172.21.15.46:0/189980915 >> v2:172.21.15.162:3300/0 conn(0x7f98400ceb70 0x7f98400cc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:19.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.267+0000 7f98455da640 1 -- 172.21.15.46:0/189980915 >> v1:172.21.15.46:6789/0 conn(0x7f9840144980 legacy=0x7f984015c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:19.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.267+0000 7f98455da640 1 -- 172.21.15.46:0/189980915 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98400ccdc0 con 0x7f98400b4370 2023-12-02T00:30:19.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.267+0000 7f983f7fe640 1 -- 172.21.15.46:0/189980915 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2414401589 0 0) 0x7f9840071770 con 0x7f9840144980 2023-12-02T00:30:19.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.268+0000 7f98455da640 1 --2- 172.21.15.46:0/189980915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98400b4370 0x7f98400b47a0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f984015b600 tx=0x7f983002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=439c55948afd0660 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:19.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.268+0000 7f983f7fe640 1 -- 172.21.15.46:0/189980915 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9830031830 con 0x7f98400b4370 2023-12-02T00:30:19.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.268+0000 7f983f7fe640 1 -- 172.21.15.46:0/189980915 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9830031990 con 0x7f98400b4370 2023-12-02T00:30:19.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.268+0000 7f983f7fe640 1 -- 172.21.15.46:0/189980915 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9830031d20 con 0x7f98400b4370 2023-12-02T00:30:19.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.268+0000 7f984703d640 1 -- 172.21.15.46:0/189980915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98400b4370 msgr2=0x7f98400b47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:19.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.268+0000 7f984703d640 1 --2- 172.21.15.46:0/189980915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98400b4370 0x7f98400b47a0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f984015b600 tx=0x7f983002b8d0 comp rx=0 tx=0).stop 2023-12-02T00:30:19.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.269+0000 7f984703d640 1 -- 172.21.15.46:0/189980915 shutdown_connections 2023-12-02T00:30:19.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.269+0000 7f984703d640 1 --2- 172.21.15.46:0/189980915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98400b4370 0x7f98400b47a0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:19.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.269+0000 7f984703d640 1 --2- 172.21.15.46:0/189980915 >> v2:172.21.15.162:3300/0 conn(0x7f98400ceb70 0x7f98400cc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:19.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.269+0000 7f984703d640 1 -- 172.21.15.46:0/189980915 >> 172.21.15.46:0/189980915 conn(0x7f984006a4d0 msgr2=0x7f984006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:19.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.269+0000 7f984703d640 1 -- 172.21.15.46:0/189980915 shutdown_connections 2023-12-02T00:30:19.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.269+0000 7f984703d640 1 -- 172.21.15.46:0/189980915 wait complete. 2023-12-02T00:30:19.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.270+0000 7f984703d640 1 Processor -- start 2023-12-02T00:30:19.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.270+0000 7f984703d640 1 -- start start 2023-12-02T00:30:19.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.271+0000 7f984703d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98400b4370 0x7f9840159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:19.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.271+0000 7f984703d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98400ceb70 0x7f9840154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:19.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.271+0000 7f984703d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9840144980 0x7f98400a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:19.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.271+0000 7f984703d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9840154ff0 con 0x7f98400ceb70 2023-12-02T00:30:19.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.271+0000 7f984703d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98401360c0 con 0x7f98400b4370 2023-12-02T00:30:19.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.271+0000 7f984703d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98401354b0 con 0x7f9840144980 2023-12-02T00:30:19.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.271+0000 7f983ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98400ceb70 0x7f9840154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:19.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.271+0000 7f983ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98400ceb70 0x7f9840154ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57158/0 (socket says 172.21.15.46:57158) 2023-12-02T00:30:19.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.271+0000 7f983ffff640 1 -- 172.21.15.46:0/3678250601 learned_addr learned my addr 172.21.15.46:0/3678250601 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:19.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.271+0000 7f9844dd9640 1 --2- 172.21.15.46:0/3678250601 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98400b4370 0x7f9840159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:19.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.271+0000 7f98455da640 1 --2- 172.21.15.46:0/3678250601 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9840144980 0x7f98400a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:19.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.271+0000 7f983ffff640 1 -- 172.21.15.46:0/3678250601 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9840144980 msgr2=0x7f98400a84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:19.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.271+0000 7f983ffff640 1 --2- 172.21.15.46:0/3678250601 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9840144980 0x7f98400a84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:19.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.271+0000 7f983ffff640 1 -- 172.21.15.46:0/3678250601 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98400b4370 msgr2=0x7f9840159cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:19.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.271+0000 7f983ffff640 1 --2- 172.21.15.46:0/3678250601 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98400b4370 0x7f9840159cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:19.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.271+0000 7f983ffff640 1 -- 172.21.15.46:0/3678250601 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9830009400 con 0x7f98400ceb70 2023-12-02T00:30:19.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.271+0000 7f983ffff640 1 --2- 172.21.15.46:0/3678250601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98400ceb70 0x7f9840154ab0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f982800d800 tx=0x7f982800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:19.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.271+0000 7f983d7fa640 1 -- 172.21.15.46:0/3678250601 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982800b6c0 con 0x7f98400ceb70 2023-12-02T00:30:19.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.271+0000 7f983d7fa640 1 -- 172.21.15.46:0/3678250601 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f982800b820 con 0x7f98400ceb70 2023-12-02T00:30:19.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.271+0000 7f984703d640 1 -- 172.21.15.46:0/3678250601 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9840135790 con 0x7f98400ceb70 2023-12-02T00:30:19.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.271+0000 7f983d7fa640 1 -- 172.21.15.46:0/3678250601 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982800bbb0 con 0x7f98400ceb70 2023-12-02T00:30:19.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.271+0000 7f984703d640 1 -- 172.21.15.46:0/3678250601 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9840135d50 con 0x7f98400ceb70 2023-12-02T00:30:19.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.272+0000 7f984703d640 1 -- 172.21.15.46:0/3678250601 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98400cbc70 con 0x7f98400ceb70 2023-12-02T00:30:19.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.273+0000 7f983d7fa640 1 -- 172.21.15.46:0/3678250601 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f982800bd10 con 0x7f98400ceb70 2023-12-02T00:30:19.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.273+0000 7f983d7fa640 1 --2- 172.21.15.46:0/3678250601 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f982006ae40 0x7f982006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:19.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.273+0000 7f983d7fa640 1 -- 172.21.15.46:0/3678250601 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f9828089c90 con 0x7f98400ceb70 2023-12-02T00:30:19.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.273+0000 7f9844dd9640 1 --2- 172.21.15.46:0/3678250601 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f982006ae40 0x7f982006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:19.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.274+0000 7f9844dd9640 1 --2- 172.21.15.46:0/3678250601 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f982006ae40 0x7f982006d300 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9838007ed0 tx=0x7f9838007e60 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:19.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.277+0000 7f983d7fa640 1 -- 172.21.15.46:0/3678250601 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9828054c40 con 0x7f98400ceb70 2023-12-02T00:30:19.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.451+0000 7f984703d640 1 -- 172.21.15.46:0/3678250601 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9840069400 con 0x7f98400ceb70 2023-12-02T00:30:19.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.452+0000 7f983d7fa640 1 -- 172.21.15.46:0/3678250601 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9828058260 con 0x7f98400ceb70 2023-12-02T00:30:19.453 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-12-02T00:30:19.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.453+0000 7f984703d640 1 -- 172.21.15.46:0/3678250601 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f982006ae40 msgr2=0x7f982006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:19.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.453+0000 7f984703d640 1 --2- 172.21.15.46:0/3678250601 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f982006ae40 0x7f982006d300 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9838007ed0 tx=0x7f9838007e60 comp rx=0 tx=0).stop 2023-12-02T00:30:19.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.453+0000 7f984703d640 1 -- 172.21.15.46:0/3678250601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98400ceb70 msgr2=0x7f9840154ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:19.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.453+0000 7f984703d640 1 --2- 172.21.15.46:0/3678250601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98400ceb70 0x7f9840154ab0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f982800d800 tx=0x7f982800dcc0 comp rx=0 tx=0).stop 2023-12-02T00:30:19.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.454+0000 7f984703d640 1 -- 172.21.15.46:0/3678250601 shutdown_connections 2023-12-02T00:30:19.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.454+0000 7f984703d640 1 --2- 172.21.15.46:0/3678250601 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f982006ae40 0x7f982006d300 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:19.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.454+0000 7f984703d640 1 --2- 172.21.15.46:0/3678250601 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9840144980 0x7f98400a84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:19.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.454+0000 7f984703d640 1 --2- 172.21.15.46:0/3678250601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98400ceb70 0x7f9840154ab0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:19.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.454+0000 7f984703d640 1 --2- 172.21.15.46:0/3678250601 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98400b4370 0x7f9840159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:19.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.454+0000 7f984703d640 1 -- 172.21.15.46:0/3678250601 >> 172.21.15.46:0/3678250601 conn(0x7f984006a4d0 msgr2=0x7f98400b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:19.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.454+0000 7f984703d640 1 -- 172.21.15.46:0/3678250601 shutdown_connections 2023-12-02T00:30:19.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.454+0000 7f984703d640 1 -- 172.21.15.46:0/3678250601 wait complete. 2023-12-02T00:30:19.466 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-02T00:30:19.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.575+0000 7f47e891d640 1 Processor -- start 2023-12-02T00:30:19.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.575+0000 7f47e891d640 1 -- start start 2023-12-02T00:30:19.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.575+0000 7f47e891d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f47e415d330 0x7f47e40b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:19.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.575+0000 7f47e891d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47e40d3900 0x7f47e40b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:19.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.575+0000 7f47e891d640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47e40b2b10 con 0x7f47e40d2910 2023-12-02T00:30:19.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.575+0000 7f47e891d640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47e40d1590 con 0x7f47e415d330 2023-12-02T00:30:19.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.575+0000 7f47e891d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47e40d1870 con 0x7f47e40d3900 2023-12-02T00:30:19.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.576+0000 7f47e2d9d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47e40d3900 0x7f47e40b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:19.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.576+0000 7f47e2d9d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47e40d3900 0x7f47e40b6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.46:39580/0 (socket says 172.21.15.46:39580) 2023-12-02T00:30:19.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.576+0000 7f47e259c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f47e415d330 0x7f47e40b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:19.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.576+0000 7f47e2d9d640 1 -- 172.21.15.46:0/656415418 learned_addr learned my addr 172.21.15.46:0/656415418 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:19.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.576+0000 7f47e2d9d640 1 -- 172.21.15.46:0/656415418 >> v2:172.21.15.162:3300/0 conn(0x7f47e415d330 msgr2=0x7f47e40b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:19.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.576+0000 7f47e2d9d640 1 --2- 172.21.15.46:0/656415418 >> v2:172.21.15.162:3300/0 conn(0x7f47e415d330 0x7f47e40b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:19.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.576+0000 7f47e2d9d640 1 -- 172.21.15.46:0/656415418 >> v1:172.21.15.162:6789/0 conn(0x7f47e40d2910 legacy=0x7f47e40d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:19.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.576+0000 7f47e2d9d640 1 -- 172.21.15.46:0/656415418 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e40d1b50 con 0x7f47e40d3900 2023-12-02T00:30:19.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.576+0000 7f47e259c640 1 --2- 172.21.15.46:0/656415418 >> v2:172.21.15.162:3300/0 conn(0x7f47e415d330 0x7f47e40b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:30:19.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.577+0000 7f47e2d9d640 1 --2- 172.21.15.46:0/656415418 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47e40d3900 0x7f47e40b6390 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f47dc009e50 tx=0x7f47dc02bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8456365c5efbe49 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:19.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.577+0000 7f47e1d9b640 1 -- 172.21.15.46:0/656415418 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47dc02ca70 con 0x7f47e40d3900 2023-12-02T00:30:19.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.577+0000 7f47e1d9b640 1 -- 172.21.15.46:0/656415418 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47dc02cbd0 con 0x7f47e40d3900 2023-12-02T00:30:19.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.577+0000 7f47e1d9b640 1 -- 172.21.15.46:0/656415418 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47dc0343f0 con 0x7f47e40d3900 2023-12-02T00:30:19.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.578+0000 7f47e891d640 1 -- 172.21.15.46:0/656415418 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47e40d3900 msgr2=0x7f47e40b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:19.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.578+0000 7f47e891d640 1 --2- 172.21.15.46:0/656415418 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47e40d3900 0x7f47e40b6390 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f47dc009e50 tx=0x7f47dc02bed0 comp rx=0 tx=0).stop 2023-12-02T00:30:19.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.578+0000 7f47e891d640 1 -- 172.21.15.46:0/656415418 shutdown_connections 2023-12-02T00:30:19.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.578+0000 7f47e891d640 1 --2- 172.21.15.46:0/656415418 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47e40d3900 0x7f47e40b6390 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:19.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.578+0000 7f47e891d640 1 --2- 172.21.15.46:0/656415418 >> v2:172.21.15.162:3300/0 conn(0x7f47e415d330 0x7f47e40b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:19.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.578+0000 7f47e891d640 1 -- 172.21.15.46:0/656415418 >> 172.21.15.46:0/656415418 conn(0x7f47e4072020 msgr2=0x7f47e4072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:19.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.578+0000 7f47e891d640 1 -- 172.21.15.46:0/656415418 shutdown_connections 2023-12-02T00:30:19.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.578+0000 7f47e891d640 1 -- 172.21.15.46:0/656415418 wait complete. 2023-12-02T00:30:19.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.579+0000 7f47e891d640 1 Processor -- start 2023-12-02T00:30:19.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.579+0000 7f47e891d640 1 -- start start 2023-12-02T00:30:19.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.580+0000 7f47e891d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47e40d2910 0x7f47e4160580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:19.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.580+0000 7f47e891d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47e40d3900 0x7f47e415ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:19.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.580+0000 7f47e891d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47e415d330 0x7f47e40c8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:19.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.580+0000 7f47e891d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e415de60 con 0x7f47e40d3900 2023-12-02T00:30:19.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.580+0000 7f47e891d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e415e140 con 0x7f47e415d330 2023-12-02T00:30:19.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.580+0000 7f47e891d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47e415e420 con 0x7f47e40d2910 2023-12-02T00:30:19.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.580+0000 7f47dbfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47e40d3900 0x7f47e415ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:19.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.580+0000 7f47dbfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47e40d3900 0x7f47e415ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57166/0 (socket says 172.21.15.46:57166) 2023-12-02T00:30:19.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.580+0000 7f47dbfff640 1 -- 172.21.15.46:0/4081221320 learned_addr learned my addr 172.21.15.46:0/4081221320 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:19.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.580+0000 7f47e2d9d640 1 --2- 172.21.15.46:0/4081221320 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47e415d330 0x7f47e40c8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:19.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.580+0000 7f47e259c640 1 --2- 172.21.15.46:0/4081221320 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47e40d2910 0x7f47e4160580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:19.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.580+0000 7f47dbfff640 1 -- 172.21.15.46:0/4081221320 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47e40d2910 msgr2=0x7f47e4160580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:19.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.580+0000 7f47dbfff640 1 --2- 172.21.15.46:0/4081221320 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47e40d2910 0x7f47e4160580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:19.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.580+0000 7f47dbfff640 1 -- 172.21.15.46:0/4081221320 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47e415d330 msgr2=0x7f47e40c8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:19.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.580+0000 7f47dbfff640 1 --2- 172.21.15.46:0/4081221320 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47e415d330 0x7f47e40c8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:19.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.581+0000 7f47dbfff640 1 -- 172.21.15.46:0/4081221320 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d4009400 con 0x7f47e40d3900 2023-12-02T00:30:19.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.581+0000 7f47e2d9d640 1 --2- 172.21.15.46:0/4081221320 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47e415d330 0x7f47e40c8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:30:19.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.581+0000 7f47dbfff640 1 --2- 172.21.15.46:0/4081221320 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47e40d3900 0x7f47e415ef00 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f47cc00c450 tx=0x7f47cc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:19.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.581+0000 7f47daffd640 1 -- 172.21.15.46:0/4081221320 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc0052d0 con 0x7f47e40d3900 2023-12-02T00:30:19.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.581+0000 7f47daffd640 1 -- 172.21.15.46:0/4081221320 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47cc005430 con 0x7f47e40d3900 2023-12-02T00:30:19.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.581+0000 7f47e891d640 1 -- 172.21.15.46:0/4081221320 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47dc009b30 con 0x7f47e40d3900 2023-12-02T00:30:19.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.581+0000 7f47daffd640 1 -- 172.21.15.46:0/4081221320 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc012870 con 0x7f47e40d3900 2023-12-02T00:30:19.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.581+0000 7f47e891d640 1 -- 172.21.15.46:0/4081221320 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47e415e9e0 con 0x7f47e40d3900 2023-12-02T00:30:19.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.582+0000 7f47e891d640 1 -- 172.21.15.46:0/4081221320 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47e4069400 con 0x7f47e40d3900 2023-12-02T00:30:19.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.582+0000 7f47daffd640 1 -- 172.21.15.46:0/4081221320 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f47cc0129d0 con 0x7f47e40d3900 2023-12-02T00:30:19.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.583+0000 7f47daffd640 1 --2- 172.21.15.46:0/4081221320 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f47c406ae40 0x7f47c406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:19.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.583+0000 7f47daffd640 1 -- 172.21.15.46:0/4081221320 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f47cc089250 con 0x7f47e40d3900 2023-12-02T00:30:19.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.583+0000 7f47e259c640 1 --2- 172.21.15.46:0/4081221320 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f47c406ae40 0x7f47c406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:19.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.584+0000 7f47e259c640 1 --2- 172.21.15.46:0/4081221320 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f47c406ae40 0x7f47c406d300 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f47d4009bc0 tx=0x7f47d4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:19.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.587+0000 7f47daffd640 1 -- 172.21.15.46:0/4081221320 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f47cc054200 con 0x7f47e40d3900 2023-12-02T00:30:19.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.765+0000 7f47e891d640 1 -- 172.21.15.46:0/4081221320 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f47e4159e60 con 0x7f47e40d3900 2023-12-02T00:30:19.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.766+0000 7f47daffd640 1 -- 172.21.15.46:0/4081221320 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v38) v1 ==== 104+0+11 (secure 0 0 0) 0x7f47cc057820 con 0x7f47e40d3900 2023-12-02T00:30:19.767 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-12-02T00:30:19.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.768+0000 7f47e891d640 1 -- 172.21.15.46:0/4081221320 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f47c406ae40 msgr2=0x7f47c406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:19.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.768+0000 7f47e891d640 1 --2- 172.21.15.46:0/4081221320 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f47c406ae40 0x7f47c406d300 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f47d4009bc0 tx=0x7f47d4008040 comp rx=0 tx=0).stop 2023-12-02T00:30:19.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.768+0000 7f47e891d640 1 -- 172.21.15.46:0/4081221320 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47e40d3900 msgr2=0x7f47e415ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:19.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.768+0000 7f47e891d640 1 --2- 172.21.15.46:0/4081221320 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47e40d3900 0x7f47e415ef00 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f47cc00c450 tx=0x7f47cc00c910 comp rx=0 tx=0).stop 2023-12-02T00:30:19.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.768+0000 7f47e891d640 1 -- 172.21.15.46:0/4081221320 shutdown_connections 2023-12-02T00:30:19.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.768+0000 7f47e891d640 1 --2- 172.21.15.46:0/4081221320 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f47c406ae40 0x7f47c406d300 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:19.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.768+0000 7f47e891d640 1 --2- 172.21.15.46:0/4081221320 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47e415d330 0x7f47e40c8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:19.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.768+0000 7f47e891d640 1 --2- 172.21.15.46:0/4081221320 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47e40d3900 0x7f47e415ef00 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:19.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.768+0000 7f47e891d640 1 --2- 172.21.15.46:0/4081221320 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47e40d2910 0x7f47e4160580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:19.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.768+0000 7f47e891d640 1 -- 172.21.15.46:0/4081221320 >> 172.21.15.46:0/4081221320 conn(0x7f47e4072020 msgr2=0x7f47e40d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:19.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.768+0000 7f47e891d640 1 -- 172.21.15.46:0/4081221320 shutdown_connections 2023-12-02T00:30:19.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.768+0000 7f47e891d640 1 -- 172.21.15.46:0/4081221320 wait complete. 2023-12-02T00:30:19.780 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-02T00:30:19.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.888+0000 7fda1c2bb640 1 Processor -- start 2023-12-02T00:30:19.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.888+0000 7fda1c2bb640 1 -- start start 2023-12-02T00:30:19.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.889+0000 7fda1c2bb640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fda140d2910 0x7fda140d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:19.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.889+0000 7fda1c2bb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda140d3730 0x7fda140b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:19.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.889+0000 7fda1c2bb640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda140b2b10 con 0x7fda1415d330 2023-12-02T00:30:19.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.889+0000 7fda1c2bb640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda140d1590 con 0x7fda140d2910 2023-12-02T00:30:19.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.889+0000 7fda1c2bb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda140d1870 con 0x7fda140d3730 2023-12-02T00:30:19.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.889+0000 7fda19856640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fda140d2910 0x7fda140d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:19.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.889+0000 7fda1a057640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fda1415d330 0x7fda140b8b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59182/0 (socket says 172.21.15.46:59182) 2023-12-02T00:30:19.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.889+0000 7fda19856640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fda140d2910 0x7fda140d31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57168/0 (socket says 172.21.15.46:57168) 2023-12-02T00:30:19.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.889+0000 7fda1a057640 1 -- 172.21.15.46:0/1903580362 learned_addr learned my addr 172.21.15.46:0/1903580362 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:30:19.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.890+0000 7fda1a858640 1 --2- 172.21.15.46:0/1903580362 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda140d3730 0x7fda140b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:19.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.890+0000 7fda19856640 1 -- 172.21.15.46:0/1903580362 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda140d3730 msgr2=0x7fda140b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:19.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.890+0000 7fda19856640 1 --2- 172.21.15.46:0/1903580362 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda140d3730 0x7fda140b6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:19.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.890+0000 7fda19856640 1 -- 172.21.15.46:0/1903580362 >> v1:172.21.15.46:6789/0 conn(0x7fda1415d330 legacy=0x7fda140b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:19.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.890+0000 7fda19856640 1 -- 172.21.15.46:0/1903580362 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda140d3b90 con 0x7fda140d2910 2023-12-02T00:30:19.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.890+0000 7fda19055640 1 -- 172.21.15.46:0/1903580362 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3855108910 0 0) 0x7fda140b2b10 con 0x7fda1415d330 2023-12-02T00:30:19.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.890+0000 7fda19856640 1 --2- 172.21.15.46:0/1903580362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda140d2910 0x7fda140d31f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fda080097a0 tx=0x7fda0802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b5c14518704f85e server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:19.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.891+0000 7fda19055640 1 -- 172.21.15.46:0/1903580362 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0802e070 con 0x7fda140d2910 2023-12-02T00:30:19.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.891+0000 7fda19055640 1 -- 172.21.15.46:0/1903580362 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda080315a0 con 0x7fda140d2910 2023-12-02T00:30:19.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.891+0000 7fda19055640 1 -- 172.21.15.46:0/1903580362 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda08031930 con 0x7fda140d2910 2023-12-02T00:30:19.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.891+0000 7fda1c2bb640 1 -- 172.21.15.46:0/1903580362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda140d2910 msgr2=0x7fda140d31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:19.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.891+0000 7fda1c2bb640 1 --2- 172.21.15.46:0/1903580362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda140d2910 0x7fda140d31f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fda080097a0 tx=0x7fda0802b840 comp rx=0 tx=0).stop 2023-12-02T00:30:19.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.891+0000 7fda1c2bb640 1 -- 172.21.15.46:0/1903580362 shutdown_connections 2023-12-02T00:30:19.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.891+0000 7fda1c2bb640 1 --2- 172.21.15.46:0/1903580362 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda140d3730 0x7fda140b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:19.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.891+0000 7fda1c2bb640 1 --2- 172.21.15.46:0/1903580362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda140d2910 0x7fda140d31f0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:19.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.891+0000 7fda1c2bb640 1 -- 172.21.15.46:0/1903580362 >> 172.21.15.46:0/1903580362 conn(0x7fda14072020 msgr2=0x7fda14072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:19.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.891+0000 7fda1c2bb640 1 -- 172.21.15.46:0/1903580362 shutdown_connections 2023-12-02T00:30:19.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.891+0000 7fda1c2bb640 1 -- 172.21.15.46:0/1903580362 wait complete. 2023-12-02T00:30:19.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.892+0000 7fda1c2bb640 1 Processor -- start 2023-12-02T00:30:19.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.892+0000 7fda1c2bb640 1 -- start start 2023-12-02T00:30:19.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.893+0000 7fda1c2bb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda140d2910 0x7fda140cad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:19.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.893+0000 7fda1c2bb640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda140d3730 0x7fda1414d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:19.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.893+0000 7fda1c2bb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda1415d330 0x7fda1415ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:19.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.893+0000 7fda1c2bb640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda140cb280 con 0x7fda140d2910 2023-12-02T00:30:19.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.893+0000 7fda1c2bb640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda1415dc60 con 0x7fda140d3730 2023-12-02T00:30:19.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.893+0000 7fda1c2bb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda1415df40 con 0x7fda1415d330 2023-12-02T00:30:19.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.893+0000 7fda1a057640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda140d2910 0x7fda140cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:19.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.893+0000 7fda1a057640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda140d2910 0x7fda140cad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57180/0 (socket says 172.21.15.46:57180) 2023-12-02T00:30:19.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.893+0000 7fda1a057640 1 -- 172.21.15.46:0/2246765676 learned_addr learned my addr 172.21.15.46:0/2246765676 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:19.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.894+0000 7fda19856640 1 --2- 172.21.15.46:0/2246765676 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda140d3730 0x7fda1414d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:19.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.894+0000 7fda1a858640 1 --2- 172.21.15.46:0/2246765676 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda1415d330 0x7fda1415ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:19.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.894+0000 7fda1a057640 1 -- 172.21.15.46:0/2246765676 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda1415d330 msgr2=0x7fda1415ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:19.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.894+0000 7fda1a057640 1 --2- 172.21.15.46:0/2246765676 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda1415d330 0x7fda1415ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:19.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.894+0000 7fda1a057640 1 -- 172.21.15.46:0/2246765676 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda140d3730 msgr2=0x7fda1414d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:19.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.894+0000 7fda1a057640 1 --2- 172.21.15.46:0/2246765676 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda140d3730 0x7fda1414d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:19.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.894+0000 7fda1a057640 1 -- 172.21.15.46:0/2246765676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda08009400 con 0x7fda140d2910 2023-12-02T00:30:19.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.894+0000 7fda19856640 1 --2- 172.21.15.46:0/2246765676 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda140d3730 0x7fda1414d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:30:19.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.894+0000 7fda1a057640 1 --2- 172.21.15.46:0/2246765676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda140d2910 0x7fda140cad40 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fda0400c7c0 tx=0x7fda0400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:19.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.894+0000 7fda12ffd640 1 -- 172.21.15.46:0/2246765676 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0400a640 con 0x7fda140d2910 2023-12-02T00:30:19.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.894+0000 7fda12ffd640 1 -- 172.21.15.46:0/2246765676 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda0400a7c0 con 0x7fda140d2910 2023-12-02T00:30:19.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.894+0000 7fda1c2bb640 1 -- 172.21.15.46:0/2246765676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda1415e220 con 0x7fda140d2910 2023-12-02T00:30:19.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.894+0000 7fda12ffd640 1 -- 172.21.15.46:0/2246765676 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0400ab70 con 0x7fda140d2910 2023-12-02T00:30:19.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.894+0000 7fda1c2bb640 1 -- 172.21.15.46:0/2246765676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda1415e7e0 con 0x7fda140d2910 2023-12-02T00:30:19.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.895+0000 7fda1c2bb640 1 -- 172.21.15.46:0/2246765676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda1415ea00 con 0x7fda140d2910 2023-12-02T00:30:19.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.895+0000 7fda12ffd640 1 -- 172.21.15.46:0/2246765676 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fda0400acd0 con 0x7fda140d2910 2023-12-02T00:30:19.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.896+0000 7fda12ffd640 1 --2- 172.21.15.46:0/2246765676 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fda0006ae40 0x7fda0006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:19.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.896+0000 7fda12ffd640 1 -- 172.21.15.46:0/2246765676 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fda04089030 con 0x7fda140d2910 2023-12-02T00:30:19.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.896+0000 7fda19856640 1 --2- 172.21.15.46:0/2246765676 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fda0006ae40 0x7fda0006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:19.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.897+0000 7fda19856640 1 --2- 172.21.15.46:0/2246765676 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fda0006ae40 0x7fda0006d300 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fda08036000 tx=0x7fda0803d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:19.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:19.900+0000 7fda12ffd640 1 -- 172.21.15.46:0/2246765676 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fda04053f50 con 0x7fda140d2910 2023-12-02T00:30:20.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.076+0000 7fda1c2bb640 1 -- 172.21.15.46:0/2246765676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fda14069400 con 0x7fda140d2910 2023-12-02T00:30:20.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.076+0000 7fda12ffd640 1 -- 172.21.15.46:0/2246765676 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v38) v1 ==== 100+0+11 (secure 0 0 0) 0x7fda040180a0 con 0x7fda140d2910 2023-12-02T00:30:20.078 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-12-02T00:30:20.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.078+0000 7fda1c2bb640 1 -- 172.21.15.46:0/2246765676 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fda0006ae40 msgr2=0x7fda0006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:20.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.078+0000 7fda1c2bb640 1 --2- 172.21.15.46:0/2246765676 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fda0006ae40 0x7fda0006d300 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fda08036000 tx=0x7fda0803d040 comp rx=0 tx=0).stop 2023-12-02T00:30:20.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.078+0000 7fda1c2bb640 1 -- 172.21.15.46:0/2246765676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda140d2910 msgr2=0x7fda140cad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:20.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.078+0000 7fda1c2bb640 1 --2- 172.21.15.46:0/2246765676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda140d2910 0x7fda140cad40 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fda0400c7c0 tx=0x7fda0400cc80 comp rx=0 tx=0).stop 2023-12-02T00:30:20.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.078+0000 7fda1c2bb640 1 -- 172.21.15.46:0/2246765676 shutdown_connections 2023-12-02T00:30:20.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.078+0000 7fda1c2bb640 1 --2- 172.21.15.46:0/2246765676 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fda0006ae40 0x7fda0006d300 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:20.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.078+0000 7fda1c2bb640 1 --2- 172.21.15.46:0/2246765676 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda1415d330 0x7fda1415ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:20.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.078+0000 7fda1c2bb640 1 --2- 172.21.15.46:0/2246765676 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda140d3730 0x7fda1414d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:20.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.078+0000 7fda1c2bb640 1 --2- 172.21.15.46:0/2246765676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda140d2910 0x7fda140cad40 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:20.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.078+0000 7fda1c2bb640 1 -- 172.21.15.46:0/2246765676 >> 172.21.15.46:0/2246765676 conn(0x7fda14072020 msgr2=0x7fda140b5600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:20.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.078+0000 7fda1c2bb640 1 -- 172.21.15.46:0/2246765676 shutdown_connections 2023-12-02T00:30:20.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.078+0000 7fda1c2bb640 1 -- 172.21.15.46:0/2246765676 wait complete. 2023-12-02T00:30:20.091 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2023-12-02T00:30:20.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.207+0000 7fa1e0efc640 1 Processor -- start 2023-12-02T00:30:20.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.207+0000 7fa1e0efc640 1 -- start start 2023-12-02T00:30:20.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.207+0000 7fa1e0efc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa1dc143490 0x7fa1dc0b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:20.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.207+0000 7fa1e0efc640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa1dc15e730 0x7fa1dc0b1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:20.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.207+0000 7fa1e0efc640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1dc15e2a0 con 0x7fa1dc15a4e0 2023-12-02T00:30:20.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.207+0000 7fa1e0efc640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1dc15b520 con 0x7fa1dc15e730 2023-12-02T00:30:20.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.207+0000 7fa1e0efc640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc143870 con 0x7fa1dc143490 2023-12-02T00:30:20.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.208+0000 7fa1dad9d640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa1dc15a4e0 0x7fa1dc15a970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59186/0 (socket says 172.21.15.46:59186) 2023-12-02T00:30:20.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.208+0000 7fa1dad9d640 1 -- 172.21.15.46:0/848934101 learned_addr learned my addr 172.21.15.46:0/848934101 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:30:20.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.208+0000 7fa1d3fff640 1 --2- 172.21.15.46:0/848934101 >> v2:172.21.15.162:3300/0 conn(0x7fa1dc15e730 0x7fa1dc0b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:20.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.208+0000 7fa1da59c640 1 --2- 172.21.15.46:0/848934101 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa1dc143490 0x7fa1dc0b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:20.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.208+0000 7fa1d9d9b640 1 -- 172.21.15.46:0/848934101 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1666565980 0 0) 0x7fa1dc15e2a0 con 0x7fa1dc15a4e0 2023-12-02T00:30:20.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.208+0000 7fa1d9d9b640 1 -- 172.21.15.46:0/848934101 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1c8003520 con 0x7fa1dc15a4e0 2023-12-02T00:30:20.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.208+0000 7fa1da59c640 1 -- 172.21.15.46:0/848934101 >> v2:172.21.15.162:3300/0 conn(0x7fa1dc15e730 msgr2=0x7fa1dc0b1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:20.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.208+0000 7fa1da59c640 1 --2- 172.21.15.46:0/848934101 >> v2:172.21.15.162:3300/0 conn(0x7fa1dc15e730 0x7fa1dc0b1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:20.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.208+0000 7fa1d9d9b640 1 -- 172.21.15.46:0/848934101 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 243882633 0 0) 0x7fa1c8003520 con 0x7fa1dc15a4e0 2023-12-02T00:30:20.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.208+0000 7fa1da59c640 1 -- 172.21.15.46:0/848934101 >> v1:172.21.15.46:6789/0 conn(0x7fa1dc15a4e0 legacy=0x7fa1dc15a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:20.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.208+0000 7fa1da59c640 1 -- 172.21.15.46:0/848934101 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1dc15b240 con 0x7fa1dc143490 2023-12-02T00:30:20.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.208+0000 7fa1d3fff640 1 --2- 172.21.15.46:0/848934101 >> v2:172.21.15.162:3300/0 conn(0x7fa1dc15e730 0x7fa1dc0b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:30:20.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.209+0000 7fa1da59c640 1 --2- 172.21.15.46:0/848934101 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa1dc143490 0x7fa1dc0b0cf0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc0097a0 tx=0x7fa1cc02b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dad40a8cc2b4a4b6 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:20.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.209+0000 7fa1d9d9b640 1 -- 172.21.15.46:0/848934101 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d40019f0 con 0x7fa1dc143490 2023-12-02T00:30:20.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.209+0000 7fa1d9d9b640 1 -- 172.21.15.46:0/848934101 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1cc02e050 con 0x7fa1dc143490 2023-12-02T00:30:20.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.209+0000 7fa1d9d9b640 1 -- 172.21.15.46:0/848934101 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc030730 con 0x7fa1dc143490 2023-12-02T00:30:20.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.210+0000 7fa1e0efc640 1 -- 172.21.15.46:0/848934101 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa1dc143490 msgr2=0x7fa1dc0b0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:20.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.210+0000 7fa1e0efc640 1 --2- 172.21.15.46:0/848934101 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa1dc143490 0x7fa1dc0b0cf0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc0097a0 tx=0x7fa1cc02b790 comp rx=0 tx=0).stop 2023-12-02T00:30:20.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.210+0000 7fa1e0efc640 1 -- 172.21.15.46:0/848934101 shutdown_connections 2023-12-02T00:30:20.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.210+0000 7fa1e0efc640 1 --2- 172.21.15.46:0/848934101 >> v2:172.21.15.162:3300/0 conn(0x7fa1dc15e730 0x7fa1dc0b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:20.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.210+0000 7fa1e0efc640 1 --2- 172.21.15.46:0/848934101 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa1dc143490 0x7fa1dc0b0cf0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:20.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.210+0000 7fa1e0efc640 1 -- 172.21.15.46:0/848934101 >> 172.21.15.46:0/848934101 conn(0x7fa1dc06a4d0 msgr2=0x7fa1dc06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:20.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.210+0000 7fa1e0efc640 1 -- 172.21.15.46:0/848934101 shutdown_connections 2023-12-02T00:30:20.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.210+0000 7fa1e0efc640 1 -- 172.21.15.46:0/848934101 wait complete. 2023-12-02T00:30:20.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.211+0000 7fa1e0efc640 1 Processor -- start 2023-12-02T00:30:20.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.211+0000 7fa1e0efc640 1 -- start start 2023-12-02T00:30:20.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.212+0000 7fa1e0efc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa1dc143490 0x7fa1dc13b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:20.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.212+0000 7fa1e0efc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa1dc15a4e0 0x7fa1dc141db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:20.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.212+0000 7fa1e0efc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1dc15e730 0x7fa1dc0ad140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:20.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.212+0000 7fa1e0efc640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc0acd20 con 0x7fa1dc15e730 2023-12-02T00:30:20.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.212+0000 7fa1e0efc640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc0ac980 con 0x7fa1dc143490 2023-12-02T00:30:20.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.212+0000 7fa1e0efc640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc14c740 con 0x7fa1dc15a4e0 2023-12-02T00:30:20.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.212+0000 7fa1dad9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1dc15e730 0x7fa1dc0ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:20.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.212+0000 7fa1dad9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1dc15e730 0x7fa1dc0ad140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57182/0 (socket says 172.21.15.46:57182) 2023-12-02T00:30:20.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.212+0000 7fa1dad9d640 1 -- 172.21.15.46:0/1281612523 learned_addr learned my addr 172.21.15.46:0/1281612523 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:20.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.212+0000 7fa1d3fff640 1 --2- 172.21.15.46:0/1281612523 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa1dc15a4e0 0x7fa1dc141db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:20.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.212+0000 7fa1da59c640 1 --2- 172.21.15.46:0/1281612523 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa1dc143490 0x7fa1dc13b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:20.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.212+0000 7fa1dad9d640 1 -- 172.21.15.46:0/1281612523 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa1dc15a4e0 msgr2=0x7fa1dc141db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:20.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.212+0000 7fa1dad9d640 1 --2- 172.21.15.46:0/1281612523 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa1dc15a4e0 0x7fa1dc141db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:20.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.212+0000 7fa1dad9d640 1 -- 172.21.15.46:0/1281612523 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa1dc143490 msgr2=0x7fa1dc13b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:20.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.212+0000 7fa1dad9d640 1 --2- 172.21.15.46:0/1281612523 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa1dc143490 0x7fa1dc13b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:20.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.212+0000 7fa1dad9d640 1 -- 172.21.15.46:0/1281612523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c4009400 con 0x7fa1dc15e730 2023-12-02T00:30:20.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.212+0000 7fa1d3fff640 1 --2- 172.21.15.46:0/1281612523 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa1dc15a4e0 0x7fa1dc141db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:30:20.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.212+0000 7fa1da59c640 1 --2- 172.21.15.46:0/1281612523 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa1dc143490 0x7fa1dc13b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:30:20.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.213+0000 7fa1dad9d640 1 --2- 172.21.15.46:0/1281612523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1dc15e730 0x7fa1dc0ad140 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa1d4001830 tx=0x7fa1d400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:20.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.213+0000 7fa1d2ffd640 1 -- 172.21.15.46:0/1281612523 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d4012400 con 0x7fa1dc15e730 2023-12-02T00:30:20.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.213+0000 7fa1d2ffd640 1 -- 172.21.15.46:0/1281612523 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d4012560 con 0x7fa1dc15e730 2023-12-02T00:30:20.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.213+0000 7fa1e0efc640 1 -- 172.21.15.46:0/1281612523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1cc009400 con 0x7fa1dc15e730 2023-12-02T00:30:20.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.213+0000 7fa1d2ffd640 1 -- 172.21.15.46:0/1281612523 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d4012910 con 0x7fa1dc15e730 2023-12-02T00:30:20.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.213+0000 7fa1e0efc640 1 -- 172.21.15.46:0/1281612523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1dc13e3c0 con 0x7fa1dc15e730 2023-12-02T00:30:20.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.214+0000 7fa1e0efc640 1 -- 172.21.15.46:0/1281612523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1dc0ace60 con 0x7fa1dc15e730 2023-12-02T00:30:20.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.214+0000 7fa1d2ffd640 1 -- 172.21.15.46:0/1281612523 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa1d4012a70 con 0x7fa1dc15e730 2023-12-02T00:30:20.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.215+0000 7fa1d2ffd640 1 --2- 172.21.15.46:0/1281612523 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa1bc06ae40 0x7fa1bc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:20.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.215+0000 7fa1d2ffd640 1 -- 172.21.15.46:0/1281612523 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fa1d408a060 con 0x7fa1dc15e730 2023-12-02T00:30:20.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.215+0000 7fa1da59c640 1 --2- 172.21.15.46:0/1281612523 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa1bc06ae40 0x7fa1bc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:20.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.216+0000 7fa1da59c640 1 --2- 172.21.15.46:0/1281612523 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa1bc06ae40 0x7fa1bc06d300 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc034000 tx=0x7fa1cc02f000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:20.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.219+0000 7fa1d2ffd640 1 -- 172.21.15.46:0/1281612523 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa1d40bce90 con 0x7fa1dc15e730 2023-12-02T00:30:20.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.395+0000 7fa1e0efc640 1 -- 172.21.15.46:0/1281612523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7fa1dc069400 con 0x7fa1dc15e730 2023-12-02T00:30:20.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.395+0000 7fa1d2ffd640 1 -- 172.21.15.46:0/1281612523 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v38) v1 ==== 97+0+11 (secure 0 0 0) 0x7fa1dc0ace60 con 0x7fa1dc15e730 2023-12-02T00:30:20.396 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-12-02T00:30:20.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.397+0000 7fa1e0efc640 1 -- 172.21.15.46:0/1281612523 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa1bc06ae40 msgr2=0x7fa1bc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:20.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.397+0000 7fa1e0efc640 1 --2- 172.21.15.46:0/1281612523 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa1bc06ae40 0x7fa1bc06d300 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc034000 tx=0x7fa1cc02f000 comp rx=0 tx=0).stop 2023-12-02T00:30:20.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.397+0000 7fa1e0efc640 1 -- 172.21.15.46:0/1281612523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1dc15e730 msgr2=0x7fa1dc0ad140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:20.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.397+0000 7fa1e0efc640 1 --2- 172.21.15.46:0/1281612523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1dc15e730 0x7fa1dc0ad140 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa1d4001830 tx=0x7fa1d400a430 comp rx=0 tx=0).stop 2023-12-02T00:30:20.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.397+0000 7fa1e0efc640 1 -- 172.21.15.46:0/1281612523 shutdown_connections 2023-12-02T00:30:20.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.397+0000 7fa1e0efc640 1 --2- 172.21.15.46:0/1281612523 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa1bc06ae40 0x7fa1bc06d300 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:20.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.397+0000 7fa1e0efc640 1 --2- 172.21.15.46:0/1281612523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1dc15e730 0x7fa1dc0ad140 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:20.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.397+0000 7fa1e0efc640 1 --2- 172.21.15.46:0/1281612523 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa1dc15a4e0 0x7fa1dc141db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:20.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.397+0000 7fa1e0efc640 1 --2- 172.21.15.46:0/1281612523 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa1dc143490 0x7fa1dc13b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:20.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.397+0000 7fa1e0efc640 1 -- 172.21.15.46:0/1281612523 >> 172.21.15.46:0/1281612523 conn(0x7fa1dc06a4d0 msgr2=0x7fa1dc0b3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:20.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.397+0000 7fa1e0efc640 1 -- 172.21.15.46:0/1281612523 shutdown_connections 2023-12-02T00:30:20.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.397+0000 7fa1e0efc640 1 -- 172.21.15.46:0/1281612523 wait complete. 2023-12-02T00:30:20.409 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-02T00:30:20.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.516+0000 7fe64176e640 1 Processor -- start 2023-12-02T00:30:20.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.516+0000 7fe64176e640 1 -- start start 2023-12-02T00:30:20.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.517+0000 7fe64176e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe63c0b4370 0x7fe63c0b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:20.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.517+0000 7fe64176e640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe63c071770 con 0x7fe63c0ceb70 2023-12-02T00:30:20.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.517+0000 7fe64176e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe63c0ccb60 con 0x7fe63c144980 2023-12-02T00:30:20.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.517+0000 7fe64176e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe63c15ced0 con 0x7fe63c0b4370 2023-12-02T00:30:20.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.517+0000 7fe63a7fc640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe63c0ceb70 0x7fe63c0cc450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59194/0 (socket says 172.21.15.46:59194) 2023-12-02T00:30:20.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.517+0000 7fe63a7fc640 1 -- 172.21.15.46:0/2563727834 learned_addr learned my addr 172.21.15.46:0/2563727834 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:30:20.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.517+0000 7fe63b7fe640 1 --2- 172.21.15.46:0/2563727834 >> v2:172.21.15.162:3300/0 conn(0x7fe63c0b4370 0x7fe63c0b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:20.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.517+0000 7fe639ffb640 1 -- 172.21.15.46:0/2563727834 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3021297119 0 0) 0x7fe63c071770 con 0x7fe63c0ceb70 2023-12-02T00:30:20.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.517+0000 7fe639ffb640 1 -- 172.21.15.46:0/2563727834 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe624003520 con 0x7fe63c0ceb70 2023-12-02T00:30:20.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.518+0000 7fe639ffb640 1 -- 172.21.15.46:0/2563727834 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1507739349 0 0) 0x7fe624003520 con 0x7fe63c0ceb70 2023-12-02T00:30:20.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.518+0000 7fe639ffb640 1 -- 172.21.15.46:0/2563727834 >> v2:172.21.15.162:3300/0 conn(0x7fe63c0b4370 msgr2=0x7fe63c0b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:20.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.518+0000 7fe639ffb640 1 --2- 172.21.15.46:0/2563727834 >> v2:172.21.15.162:3300/0 conn(0x7fe63c0b4370 0x7fe63c0b47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:20.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.518+0000 7fe639ffb640 1 -- 172.21.15.46:0/2563727834 >> v1:172.21.15.162:6789/0 conn(0x7fe63c144980 legacy=0x7fe63c15c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:20.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.518+0000 7fe639ffb640 1 -- 172.21.15.46:0/2563727834 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe63c0b4ce0 con 0x7fe63c0ceb70 2023-12-02T00:30:20.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.518+0000 7fe639ffb640 1 -- 172.21.15.46:0/2563727834 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7fe630001670 con 0x7fe63c0ceb70 2023-12-02T00:30:20.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.518+0000 7fe639ffb640 1 -- 172.21.15.46:0/2563727834 >> v1:172.21.15.46:6789/0 conn(0x7fe63c0ceb70 legacy=0x7fe63c0cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:20.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.518+0000 7fe639ffb640 1 --2- 172.21.15.46:0/2563727834 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe624003970 0x7fe624003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:20.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.518+0000 7fe639ffb640 1 --2- 172.21.15.46:0/2563727834 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe6240047c0 0x7fe624004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:20.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.518+0000 7fe639ffb640 1 --2- 172.21.15.46:0/2563727834 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe624005180 0x7fe62400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:20.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.518+0000 7fe639ffb640 1 -- 172.21.15.46:0/2563727834 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe63c15ced0 con 0x7fe624003970 2023-12-02T00:30:20.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.518+0000 7fe639ffb640 1 -- 172.21.15.46:0/2563727834 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe62400c050 con 0x7fe624005180 2023-12-02T00:30:20.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.518+0000 7fe639ffb640 1 -- 172.21.15.46:0/2563727834 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe62400c210 con 0x7fe6240047c0 2023-12-02T00:30:20.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.518+0000 7fe63affd640 1 --2- 172.21.15.46:0/2563727834 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe624003970 0x7fe624003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:20.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.518+0000 7fe63b7fe640 1 --2- 172.21.15.46:0/2563727834 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe624005180 0x7fe62400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:20.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.518+0000 7fe63a7fc640 1 --2- 172.21.15.46:0/2563727834 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe6240047c0 0x7fe624004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:20.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.518+0000 7fe63affd640 1 -- 172.21.15.46:0/2563727834 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe6240047c0 msgr2=0x7fe624004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:20.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.518+0000 7fe63affd640 1 --2- 172.21.15.46:0/2563727834 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe6240047c0 0x7fe624004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:20.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.518+0000 7fe63affd640 1 -- 172.21.15.46:0/2563727834 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe624005180 msgr2=0x7fe62400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:20.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.518+0000 7fe63affd640 1 --2- 172.21.15.46:0/2563727834 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe624005180 0x7fe62400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:20.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.518+0000 7fe63b7fe640 1 -- 172.21.15.46:0/2563727834 reap_dead start 2023-12-02T00:30:20.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.518+0000 7fe63affd640 1 -- 172.21.15.46:0/2563727834 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe624003520 con 0x7fe624003970 2023-12-02T00:30:20.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.519+0000 7fe63affd640 1 --2- 172.21.15.46:0/2563727834 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe624003970 0x7fe624003db0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fe634001750 tx=0x7fe6240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15d783e418ca23f6 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:20.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.519+0000 7fe639ffb640 1 -- 172.21.15.46:0/2563727834 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6340052d0 con 0x7fe624003970 2023-12-02T00:30:20.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.519+0000 7fe639ffb640 1 -- 172.21.15.46:0/2563727834 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe63402e050 con 0x7fe624003970 2023-12-02T00:30:20.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.519+0000 7fe639ffb640 1 -- 172.21.15.46:0/2563727834 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6340035f0 con 0x7fe624003970 2023-12-02T00:30:20.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.520+0000 7fe64176e640 1 -- 172.21.15.46:0/2563727834 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe624003970 msgr2=0x7fe624003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:20.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.520+0000 7fe64176e640 1 --2- 172.21.15.46:0/2563727834 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe624003970 0x7fe624003db0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fe634001750 tx=0x7fe6240057f0 comp rx=0 tx=0).stop 2023-12-02T00:30:20.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.520+0000 7fe64176e640 1 -- 172.21.15.46:0/2563727834 shutdown_connections 2023-12-02T00:30:20.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.520+0000 7fe64176e640 1 --2- 172.21.15.46:0/2563727834 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe624003970 0x7fe624003db0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:20.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.520+0000 7fe64176e640 1 -- 172.21.15.46:0/2563727834 >> 172.21.15.46:0/2563727834 conn(0x7fe63c06a4d0 msgr2=0x7fe63c06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:20.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.520+0000 7fe64176e640 1 -- 172.21.15.46:0/2563727834 shutdown_connections 2023-12-02T00:30:20.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.520+0000 7fe64176e640 1 -- 172.21.15.46:0/2563727834 wait complete. 2023-12-02T00:30:20.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.521+0000 7fe64176e640 1 Processor -- start 2023-12-02T00:30:20.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.521+0000 7fe64176e640 1 -- start start 2023-12-02T00:30:20.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.522+0000 7fe64176e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe624003970 0x7fe63c159110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:20.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.522+0000 7fe64176e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe63c159650 0x7fe63c1f0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:20.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.522+0000 7fe64176e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe63c1f1070 0x7fe63c0c5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:20.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.522+0000 7fe64176e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe63c0ca3e0 con 0x7fe624003970 2023-12-02T00:30:20.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.522+0000 7fe64176e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe63c1f4e60 con 0x7fe63c1f1070 2023-12-02T00:30:20.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.522+0000 7fe64176e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe63c159ab0 con 0x7fe63c159650 2023-12-02T00:30:20.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.522+0000 7fe63affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe624003970 0x7fe63c159110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:20.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.522+0000 7fe63affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe624003970 0x7fe63c159110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57198/0 (socket says 172.21.15.46:57198) 2023-12-02T00:30:20.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.522+0000 7fe63affd640 1 -- 172.21.15.46:0/3165157766 learned_addr learned my addr 172.21.15.46:0/3165157766 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:20.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.522+0000 7fe63affd640 1 -- 172.21.15.46:0/3165157766 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe63c159650 msgr2=0x7fe63c1f0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:20.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.522+0000 7fe63b7fe640 1 --2- 172.21.15.46:0/3165157766 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe63c1f1070 0x7fe63c0c5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:20.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.522+0000 7fe63a7fc640 1 --2- 172.21.15.46:0/3165157766 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe63c159650 0x7fe63c1f0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:20.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.522+0000 7fe63affd640 1 --2- 172.21.15.46:0/3165157766 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe63c159650 0x7fe63c1f0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:20.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.522+0000 7fe63affd640 1 -- 172.21.15.46:0/3165157766 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe63c1f1070 msgr2=0x7fe63c0c5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:20.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.522+0000 7fe63affd640 1 --2- 172.21.15.46:0/3165157766 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe63c1f1070 0x7fe63c0c5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:20.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.522+0000 7fe63affd640 1 -- 172.21.15.46:0/3165157766 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe63c1f40a0 con 0x7fe624003970 2023-12-02T00:30:20.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.522+0000 7fe63b7fe640 1 --2- 172.21.15.46:0/3165157766 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe63c1f1070 0x7fe63c0c5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:30:20.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.522+0000 7fe63a7fc640 1 --2- 172.21.15.46:0/3165157766 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe63c159650 0x7fe63c1f0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:30:20.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.522+0000 7fe63affd640 1 --2- 172.21.15.46:0/3165157766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe624003970 0x7fe63c159110 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe63402bc00 tx=0x7fe634001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:20.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.522+0000 7fe6237fe640 1 -- 172.21.15.46:0/3165157766 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6340076c0 con 0x7fe624003970 2023-12-02T00:30:20.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.523+0000 7fe6237fe640 1 -- 172.21.15.46:0/3165157766 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe634007a30 con 0x7fe624003970 2023-12-02T00:30:20.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.523+0000 7fe6237fe640 1 -- 172.21.15.46:0/3165157766 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe634006be0 con 0x7fe624003970 2023-12-02T00:30:20.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.523+0000 7fe64176e640 1 -- 172.21.15.46:0/3165157766 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe63c1f4380 con 0x7fe624003970 2023-12-02T00:30:20.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.523+0000 7fe64176e640 1 -- 172.21.15.46:0/3165157766 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe63c1f4940 con 0x7fe624003970 2023-12-02T00:30:20.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.523+0000 7fe64176e640 1 -- 172.21.15.46:0/3165157766 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe63c1f4b00 con 0x7fe624003970 2023-12-02T00:30:20.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.524+0000 7fe6237fe640 1 -- 172.21.15.46:0/3165157766 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe634006d40 con 0x7fe624003970 2023-12-02T00:30:20.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.524+0000 7fe6237fe640 1 --2- 172.21.15.46:0/3165157766 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe62806ae40 0x7fe62806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:20.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.524+0000 7fe6237fe640 1 -- 172.21.15.46:0/3165157766 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fe6340ab3b0 con 0x7fe624003970 2023-12-02T00:30:20.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.525+0000 7fe63a7fc640 1 --2- 172.21.15.46:0/3165157766 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe62806ae40 0x7fe62806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:20.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.525+0000 7fe63a7fc640 1 --2- 172.21.15.46:0/3165157766 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe62806ae40 0x7fe62806d300 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe630001200 tx=0x7fe630000f30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:20.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.529+0000 7fe6237fe640 1 -- 172.21.15.46:0/3165157766 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe6340762d0 con 0x7fe624003970 2023-12-02T00:30:20.532 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-02T00:30:20.537 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-02T00:30:20.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.719+0000 7fe64176e640 1 -- 172.21.15.46:0/3165157766 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe63c069400 con 0x7fe624003970 2023-12-02T00:30:20.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.719+0000 7fe6237fe640 1 -- 172.21.15.46:0/3165157766 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v38) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe6340798f0 con 0x7fe624003970 2023-12-02T00:30:20.721 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2023-12-02T00:30:20.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.722+0000 7fe64176e640 1 -- 172.21.15.46:0/3165157766 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe62806ae40 msgr2=0x7fe62806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:20.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.722+0000 7fe64176e640 1 --2- 172.21.15.46:0/3165157766 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe62806ae40 0x7fe62806d300 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe630001200 tx=0x7fe630000f30 comp rx=0 tx=0).stop 2023-12-02T00:30:20.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.722+0000 7fe64176e640 1 -- 172.21.15.46:0/3165157766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe624003970 msgr2=0x7fe63c159110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:20.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.722+0000 7fe64176e640 1 --2- 172.21.15.46:0/3165157766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe624003970 0x7fe63c159110 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe63402bc00 tx=0x7fe634001120 comp rx=0 tx=0).stop 2023-12-02T00:30:20.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.722+0000 7fe64176e640 1 -- 172.21.15.46:0/3165157766 shutdown_connections 2023-12-02T00:30:20.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.722+0000 7fe64176e640 1 --2- 172.21.15.46:0/3165157766 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe62806ae40 0x7fe62806d300 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:20.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.722+0000 7fe64176e640 1 --2- 172.21.15.46:0/3165157766 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe63c1f1070 0x7fe63c0c5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:20.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.722+0000 7fe64176e640 1 --2- 172.21.15.46:0/3165157766 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe63c159650 0x7fe63c1f0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:20.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.722+0000 7fe64176e640 1 --2- 172.21.15.46:0/3165157766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe624003970 0x7fe63c159110 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:20.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.722+0000 7fe64176e640 1 -- 172.21.15.46:0/3165157766 >> 172.21.15.46:0/3165157766 conn(0x7fe63c06a4d0 msgr2=0x7fe63c15a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:20.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.722+0000 7fe64176e640 1 -- 172.21.15.46:0/3165157766 shutdown_connections 2023-12-02T00:30:20.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:20.722+0000 7fe64176e640 1 -- 172.21.15.46:0/3165157766 wait complete. 2023-12-02T00:30:20.733 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi046.front.sepia.ceph.com... 2023-12-02T00:30:20.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:30:20.734 DEBUG:teuthology.orchestra.run.smithi046:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-12-02T00:30:20.764 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[105675]: fuse finished with error 0 and tester_r 0 2023-12-02T00:30:20.765 INFO:teuthology.orchestra.run:waiting for 300 2023-12-02T00:30:26.867 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2023-12-02T00:30:26.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:30:26.867 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-02T00:30:26.887 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds 'allow rwp' mon 'allow r' osd 'allow rw pool=cephfs_data,allow rw pool=data_new' 2023-12-02T00:30:27.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.050+0000 7f2b940b4640 1 Processor -- start 2023-12-02T00:30:27.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.050+0000 7f2b940b4640 1 -- start start 2023-12-02T00:30:27.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.051+0000 7f2b940b4640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2b8c0b1300 0x7f2b8c15de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:27.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.051+0000 7f2b940b4640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b8c071d20 con 0x7f2b8c159920 2023-12-02T00:30:27.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.051+0000 7f2b940b4640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b8c0cdcd0 con 0x7f2b8c071010 2023-12-02T00:30:27.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.051+0000 7f2b940b4640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b8c159d00 con 0x7f2b8c0b1300 2023-12-02T00:30:27.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.051+0000 7f2b91e50640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2b8c159920 0x7f2b8c0ce160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:44620/0 (socket says 172.21.15.46:44620) 2023-12-02T00:30:27.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.051+0000 7f2b91e50640 1 -- 172.21.15.46:0/3871219327 learned_addr learned my addr 172.21.15.46:0/3871219327 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:30:27.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.051+0000 7f2b92651640 1 --2- 172.21.15.46:0/3871219327 >> v2:172.21.15.162:3300/0 conn(0x7f2b8c0b1300 0x7f2b8c15de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:27.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.051+0000 7f2b90e4e640 1 -- 172.21.15.46:0/3871219327 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2782093648 0 0) 0x7f2b88001110 con 0x7f2b8c159920 2023-12-02T00:30:27.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.051+0000 7f2b90e4e640 1 -- 172.21.15.46:0/3871219327 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b78004630 con 0x7f2b8c159920 2023-12-02T00:30:27.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.051+0000 7f2b92651640 1 -- 172.21.15.46:0/3871219327 >> v1:172.21.15.162:6789/0 conn(0x7f2b8c071010 legacy=0x7f2b8c0b0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:27.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.051+0000 7f2b90e4e640 1 -- 172.21.15.46:0/3871219327 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3959314705 0 0) 0x7f2b78004630 con 0x7f2b8c159920 2023-12-02T00:30:27.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.051+0000 7f2b92651640 1 -- 172.21.15.46:0/3871219327 >> v1:172.21.15.46:6789/0 conn(0x7f2b8c159920 legacy=0x7f2b8c0ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:27.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.051+0000 7f2b92651640 1 -- 172.21.15.46:0/3871219327 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b8c0ce870 con 0x7f2b8c0b1300 2023-12-02T00:30:27.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.052+0000 7f2b92651640 1 --2- 172.21.15.46:0/3871219327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b8c0b1300 0x7f2b8c15de70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2b84009d90 tx=0x7f2b8402bbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42c86e27de4f5904 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:27.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.052+0000 7f2b90e4e640 1 -- 172.21.15.46:0/3871219327 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b88001800 con 0x7f2b8c0b1300 2023-12-02T00:30:27.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.053+0000 7f2b90e4e640 1 -- 172.21.15.46:0/3871219327 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b8402d720 con 0x7f2b8c0b1300 2023-12-02T00:30:27.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.053+0000 7f2b90e4e640 1 -- 172.21.15.46:0/3871219327 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8402dab0 con 0x7f2b8c0b1300 2023-12-02T00:30:27.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.053+0000 7f2b940b4640 1 -- 172.21.15.46:0/3871219327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b8c0b1300 msgr2=0x7f2b8c15de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:27.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.053+0000 7f2b940b4640 1 --2- 172.21.15.46:0/3871219327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b8c0b1300 0x7f2b8c15de70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2b84009d90 tx=0x7f2b8402bbe0 comp rx=0 tx=0).stop 2023-12-02T00:30:27.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.053+0000 7f2b940b4640 1 -- 172.21.15.46:0/3871219327 shutdown_connections 2023-12-02T00:30:27.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.053+0000 7f2b940b4640 1 --2- 172.21.15.46:0/3871219327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b8c0b1300 0x7f2b8c15de70 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:27.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.053+0000 7f2b940b4640 1 -- 172.21.15.46:0/3871219327 >> 172.21.15.46:0/3871219327 conn(0x7f2b8c06d9c0 msgr2=0x7f2b8c06a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:27.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.053+0000 7f2b940b4640 1 -- 172.21.15.46:0/3871219327 shutdown_connections 2023-12-02T00:30:27.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.053+0000 7f2b940b4640 1 -- 172.21.15.46:0/3871219327 wait complete. 2023-12-02T00:30:27.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.054+0000 7f2b940b4640 1 Processor -- start 2023-12-02T00:30:27.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.054+0000 7f2b940b4640 1 -- start start 2023-12-02T00:30:27.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.055+0000 7f2b940b4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2b8c071010 0x7f2b8c153870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:27.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.055+0000 7f2b940b4640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b8c0b1300 0x7f2b8c1f16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:27.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.055+0000 7f2b940b4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b8c159920 0x7f2b8c0ac130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:27.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.055+0000 7f2b940b4640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8c1f1c10 con 0x7f2b8c159920 2023-12-02T00:30:27.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.055+0000 7f2b940b4640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8c15c290 con 0x7f2b8c0b1300 2023-12-02T00:30:27.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.055+0000 7f2b940b4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8c1f1e10 con 0x7f2b8c071010 2023-12-02T00:30:27.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.055+0000 7f2b92651640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b8c159920 0x7f2b8c0ac130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:27.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.055+0000 7f2b92651640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b8c159920 0x7f2b8c0ac130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40776/0 (socket says 172.21.15.46:40776) 2023-12-02T00:30:27.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.055+0000 7f2b92651640 1 -- 172.21.15.46:0/616328208 learned_addr learned my addr 172.21.15.46:0/616328208 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:27.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.055+0000 7f2b9164f640 1 --2- 172.21.15.46:0/616328208 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b8c0b1300 0x7f2b8c1f16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:27.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.055+0000 7f2b91e50640 1 --2- 172.21.15.46:0/616328208 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2b8c071010 0x7f2b8c153870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:27.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.056+0000 7f2b92651640 1 -- 172.21.15.46:0/616328208 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2b8c071010 msgr2=0x7f2b8c153870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:27.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.056+0000 7f2b92651640 1 --2- 172.21.15.46:0/616328208 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2b8c071010 0x7f2b8c153870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:27.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.056+0000 7f2b92651640 1 -- 172.21.15.46:0/616328208 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b8c0b1300 msgr2=0x7f2b8c1f16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:27.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.056+0000 7f2b92651640 1 --2- 172.21.15.46:0/616328208 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b8c0b1300 0x7f2b8c1f16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:27.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.056+0000 7f2b92651640 1 -- 172.21.15.46:0/616328208 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b84009a70 con 0x7f2b8c159920 2023-12-02T00:30:27.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.056+0000 7f2b91e50640 1 --2- 172.21.15.46:0/616328208 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2b8c071010 0x7f2b8c153870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:30:27.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.056+0000 7f2b92651640 1 --2- 172.21.15.46:0/616328208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b8c159920 0x7f2b8c0ac130 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f2b84037040 tx=0x7f2b84004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:27.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.056+0000 7f2b827fc640 1 -- 172.21.15.46:0/616328208 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b84035be0 con 0x7f2b8c159920 2023-12-02T00:30:27.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.056+0000 7f2b827fc640 1 -- 172.21.15.46:0/616328208 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b84035d60 con 0x7f2b8c159920 2023-12-02T00:30:27.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.056+0000 7f2b940b4640 1 -- 172.21.15.46:0/616328208 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b8c1f20f0 con 0x7f2b8c159920 2023-12-02T00:30:27.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.056+0000 7f2b827fc640 1 -- 172.21.15.46:0/616328208 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b84003bd0 con 0x7f2b8c159920 2023-12-02T00:30:27.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.056+0000 7f2b940b4640 1 -- 172.21.15.46:0/616328208 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b8c1f26b0 con 0x7f2b8c159920 2023-12-02T00:30:27.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.057+0000 7f2b940b4640 1 -- 172.21.15.46:0/616328208 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b8c1f28a0 con 0x7f2b8c159920 2023-12-02T00:30:27.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.057+0000 7f2b827fc640 1 -- 172.21.15.46:0/616328208 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2b84003d30 con 0x7f2b8c159920 2023-12-02T00:30:27.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.058+0000 7f2b827fc640 1 --2- 172.21.15.46:0/616328208 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2b7006ae70 0x7f2b7006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:27.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.058+0000 7f2b827fc640 1 -- 172.21.15.46:0/616328208 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f2b84031070 con 0x7f2b8c159920 2023-12-02T00:30:27.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.058+0000 7f2b91e50640 1 --2- 172.21.15.46:0/616328208 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2b7006ae70 0x7f2b7006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:27.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.059+0000 7f2b91e50640 1 --2- 172.21.15.46:0/616328208 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2b7006ae70 0x7f2b7006d330 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2b88000f60 tx=0x7f2b88001260 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:27.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.062+0000 7f2b827fc640 1 -- 172.21.15.46:0/616328208 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2b84077470 con 0x7f2b8c159920 2023-12-02T00:30:27.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.308+0000 7f2b940b4640 1 -- 172.21.15.46:0/616328208 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rwp", "mon", "allow r", "osd", "allow rw pool=cephfs_data,allow rw pool=data_new"]} v 0) v1 -- 0x7f2b8c069400 con 0x7f2b8c159920 2023-12-02T00:30:27.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.311+0000 7f2b827fc640 1 -- 172.21.15.46:0/616328208 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rwp", "mon", "allow r", "osd", "allow rw pool=cephfs_data,allow rw pool=data_new"]}]=0 updated caps for client.0 v15) v1 ==== 211+0+168 (secure 0 0 0) 0x7f2b8407aa90 con 0x7f2b8c159920 2023-12-02T00:30:27.312 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2023-12-02T00:30:27.312 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAJempl5Lx1DhAAjpK6ajoQ5jQiUr/EMpFeLg== 2023-12-02T00:30:27.312 INFO:teuthology.orchestra.run.smithi046.stdout: caps mds = "allow rwp" 2023-12-02T00:30:27.312 INFO:teuthology.orchestra.run.smithi046.stdout: caps mon = "allow r" 2023-12-02T00:30:27.312 INFO:teuthology.orchestra.run.smithi046.stdout: caps osd = "allow rw pool=cephfs_data,allow rw pool=data_new" 2023-12-02T00:30:27.312 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2023-12-02T00:30:27.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.313+0000 7f2b940b4640 1 -- 172.21.15.46:0/616328208 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2b7006ae70 msgr2=0x7f2b7006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:27.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.313+0000 7f2b940b4640 1 --2- 172.21.15.46:0/616328208 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2b7006ae70 0x7f2b7006d330 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2b88000f60 tx=0x7f2b88001260 comp rx=0 tx=0).stop 2023-12-02T00:30:27.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.313+0000 7f2b940b4640 1 -- 172.21.15.46:0/616328208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b8c159920 msgr2=0x7f2b8c0ac130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:27.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.313+0000 7f2b940b4640 1 --2- 172.21.15.46:0/616328208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b8c159920 0x7f2b8c0ac130 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f2b84037040 tx=0x7f2b84004a40 comp rx=0 tx=0).stop 2023-12-02T00:30:27.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.313+0000 7f2b940b4640 1 -- 172.21.15.46:0/616328208 shutdown_connections 2023-12-02T00:30:27.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.313+0000 7f2b940b4640 1 --2- 172.21.15.46:0/616328208 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2b7006ae70 0x7f2b7006d330 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:27.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.313+0000 7f2b940b4640 1 --2- 172.21.15.46:0/616328208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b8c159920 0x7f2b8c0ac130 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:27.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.313+0000 7f2b940b4640 1 --2- 172.21.15.46:0/616328208 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b8c0b1300 0x7f2b8c1f16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:27.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.313+0000 7f2b940b4640 1 --2- 172.21.15.46:0/616328208 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2b8c071010 0x7f2b8c153870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:27.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.313+0000 7f2b940b4640 1 -- 172.21.15.46:0/616328208 >> 172.21.15.46:0/616328208 conn(0x7f2b8c06d9c0 msgr2=0x7f2b8c15c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:27.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.313+0000 7f2b940b4640 1 -- 172.21.15.46:0/616328208 shutdown_connections 2023-12-02T00:30:27.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.313+0000 7f2b940b4640 1 -- 172.21.15.46:0/616328208 wait complete. 2023-12-02T00:30:27.325 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-02T00:30:27.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.429+0000 7fb199b39640 1 Processor -- start 2023-12-02T00:30:27.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.429+0000 7fb199b39640 1 -- start start 2023-12-02T00:30:27.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.430+0000 7fb199b39640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb19415d330 0x7fb1940b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:27.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.430+0000 7fb199b39640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb1940d2910 0x7fb1940d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:27.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.430+0000 7fb199b39640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1940b2b10 con 0x7fb1940d2d90 2023-12-02T00:30:27.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.430+0000 7fb199b39640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1940d1590 con 0x7fb1940d2910 2023-12-02T00:30:27.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.430+0000 7fb199b39640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1940d1870 con 0x7fb19415d330 2023-12-02T00:30:27.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.430+0000 7fb192ffd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb1940d2910 0x7fb1940d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:27.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.430+0000 7fb192ffd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb1940d2910 0x7fb1940d31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40790/0 (socket says 172.21.15.46:40790) 2023-12-02T00:30:27.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.430+0000 7fb192ffd640 1 -- 172.21.15.46:0/3927573749 learned_addr learned my addr 172.21.15.46:0/3927573749 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:27.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.431+0000 7fb1937fe640 1 --2- 172.21.15.46:0/3927573749 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb19415d330 0x7fb1940b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:27.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.431+0000 7fb192ffd640 1 -- 172.21.15.46:0/3927573749 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb19415d330 msgr2=0x7fb1940b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:27.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.431+0000 7fb192ffd640 1 --2- 172.21.15.46:0/3927573749 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb19415d330 0x7fb1940b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:27.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.431+0000 7fb192ffd640 1 -- 172.21.15.46:0/3927573749 >> v1:172.21.15.162:6789/0 conn(0x7fb1940d2d90 legacy=0x7fb1940b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:27.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.431+0000 7fb192ffd640 1 -- 172.21.15.46:0/3927573749 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1940d1b50 con 0x7fb1940d2910 2023-12-02T00:30:27.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.432+0000 7fb192ffd640 1 --2- 172.21.15.46:0/3927573749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1940d2910 0x7fb1940d31f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fb188009870 tx=0x7fb18802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=583824103ab03f5c server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:27.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.432+0000 7fb1927fc640 1 -- 172.21.15.46:0/3927573749 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb188031640 con 0x7fb1940d2910 2023-12-02T00:30:27.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.432+0000 7fb1927fc640 1 -- 172.21.15.46:0/3927573749 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1880317a0 con 0x7fb1940d2910 2023-12-02T00:30:27.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.432+0000 7fb1927fc640 1 -- 172.21.15.46:0/3927573749 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb188031b30 con 0x7fb1940d2910 2023-12-02T00:30:27.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.432+0000 7fb199b39640 1 -- 172.21.15.46:0/3927573749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1940d2910 msgr2=0x7fb1940d31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:27.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.432+0000 7fb199b39640 1 --2- 172.21.15.46:0/3927573749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1940d2910 0x7fb1940d31f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fb188009870 tx=0x7fb18802b8f0 comp rx=0 tx=0).stop 2023-12-02T00:30:27.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.432+0000 7fb199b39640 1 -- 172.21.15.46:0/3927573749 shutdown_connections 2023-12-02T00:30:27.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.432+0000 7fb199b39640 1 --2- 172.21.15.46:0/3927573749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1940d2910 0x7fb1940d31f0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:27.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.432+0000 7fb199b39640 1 --2- 172.21.15.46:0/3927573749 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb19415d330 0x7fb1940b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:27.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.432+0000 7fb199b39640 1 -- 172.21.15.46:0/3927573749 >> 172.21.15.46:0/3927573749 conn(0x7fb194072020 msgr2=0x7fb194072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:27.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.432+0000 7fb199b39640 1 -- 172.21.15.46:0/3927573749 shutdown_connections 2023-12-02T00:30:27.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.432+0000 7fb199b39640 1 -- 172.21.15.46:0/3927573749 wait complete. 2023-12-02T00:30:27.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.433+0000 7fb199b39640 1 Processor -- start 2023-12-02T00:30:27.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.434+0000 7fb199b39640 1 -- start start 2023-12-02T00:30:27.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.434+0000 7fb199b39640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb1940d2910 0x7fb194160330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:27.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.434+0000 7fb199b39640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb1940d2d90 0x7fb1940c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:27.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.434+0000 7fb199b39640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb19415d330 0x7fb1940ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:27.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.434+0000 7fb199b39640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1940c8ac0 con 0x7fb19415d330 2023-12-02T00:30:27.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.434+0000 7fb199b39640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb194152800 con 0x7fb1940d2d90 2023-12-02T00:30:27.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.434+0000 7fb199b39640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1940c7bf0 con 0x7fb1940d2910 2023-12-02T00:30:27.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.435+0000 7fb193fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb19415d330 0x7fb1940ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:27.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.435+0000 7fb193fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb19415d330 0x7fb1940ccdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40796/0 (socket says 172.21.15.46:40796) 2023-12-02T00:30:27.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.435+0000 7fb193fff640 1 -- 172.21.15.46:0/994307972 learned_addr learned my addr 172.21.15.46:0/994307972 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:27.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.435+0000 7fb192ffd640 1 --2- 172.21.15.46:0/994307972 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb1940d2d90 0x7fb1940c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:27.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.435+0000 7fb1937fe640 1 --2- 172.21.15.46:0/994307972 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb1940d2910 0x7fb194160330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:27.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.435+0000 7fb193fff640 1 -- 172.21.15.46:0/994307972 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb1940d2910 msgr2=0x7fb194160330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:27.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.435+0000 7fb193fff640 1 --2- 172.21.15.46:0/994307972 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb1940d2910 0x7fb194160330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:27.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.435+0000 7fb193fff640 1 -- 172.21.15.46:0/994307972 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb1940d2d90 msgr2=0x7fb1940c8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:27.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.435+0000 7fb193fff640 1 --2- 172.21.15.46:0/994307972 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb1940d2d90 0x7fb1940c8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:27.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.435+0000 7fb193fff640 1 -- 172.21.15.46:0/994307972 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1880094d0 con 0x7fb19415d330 2023-12-02T00:30:27.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.435+0000 7fb193fff640 1 --2- 172.21.15.46:0/994307972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb19415d330 0x7fb1940ccdf0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fb18c00c730 tx=0x7fb18c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:27.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.435+0000 7fb17bfff640 1 -- 172.21.15.46:0/994307972 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18c003b70 con 0x7fb19415d330 2023-12-02T00:30:27.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.435+0000 7fb17bfff640 1 -- 172.21.15.46:0/994307972 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb18c003cd0 con 0x7fb19415d330 2023-12-02T00:30:27.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.435+0000 7fb199b39640 1 -- 172.21.15.46:0/994307972 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1940c7e40 con 0x7fb19415d330 2023-12-02T00:30:27.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.435+0000 7fb17bfff640 1 -- 172.21.15.46:0/994307972 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18c00aa90 con 0x7fb19415d330 2023-12-02T00:30:27.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.435+0000 7fb199b39640 1 -- 172.21.15.46:0/994307972 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1940c8400 con 0x7fb19415d330 2023-12-02T00:30:27.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.436+0000 7fb199b39640 1 -- 172.21.15.46:0/994307972 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1940c8c00 con 0x7fb19415d330 2023-12-02T00:30:27.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.437+0000 7fb17bfff640 1 -- 172.21.15.46:0/994307972 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb18c00abf0 con 0x7fb19415d330 2023-12-02T00:30:27.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.437+0000 7fb17bfff640 1 --2- 172.21.15.46:0/994307972 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb18006ae40 0x7fb18006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:27.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.437+0000 7fb17bfff640 1 -- 172.21.15.46:0/994307972 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fb18c089400 con 0x7fb19415d330 2023-12-02T00:30:27.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.438+0000 7fb1937fe640 1 --2- 172.21.15.46:0/994307972 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb18006ae40 0x7fb18006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:27.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.438+0000 7fb1937fe640 1 --2- 172.21.15.46:0/994307972 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb18006ae40 0x7fb18006d300 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb184006fd0 tx=0x7fb184008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:27.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.442+0000 7fb17bfff640 1 -- 172.21.15.46:0/994307972 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb18c054460 con 0x7fb19415d330 2023-12-02T00:30:27.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.654+0000 7fb199b39640 1 -- 172.21.15.46:0/994307972 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb194069400 con 0x7fb19415d330 2023-12-02T00:30:27.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.654+0000 7fb17bfff640 1 -- 172.21.15.46:0/994307972 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) v1 ==== 53+0+82 (secure 0 0 0) 0x7fb18c057a80 con 0x7fb19415d330 2023-12-02T00:30:27.655 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data data_new ] 2023-12-02T00:30:27.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.656+0000 7fb199b39640 1 -- 172.21.15.46:0/994307972 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb18006ae40 msgr2=0x7fb18006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:27.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.656+0000 7fb199b39640 1 --2- 172.21.15.46:0/994307972 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb18006ae40 0x7fb18006d300 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb184006fd0 tx=0x7fb184008040 comp rx=0 tx=0).stop 2023-12-02T00:30:27.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.656+0000 7fb199b39640 1 -- 172.21.15.46:0/994307972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb19415d330 msgr2=0x7fb1940ccdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:27.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.656+0000 7fb199b39640 1 --2- 172.21.15.46:0/994307972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb19415d330 0x7fb1940ccdf0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fb18c00c730 tx=0x7fb18c00cbf0 comp rx=0 tx=0).stop 2023-12-02T00:30:27.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.656+0000 7fb199b39640 1 -- 172.21.15.46:0/994307972 shutdown_connections 2023-12-02T00:30:27.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.656+0000 7fb199b39640 1 --2- 172.21.15.46:0/994307972 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb18006ae40 0x7fb18006d300 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:27.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.656+0000 7fb199b39640 1 --2- 172.21.15.46:0/994307972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb19415d330 0x7fb1940ccdf0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:27.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.656+0000 7fb199b39640 1 --2- 172.21.15.46:0/994307972 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb1940d2d90 0x7fb1940c8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:27.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.656+0000 7fb199b39640 1 --2- 172.21.15.46:0/994307972 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb1940d2910 0x7fb194160330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:27.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.656+0000 7fb199b39640 1 -- 172.21.15.46:0/994307972 >> 172.21.15.46:0/994307972 conn(0x7fb194072020 msgr2=0x7fb194072400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:27.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.656+0000 7fb199b39640 1 -- 172.21.15.46:0/994307972 shutdown_connections 2023-12-02T00:30:27.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:27.656+0000 7fb199b39640 1 -- 172.21.15.46:0/994307972 wait complete. 2023-12-02T00:30:27.668 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-02T00:30:27.668 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-02T00:30:27.668 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2023-12-02T00:30:27.668 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-02T00:30:27.668 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-02T00:30:27.668 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-02T00:30:27.668 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-02T00:30:27.668 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-02T00:30:27.668 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-02T00:30:27.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:30:27.669 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2023-12-02T00:30:27.684 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-02T00:30:27.684 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-02T00:30:27.684 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2023-12-02T00:30:27.684 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:30:27.684 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2023-12-02T00:30:27.684 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:30:27.685 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-02T00:30:27.685 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2023-12-02T00:30:27.685 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2023-12-02T00:30:27.685 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-02T00:30:27.685 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2023-12-02T00:30:27.685 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2023-12-02T00:30:27.685 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-02T00:30:27.685 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2023-12-02T00:30:27.685 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2023-12-02T00:30:27.685 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-12-02T00:30:27.685 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42551sec preferred_lft 42551sec 2023-12-02T00:30:27.685 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link noprefixroute 2023-12-02T00:30:27.685 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:30:27.685 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-02T00:30:27.686 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2023-12-02T00:30:27.686 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2023-12-02T00:30:27.686 INFO:teuthology.orchestra.run.smithi046.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-02T00:30:27.686 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether b6:1a:8b:80:15:f1 brd ff:ff:ff:ff:ff:ff 2023-12-02T00:30:27.686 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-02T00:30:27.686 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:30:27.686 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::eca8:9ff:feb3:1726/64 scope link 2023-12-02T00:30:27.686 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:30:27.686 INFO:teuthology.orchestra.run.smithi046.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-02T00:30:27.686 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether b6:1a:8b:80:15:f1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-02T00:30:27.686 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::b41a:8bff:fe80:15f1/64 scope link 2023-12-02T00:30:27.686 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:30:27.686 INFO:teuthology.orchestra.run.smithi046.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-02T00:30:27.686 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether c6:93:a5:c0:e2:af brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-02T00:30:27.686 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c493:a5ff:fec0:e2af/64 scope link 2023-12-02T00:30:27.687 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:30:27.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:30:27.687 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-02T00:30:27.754 INFO:teuthology.orchestra.run.smithi046.stdout:1 2023-12-02T00:30:27.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:30:27.756 DEBUG:teuthology.orchestra.run.smithi046:> route 2023-12-02T00:30:27.812 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2023-12-02T00:30:27.812 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-02T00:30:27.812 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-12-02T00:30:27.812 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-12-02T00:30:27.812 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-02T00:30:27.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:30:27.812 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2023-12-02T00:30:27.813 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2023-12-02T00:30:27.813 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-02T00:30:27.813 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-02T00:30:27.813 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-02T00:30:27.813 DEBUG:teuthology.orchestra.run.smithi046:> ') 2023-12-02T00:30:27.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:30:27.919 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2023-12-02T00:30:27.933 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-02T00:30:27.933 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-02T00:30:27.934 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-02T00:30:27.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:30:27.934 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2023-12-02T00:30:28.005 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-12-02T00:30:28.005 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-02T00:30:28.005 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-02T00:30:28.060 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-02T00:30:28.060 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-02T00:30:28.061 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-02T00:30:28.115 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2023-12-02T00:30:28.187 DEBUG:teuthology.orchestra.run.smithi046:> cat /proc/self/mounts | awk '{print $2}' 2023-12-02T00:30:28.243 INFO:teuthology.orchestra.run.smithi046.stdout:/proc 2023-12-02T00:30:28.243 INFO:teuthology.orchestra.run.smithi046.stdout:/sys 2023-12-02T00:30:28.243 INFO:teuthology.orchestra.run.smithi046.stdout:/dev 2023-12-02T00:30:28.243 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/security 2023-12-02T00:30:28.243 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/shm 2023-12-02T00:30:28.243 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/pts 2023-12-02T00:30:28.243 INFO:teuthology.orchestra.run.smithi046.stdout:/run 2023-12-02T00:30:28.243 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/cgroup 2023-12-02T00:30:28.243 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/pstore 2023-12-02T00:30:28.243 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/bpf 2023-12-02T00:30:28.243 INFO:teuthology.orchestra.run.smithi046.stdout:/ 2023-12-02T00:30:28.244 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/selinux 2023-12-02T00:30:28.244 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2023-12-02T00:30:28.244 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/mqueue 2023-12-02T00:30:28.244 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/hugepages 2023-12-02T00:30:28.244 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/debug 2023-12-02T00:30:28.244 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/tracing 2023-12-02T00:30:28.244 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/fuse/connections 2023-12-02T00:30:28.244 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/config 2023-12-02T00:30:28.244 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-sysctl.service 2023-12-02T00:30:28.244 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-02T00:30:28.244 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-02T00:30:28.244 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/nfs/rpc_pipefs 2023-12-02T00:30:28.244 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/1000 2023-12-02T00:30:28.244 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph 2023-12-02T00:30:28.244 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-0 2023-12-02T00:30:28.244 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-1 2023-12-02T00:30:28.245 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-2 2023-12-02T00:30:28.245 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-3 2023-12-02T00:30:28.245 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns 2023-12-02T00:30:28.245 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-02T00:30:28.245 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-02T00:30:28.245 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-02T00:30:28.245 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-02T00:30:28.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:30:28.245 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2023-12-02T00:30:28.298 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-12-02T00:30:28.298 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-12-02T00:30:28.343 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2023-12-02T00:30:28.374 DEBUG:teuthology.orchestra.run.smithi046:> cat /proc/self/mounts | awk '{print $2}' 2023-12-02T00:30:28.430 INFO:teuthology.orchestra.run.smithi046.stdout:/proc 2023-12-02T00:30:28.431 INFO:teuthology.orchestra.run.smithi046.stdout:/sys 2023-12-02T00:30:28.431 INFO:teuthology.orchestra.run.smithi046.stdout:/dev 2023-12-02T00:30:28.431 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/security 2023-12-02T00:30:28.431 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/shm 2023-12-02T00:30:28.431 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/pts 2023-12-02T00:30:28.431 INFO:teuthology.orchestra.run.smithi046.stdout:/run 2023-12-02T00:30:28.431 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/cgroup 2023-12-02T00:30:28.431 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/pstore 2023-12-02T00:30:28.432 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/bpf 2023-12-02T00:30:28.432 INFO:teuthology.orchestra.run.smithi046.stdout:/ 2023-12-02T00:30:28.432 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/selinux 2023-12-02T00:30:28.432 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2023-12-02T00:30:28.432 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/mqueue 2023-12-02T00:30:28.432 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/hugepages 2023-12-02T00:30:28.432 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/debug 2023-12-02T00:30:28.433 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/tracing 2023-12-02T00:30:28.433 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/fuse/connections 2023-12-02T00:30:28.433 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/config 2023-12-02T00:30:28.433 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-sysctl.service 2023-12-02T00:30:28.433 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-02T00:30:28.433 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-02T00:30:28.433 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/nfs/rpc_pipefs 2023-12-02T00:30:28.433 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/1000 2023-12-02T00:30:28.433 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph 2023-12-02T00:30:28.433 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-0 2023-12-02T00:30:28.434 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-1 2023-12-02T00:30:28.434 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-2 2023-12-02T00:30:28.434 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-3 2023-12-02T00:30:28.434 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns 2023-12-02T00:30:28.434 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-02T00:30:28.434 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-02T00:30:28.434 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-02T00:30:28.434 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-02T00:30:28.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:30:28.435 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2023-12-02T00:30:28.437 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:2023-12-02T00:30:28.436+0000 7f1689a68480 -1 init, newargv = 0x55717fe9a3c0 newargc=15 2023-12-02T00:30:28.437 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[106757]: starting ceph client 2023-12-02T00:30:28.441 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[106757]: starting fuse 2023-12-02T00:30:28.452 INFO:teuthology.orchestra.run.smithi046.stdout:41 2023-12-02T00:30:28.453 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-12-02T00:30:28.453 DEBUG:teuthology.orchestra.run.smithi046:> sudo stdin-killer -- python3 -c ' 2023-12-02T00:30:28.453 DEBUG:teuthology.orchestra.run.smithi046:> import glob 2023-12-02T00:30:28.454 DEBUG:teuthology.orchestra.run.smithi046:> import re 2023-12-02T00:30:28.454 DEBUG:teuthology.orchestra.run.smithi046:> import os 2023-12-02T00:30:28.454 DEBUG:teuthology.orchestra.run.smithi046:> import subprocess 2023-12-02T00:30:28.454 DEBUG:teuthology.orchestra.run.smithi046:> 2023-12-02T00:30:28.454 DEBUG:teuthology.orchestra.run.smithi046:> def _find_admin_socket(client_name): 2023-12-02T00:30:28.454 DEBUG:teuthology.orchestra.run.smithi046:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-12-02T00:30:28.454 DEBUG:teuthology.orchestra.run.smithi046:> files = glob.glob(asok_path) 2023-12-02T00:30:28.454 DEBUG:teuthology.orchestra.run.smithi046:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-12-02T00:30:28.455 DEBUG:teuthology.orchestra.run.smithi046:> 2023-12-02T00:30:28.455 DEBUG:teuthology.orchestra.run.smithi046:> # Given a non-glob path, it better be there 2023-12-02T00:30:28.455 DEBUG:teuthology.orchestra.run.smithi046:> if "*" not in asok_path: 2023-12-02T00:30:28.455 DEBUG:teuthology.orchestra.run.smithi046:> assert(len(files) == 1) 2023-12-02T00:30:28.455 DEBUG:teuthology.orchestra.run.smithi046:> return files[0] 2023-12-02T00:30:28.455 DEBUG:teuthology.orchestra.run.smithi046:> 2023-12-02T00:30:28.455 DEBUG:teuthology.orchestra.run.smithi046:> for f in files: 2023-12-02T00:30:28.455 DEBUG:teuthology.orchestra.run.smithi046:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-02T00:30:28.455 DEBUG:teuthology.orchestra.run.smithi046:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-02T00:30:28.455 DEBUG:teuthology.orchestra.run.smithi046:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-02T00:30:28.456 DEBUG:teuthology.orchestra.run.smithi046:> contents = proc_f.read() 2023-12-02T00:30:28.456 DEBUG:teuthology.orchestra.run.smithi046:> if mountpoint in contents: 2023-12-02T00:30:28.456 DEBUG:teuthology.orchestra.run.smithi046:> return f 2023-12-02T00:30:28.456 DEBUG:teuthology.orchestra.run.smithi046:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-02T00:30:28.456 DEBUG:teuthology.orchestra.run.smithi046:> 2023-12-02T00:30:28.456 DEBUG:teuthology.orchestra.run.smithi046:> print(_find_admin_socket("client.0")) 2023-12-02T00:30:28.456 DEBUG:teuthology.orchestra.run.smithi046:> ' 2023-12-02T00:30:28.578 INFO:teuthology.orchestra.run.smithi046.stdout:/var/run/ceph/ceph-client.0.106757.asok 2023-12-02T00:30:28.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-12-02T00:30:28.588 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.106757.asok 2023-12-02T00:30:28.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:30:28.589 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.106757.asok status 2023-12-02T00:30:28.732 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-12-02T00:30:28.732 INFO:teuthology.orchestra.run.smithi046.stdout: "metadata": { 2023-12-02T00:30:28.732 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_sha1": "bed7681480ccb0192b91a35e691658867e1e96b7", 2023-12-02T00:30:28.732 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_version": "ceph version 18.0.0-7631-gbed76814 (bed7681480ccb0192b91a35e691658867e1e96b7) reef (dev)", 2023-12-02T00:30:28.733 INFO:teuthology.orchestra.run.smithi046.stdout: "entity_id": "0", 2023-12-02T00:30:28.733 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": "smithi046", 2023-12-02T00:30:28.733 INFO:teuthology.orchestra.run.smithi046.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-12-02T00:30:28.733 INFO:teuthology.orchestra.run.smithi046.stdout: "pid": "106757", 2023-12-02T00:30:28.733 INFO:teuthology.orchestra.run.smithi046.stdout: "root": "/" 2023-12-02T00:30:28.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:30:28.733 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_count": 0, 2023-12-02T00:30:28.733 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_pinned_count": 0, 2023-12-02T00:30:28.734 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 5730, 2023-12-02T00:30:28.734 INFO:teuthology.orchestra.run.smithi046.stdout: "inst": { 2023-12-02T00:30:28.734 INFO:teuthology.orchestra.run.smithi046.stdout: "name": { 2023-12-02T00:30:28.734 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "client", 2023-12-02T00:30:28.734 INFO:teuthology.orchestra.run.smithi046.stdout: "num": 5730 2023-12-02T00:30:28.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:30:28.734 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2023-12-02T00:30:28.735 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2023-12-02T00:30:28.735 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2023-12-02T00:30:28.735 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2402964533 2023-12-02T00:30:28.735 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:30:28.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:30:28.735 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2023-12-02T00:30:28.735 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2023-12-02T00:30:28.735 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2023-12-02T00:30:28.735 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2402964533 2023-12-02T00:30:28.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:30:28.736 INFO:teuthology.orchestra.run.smithi046.stdout: "inst_str": "client.5730 192.168.0.1:0/2402964533", 2023-12-02T00:30:28.736 INFO:teuthology.orchestra.run.smithi046.stdout: "addr_str": "192.168.0.1:0/2402964533", 2023-12-02T00:30:28.736 INFO:teuthology.orchestra.run.smithi046.stdout: "inode_count": 1, 2023-12-02T00:30:28.736 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_epoch": 14, 2023-12-02T00:30:28.736 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch": 38, 2023-12-02T00:30:28.736 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch_barrier": 0, 2023-12-02T00:30:28.736 INFO:teuthology.orchestra.run.smithi046.stdout: "blocklisted": false, 2023-12-02T00:30:28.737 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs" 2023-12-02T00:30:28.737 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-12-02T00:30:28.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:30:28.741 DEBUG:teuthology.orchestra.run.smithi046:> stat --file-system '--printf=%T 2023-12-02T00:30:28.741 DEBUG:teuthology.orchestra.run.smithi046:> ' -- /home/ubuntu/cephtest/mnt.0 2023-12-02T00:30:28.757 INFO:teuthology.orchestra.run.smithi046.stdout:fuseblk 2023-12-02T00:30:28.757 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-12-02T00:30:28.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:30:28.757 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-02T00:30:28.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:30:28.827 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.file.layout.pool -v data_new layoutfile) 2023-12-02T00:30:28.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:30:28.885 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.layout.pool -v data_new layoutdir) 2023-12-02T00:30:28.944 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-02T00:30:28.944 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-02T00:30:29.002 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:30:29.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.147+0000 7ff653d5a640 1 Processor -- start 2023-12-02T00:30:29.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.148+0000 7ff653d5a640 1 -- start start 2023-12-02T00:30:29.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.148+0000 7ff653d5a640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff64c15d530 0x7ff64c0d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:29.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.148+0000 7ff653d5a640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff64c0d1230 0x7ff64c0d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:29.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.148+0000 7ff653d5a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff64c0d1c20 0x7ff64c0cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:29.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.148+0000 7ff653d5a640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff64c0d20b0 con 0x7ff64c0d1230 2023-12-02T00:30:29.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.148+0000 7ff653d5a640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff64c0cf0a0 con 0x7ff64c15d530 2023-12-02T00:30:29.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.148+0000 7ff653d5a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff64c0b2a80 con 0x7ff64c0d1c20 2023-12-02T00:30:29.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.149+0000 7ff6512f5640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff64c0d1230 0x7ff64c0d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:29.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.149+0000 7ff6512f5640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff64c0d1230 0x7ff64c0d16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40798/0 (socket says 172.21.15.46:40798) 2023-12-02T00:30:29.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.149+0000 7ff6512f5640 1 -- 172.21.15.46:0/3097565383 learned_addr learned my addr 172.21.15.46:0/3097565383 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:29.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.149+0000 7ff651af6640 1 --2- 172.21.15.46:0/3097565383 >> v2:172.21.15.162:3300/0 conn(0x7ff64c15d530 0x7ff64c0d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:29.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.149+0000 7ff6522f7640 1 --2- 172.21.15.46:0/3097565383 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff64c0d1c20 0x7ff64c0cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:29.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.149+0000 7ff6512f5640 1 -- 172.21.15.46:0/3097565383 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff64c0d1c20 msgr2=0x7ff64c0cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:29.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.149+0000 7ff6512f5640 1 --2- 172.21.15.46:0/3097565383 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff64c0d1c20 0x7ff64c0cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:29.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.149+0000 7ff6512f5640 1 -- 172.21.15.46:0/3097565383 >> v2:172.21.15.162:3300/0 conn(0x7ff64c15d530 msgr2=0x7ff64c0d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:29.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.149+0000 7ff6512f5640 1 --2- 172.21.15.46:0/3097565383 >> v2:172.21.15.162:3300/0 conn(0x7ff64c15d530 0x7ff64c0d0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:29.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.149+0000 7ff6512f5640 1 -- 172.21.15.46:0/3097565383 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff64c0ceae0 con 0x7ff64c0d1230 2023-12-02T00:30:29.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.150+0000 7ff6512f5640 1 --2- 172.21.15.46:0/3097565383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff64c0d1230 0x7ff64c0d16e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff64c08d6b0 tx=0x7ff64002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42fcc56536150981 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:29.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.150+0000 7ff650af4640 1 -- 172.21.15.46:0/3097565383 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6400317e0 con 0x7ff64c0d1230 2023-12-02T00:30:29.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.150+0000 7ff650af4640 1 -- 172.21.15.46:0/3097565383 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff640031940 con 0x7ff64c0d1230 2023-12-02T00:30:29.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.150+0000 7ff650af4640 1 -- 172.21.15.46:0/3097565383 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff640031cd0 con 0x7ff64c0d1230 2023-12-02T00:30:29.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.150+0000 7ff653d5a640 1 -- 172.21.15.46:0/3097565383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff64c0d1230 msgr2=0x7ff64c0d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:29.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.150+0000 7ff653d5a640 1 --2- 172.21.15.46:0/3097565383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff64c0d1230 0x7ff64c0d16e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff64c08d6b0 tx=0x7ff64002b880 comp rx=0 tx=0).stop 2023-12-02T00:30:29.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.151+0000 7ff653d5a640 1 -- 172.21.15.46:0/3097565383 shutdown_connections 2023-12-02T00:30:29.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.151+0000 7ff653d5a640 1 --2- 172.21.15.46:0/3097565383 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff64c0d1c20 0x7ff64c0cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:29.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.151+0000 7ff653d5a640 1 --2- 172.21.15.46:0/3097565383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff64c0d1230 0x7ff64c0d16e0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:29.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.151+0000 7ff653d5a640 1 --2- 172.21.15.46:0/3097565383 >> v2:172.21.15.162:3300/0 conn(0x7ff64c15d530 0x7ff64c0d0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:29.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.151+0000 7ff653d5a640 1 -- 172.21.15.46:0/3097565383 >> 172.21.15.46:0/3097565383 conn(0x7ff64c09c480 msgr2=0x7ff64c0d3120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:29.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.151+0000 7ff653d5a640 1 -- 172.21.15.46:0/3097565383 shutdown_connections 2023-12-02T00:30:29.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.151+0000 7ff653d5a640 1 -- 172.21.15.46:0/3097565383 wait complete. 2023-12-02T00:30:29.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.152+0000 7ff653d5a640 1 Processor -- start 2023-12-02T00:30:29.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.152+0000 7ff653d5a640 1 -- start start 2023-12-02T00:30:29.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.152+0000 7ff653d5a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff64c0d1230 0x7ff64c154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:29.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.152+0000 7ff653d5a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff64c0d1c20 0x7ff64c0cbd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:29.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.152+0000 7ff653d5a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff64c15d530 0x7ff64c0cc250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:29.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.153+0000 7ff653d5a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff64c1f4c10 con 0x7ff64c0d1c20 2023-12-02T00:30:29.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.153+0000 7ff653d5a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff64c1f4600 con 0x7ff64c0d1230 2023-12-02T00:30:29.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.153+0000 7ff653d5a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff64c1f48e0 con 0x7ff64c15d530 2023-12-02T00:30:29.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.153+0000 7ff6512f5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff64c0d1c20 0x7ff64c0cbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:29.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.153+0000 7ff6512f5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff64c0d1c20 0x7ff64c0cbd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40810/0 (socket says 172.21.15.46:40810) 2023-12-02T00:30:29.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.153+0000 7ff6512f5640 1 -- 172.21.15.46:0/2749213038 learned_addr learned my addr 172.21.15.46:0/2749213038 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:29.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.153+0000 7ff651af6640 1 --2- 172.21.15.46:0/2749213038 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff64c0d1230 0x7ff64c154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:29.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.153+0000 7ff6522f7640 1 --2- 172.21.15.46:0/2749213038 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff64c15d530 0x7ff64c0cc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:29.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.153+0000 7ff6512f5640 1 -- 172.21.15.46:0/2749213038 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff64c15d530 msgr2=0x7ff64c0cc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:29.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.153+0000 7ff6512f5640 1 --2- 172.21.15.46:0/2749213038 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff64c15d530 0x7ff64c0cc250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:29.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.153+0000 7ff6512f5640 1 -- 172.21.15.46:0/2749213038 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff64c0d1230 msgr2=0x7ff64c154180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:29.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.153+0000 7ff6512f5640 1 --2- 172.21.15.46:0/2749213038 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff64c0d1230 0x7ff64c154180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:29.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.153+0000 7ff6512f5640 1 -- 172.21.15.46:0/2749213038 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6400094d0 con 0x7ff64c0d1c20 2023-12-02T00:30:29.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.153+0000 7ff651af6640 1 --2- 172.21.15.46:0/2749213038 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff64c0d1230 0x7ff64c154180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:30:29.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.153+0000 7ff6512f5640 1 --2- 172.21.15.46:0/2749213038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff64c0d1c20 0x7ff64c0cbd10 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff640000c00 tx=0x7ff6400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:29.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.153+0000 7ff64a7fc640 1 -- 172.21.15.46:0/2749213038 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff640032880 con 0x7ff64c0d1c20 2023-12-02T00:30:29.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.153+0000 7ff64a7fc640 1 -- 172.21.15.46:0/2749213038 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6400329e0 con 0x7ff64c0d1c20 2023-12-02T00:30:29.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.153+0000 7ff64a7fc640 1 -- 172.21.15.46:0/2749213038 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff640032d90 con 0x7ff64c0d1c20 2023-12-02T00:30:29.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.153+0000 7ff653d5a640 1 -- 172.21.15.46:0/2749213038 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff64c0b5d40 con 0x7ff64c0d1c20 2023-12-02T00:30:29.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.153+0000 7ff653d5a640 1 -- 172.21.15.46:0/2749213038 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff64c0b6300 con 0x7ff64c0d1c20 2023-12-02T00:30:29.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.154+0000 7ff653d5a640 1 -- 172.21.15.46:0/2749213038 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff64c0b1f70 con 0x7ff64c0d1c20 2023-12-02T00:30:29.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.155+0000 7ff64a7fc640 1 -- 172.21.15.46:0/2749213038 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff64004e450 con 0x7ff64c0d1c20 2023-12-02T00:30:29.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.155+0000 7ff64a7fc640 1 --2- 172.21.15.46:0/2749213038 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff63806ae40 0x7ff63806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:29.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.155+0000 7ff64a7fc640 1 -- 172.21.15.46:0/2749213038 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7ff6400ab7b0 con 0x7ff64c0d1c20 2023-12-02T00:30:29.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.155+0000 7ff651af6640 1 --2- 172.21.15.46:0/2749213038 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff63806ae40 0x7ff63806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:29.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.156+0000 7ff651af6640 1 --2- 172.21.15.46:0/2749213038 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff63806ae40 0x7ff63806d300 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff64c1f6fa0 tx=0x7ff644008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:29.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.159+0000 7ff64a7fc640 1 -- 172.21.15.46:0/2749213038 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff640076840 con 0x7ff64c0d1c20 2023-12-02T00:30:29.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.339+0000 7ff653d5a640 1 -- 172.21.15.46:0/2749213038 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff64c04ed60 con 0x7ff64c0d1c20 2023-12-02T00:30:29.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.341+0000 7ff64a7fc640 1 -- 172.21.15.46:0/2749213038 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+18324 (secure 0 0 0) 0x7ff640079e60 con 0x7ff64c0d1c20 2023-12-02T00:30:29.342 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:30:29.342 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":38,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:30:10.714057+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-02T00:29:55.676572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-02T00:29:56.244869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"data_new","create_time":"2023-12-02T00:30:08.897990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:30:29.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.342+0000 7ff653d5a640 1 -- 172.21.15.46:0/2749213038 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff63806ae40 msgr2=0x7ff63806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:29.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.342+0000 7ff653d5a640 1 --2- 172.21.15.46:0/2749213038 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff63806ae40 0x7ff63806d300 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff64c1f6fa0 tx=0x7ff644008040 comp rx=0 tx=0).stop 2023-12-02T00:30:29.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.343+0000 7ff653d5a640 1 -- 172.21.15.46:0/2749213038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff64c0d1c20 msgr2=0x7ff64c0cbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:29.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.343+0000 7ff653d5a640 1 --2- 172.21.15.46:0/2749213038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff64c0d1c20 0x7ff64c0cbd10 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff640000c00 tx=0x7ff6400046c0 comp rx=0 tx=0).stop 2023-12-02T00:30:29.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.343+0000 7ff653d5a640 1 -- 172.21.15.46:0/2749213038 shutdown_connections 2023-12-02T00:30:29.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.343+0000 7ff653d5a640 1 --2- 172.21.15.46:0/2749213038 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff63806ae40 0x7ff63806d300 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:29.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.343+0000 7ff653d5a640 1 --2- 172.21.15.46:0/2749213038 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff64c15d530 0x7ff64c0cc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:29.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.343+0000 7ff653d5a640 1 --2- 172.21.15.46:0/2749213038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff64c0d1c20 0x7ff64c0cbd10 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:29.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.343+0000 7ff653d5a640 1 --2- 172.21.15.46:0/2749213038 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff64c0d1230 0x7ff64c154180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:29.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.343+0000 7ff653d5a640 1 -- 172.21.15.46:0/2749213038 >> 172.21.15.46:0/2749213038 conn(0x7ff64c09c480 msgr2=0x7ff64c0b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:29.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.343+0000 7ff653d5a640 1 -- 172.21.15.46:0/2749213038 shutdown_connections 2023-12-02T00:30:29.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.343+0000 7ff653d5a640 1 -- 172.21.15.46:0/2749213038 wait complete. 2023-12-02T00:30:29.355 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-02T00:28:37.213645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-02T00:29:55.676572+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-12-02T00:29:56.244869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'data_new', 'create_time': '2023-12-02T00:30:08.897990+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-02T00:30:29.355 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-02T00:30:29.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.463+0000 7f314c609640 1 Processor -- start 2023-12-02T00:30:29.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.463+0000 7f314c609640 1 -- start start 2023-12-02T00:30:29.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.464+0000 7f314c609640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f31440d2910 0x7f31440d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:29.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.464+0000 7f314c609640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f31440d3730 0x7f31440b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:29.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.464+0000 7f314c609640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31440b2b10 con 0x7f314415d330 2023-12-02T00:30:29.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.464+0000 7f314c609640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31440d3bc0 con 0x7f31440d2910 2023-12-02T00:30:29.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.464+0000 7f314c609640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31440d1590 con 0x7f31440d3730 2023-12-02T00:30:29.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.464+0000 7f3149ba4640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f31440d2910 0x7f31440d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:29.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.464+0000 7f314a3a5640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f314415d330 0x7f31440b8b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:44628/0 (socket says 172.21.15.46:44628) 2023-12-02T00:30:29.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.465+0000 7f314a3a5640 1 -- 172.21.15.46:0/3592188964 learned_addr learned my addr 172.21.15.46:0/3592188964 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:30:29.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.465+0000 7f314aba6640 1 --2- 172.21.15.46:0/3592188964 >> v2:172.21.15.162:3300/0 conn(0x7f31440d3730 0x7f31440b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:29.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.465+0000 7f3149ba4640 1 -- 172.21.15.46:0/3592188964 >> v2:172.21.15.162:3300/0 conn(0x7f31440d3730 msgr2=0x7f31440b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:29.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.465+0000 7f3149ba4640 1 --2- 172.21.15.46:0/3592188964 >> v2:172.21.15.162:3300/0 conn(0x7f31440d3730 0x7f31440b6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:29.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.465+0000 7f3149ba4640 1 -- 172.21.15.46:0/3592188964 >> v1:172.21.15.46:6789/0 conn(0x7f314415d330 legacy=0x7f31440b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:29.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.465+0000 7f3149ba4640 1 -- 172.21.15.46:0/3592188964 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31440d1870 con 0x7f31440d2910 2023-12-02T00:30:29.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.465+0000 7f31493a3640 1 -- 172.21.15.46:0/3592188964 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 755416875 0 0) 0x7f31440b2b10 con 0x7f314415d330 2023-12-02T00:30:29.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.466+0000 7f3149ba4640 1 --2- 172.21.15.46:0/3592188964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31440d2910 0x7f31440d31f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f314406c500 tx=0x7f313802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87d3ee5fb6401cf9 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:29.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.466+0000 7f31493a3640 1 -- 172.21.15.46:0/3592188964 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313802e070 con 0x7f31440d2910 2023-12-02T00:30:29.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.466+0000 7f31493a3640 1 -- 172.21.15.46:0/3592188964 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31380315a0 con 0x7f31440d2910 2023-12-02T00:30:29.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.466+0000 7f31493a3640 1 -- 172.21.15.46:0/3592188964 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3138031930 con 0x7f31440d2910 2023-12-02T00:30:29.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.466+0000 7f314c609640 1 -- 172.21.15.46:0/3592188964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31440d2910 msgr2=0x7f31440d31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:29.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.466+0000 7f314c609640 1 --2- 172.21.15.46:0/3592188964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31440d2910 0x7f31440d31f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f314406c500 tx=0x7f313802b840 comp rx=0 tx=0).stop 2023-12-02T00:30:29.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.466+0000 7f314c609640 1 -- 172.21.15.46:0/3592188964 shutdown_connections 2023-12-02T00:30:29.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.466+0000 7f314c609640 1 --2- 172.21.15.46:0/3592188964 >> v2:172.21.15.162:3300/0 conn(0x7f31440d3730 0x7f31440b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:29.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.466+0000 7f314c609640 1 --2- 172.21.15.46:0/3592188964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31440d2910 0x7f31440d31f0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:29.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.466+0000 7f314c609640 1 -- 172.21.15.46:0/3592188964 >> 172.21.15.46:0/3592188964 conn(0x7f3144072020 msgr2=0x7f3144072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:29.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.466+0000 7f314c609640 1 -- 172.21.15.46:0/3592188964 shutdown_connections 2023-12-02T00:30:29.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.466+0000 7f314c609640 1 -- 172.21.15.46:0/3592188964 wait complete. 2023-12-02T00:30:29.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.467+0000 7f314c609640 1 Processor -- start 2023-12-02T00:30:29.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.468+0000 7f314c609640 1 -- start start 2023-12-02T00:30:29.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.468+0000 7f314c609640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31440d2910 0x7f3144153560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:29.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.468+0000 7f314c609640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31440d3730 0x7f31440c8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:29.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.468+0000 7f314c609640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f314415d330 0x7f314415f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:29.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.468+0000 7f314c609640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31440c8680 con 0x7f31440d2910 2023-12-02T00:30:29.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.468+0000 7f314c609640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31440c70a0 con 0x7f314415d330 2023-12-02T00:30:29.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.468+0000 7f314c609640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31440c7380 con 0x7f31440d3730 2023-12-02T00:30:29.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.468+0000 7f314a3a5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31440d2910 0x7f3144153560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:29.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.468+0000 7f314a3a5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31440d2910 0x7f3144153560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40840/0 (socket says 172.21.15.46:40840) 2023-12-02T00:30:29.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.468+0000 7f314a3a5640 1 -- 172.21.15.46:0/466012697 learned_addr learned my addr 172.21.15.46:0/466012697 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:29.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.468+0000 7f314aba6640 1 --2- 172.21.15.46:0/466012697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f314415d330 0x7f314415f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:29.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.468+0000 7f3149ba4640 1 --2- 172.21.15.46:0/466012697 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31440d3730 0x7f31440c8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:29.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.469+0000 7f314a3a5640 1 -- 172.21.15.46:0/466012697 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31440d3730 msgr2=0x7f31440c8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:29.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.469+0000 7f314a3a5640 1 --2- 172.21.15.46:0/466012697 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31440d3730 0x7f31440c8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:29.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.469+0000 7f314a3a5640 1 -- 172.21.15.46:0/466012697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f314415d330 msgr2=0x7f314415f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:29.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.469+0000 7f314a3a5640 1 --2- 172.21.15.46:0/466012697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f314415d330 0x7f314415f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:29.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.469+0000 7f314a3a5640 1 -- 172.21.15.46:0/466012697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3138009400 con 0x7f31440d2910 2023-12-02T00:30:29.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.469+0000 7f314a3a5640 1 --2- 172.21.15.46:0/466012697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31440d2910 0x7f3144153560 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f313400c7c0 tx=0x7f313400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:29.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.469+0000 7f3142ffd640 1 -- 172.21.15.46:0/466012697 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313400a640 con 0x7f31440d2910 2023-12-02T00:30:29.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.469+0000 7f3142ffd640 1 -- 172.21.15.46:0/466012697 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f313400a7c0 con 0x7f31440d2910 2023-12-02T00:30:29.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.469+0000 7f314c609640 1 -- 172.21.15.46:0/466012697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31440c7660 con 0x7f31440d2910 2023-12-02T00:30:29.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.469+0000 7f3142ffd640 1 -- 172.21.15.46:0/466012697 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313400ab70 con 0x7f31440d2910 2023-12-02T00:30:29.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.469+0000 7f314c609640 1 -- 172.21.15.46:0/466012697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31440c7c20 con 0x7f31440d2910 2023-12-02T00:30:29.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.470+0000 7f314c609640 1 -- 172.21.15.46:0/466012697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31440c7e10 con 0x7f31440d2910 2023-12-02T00:30:29.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.470+0000 7f3142ffd640 1 -- 172.21.15.46:0/466012697 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f313400acd0 con 0x7f31440d2910 2023-12-02T00:30:29.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.471+0000 7f3142ffd640 1 --2- 172.21.15.46:0/466012697 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f313006ae70 0x7f313006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:29.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.471+0000 7f3142ffd640 1 -- 172.21.15.46:0/466012697 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f3134088bc0 con 0x7f31440d2910 2023-12-02T00:30:29.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.471+0000 7f3149ba4640 1 --2- 172.21.15.46:0/466012697 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f313006ae70 0x7f313006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:29.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.472+0000 7f3149ba4640 1 --2- 172.21.15.46:0/466012697 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f313006ae70 0x7f313006d330 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3138036000 tx=0x7f313803d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:29.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.475+0000 7f3142ffd640 1 -- 172.21.15.46:0/466012697 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3134053b00 con 0x7f31440d2910 2023-12-02T00:30:29.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.650+0000 7f314c609640 1 -- 172.21.15.46:0/466012697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3144069400 con 0x7f31440d2910 2023-12-02T00:30:29.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.651+0000 7f3142ffd640 1 -- 172.21.15.46:0/466012697 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3134018070 con 0x7f31440d2910 2023-12-02T00:30:29.652 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-12-02T00:30:29.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.653+0000 7f314c609640 1 -- 172.21.15.46:0/466012697 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f313006ae70 msgr2=0x7f313006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:29.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.653+0000 7f314c609640 1 --2- 172.21.15.46:0/466012697 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f313006ae70 0x7f313006d330 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3138036000 tx=0x7f313803d040 comp rx=0 tx=0).stop 2023-12-02T00:30:29.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.653+0000 7f314c609640 1 -- 172.21.15.46:0/466012697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31440d2910 msgr2=0x7f3144153560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:29.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.653+0000 7f314c609640 1 --2- 172.21.15.46:0/466012697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31440d2910 0x7f3144153560 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f313400c7c0 tx=0x7f313400cc80 comp rx=0 tx=0).stop 2023-12-02T00:30:29.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.653+0000 7f314c609640 1 -- 172.21.15.46:0/466012697 shutdown_connections 2023-12-02T00:30:29.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.653+0000 7f314c609640 1 --2- 172.21.15.46:0/466012697 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f313006ae70 0x7f313006d330 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:29.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.653+0000 7f314c609640 1 --2- 172.21.15.46:0/466012697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f314415d330 0x7f314415f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:29.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.653+0000 7f314c609640 1 --2- 172.21.15.46:0/466012697 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31440d3730 0x7f31440c8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:29.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.653+0000 7f314c609640 1 --2- 172.21.15.46:0/466012697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31440d2910 0x7f3144153560 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:29.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.653+0000 7f314c609640 1 -- 172.21.15.46:0/466012697 >> 172.21.15.46:0/466012697 conn(0x7f3144072020 msgr2=0x7f31440b6be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:29.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.653+0000 7f314c609640 1 -- 172.21.15.46:0/466012697 shutdown_connections 2023-12-02T00:30:29.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.653+0000 7f314c609640 1 -- 172.21.15.46:0/466012697 wait complete. 2023-12-02T00:30:29.665 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-02T00:30:29.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.774+0000 7f1ee6c75640 1 Processor -- start 2023-12-02T00:30:29.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.774+0000 7f1ee6c75640 1 -- start start 2023-12-02T00:30:29.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.775+0000 7f1ee6c75640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1ee0150940 0x7f1ee015d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:29.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.775+0000 7f1ee6c75640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ee00b49f0 con 0x7f1ee015dec0 2023-12-02T00:30:29.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.775+0000 7f1ee6c75640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ee00b3560 con 0x7f1ee0150560 2023-12-02T00:30:29.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.775+0000 7f1ee6c75640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ee00b3de0 con 0x7f1ee0150940 2023-12-02T00:30:29.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.775+0000 7f1ee5212640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1ee015dec0 0x7f1ee00ce0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:44642/0 (socket says 172.21.15.46:44642) 2023-12-02T00:30:29.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.775+0000 7f1ee5212640 1 -- 172.21.15.46:0/3533866440 learned_addr learned my addr 172.21.15.46:0/3533866440 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:30:29.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.775+0000 7f1edffff640 1 --2- 172.21.15.46:0/3533866440 >> v2:172.21.15.162:3300/0 conn(0x7f1ee0150940 0x7f1ee015d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:29.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.775+0000 7f1edf7fe640 1 -- 172.21.15.46:0/3533866440 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1078205905 0 0) 0x7f1ee00b49f0 con 0x7f1ee015dec0 2023-12-02T00:30:29.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.775+0000 7f1edf7fe640 1 -- 172.21.15.46:0/3533866440 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ecc003520 con 0x7f1ee015dec0 2023-12-02T00:30:29.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.776+0000 7f1edf7fe640 1 -- 172.21.15.46:0/3533866440 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2072051540 0 0) 0x7f1ecc003520 con 0x7f1ee015dec0 2023-12-02T00:30:29.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.776+0000 7f1edf7fe640 1 -- 172.21.15.46:0/3533866440 >> v2:172.21.15.162:3300/0 conn(0x7f1ee0150940 msgr2=0x7f1ee015d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:29.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.776+0000 7f1edf7fe640 1 --2- 172.21.15.46:0/3533866440 >> v2:172.21.15.162:3300/0 conn(0x7f1ee0150940 0x7f1ee015d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:29.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.776+0000 7f1edf7fe640 1 -- 172.21.15.46:0/3533866440 >> v1:172.21.15.162:6789/0 conn(0x7f1ee0150560 legacy=0x7f1ee00b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:29.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.776+0000 7f1edf7fe640 1 -- 172.21.15.46:0/3533866440 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee00b40c0 con 0x7f1ee015dec0 2023-12-02T00:30:29.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.776+0000 7f1edf7fe640 1 -- 172.21.15.46:0/3533866440 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7f1ed0001670 con 0x7f1ee015dec0 2023-12-02T00:30:29.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.776+0000 7f1edf7fe640 1 -- 172.21.15.46:0/3533866440 >> v1:172.21.15.46:6789/0 conn(0x7f1ee015dec0 legacy=0x7f1ee00ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:29.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.776+0000 7f1edf7fe640 1 --2- 172.21.15.46:0/3533866440 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ecc003970 0x7f1ecc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:29.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.776+0000 7f1edf7fe640 1 --2- 172.21.15.46:0/3533866440 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ecc0047c0 0x7f1ecc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:29.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.776+0000 7f1edf7fe640 1 --2- 172.21.15.46:0/3533866440 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ecc005180 0x7f1ecc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:29.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.776+0000 7f1edf7fe640 1 -- 172.21.15.46:0/3533866440 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee00b3de0 con 0x7f1ecc005180 2023-12-02T00:30:29.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.776+0000 7f1edf7fe640 1 -- 172.21.15.46:0/3533866440 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ecc00c020 con 0x7f1ecc003970 2023-12-02T00:30:29.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.776+0000 7f1edf7fe640 1 -- 172.21.15.46:0/3533866440 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ecc00c1e0 con 0x7f1ecc0047c0 2023-12-02T00:30:29.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.776+0000 7f1ee5212640 1 --2- 172.21.15.46:0/3533866440 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ecc005180 0x7f1ecc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:29.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.776+0000 7f1ee4a11640 1 --2- 172.21.15.46:0/3533866440 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ecc003970 0x7f1ecc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:29.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.776+0000 7f1edffff640 1 --2- 172.21.15.46:0/3533866440 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ecc0047c0 0x7f1ecc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:29.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.776+0000 7f1ee5212640 1 -- 172.21.15.46:0/3533866440 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ecc0047c0 msgr2=0x7f1ecc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:29.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.776+0000 7f1ee5212640 1 --2- 172.21.15.46:0/3533866440 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ecc0047c0 0x7f1ecc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:29.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.776+0000 7f1ee5212640 1 -- 172.21.15.46:0/3533866440 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ecc003970 msgr2=0x7f1ecc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:29.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.776+0000 7f1ee5212640 1 --2- 172.21.15.46:0/3533866440 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ecc003970 0x7f1ecc003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:29.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.776+0000 7f1ee5212640 1 -- 172.21.15.46:0/3533866440 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ecc003520 con 0x7f1ecc005180 2023-12-02T00:30:29.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.777+0000 7f1ee5212640 1 -- 172.21.15.46:0/3533866440 reap_dead start 2023-12-02T00:30:29.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.777+0000 7f1ee5212640 1 --2- 172.21.15.46:0/3533866440 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ecc005180 0x7f1ecc00b980 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1ed000c240 tx=0x7f1ecc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4d647394c667e57 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:29.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.777+0000 7f1edf7fe640 1 -- 172.21.15.46:0/3533866440 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed0005ad0 con 0x7f1ecc005180 2023-12-02T00:30:29.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.777+0000 7f1edf7fe640 1 -- 172.21.15.46:0/3533866440 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed0005c70 con 0x7f1ecc005180 2023-12-02T00:30:29.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.777+0000 7f1edf7fe640 1 -- 172.21.15.46:0/3533866440 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed002c8b0 con 0x7f1ecc005180 2023-12-02T00:30:29.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.778+0000 7f1ee6c75640 1 -- 172.21.15.46:0/3533866440 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ecc005180 msgr2=0x7f1ecc00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:29.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.778+0000 7f1ee6c75640 1 --2- 172.21.15.46:0/3533866440 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ecc005180 0x7f1ecc00b980 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1ed000c240 tx=0x7f1ecc0057f0 comp rx=0 tx=0).stop 2023-12-02T00:30:29.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.778+0000 7f1ee6c75640 1 -- 172.21.15.46:0/3533866440 shutdown_connections 2023-12-02T00:30:29.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.778+0000 7f1ee6c75640 1 --2- 172.21.15.46:0/3533866440 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ecc005180 0x7f1ecc00b980 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:29.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.778+0000 7f1ee6c75640 1 -- 172.21.15.46:0/3533866440 >> 172.21.15.46:0/3533866440 conn(0x7f1ee006a4d0 msgr2=0x7f1ee006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:29.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.778+0000 7f1ee6c75640 1 -- 172.21.15.46:0/3533866440 shutdown_connections 2023-12-02T00:30:29.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.778+0000 7f1ee6c75640 1 -- 172.21.15.46:0/3533866440 wait complete. 2023-12-02T00:30:29.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.779+0000 7f1ee6c75640 1 Processor -- start 2023-12-02T00:30:29.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.779+0000 7f1ee6c75640 1 -- start start 2023-12-02T00:30:29.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.780+0000 7f1ee6c75640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ecc005180 0x7f1ee015bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:29.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.780+0000 7f1ee6c75640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ee01400e0 0x7f1ee014c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:29.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.780+0000 7f1ee6c75640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ee00cb270 0x7f1ee00cb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:29.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.780+0000 7f1ee6c75640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee00ac8e0 con 0x7f1ee01400e0 2023-12-02T00:30:29.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.780+0000 7f1ee6c75640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee00cbc60 con 0x7f1ee00cb270 2023-12-02T00:30:29.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.780+0000 7f1ee6c75640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee01f2c20 con 0x7f1ecc005180 2023-12-02T00:30:29.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.780+0000 7f1edffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ee01400e0 0x7f1ee014c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:29.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.780+0000 7f1edffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ee01400e0 0x7f1ee014c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40852/0 (socket says 172.21.15.46:40852) 2023-12-02T00:30:29.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.780+0000 7f1edffff640 1 -- 172.21.15.46:0/1982452094 learned_addr learned my addr 172.21.15.46:0/1982452094 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:29.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.780+0000 7f1ee5212640 1 --2- 172.21.15.46:0/1982452094 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ee00cb270 0x7f1ee00cb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:29.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.780+0000 7f1ee4a11640 1 --2- 172.21.15.46:0/1982452094 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ecc005180 0x7f1ee015bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:29.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.780+0000 7f1edffff640 1 -- 172.21.15.46:0/1982452094 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ecc005180 msgr2=0x7f1ee015bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:29.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.780+0000 7f1edffff640 1 --2- 172.21.15.46:0/1982452094 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ecc005180 0x7f1ee015bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:29.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.780+0000 7f1edffff640 1 -- 172.21.15.46:0/1982452094 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ee00cb270 msgr2=0x7f1ee00cb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:29.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.780+0000 7f1edffff640 1 --2- 172.21.15.46:0/1982452094 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ee00cb270 0x7f1ee00cb720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:29.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.780+0000 7f1edffff640 1 -- 172.21.15.46:0/1982452094 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee00ace30 con 0x7f1ee01400e0 2023-12-02T00:30:29.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.780+0000 7f1edffff640 1 --2- 172.21.15.46:0/1982452094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ee01400e0 0x7f1ee014c7e0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8011820 tx=0x7f1ec8011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:29.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.781+0000 7f1edd7fa640 1 -- 172.21.15.46:0/1982452094 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec800fd30 con 0x7f1ee01400e0 2023-12-02T00:30:29.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.781+0000 7f1edd7fa640 1 -- 172.21.15.46:0/1982452094 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ec800fe90 con 0x7f1ee01400e0 2023-12-02T00:30:29.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.781+0000 7f1ee6c75640 1 -- 172.21.15.46:0/1982452094 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ee015c8e0 con 0x7f1ee01400e0 2023-12-02T00:30:29.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.781+0000 7f1edd7fa640 1 -- 172.21.15.46:0/1982452094 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec801c6c0 con 0x7f1ee01400e0 2023-12-02T00:30:29.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.781+0000 7f1ee6c75640 1 -- 172.21.15.46:0/1982452094 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ee00acac0 con 0x7f1ee01400e0 2023-12-02T00:30:29.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.782+0000 7f1ee6c75640 1 -- 172.21.15.46:0/1982452094 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ee015ce00 con 0x7f1ee01400e0 2023-12-02T00:30:29.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.782+0000 7f1edd7fa640 1 -- 172.21.15.46:0/1982452094 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1ec801c820 con 0x7f1ee01400e0 2023-12-02T00:30:29.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.783+0000 7f1edd7fa640 1 --2- 172.21.15.46:0/1982452094 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f1ec006ae40 0x7f1ec006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:29.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.783+0000 7f1edd7fa640 1 -- 172.21.15.46:0/1982452094 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f1ec8017070 con 0x7f1ee01400e0 2023-12-02T00:30:29.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.783+0000 7f1ee4a11640 1 --2- 172.21.15.46:0/1982452094 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f1ec006ae40 0x7f1ec006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:29.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.783+0000 7f1ee4a11640 1 --2- 172.21.15.46:0/1982452094 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f1ec006ae40 0x7f1ec006d300 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8000f80 tx=0x7f1ed8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:29.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.787+0000 7f1edd7fa640 1 -- 172.21.15.46:0/1982452094 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1ee015ce00 con 0x7f1ee01400e0 2023-12-02T00:30:29.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.966+0000 7f1ee6c75640 1 -- 172.21.15.46:0/1982452094 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1ee015ce00 con 0x7f1ee01400e0 2023-12-02T00:30:29.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.967+0000 7f1edd7fa640 1 -- 172.21.15.46:0/1982452094 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v38) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1ee015ce00 con 0x7f1ee01400e0 2023-12-02T00:30:29.968 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-12-02T00:30:29.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.969+0000 7f1ee6c75640 1 -- 172.21.15.46:0/1982452094 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f1ec006ae40 msgr2=0x7f1ec006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:29.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.969+0000 7f1ee6c75640 1 --2- 172.21.15.46:0/1982452094 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f1ec006ae40 0x7f1ec006d300 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8000f80 tx=0x7f1ed8009040 comp rx=0 tx=0).stop 2023-12-02T00:30:29.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.969+0000 7f1ee6c75640 1 -- 172.21.15.46:0/1982452094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ee01400e0 msgr2=0x7f1ee014c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:29.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.969+0000 7f1ee6c75640 1 --2- 172.21.15.46:0/1982452094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ee01400e0 0x7f1ee014c7e0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8011820 tx=0x7f1ec8011ce0 comp rx=0 tx=0).stop 2023-12-02T00:30:29.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.969+0000 7f1ee6c75640 1 -- 172.21.15.46:0/1982452094 shutdown_connections 2023-12-02T00:30:29.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.969+0000 7f1ee6c75640 1 --2- 172.21.15.46:0/1982452094 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f1ec006ae40 0x7f1ec006d300 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:29.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.969+0000 7f1ee6c75640 1 --2- 172.21.15.46:0/1982452094 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ee00cb270 0x7f1ee00cb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:29.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.969+0000 7f1ee6c75640 1 --2- 172.21.15.46:0/1982452094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ee01400e0 0x7f1ee014c7e0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:29.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.969+0000 7f1ee6c75640 1 --2- 172.21.15.46:0/1982452094 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ecc005180 0x7f1ee015bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:29.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.969+0000 7f1ee6c75640 1 -- 172.21.15.46:0/1982452094 >> 172.21.15.46:0/1982452094 conn(0x7f1ee006a4d0 msgr2=0x7f1ee00b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:29.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.969+0000 7f1ee6c75640 1 -- 172.21.15.46:0/1982452094 shutdown_connections 2023-12-02T00:30:29.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:29.969+0000 7f1ee6c75640 1 -- 172.21.15.46:0/1982452094 wait complete. 2023-12-02T00:30:29.982 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-02T00:30:30.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.090+0000 7fe92f4b4640 1 Processor -- start 2023-12-02T00:30:30.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.091+0000 7fe92f4b4640 1 -- start start 2023-12-02T00:30:30.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.091+0000 7fe92f4b4640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe92806bbf0 0x7fe92806d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:30.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.091+0000 7fe92f4b4640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9280adae0 con 0x7fe9280ad640 2023-12-02T00:30:30.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.091+0000 7fe92f4b4640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9280addc0 con 0x7fe92815d1d0 2023-12-02T00:30:30.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.091+0000 7fe92f4b4640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9280ae0a0 con 0x7fe92806bbf0 2023-12-02T00:30:30.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.092+0000 7fe92ca4f640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe9280ad640 0x7fe92806d0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:44650/0 (socket says 172.21.15.46:44650) 2023-12-02T00:30:30.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.092+0000 7fe92ca4f640 1 -- 172.21.15.46:0/3270491709 learned_addr learned my addr 172.21.15.46:0/3270491709 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:30:30.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.092+0000 7fe92da51640 1 --2- 172.21.15.46:0/3270491709 >> v2:172.21.15.162:3300/0 conn(0x7fe92806bbf0 0x7fe92806d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:30.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.092+0000 7fe927fff640 1 -- 172.21.15.46:0/3270491709 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 434106003 0 0) 0x7fe9280adae0 con 0x7fe9280ad640 2023-12-02T00:30:30.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.092+0000 7fe927fff640 1 -- 172.21.15.46:0/3270491709 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe910003520 con 0x7fe9280ad640 2023-12-02T00:30:30.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.092+0000 7fe927fff640 1 -- 172.21.15.46:0/3270491709 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 721068471 0 0) 0x7fe910003520 con 0x7fe9280ad640 2023-12-02T00:30:30.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.092+0000 7fe927fff640 1 -- 172.21.15.46:0/3270491709 >> v2:172.21.15.162:3300/0 conn(0x7fe92806bbf0 msgr2=0x7fe92806d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:30.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.092+0000 7fe927fff640 1 --2- 172.21.15.46:0/3270491709 >> v2:172.21.15.162:3300/0 conn(0x7fe92806bbf0 0x7fe92806d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:30.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.092+0000 7fe927fff640 1 -- 172.21.15.46:0/3270491709 >> v1:172.21.15.162:6789/0 conn(0x7fe92815d1d0 legacy=0x7fe92806b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:30.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.092+0000 7fe927fff640 1 -- 172.21.15.46:0/3270491709 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9280ae380 con 0x7fe9280ad640 2023-12-02T00:30:30.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.092+0000 7fe927fff640 1 -- 172.21.15.46:0/3270491709 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7fe91c001670 con 0x7fe9280ad640 2023-12-02T00:30:30.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.092+0000 7fe927fff640 1 -- 172.21.15.46:0/3270491709 >> v1:172.21.15.46:6789/0 conn(0x7fe9280ad640 legacy=0x7fe92806d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:30.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.092+0000 7fe927fff640 1 --2- 172.21.15.46:0/3270491709 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe910003970 0x7fe910003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:30.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.092+0000 7fe927fff640 1 --2- 172.21.15.46:0/3270491709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe910004370 0x7fe910004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:30.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.092+0000 7fe927fff640 1 --2- 172.21.15.46:0/3270491709 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe910005040 0x7fe91000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:30.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.092+0000 7fe927fff640 1 -- 172.21.15.46:0/3270491709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9280ae0a0 con 0x7fe910004370 2023-12-02T00:30:30.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.092+0000 7fe927fff640 1 -- 172.21.15.46:0/3270491709 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91000bfa0 con 0x7fe910003970 2023-12-02T00:30:30.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.092+0000 7fe927fff640 1 -- 172.21.15.46:0/3270491709 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe91000c160 con 0x7fe910005040 2023-12-02T00:30:30.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.092+0000 7fe92ca4f640 1 --2- 172.21.15.46:0/3270491709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe910004370 0x7fe910004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:30.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.092+0000 7fe92d250640 1 --2- 172.21.15.46:0/3270491709 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe910003970 0x7fe910003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:30.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.093+0000 7fe92da51640 1 --2- 172.21.15.46:0/3270491709 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe910005040 0x7fe91000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:30.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.093+0000 7fe92ca4f640 1 -- 172.21.15.46:0/3270491709 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe910005040 msgr2=0x7fe91000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:30.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.093+0000 7fe92ca4f640 1 --2- 172.21.15.46:0/3270491709 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe910005040 0x7fe91000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:30.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.093+0000 7fe92ca4f640 1 -- 172.21.15.46:0/3270491709 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe910003970 msgr2=0x7fe910003d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:30.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.093+0000 7fe92ca4f640 1 --2- 172.21.15.46:0/3270491709 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe910003970 0x7fe910003d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:30.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.093+0000 7fe92ca4f640 1 -- 172.21.15.46:0/3270491709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe910003520 con 0x7fe910004370 2023-12-02T00:30:30.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.093+0000 7fe92da51640 1 --2- 172.21.15.46:0/3270491709 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe910005040 0x7fe91000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:30:30.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.093+0000 7fe92da51640 1 -- 172.21.15.46:0/3270491709 reap_dead start 2023-12-02T00:30:30.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.093+0000 7fe92ca4f640 1 --2- 172.21.15.46:0/3270491709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe910004370 0x7fe910004840 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe91c00c190 tx=0x7fe9100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f937d8299dcb2e9 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:30.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.093+0000 7fe927fff640 1 -- 172.21.15.46:0/3270491709 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91c005ad0 con 0x7fe910004370 2023-12-02T00:30:30.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.094+0000 7fe927fff640 1 -- 172.21.15.46:0/3270491709 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe91c005c50 con 0x7fe910004370 2023-12-02T00:30:30.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.094+0000 7fe927fff640 1 -- 172.21.15.46:0/3270491709 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91c02c7c0 con 0x7fe910004370 2023-12-02T00:30:30.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.094+0000 7fe92f4b4640 1 -- 172.21.15.46:0/3270491709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe910004370 msgr2=0x7fe910004840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:30.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.094+0000 7fe92f4b4640 1 --2- 172.21.15.46:0/3270491709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe910004370 0x7fe910004840 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe91c00c190 tx=0x7fe9100057f0 comp rx=0 tx=0).stop 2023-12-02T00:30:30.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.094+0000 7fe92f4b4640 1 -- 172.21.15.46:0/3270491709 shutdown_connections 2023-12-02T00:30:30.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.094+0000 7fe92f4b4640 1 --2- 172.21.15.46:0/3270491709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe910004370 0x7fe910004840 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:30.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.094+0000 7fe92f4b4640 1 -- 172.21.15.46:0/3270491709 >> 172.21.15.46:0/3270491709 conn(0x7fe928070e70 msgr2=0x7fe928071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:30.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.094+0000 7fe92f4b4640 1 -- 172.21.15.46:0/3270491709 shutdown_connections 2023-12-02T00:30:30.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.094+0000 7fe92f4b4640 1 -- 172.21.15.46:0/3270491709 wait complete. 2023-12-02T00:30:30.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.095+0000 7fe92f4b4640 1 Processor -- start 2023-12-02T00:30:30.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.095+0000 7fe92f4b4640 1 -- start start 2023-12-02T00:30:30.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.096+0000 7fe92f4b4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe910004370 0x7fe92815e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:30.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.096+0000 7fe92f4b4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9281507c0 0x7fe928153f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:30.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.096+0000 7fe92f4b4640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9281593e0 0x7fe928159860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:30.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.096+0000 7fe92f4b4640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe928155bd0 con 0x7fe9281507c0 2023-12-02T00:30:30.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.096+0000 7fe92f4b4640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe928159da0 con 0x7fe9281593e0 2023-12-02T00:30:30.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.096+0000 7fe92f4b4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe92815edb0 con 0x7fe910004370 2023-12-02T00:30:30.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.096+0000 7fe92ca4f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9281507c0 0x7fe928153f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:30.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.096+0000 7fe92ca4f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9281507c0 0x7fe928153f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40860/0 (socket says 172.21.15.46:40860) 2023-12-02T00:30:30.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.096+0000 7fe92ca4f640 1 -- 172.21.15.46:0/533250006 learned_addr learned my addr 172.21.15.46:0/533250006 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:30.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.096+0000 7fe92d250640 1 --2- 172.21.15.46:0/533250006 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe910004370 0x7fe92815e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:30.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.096+0000 7fe92da51640 1 --2- 172.21.15.46:0/533250006 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9281593e0 0x7fe928159860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:30.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.096+0000 7fe92ca4f640 1 -- 172.21.15.46:0/533250006 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe910004370 msgr2=0x7fe92815e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:30.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.096+0000 7fe92ca4f640 1 --2- 172.21.15.46:0/533250006 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe910004370 0x7fe92815e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:30.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.096+0000 7fe92ca4f640 1 -- 172.21.15.46:0/533250006 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9281593e0 msgr2=0x7fe928159860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:30.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.096+0000 7fe92ca4f640 1 --2- 172.21.15.46:0/533250006 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9281593e0 0x7fe928159860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:30.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.096+0000 7fe92ca4f640 1 -- 172.21.15.46:0/533250006 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe92815f090 con 0x7fe9281507c0 2023-12-02T00:30:30.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.097+0000 7fe92d250640 1 --2- 172.21.15.46:0/533250006 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe910004370 0x7fe92815e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:30:30.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.097+0000 7fe92ca4f640 1 --2- 172.21.15.46:0/533250006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9281507c0 0x7fe928153f00 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe91c02c360 tx=0x7fe91c02e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:30.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.097+0000 7fe925ffb640 1 -- 172.21.15.46:0/533250006 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91c03c040 con 0x7fe9281507c0 2023-12-02T00:30:30.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.097+0000 7fe925ffb640 1 -- 172.21.15.46:0/533250006 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe91c02cc90 con 0x7fe9281507c0 2023-12-02T00:30:30.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.097+0000 7fe925ffb640 1 -- 172.21.15.46:0/533250006 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91c007070 con 0x7fe9281507c0 2023-12-02T00:30:30.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.097+0000 7fe92f4b4640 1 -- 172.21.15.46:0/533250006 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe92815f370 con 0x7fe9281507c0 2023-12-02T00:30:30.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.097+0000 7fe92f4b4640 1 -- 172.21.15.46:0/533250006 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe928150c40 con 0x7fe9281507c0 2023-12-02T00:30:30.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.098+0000 7fe92f4b4640 1 -- 172.21.15.46:0/533250006 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe928155d10 con 0x7fe9281507c0 2023-12-02T00:30:30.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.098+0000 7fe925ffb640 1 -- 172.21.15.46:0/533250006 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe91c032070 con 0x7fe9281507c0 2023-12-02T00:30:30.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.099+0000 7fe925ffb640 1 --2- 172.21.15.46:0/533250006 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe91406ae40 0x7fe91406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:30.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.099+0000 7fe925ffb640 1 -- 172.21.15.46:0/533250006 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fe91c0aabf0 con 0x7fe9281507c0 2023-12-02T00:30:30.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.099+0000 7fe92d250640 1 --2- 172.21.15.46:0/533250006 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe91406ae40 0x7fe91406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:30.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.100+0000 7fe92d250640 1 --2- 172.21.15.46:0/533250006 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe91406ae40 0x7fe91406d300 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe920003640 tx=0x7fe920009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:30.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.103+0000 7fe925ffb640 1 -- 172.21.15.46:0/533250006 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe928155d10 con 0x7fe9281507c0 2023-12-02T00:30:30.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.279+0000 7fe92f4b4640 1 -- 172.21.15.46:0/533250006 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe928155d10 con 0x7fe9281507c0 2023-12-02T00:30:30.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.280+0000 7fe925ffb640 1 -- 172.21.15.46:0/533250006 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v38) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe928155d10 con 0x7fe9281507c0 2023-12-02T00:30:30.281 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-12-02T00:30:30.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.282+0000 7fe92f4b4640 1 -- 172.21.15.46:0/533250006 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe91406ae40 msgr2=0x7fe91406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:30.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.282+0000 7fe92f4b4640 1 --2- 172.21.15.46:0/533250006 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe91406ae40 0x7fe91406d300 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe920003640 tx=0x7fe920009040 comp rx=0 tx=0).stop 2023-12-02T00:30:30.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.282+0000 7fe92f4b4640 1 -- 172.21.15.46:0/533250006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9281507c0 msgr2=0x7fe928153f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:30.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.282+0000 7fe92f4b4640 1 --2- 172.21.15.46:0/533250006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9281507c0 0x7fe928153f00 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe91c02c360 tx=0x7fe91c02e070 comp rx=0 tx=0).stop 2023-12-02T00:30:30.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.282+0000 7fe92f4b4640 1 -- 172.21.15.46:0/533250006 shutdown_connections 2023-12-02T00:30:30.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.282+0000 7fe92f4b4640 1 --2- 172.21.15.46:0/533250006 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe91406ae40 0x7fe91406d300 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:30.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.282+0000 7fe92f4b4640 1 --2- 172.21.15.46:0/533250006 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9281593e0 0x7fe928159860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:30.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.282+0000 7fe92f4b4640 1 --2- 172.21.15.46:0/533250006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9281507c0 0x7fe928153f00 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:30.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.282+0000 7fe92f4b4640 1 --2- 172.21.15.46:0/533250006 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe910004370 0x7fe92815e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:30.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.282+0000 7fe92f4b4640 1 -- 172.21.15.46:0/533250006 >> 172.21.15.46:0/533250006 conn(0x7fe928070e70 msgr2=0x7fe9280b0100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:30.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.282+0000 7fe92f4b4640 1 -- 172.21.15.46:0/533250006 shutdown_connections 2023-12-02T00:30:30.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.282+0000 7fe92f4b4640 1 -- 172.21.15.46:0/533250006 wait complete. 2023-12-02T00:30:30.294 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2023-12-02T00:30:30.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.403+0000 7f2edd4d4640 1 Processor -- start 2023-12-02T00:30:30.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.403+0000 7f2edd4d4640 1 -- start start 2023-12-02T00:30:30.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.404+0000 7f2edd4d4640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2ed8160f70 0x7f2ed80cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:30.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.404+0000 7f2edd4d4640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2ed80ce480 0x7f2ed80d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:30.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.404+0000 7f2edd4d4640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ed81614e0 con 0x7f2ed815d2d0 2023-12-02T00:30:30.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.404+0000 7f2edd4d4640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ed81617c0 con 0x7f2ed80ce480 2023-12-02T00:30:30.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.404+0000 7f2edd4d4640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ed8161aa0 con 0x7f2ed8160f70 2023-12-02T00:30:30.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.404+0000 7f2ed77fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2ed80ce480 0x7f2ed80d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:30.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.404+0000 7f2ed77fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2ed80ce480 0x7f2ed80d1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40874/0 (socket says 172.21.15.46:40874) 2023-12-02T00:30:30.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.404+0000 7f2ed77fe640 1 -- 172.21.15.46:0/1850956371 learned_addr learned my addr 172.21.15.46:0/1850956371 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:30.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.404+0000 7f2ed67fc640 1 --2- 172.21.15.46:0/1850956371 >> v2:172.21.15.162:3300/0 conn(0x7f2ed8160f70 0x7f2ed80cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:30.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.404+0000 7f2ed77fe640 1 -- 172.21.15.46:0/1850956371 >> v2:172.21.15.162:3300/0 conn(0x7f2ed8160f70 msgr2=0x7f2ed80cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:30.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.404+0000 7f2ed77fe640 1 --2- 172.21.15.46:0/1850956371 >> v2:172.21.15.162:3300/0 conn(0x7f2ed8160f70 0x7f2ed80cdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:30.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.404+0000 7f2ed77fe640 1 -- 172.21.15.46:0/1850956371 >> v1:172.21.15.162:6789/0 conn(0x7f2ed815d2d0 legacy=0x7f2ed80b8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:30.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.404+0000 7f2ed77fe640 1 -- 172.21.15.46:0/1850956371 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed8161d80 con 0x7f2ed80ce480 2023-12-02T00:30:30.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.405+0000 7f2ed77fe640 1 --2- 172.21.15.46:0/1850956371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ed80ce480 0x7f2ed80d1af0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f2ed80cdb80 tx=0x7f2ed002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69182f94415830d3 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:30.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.405+0000 7f2ed5ffb640 1 -- 172.21.15.46:0/1850956371 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed002ca80 con 0x7f2ed80ce480 2023-12-02T00:30:30.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.405+0000 7f2ed5ffb640 1 -- 172.21.15.46:0/1850956371 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ed002cbe0 con 0x7f2ed80ce480 2023-12-02T00:30:30.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.405+0000 7f2ed5ffb640 1 -- 172.21.15.46:0/1850956371 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed00343f0 con 0x7f2ed80ce480 2023-12-02T00:30:30.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.405+0000 7f2edd4d4640 1 -- 172.21.15.46:0/1850956371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ed80ce480 msgr2=0x7f2ed80d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:30.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.405+0000 7f2edd4d4640 1 --2- 172.21.15.46:0/1850956371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ed80ce480 0x7f2ed80d1af0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f2ed80cdb80 tx=0x7f2ed002bee0 comp rx=0 tx=0).stop 2023-12-02T00:30:30.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.405+0000 7f2edd4d4640 1 -- 172.21.15.46:0/1850956371 shutdown_connections 2023-12-02T00:30:30.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.405+0000 7f2edd4d4640 1 --2- 172.21.15.46:0/1850956371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ed80ce480 0x7f2ed80d1af0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:30.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.405+0000 7f2edd4d4640 1 --2- 172.21.15.46:0/1850956371 >> v2:172.21.15.162:3300/0 conn(0x7f2ed8160f70 0x7f2ed80cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:30.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.405+0000 7f2edd4d4640 1 -- 172.21.15.46:0/1850956371 >> 172.21.15.46:0/1850956371 conn(0x7f2ed8072230 msgr2=0x7f2ed80cfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:30.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.406+0000 7f2edd4d4640 1 -- 172.21.15.46:0/1850956371 shutdown_connections 2023-12-02T00:30:30.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.406+0000 7f2edd4d4640 1 -- 172.21.15.46:0/1850956371 wait complete. 2023-12-02T00:30:30.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.407+0000 7f2edd4d4640 1 Processor -- start 2023-12-02T00:30:30.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.407+0000 7f2edd4d4640 1 -- start start 2023-12-02T00:30:30.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.408+0000 7f2edd4d4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ed80ce480 0x7f2ed80c94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:30.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.408+0000 7f2edd4d4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ed815d2d0 0x7f2ed80c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:30.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.408+0000 7f2edd4d4640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ed8160f70 0x7f2ed80d31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:30.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.408+0000 7f2edd4d4640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed80cb480 con 0x7f2ed815d2d0 2023-12-02T00:30:30.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.408+0000 7f2edd4d4640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed80ca800 con 0x7f2ed8160f70 2023-12-02T00:30:30.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.408+0000 7f2edd4d4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed80caae0 con 0x7f2ed80ce480 2023-12-02T00:30:30.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.408+0000 7f2ed67fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ed815d2d0 0x7f2ed80c5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:30.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.408+0000 7f2ed67fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ed815d2d0 0x7f2ed80c5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40878/0 (socket says 172.21.15.46:40878) 2023-12-02T00:30:30.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.408+0000 7f2ed67fc640 1 -- 172.21.15.46:0/3890356664 learned_addr learned my addr 172.21.15.46:0/3890356664 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:30.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.408+0000 7f2ed77fe640 1 --2- 172.21.15.46:0/3890356664 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ed8160f70 0x7f2ed80d31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:30.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.408+0000 7f2ed6ffd640 1 --2- 172.21.15.46:0/3890356664 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ed80ce480 0x7f2ed80c94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:30.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.408+0000 7f2ed67fc640 1 -- 172.21.15.46:0/3890356664 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ed80ce480 msgr2=0x7f2ed80c94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:30.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.408+0000 7f2ed67fc640 1 --2- 172.21.15.46:0/3890356664 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ed80ce480 0x7f2ed80c94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:30.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.408+0000 7f2ed67fc640 1 -- 172.21.15.46:0/3890356664 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ed8160f70 msgr2=0x7f2ed80d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:30.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.408+0000 7f2ed67fc640 1 --2- 172.21.15.46:0/3890356664 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ed8160f70 0x7f2ed80d31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:30.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.408+0000 7f2ed67fc640 1 -- 172.21.15.46:0/3890356664 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed0009b30 con 0x7f2ed815d2d0 2023-12-02T00:30:30.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.408+0000 7f2ed77fe640 1 --2- 172.21.15.46:0/3890356664 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ed8160f70 0x7f2ed80d31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:30:30.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.408+0000 7f2ed67fc640 1 --2- 172.21.15.46:0/3890356664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ed815d2d0 0x7f2ed80c5ef0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc00d5f0 tx=0x7f2ecc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:30.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.408+0000 7f2ebf7fe640 1 -- 172.21.15.46:0/3890356664 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc0125f0 con 0x7f2ed815d2d0 2023-12-02T00:30:30.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.408+0000 7f2ebf7fe640 1 -- 172.21.15.46:0/3890356664 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ecc012750 con 0x7f2ed815d2d0 2023-12-02T00:30:30.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.408+0000 7f2ebf7fe640 1 -- 172.21.15.46:0/3890356664 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc012b00 con 0x7f2ed815d2d0 2023-12-02T00:30:30.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.408+0000 7f2edd4d4640 1 -- 172.21.15.46:0/3890356664 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ed80cadc0 con 0x7f2ed815d2d0 2023-12-02T00:30:30.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.408+0000 7f2edd4d4640 1 -- 172.21.15.46:0/3890356664 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ed8157340 con 0x7f2ed815d2d0 2023-12-02T00:30:30.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.409+0000 7f2edd4d4640 1 -- 172.21.15.46:0/3890356664 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ed80cb5c0 con 0x7f2ed815d2d0 2023-12-02T00:30:30.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.410+0000 7f2ebf7fe640 1 -- 172.21.15.46:0/3890356664 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2ecc012c60 con 0x7f2ed815d2d0 2023-12-02T00:30:30.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.410+0000 7f2ebf7fe640 1 --2- 172.21.15.46:0/3890356664 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2ec406ae40 0x7f2ec406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:30.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.410+0000 7f2ebf7fe640 1 -- 172.21.15.46:0/3890356664 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f2ecc089d60 con 0x7f2ed815d2d0 2023-12-02T00:30:30.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.410+0000 7f2ed6ffd640 1 --2- 172.21.15.46:0/3890356664 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2ec406ae40 0x7f2ec406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:30.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.411+0000 7f2ed6ffd640 1 --2- 172.21.15.46:0/3890356664 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2ec406ae40 0x7f2ec406d300 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8160ed0 tx=0x7f2ec8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:30.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.414+0000 7f2ebf7fe640 1 -- 172.21.15.46:0/3890356664 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2ed80cb5c0 con 0x7f2ed815d2d0 2023-12-02T00:30:30.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.594+0000 7f2edd4d4640 1 -- 172.21.15.46:0/3890356664 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7f2ed80cb5c0 con 0x7f2ed815d2d0 2023-12-02T00:30:30.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.594+0000 7f2ebf7fe640 1 -- 172.21.15.46:0/3890356664 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v38) v1 ==== 97+0+11 (secure 0 0 0) 0x7f2ed80cb5c0 con 0x7f2ed815d2d0 2023-12-02T00:30:30.596 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-12-02T00:30:30.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.596+0000 7f2edd4d4640 1 -- 172.21.15.46:0/3890356664 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2ec406ae40 msgr2=0x7f2ec406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:30.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.596+0000 7f2edd4d4640 1 --2- 172.21.15.46:0/3890356664 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2ec406ae40 0x7f2ec406d300 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8160ed0 tx=0x7f2ec8009040 comp rx=0 tx=0).stop 2023-12-02T00:30:30.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.596+0000 7f2edd4d4640 1 -- 172.21.15.46:0/3890356664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ed815d2d0 msgr2=0x7f2ed80c5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:30.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.596+0000 7f2edd4d4640 1 --2- 172.21.15.46:0/3890356664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ed815d2d0 0x7f2ed80c5ef0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc00d5f0 tx=0x7f2ecc00dab0 comp rx=0 tx=0).stop 2023-12-02T00:30:30.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.596+0000 7f2edd4d4640 1 -- 172.21.15.46:0/3890356664 shutdown_connections 2023-12-02T00:30:30.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.596+0000 7f2edd4d4640 1 --2- 172.21.15.46:0/3890356664 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2ec406ae40 0x7f2ec406d300 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:30.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.596+0000 7f2edd4d4640 1 --2- 172.21.15.46:0/3890356664 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ed8160f70 0x7f2ed80d31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:30.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.596+0000 7f2edd4d4640 1 --2- 172.21.15.46:0/3890356664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ed815d2d0 0x7f2ed80c5ef0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:30.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.596+0000 7f2edd4d4640 1 --2- 172.21.15.46:0/3890356664 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ed80ce480 0x7f2ed80c94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:30.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.596+0000 7f2edd4d4640 1 -- 172.21.15.46:0/3890356664 >> 172.21.15.46:0/3890356664 conn(0x7f2ed8072230 msgr2=0x7f2ed815c500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:30.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.596+0000 7f2edd4d4640 1 -- 172.21.15.46:0/3890356664 shutdown_connections 2023-12-02T00:30:30.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.597+0000 7f2edd4d4640 1 -- 172.21.15.46:0/3890356664 wait complete. 2023-12-02T00:30:30.609 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-02T00:30:30.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.720+0000 7f5e28133640 1 Processor -- start 2023-12-02T00:30:30.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.720+0000 7f5e28133640 1 -- start start 2023-12-02T00:30:30.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.721+0000 7f5e28133640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5e2006ea50 0x7f5e2006ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:30.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.721+0000 7f5e28133640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e20161d20 con 0x7f5e20160fb0 2023-12-02T00:30:30.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.721+0000 7f5e28133640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e2006f370 con 0x7f5e200d0ed0 2023-12-02T00:30:30.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.721+0000 7f5e28133640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e20161b20 con 0x7f5e2006ea50 2023-12-02T00:30:30.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.721+0000 7f5e256ce640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5e20160fb0 0x7f5e20161410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:44654/0 (socket says 172.21.15.46:44654) 2023-12-02T00:30:30.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.721+0000 7f5e256ce640 1 -- 172.21.15.46:0/629795382 learned_addr learned my addr 172.21.15.46:0/629795382 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:30:30.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.721+0000 7f5e25ecf640 1 --2- 172.21.15.46:0/629795382 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5e2006ea50 0x7f5e2006ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:30.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.721+0000 7f5e24ecd640 1 -- 172.21.15.46:0/629795382 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2537507570 0 0) 0x7f5e20161d20 con 0x7f5e20160fb0 2023-12-02T00:30:30.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.721+0000 7f5e24ecd640 1 -- 172.21.15.46:0/629795382 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e0c003520 con 0x7f5e20160fb0 2023-12-02T00:30:30.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.721+0000 7f5e24ecd640 1 -- 172.21.15.46:0/629795382 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2854801617 0 0) 0x7f5e2006f370 con 0x7f5e200d0ed0 2023-12-02T00:30:30.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.721+0000 7f5e24ecd640 1 -- 172.21.15.46:0/629795382 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e20161d20 con 0x7f5e200d0ed0 2023-12-02T00:30:30.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.721+0000 7f5e25ecf640 1 -- 172.21.15.46:0/629795382 >> v1:172.21.15.162:6789/0 conn(0x7f5e200d0ed0 legacy=0x7f5e200d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:30.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.721+0000 7f5e25ecf640 1 -- 172.21.15.46:0/629795382 >> v1:172.21.15.46:6789/0 conn(0x7f5e20160fb0 legacy=0x7f5e20161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:30.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.721+0000 7f5e24ecd640 1 -- 172.21.15.46:0/629795382 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2314373281 0 0) 0x7f5e0c003520 con 0x7f5e20160fb0 2023-12-02T00:30:30.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.721+0000 7f5e25ecf640 1 -- 172.21.15.46:0/629795382 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e2015e760 con 0x7f5e2006ea50 2023-12-02T00:30:30.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.722+0000 7f5e25ecf640 1 --2- 172.21.15.46:0/629795382 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5e2006ea50 0x7f5e2006ee30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c0097d0 tx=0x7f5e1c02b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86c1a91e8d87d416 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:30.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.722+0000 7f5e24ecd640 1 -- 172.21.15.46:0/629795382 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e10001670 con 0x7f5e2006ea50 2023-12-02T00:30:30.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.722+0000 7f5e24ecd640 1 -- 172.21.15.46:0/629795382 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e1c02e050 con 0x7f5e2006ea50 2023-12-02T00:30:30.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.722+0000 7f5e24ecd640 1 -- 172.21.15.46:0/629795382 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c030710 con 0x7f5e2006ea50 2023-12-02T00:30:30.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.723+0000 7f5e28133640 1 -- 172.21.15.46:0/629795382 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5e2006ea50 msgr2=0x7f5e2006ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:30.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.723+0000 7f5e28133640 1 --2- 172.21.15.46:0/629795382 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5e2006ea50 0x7f5e2006ee30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c0097d0 tx=0x7f5e1c02b800 comp rx=0 tx=0).stop 2023-12-02T00:30:30.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.723+0000 7f5e28133640 1 -- 172.21.15.46:0/629795382 shutdown_connections 2023-12-02T00:30:30.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.723+0000 7f5e28133640 1 --2- 172.21.15.46:0/629795382 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5e2006ea50 0x7f5e2006ee30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:30.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.723+0000 7f5e28133640 1 -- 172.21.15.46:0/629795382 >> 172.21.15.46:0/629795382 conn(0x7f5e2006a6b0 msgr2=0x7f5e2006aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:30.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.723+0000 7f5e28133640 1 -- 172.21.15.46:0/629795382 shutdown_connections 2023-12-02T00:30:30.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.723+0000 7f5e28133640 1 -- 172.21.15.46:0/629795382 wait complete. 2023-12-02T00:30:30.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.724+0000 7f5e28133640 1 Processor -- start 2023-12-02T00:30:30.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.724+0000 7f5e28133640 1 -- start start 2023-12-02T00:30:30.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.725+0000 7f5e28133640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e2006ea50 0x7f5e201f2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:30.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.725+0000 7f5e28133640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e200d0ed0 0x7f5e201596c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:30.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.725+0000 7f5e28133640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5e20160fb0 0x7f5e20159c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:30.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.725+0000 7f5e28133640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2015a7d0 con 0x7f5e200d0ed0 2023-12-02T00:30:30.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.725+0000 7f5e28133640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2015a210 con 0x7f5e2006ea50 2023-12-02T00:30:30.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.725+0000 7f5e28133640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2015a4f0 con 0x7f5e20160fb0 2023-12-02T00:30:30.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.725+0000 7f5e256ce640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e200d0ed0 0x7f5e201596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:30.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.725+0000 7f5e256ce640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e200d0ed0 0x7f5e201596c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40886/0 (socket says 172.21.15.46:40886) 2023-12-02T00:30:30.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.725+0000 7f5e256ce640 1 -- 172.21.15.46:0/523592546 learned_addr learned my addr 172.21.15.46:0/523592546 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:30.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.725+0000 7f5e25ecf640 1 --2- 172.21.15.46:0/523592546 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e2006ea50 0x7f5e201f2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:30.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.725+0000 7f5e266d0640 1 --2- 172.21.15.46:0/523592546 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5e20160fb0 0x7f5e20159c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:30.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.725+0000 7f5e256ce640 1 -- 172.21.15.46:0/523592546 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5e20160fb0 msgr2=0x7f5e20159c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:30.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.725+0000 7f5e256ce640 1 --2- 172.21.15.46:0/523592546 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5e20160fb0 0x7f5e20159c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:30.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.725+0000 7f5e256ce640 1 -- 172.21.15.46:0/523592546 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e2006ea50 msgr2=0x7f5e201f2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:30.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.725+0000 7f5e256ce640 1 --2- 172.21.15.46:0/523592546 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e2006ea50 0x7f5e201f2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:30.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.725+0000 7f5e256ce640 1 -- 172.21.15.46:0/523592546 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e1c009400 con 0x7f5e200d0ed0 2023-12-02T00:30:30.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.725+0000 7f5e25ecf640 1 --2- 172.21.15.46:0/523592546 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e2006ea50 0x7f5e201f2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:30:30.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.725+0000 7f5e256ce640 1 --2- 172.21.15.46:0/523592546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e200d0ed0 0x7f5e201596c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f5e1000c750 tx=0x7f5e1000cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:30.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.726+0000 7f5e167fc640 1 -- 172.21.15.46:0/523592546 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e10005ad0 con 0x7f5e200d0ed0 2023-12-02T00:30:30.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.726+0000 7f5e167fc640 1 -- 172.21.15.46:0/523592546 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e10005c30 con 0x7f5e200d0ed0 2023-12-02T00:30:30.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.726+0000 7f5e167fc640 1 -- 172.21.15.46:0/523592546 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e10007070 con 0x7f5e200d0ed0 2023-12-02T00:30:30.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.726+0000 7f5e28133640 1 -- 172.21.15.46:0/523592546 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e2013d7e0 con 0x7f5e200d0ed0 2023-12-02T00:30:30.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.726+0000 7f5e28133640 1 -- 172.21.15.46:0/523592546 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e20158cf0 con 0x7f5e200d0ed0 2023-12-02T00:30:30.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.727+0000 7f5e28133640 1 -- 172.21.15.46:0/523592546 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e2013d640 con 0x7f5e200d0ed0 2023-12-02T00:30:30.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.727+0000 7f5e167fc640 1 -- 172.21.15.46:0/523592546 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5e100071d0 con 0x7f5e200d0ed0 2023-12-02T00:30:30.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.728+0000 7f5e167fc640 1 --2- 172.21.15.46:0/523592546 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5e0406ae40 0x7f5e0406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:30.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.728+0000 7f5e167fc640 1 -- 172.21.15.46:0/523592546 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f5e10089500 con 0x7f5e200d0ed0 2023-12-02T00:30:30.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.728+0000 7f5e25ecf640 1 --2- 172.21.15.46:0/523592546 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5e0406ae40 0x7f5e0406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:30.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.728+0000 7f5e25ecf640 1 --2- 172.21.15.46:0/523592546 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5e0406ae40 0x7f5e0406d300 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c034000 tx=0x7f5e1c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:30.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.732+0000 7f5e167fc640 1 -- 172.21.15.46:0/523592546 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5e2013d640 con 0x7f5e200d0ed0 2023-12-02T00:30:30.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.922+0000 7f5e28133640 1 -- 172.21.15.46:0/523592546 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5e2013dd00 con 0x7f5e200d0ed0 2023-12-02T00:30:30.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.922+0000 7f5e167fc640 1 -- 172.21.15.46:0/523592546 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v38) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5e2013dd00 con 0x7f5e200d0ed0 2023-12-02T00:30:30.923 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2023-12-02T00:30:30.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.924+0000 7f5e28133640 1 -- 172.21.15.46:0/523592546 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5e0406ae40 msgr2=0x7f5e0406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:30.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.924+0000 7f5e28133640 1 --2- 172.21.15.46:0/523592546 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5e0406ae40 0x7f5e0406d300 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c034000 tx=0x7f5e1c008030 comp rx=0 tx=0).stop 2023-12-02T00:30:30.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.924+0000 7f5e28133640 1 -- 172.21.15.46:0/523592546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e200d0ed0 msgr2=0x7f5e201596c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:30.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.924+0000 7f5e28133640 1 --2- 172.21.15.46:0/523592546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e200d0ed0 0x7f5e201596c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f5e1000c750 tx=0x7f5e1000cc10 comp rx=0 tx=0).stop 2023-12-02T00:30:30.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.924+0000 7f5e28133640 1 -- 172.21.15.46:0/523592546 shutdown_connections 2023-12-02T00:30:30.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.924+0000 7f5e28133640 1 --2- 172.21.15.46:0/523592546 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5e0406ae40 0x7f5e0406d300 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:30.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.924+0000 7f5e28133640 1 --2- 172.21.15.46:0/523592546 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5e20160fb0 0x7f5e20159c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:30.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.924+0000 7f5e28133640 1 --2- 172.21.15.46:0/523592546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e200d0ed0 0x7f5e201596c0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:30.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.924+0000 7f5e28133640 1 --2- 172.21.15.46:0/523592546 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e2006ea50 0x7f5e201f2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:30.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.924+0000 7f5e28133640 1 -- 172.21.15.46:0/523592546 >> 172.21.15.46:0/523592546 conn(0x7f5e2006a6b0 msgr2=0x7f5e200b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:30.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.924+0000 7f5e28133640 1 -- 172.21.15.46:0/523592546 shutdown_connections 2023-12-02T00:30:30.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:30.924+0000 7f5e28133640 1 -- 172.21.15.46:0/523592546 wait complete. 2023-12-02T00:30:30.936 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-02T00:30:30.936 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-02T00:30:30.952 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:30:31.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.099+0000 7fd4ece64640 1 Processor -- start 2023-12-02T00:30:31.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.099+0000 7fd4ece64640 1 -- start start 2023-12-02T00:30:31.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.099+0000 7fd4ece64640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4e8160fb0 0x7fd4e8161440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:31.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.099+0000 7fd4ece64640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd4e80d0ed0 0x7fd4e80d1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:31.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.099+0000 7fd4ece64640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4e8161d20 con 0x7fd4e806ea50 2023-12-02T00:30:31.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.099+0000 7fd4ece64640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4e815e760 con 0x7fd4e80d0ed0 2023-12-02T00:30:31.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.099+0000 7fd4ece64640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e815d9a0 con 0x7fd4e8160fb0 2023-12-02T00:30:31.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.100+0000 7fd4dffff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4e8160fb0 0x7fd4e8161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:31.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.100+0000 7fd4e6d9d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd4e80d0ed0 0x7fd4e80d1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:31.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.100+0000 7fd4e659c640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fd4e806ea50 0x7fd4e806ee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.46:56912/0 (socket says 172.21.15.46:56912) 2023-12-02T00:30:31.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.100+0000 7fd4e659c640 1 -- 172.21.15.46:0/1936830039 learned_addr learned my addr 172.21.15.46:0/1936830039 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:30:31.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.100+0000 7fd4e5d9b640 1 -- 172.21.15.46:0/1936830039 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1711458491 0 0) 0x7fd4e8161d20 con 0x7fd4e806ea50 2023-12-02T00:30:31.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.100+0000 7fd4e5d9b640 1 -- 172.21.15.46:0/1936830039 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4d4003520 con 0x7fd4e806ea50 2023-12-02T00:30:31.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.100+0000 7fd4e6d9d640 1 -- 172.21.15.46:0/1936830039 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4e8160fb0 msgr2=0x7fd4e8161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:31.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.100+0000 7fd4e6d9d640 1 --2- 172.21.15.46:0/1936830039 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4e8160fb0 0x7fd4e8161440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:31.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.100+0000 7fd4e6d9d640 1 -- 172.21.15.46:0/1936830039 >> v1:172.21.15.162:6789/0 conn(0x7fd4e806ea50 legacy=0x7fd4e806ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:31.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.100+0000 7fd4e6d9d640 1 -- 172.21.15.46:0/1936830039 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e815dc80 con 0x7fd4e80d0ed0 2023-12-02T00:30:31.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.100+0000 7fd4dffff640 1 --2- 172.21.15.46:0/1936830039 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4e8160fb0 0x7fd4e8161440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:30:31.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.101+0000 7fd4e6d9d640 1 --2- 172.21.15.46:0/1936830039 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4e80d0ed0 0x7fd4e80d1360 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0009d90 tx=0x7fd4e002bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=628faa3c01a596b0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:31.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.101+0000 7fd4e5d9b640 1 -- 172.21.15.46:0/1936830039 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e002cc50 con 0x7fd4e80d0ed0 2023-12-02T00:30:31.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.101+0000 7fd4e5d9b640 1 -- 172.21.15.46:0/1936830039 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e002cdb0 con 0x7fd4e80d0ed0 2023-12-02T00:30:31.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.101+0000 7fd4e5d9b640 1 -- 172.21.15.46:0/1936830039 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e0034660 con 0x7fd4e80d0ed0 2023-12-02T00:30:31.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.102+0000 7fd4ece64640 1 -- 172.21.15.46:0/1936830039 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4e80d0ed0 msgr2=0x7fd4e80d1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:31.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.102+0000 7fd4ece64640 1 --2- 172.21.15.46:0/1936830039 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4e80d0ed0 0x7fd4e80d1360 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0009d90 tx=0x7fd4e002bea0 comp rx=0 tx=0).stop 2023-12-02T00:30:31.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.102+0000 7fd4ece64640 1 -- 172.21.15.46:0/1936830039 shutdown_connections 2023-12-02T00:30:31.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.102+0000 7fd4ece64640 1 --2- 172.21.15.46:0/1936830039 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4e8160fb0 0x7fd4e8161440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:31.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.102+0000 7fd4ece64640 1 --2- 172.21.15.46:0/1936830039 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4e80d0ed0 0x7fd4e80d1360 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:31.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.102+0000 7fd4ece64640 1 -- 172.21.15.46:0/1936830039 >> 172.21.15.46:0/1936830039 conn(0x7fd4e806a6b0 msgr2=0x7fd4e806aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:31.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.102+0000 7fd4ece64640 1 -- 172.21.15.46:0/1936830039 shutdown_connections 2023-12-02T00:30:31.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.102+0000 7fd4ece64640 1 -- 172.21.15.46:0/1936830039 wait complete. 2023-12-02T00:30:31.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.103+0000 7fd4ece64640 1 Processor -- start 2023-12-02T00:30:31.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.103+0000 7fd4ece64640 1 -- start start 2023-12-02T00:30:31.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.104+0000 7fd4ece64640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4e806ea50 0x7fd4e806c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:31.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.104+0000 7fd4ece64640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4e80d0ed0 0x7fd4e8157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:31.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.104+0000 7fd4ece64640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4e8160fb0 0x7fd4e815a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:31.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.104+0000 7fd4ece64640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e8159db0 con 0x7fd4e8160fb0 2023-12-02T00:30:31.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.104+0000 7fd4ece64640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e8158df0 con 0x7fd4e80d0ed0 2023-12-02T00:30:31.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.104+0000 7fd4ece64640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e8158830 con 0x7fd4e806ea50 2023-12-02T00:30:31.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.104+0000 7fd4e6d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4e8160fb0 0x7fd4e815a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:31.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.104+0000 7fd4e6d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4e8160fb0 0x7fd4e815a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40898/0 (socket says 172.21.15.46:40898) 2023-12-02T00:30:31.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.104+0000 7fd4e6d9d640 1 -- 172.21.15.46:0/213804662 learned_addr learned my addr 172.21.15.46:0/213804662 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:31.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.104+0000 7fd4dffff640 1 --2- 172.21.15.46:0/213804662 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4e80d0ed0 0x7fd4e8157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:31.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.104+0000 7fd4e659c640 1 --2- 172.21.15.46:0/213804662 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4e806ea50 0x7fd4e806c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:31.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.104+0000 7fd4e6d9d640 1 -- 172.21.15.46:0/213804662 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4e806ea50 msgr2=0x7fd4e806c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:31.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.104+0000 7fd4e6d9d640 1 --2- 172.21.15.46:0/213804662 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4e806ea50 0x7fd4e806c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:31.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.104+0000 7fd4e6d9d640 1 -- 172.21.15.46:0/213804662 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4e80d0ed0 msgr2=0x7fd4e8157ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:31.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.104+0000 7fd4e6d9d640 1 --2- 172.21.15.46:0/213804662 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4e80d0ed0 0x7fd4e8157ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:31.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.104+0000 7fd4e6d9d640 1 -- 172.21.15.46:0/213804662 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d0009400 con 0x7fd4e8160fb0 2023-12-02T00:30:31.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.104+0000 7fd4e659c640 1 --2- 172.21.15.46:0/213804662 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4e806ea50 0x7fd4e806c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:30:31.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.104+0000 7fd4dffff640 1 --2- 172.21.15.46:0/213804662 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4e80d0ed0 0x7fd4e8157ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:30:31.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.105+0000 7fd4e6d9d640 1 --2- 172.21.15.46:0/213804662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4e8160fb0 0x7fd4e815a240 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0009d90 tx=0x7fd4e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:31.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.105+0000 7fd4deffd640 1 -- 172.21.15.46:0/213804662 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e0040650 con 0x7fd4e8160fb0 2023-12-02T00:30:31.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.105+0000 7fd4deffd640 1 -- 172.21.15.46:0/213804662 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e00407b0 con 0x7fd4e8160fb0 2023-12-02T00:30:31.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.105+0000 7fd4ece64640 1 -- 172.21.15.46:0/213804662 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4e0009a70 con 0x7fd4e8160fb0 2023-12-02T00:30:31.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.105+0000 7fd4deffd640 1 -- 172.21.15.46:0/213804662 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e0040b60 con 0x7fd4e8160fb0 2023-12-02T00:30:31.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.105+0000 7fd4ece64640 1 -- 172.21.15.46:0/213804662 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4e8159a40 con 0x7fd4e8160fb0 2023-12-02T00:30:31.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.106+0000 7fd4ece64640 1 -- 172.21.15.46:0/213804662 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4e8159f20 con 0x7fd4e8160fb0 2023-12-02T00:30:31.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.106+0000 7fd4deffd640 1 -- 172.21.15.46:0/213804662 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd4e0040cc0 con 0x7fd4e8160fb0 2023-12-02T00:30:31.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.107+0000 7fd4deffd640 1 --2- 172.21.15.46:0/213804662 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd4c806ae40 0x7fd4c806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:31.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.107+0000 7fd4deffd640 1 -- 172.21.15.46:0/213804662 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fd4e00ac3b0 con 0x7fd4e8160fb0 2023-12-02T00:30:31.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.107+0000 7fd4e659c640 1 --2- 172.21.15.46:0/213804662 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd4c806ae40 0x7fd4c806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:31.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.107+0000 7fd4e659c640 1 --2- 172.21.15.46:0/213804662 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd4c806ae40 0x7fd4c806d300 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd4d80011e0 tx=0x7fd4d8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:31.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.111+0000 7fd4deffd640 1 -- 172.21.15.46:0/213804662 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd4e00772d0 con 0x7fd4e8160fb0 2023-12-02T00:30:31.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.289+0000 7fd4ece64640 1 -- 172.21.15.46:0/213804662 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd4e8069400 con 0x7fd4e8160fb0 2023-12-02T00:30:31.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.291+0000 7fd4deffd640 1 -- 172.21.15.46:0/213804662 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+18324 (secure 0 0 0) 0x7fd4e0009450 con 0x7fd4e8160fb0 2023-12-02T00:30:31.292 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:30:31.292 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":38,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:30:10.714057+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-02T00:29:55.676572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-02T00:29:56.244869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"data_new","create_time":"2023-12-02T00:30:08.897990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:30:31.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.293+0000 7fd4ece64640 1 -- 172.21.15.46:0/213804662 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd4c806ae40 msgr2=0x7fd4c806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:31.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.293+0000 7fd4ece64640 1 --2- 172.21.15.46:0/213804662 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd4c806ae40 0x7fd4c806d300 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd4d80011e0 tx=0x7fd4d8009040 comp rx=0 tx=0).stop 2023-12-02T00:30:31.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.293+0000 7fd4ece64640 1 -- 172.21.15.46:0/213804662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4e8160fb0 msgr2=0x7fd4e815a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:31.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.293+0000 7fd4ece64640 1 --2- 172.21.15.46:0/213804662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4e8160fb0 0x7fd4e815a240 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0009d90 tx=0x7fd4e0004a40 comp rx=0 tx=0).stop 2023-12-02T00:30:31.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.293+0000 7fd4ece64640 1 -- 172.21.15.46:0/213804662 shutdown_connections 2023-12-02T00:30:31.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.293+0000 7fd4ece64640 1 --2- 172.21.15.46:0/213804662 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd4c806ae40 0x7fd4c806d300 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:31.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.293+0000 7fd4ece64640 1 --2- 172.21.15.46:0/213804662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4e8160fb0 0x7fd4e815a240 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:31.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.293+0000 7fd4ece64640 1 --2- 172.21.15.46:0/213804662 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4e80d0ed0 0x7fd4e8157ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:31.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.293+0000 7fd4ece64640 1 --2- 172.21.15.46:0/213804662 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4e806ea50 0x7fd4e806c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:31.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.293+0000 7fd4ece64640 1 -- 172.21.15.46:0/213804662 >> 172.21.15.46:0/213804662 conn(0x7fd4e806a6b0 msgr2=0x7fd4e806aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:31.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.293+0000 7fd4ece64640 1 -- 172.21.15.46:0/213804662 shutdown_connections 2023-12-02T00:30:31.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.293+0000 7fd4ece64640 1 -- 172.21.15.46:0/213804662 wait complete. 2023-12-02T00:30:31.305 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-02T00:28:37.213645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-02T00:29:55.676572+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-12-02T00:29:56.244869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'data_new', 'create_time': '2023-12-02T00:30:08.897990+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-02T00:30:31.305 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-02T00:30:31.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.413+0000 7f3f707de640 1 Processor -- start 2023-12-02T00:30:31.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.413+0000 7f3f707de640 1 -- start start 2023-12-02T00:30:31.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.414+0000 7f3f707de640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3f6815d330 0x7f3f680b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:31.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.414+0000 7f3f707de640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3f680d2d90 0x7f3f680b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:31.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.414+0000 7f3f707de640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f680b2b10 con 0x7f3f680d2910 2023-12-02T00:30:31.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.414+0000 7f3f707de640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f680d1590 con 0x7f3f680d2d90 2023-12-02T00:30:31.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.414+0000 7f3f707de640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f680d1870 con 0x7f3f6815d330 2023-12-02T00:30:31.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.414+0000 7f3f6ed7b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3f680d2d90 0x7f3f680b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:31.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.414+0000 7f3f6ed7b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3f680d2d90 0x7f3f680b6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40900/0 (socket says 172.21.15.46:40900) 2023-12-02T00:30:31.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.414+0000 7f3f6ed7b640 1 -- 172.21.15.46:0/1349534979 learned_addr learned my addr 172.21.15.46:0/1349534979 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:31.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.415+0000 7f3f6e57a640 1 --2- 172.21.15.46:0/1349534979 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3f6815d330 0x7f3f680b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:31.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.415+0000 7f3f6ed7b640 1 -- 172.21.15.46:0/1349534979 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3f6815d330 msgr2=0x7f3f680b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:31.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.415+0000 7f3f6ed7b640 1 --2- 172.21.15.46:0/1349534979 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3f6815d330 0x7f3f680b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:31.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.415+0000 7f3f6ed7b640 1 -- 172.21.15.46:0/1349534979 >> v1:172.21.15.162:6789/0 conn(0x7f3f680d2910 legacy=0x7f3f680d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:31.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.415+0000 7f3f6ed7b640 1 -- 172.21.15.46:0/1349534979 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f680d1b50 con 0x7f3f680d2d90 2023-12-02T00:30:31.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.415+0000 7f3f6ed7b640 1 --2- 172.21.15.46:0/1349534979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f680d2d90 0x7f3f680b6390 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f3f60009870 tx=0x7f3f6002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b6fd705c95ca7f1 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:31.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.416+0000 7f3f6d578640 1 -- 172.21.15.46:0/1349534979 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f60031640 con 0x7f3f680d2d90 2023-12-02T00:30:31.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.416+0000 7f3f6d578640 1 -- 172.21.15.46:0/1349534979 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f600317a0 con 0x7f3f680d2d90 2023-12-02T00:30:31.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.416+0000 7f3f6d578640 1 -- 172.21.15.46:0/1349534979 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f60031b30 con 0x7f3f680d2d90 2023-12-02T00:30:31.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.416+0000 7f3f707de640 1 -- 172.21.15.46:0/1349534979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f680d2d90 msgr2=0x7f3f680b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:31.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.416+0000 7f3f707de640 1 --2- 172.21.15.46:0/1349534979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f680d2d90 0x7f3f680b6390 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f3f60009870 tx=0x7f3f6002b8f0 comp rx=0 tx=0).stop 2023-12-02T00:30:31.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.416+0000 7f3f707de640 1 -- 172.21.15.46:0/1349534979 shutdown_connections 2023-12-02T00:30:31.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.416+0000 7f3f707de640 1 --2- 172.21.15.46:0/1349534979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f680d2d90 0x7f3f680b6390 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:31.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.416+0000 7f3f707de640 1 --2- 172.21.15.46:0/1349534979 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3f6815d330 0x7f3f680b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:31.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.416+0000 7f3f707de640 1 -- 172.21.15.46:0/1349534979 >> 172.21.15.46:0/1349534979 conn(0x7f3f68072020 msgr2=0x7f3f68072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:31.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.416+0000 7f3f707de640 1 -- 172.21.15.46:0/1349534979 shutdown_connections 2023-12-02T00:30:31.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.416+0000 7f3f707de640 1 -- 172.21.15.46:0/1349534979 wait complete. 2023-12-02T00:30:31.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.417+0000 7f3f707de640 1 Processor -- start 2023-12-02T00:30:31.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.418+0000 7f3f707de640 1 -- start start 2023-12-02T00:30:31.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.418+0000 7f3f707de640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3f680d2910 0x7f3f68150d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:31.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.418+0000 7f3f707de640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f680d2d90 0x7f3f680c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:31.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.418+0000 7f3f707de640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f6815d330 0x7f3f6815fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:31.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.418+0000 7f3f707de640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6815eaf0 con 0x7f3f680d2d90 2023-12-02T00:30:31.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.418+0000 7f3f707de640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f68151260 con 0x7f3f6815d330 2023-12-02T00:30:31.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.418+0000 7f3f707de640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6815dee0 con 0x7f3f680d2910 2023-12-02T00:30:31.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.418+0000 7f3f6dd79640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f680d2d90 0x7f3f680c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:31.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.418+0000 7f3f6dd79640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f680d2d90 0x7f3f680c8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40906/0 (socket says 172.21.15.46:40906) 2023-12-02T00:30:31.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.418+0000 7f3f6dd79640 1 -- 172.21.15.46:0/3860946282 learned_addr learned my addr 172.21.15.46:0/3860946282 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:31.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.419+0000 7f3f6ed7b640 1 --2- 172.21.15.46:0/3860946282 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f6815d330 0x7f3f6815fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:31.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.419+0000 7f3f6e57a640 1 --2- 172.21.15.46:0/3860946282 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3f680d2910 0x7f3f68150d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:31.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.419+0000 7f3f6dd79640 1 -- 172.21.15.46:0/3860946282 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3f680d2910 msgr2=0x7f3f68150d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:31.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.419+0000 7f3f6dd79640 1 --2- 172.21.15.46:0/3860946282 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3f680d2910 0x7f3f68150d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:31.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.419+0000 7f3f6dd79640 1 -- 172.21.15.46:0/3860946282 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f6815d330 msgr2=0x7f3f6815fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:31.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.419+0000 7f3f6dd79640 1 --2- 172.21.15.46:0/3860946282 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f6815d330 0x7f3f6815fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:31.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.419+0000 7f3f6dd79640 1 -- 172.21.15.46:0/3860946282 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f600094d0 con 0x7f3f680d2d90 2023-12-02T00:30:31.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.419+0000 7f3f6ed7b640 1 --2- 172.21.15.46:0/3860946282 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f6815d330 0x7f3f6815fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:30:31.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.419+0000 7f3f6dd79640 1 --2- 172.21.15.46:0/3860946282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f680d2d90 0x7f3f680c8d90 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c00c400 tx=0x7f3f5c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:31.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.419+0000 7f3f66ffd640 1 -- 172.21.15.46:0/3860946282 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c0052d0 con 0x7f3f680d2d90 2023-12-02T00:30:31.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.419+0000 7f3f66ffd640 1 -- 172.21.15.46:0/3860946282 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f5c005430 con 0x7f3f680d2d90 2023-12-02T00:30:31.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.419+0000 7f3f66ffd640 1 -- 172.21.15.46:0/3860946282 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c003830 con 0x7f3f680d2d90 2023-12-02T00:30:31.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.419+0000 7f3f707de640 1 -- 172.21.15.46:0/3860946282 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f6815e1c0 con 0x7f3f680d2d90 2023-12-02T00:30:31.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.419+0000 7f3f707de640 1 -- 172.21.15.46:0/3860946282 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f6815e780 con 0x7f3f680d2d90 2023-12-02T00:30:31.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.420+0000 7f3f707de640 1 -- 172.21.15.46:0/3860946282 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f6815e940 con 0x7f3f680d2d90 2023-12-02T00:30:31.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.421+0000 7f3f66ffd640 1 -- 172.21.15.46:0/3860946282 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3f5c003990 con 0x7f3f680d2d90 2023-12-02T00:30:31.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.421+0000 7f3f66ffd640 1 --2- 172.21.15.46:0/3860946282 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3f5406ae40 0x7f3f5406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:31.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.421+0000 7f3f66ffd640 1 -- 172.21.15.46:0/3860946282 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f3f5c088bc0 con 0x7f3f680d2d90 2023-12-02T00:30:31.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.421+0000 7f3f6e57a640 1 --2- 172.21.15.46:0/3860946282 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3f5406ae40 0x7f3f5406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:31.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.422+0000 7f3f6e57a640 1 --2- 172.21.15.46:0/3860946282 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3f5406ae40 0x7f3f5406d300 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3f580113e0 tx=0x7f3f5800b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:31.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.425+0000 7f3f66ffd640 1 -- 172.21.15.46:0/3860946282 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3f5c053c20 con 0x7f3f680d2d90 2023-12-02T00:30:31.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.600+0000 7f3f707de640 1 -- 172.21.15.46:0/3860946282 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3f68069400 con 0x7f3f680d2d90 2023-12-02T00:30:31.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.600+0000 7f3f66ffd640 1 -- 172.21.15.46:0/3860946282 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3f5c0170a0 con 0x7f3f680d2d90 2023-12-02T00:30:31.601 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-12-02T00:30:31.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.602+0000 7f3f707de640 1 -- 172.21.15.46:0/3860946282 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3f5406ae40 msgr2=0x7f3f5406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:31.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.602+0000 7f3f707de640 1 --2- 172.21.15.46:0/3860946282 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3f5406ae40 0x7f3f5406d300 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3f580113e0 tx=0x7f3f5800b040 comp rx=0 tx=0).stop 2023-12-02T00:30:31.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.602+0000 7f3f707de640 1 -- 172.21.15.46:0/3860946282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f680d2d90 msgr2=0x7f3f680c8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:31.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.602+0000 7f3f707de640 1 --2- 172.21.15.46:0/3860946282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f680d2d90 0x7f3f680c8d90 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c00c400 tx=0x7f3f5c00c8c0 comp rx=0 tx=0).stop 2023-12-02T00:30:31.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.602+0000 7f3f707de640 1 -- 172.21.15.46:0/3860946282 shutdown_connections 2023-12-02T00:30:31.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.602+0000 7f3f707de640 1 --2- 172.21.15.46:0/3860946282 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3f5406ae40 0x7f3f5406d300 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:31.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.602+0000 7f3f707de640 1 --2- 172.21.15.46:0/3860946282 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f6815d330 0x7f3f6815fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:31.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.602+0000 7f3f707de640 1 --2- 172.21.15.46:0/3860946282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f680d2d90 0x7f3f680c8d90 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:31.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.602+0000 7f3f707de640 1 --2- 172.21.15.46:0/3860946282 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3f680d2910 0x7f3f68150d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:31.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.602+0000 7f3f707de640 1 -- 172.21.15.46:0/3860946282 >> 172.21.15.46:0/3860946282 conn(0x7f3f68072020 msgr2=0x7f3f680b2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:31.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.602+0000 7f3f707de640 1 -- 172.21.15.46:0/3860946282 shutdown_connections 2023-12-02T00:30:31.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.602+0000 7f3f707de640 1 -- 172.21.15.46:0/3860946282 wait complete. 2023-12-02T00:30:31.614 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-02T00:30:31.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.723+0000 7ff41cde5640 1 Processor -- start 2023-12-02T00:30:31.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.723+0000 7ff41cde5640 1 -- start start 2023-12-02T00:30:31.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.724+0000 7ff41cde5640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff418155790 0x7ff41806f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:31.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.724+0000 7ff41cde5640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff418155b70 con 0x7ff41806abe0 2023-12-02T00:30:31.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.724+0000 7ff41cde5640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff41806f3b0 con 0x7ff41815b5e0 2023-12-02T00:30:31.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.724+0000 7ff41cde5640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff41806ee30 con 0x7ff418155790 2023-12-02T00:30:31.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.724+0000 7ff416d9d640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff41806abe0 0x7ff4180714a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:44662/0 (socket says 172.21.15.46:44662) 2023-12-02T00:30:31.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.724+0000 7ff416d9d640 1 -- 172.21.15.46:0/3756149439 learned_addr learned my addr 172.21.15.46:0/3756149439 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:30:31.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.724+0000 7ff41659c640 1 --2- 172.21.15.46:0/3756149439 >> v2:172.21.15.162:3300/0 conn(0x7ff418155790 0x7ff41806f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:31.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.724+0000 7ff41559a640 1 -- 172.21.15.46:0/3756149439 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 708173921 0 0) 0x7ff418155b70 con 0x7ff41806abe0 2023-12-02T00:30:31.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.724+0000 7ff41559a640 1 -- 172.21.15.46:0/3756149439 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff400003520 con 0x7ff41806abe0 2023-12-02T00:30:31.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.725+0000 7ff41559a640 1 -- 172.21.15.46:0/3756149439 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3709738223 0 0) 0x7ff41806f3b0 con 0x7ff41815b5e0 2023-12-02T00:30:31.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.725+0000 7ff41559a640 1 -- 172.21.15.46:0/3756149439 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff418155b70 con 0x7ff41815b5e0 2023-12-02T00:30:31.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.725+0000 7ff41559a640 1 -- 172.21.15.46:0/3756149439 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3072359694 0 0) 0x7ff400003520 con 0x7ff41806abe0 2023-12-02T00:30:31.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.725+0000 7ff41659c640 1 -- 172.21.15.46:0/3756149439 >> v1:172.21.15.162:6789/0 conn(0x7ff41815b5e0 legacy=0x7ff41806a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:31.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.725+0000 7ff41659c640 1 -- 172.21.15.46:0/3756149439 >> v1:172.21.15.46:6789/0 conn(0x7ff41806abe0 legacy=0x7ff4180714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:31.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.725+0000 7ff41659c640 1 -- 172.21.15.46:0/3756149439 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff41815c170 con 0x7ff418155790 2023-12-02T00:30:31.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.725+0000 7ff41659c640 1 --2- 172.21.15.46:0/3756149439 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff418155790 0x7ff41806f840 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff4100097a0 tx=0x7ff41002b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fbf41368ba8611e server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:31.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.726+0000 7ff41559a640 1 -- 172.21.15.46:0/3756149439 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41002f050 con 0x7ff418155790 2023-12-02T00:30:31.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.726+0000 7ff41559a640 1 -- 172.21.15.46:0/3756149439 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff410031500 con 0x7ff418155790 2023-12-02T00:30:31.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.726+0000 7ff41559a640 1 -- 172.21.15.46:0/3756149439 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4080019f0 con 0x7ff418155790 2023-12-02T00:30:31.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.726+0000 7ff41cde5640 1 -- 172.21.15.46:0/3756149439 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff418155790 msgr2=0x7ff41806f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:31.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.726+0000 7ff41cde5640 1 --2- 172.21.15.46:0/3756149439 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff418155790 0x7ff41806f840 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff4100097a0 tx=0x7ff41002b820 comp rx=0 tx=0).stop 2023-12-02T00:30:31.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.726+0000 7ff41cde5640 1 -- 172.21.15.46:0/3756149439 shutdown_connections 2023-12-02T00:30:31.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.726+0000 7ff41cde5640 1 --2- 172.21.15.46:0/3756149439 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff418155790 0x7ff41806f840 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:31.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.726+0000 7ff41cde5640 1 -- 172.21.15.46:0/3756149439 >> 172.21.15.46:0/3756149439 conn(0x7ff4180b1e50 msgr2=0x7ff4180b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:31.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.726+0000 7ff41cde5640 1 -- 172.21.15.46:0/3756149439 shutdown_connections 2023-12-02T00:30:31.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.726+0000 7ff41cde5640 1 -- 172.21.15.46:0/3756149439 wait complete. 2023-12-02T00:30:31.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.728+0000 7ff41cde5640 1 Processor -- start 2023-12-02T00:30:31.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.728+0000 7ff41cde5640 1 -- start start 2023-12-02T00:30:31.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.728+0000 7ff41cde5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff41806abe0 0x7ff418158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:31.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.728+0000 7ff41cde5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff418155790 0x7ff418150f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:31.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.728+0000 7ff41cde5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff41815b5e0 0x7ff4181f0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:31.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.728+0000 7ff41cde5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff418152060 con 0x7ff41806abe0 2023-12-02T00:30:31.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.728+0000 7ff41cde5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff418158940 con 0x7ff41815b5e0 2023-12-02T00:30:31.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.728+0000 7ff41cde5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4181e3dd0 con 0x7ff418155790 2023-12-02T00:30:31.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.728+0000 7ff41659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff41806abe0 0x7ff418158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:31.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.729+0000 7ff41659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff41806abe0 0x7ff418158400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40920/0 (socket says 172.21.15.46:40920) 2023-12-02T00:30:31.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.729+0000 7ff41659c640 1 -- 172.21.15.46:0/2199970 learned_addr learned my addr 172.21.15.46:0/2199970 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:31.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.729+0000 7ff416d9d640 1 --2- 172.21.15.46:0/2199970 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff41815b5e0 0x7ff4181f0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:31.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.729+0000 7ff415d9b640 1 --2- 172.21.15.46:0/2199970 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff418155790 0x7ff418150f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:31.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.729+0000 7ff41659c640 1 -- 172.21.15.46:0/2199970 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff418155790 msgr2=0x7ff418150f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:31.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.729+0000 7ff41659c640 1 --2- 172.21.15.46:0/2199970 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff418155790 0x7ff418150f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:31.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.729+0000 7ff41659c640 1 -- 172.21.15.46:0/2199970 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff41815b5e0 msgr2=0x7ff4181f0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:31.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.729+0000 7ff41659c640 1 --2- 172.21.15.46:0/2199970 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff41815b5e0 0x7ff4181f0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:31.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.729+0000 7ff41659c640 1 -- 172.21.15.46:0/2199970 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff410009400 con 0x7ff41806abe0 2023-12-02T00:30:31.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.729+0000 7ff415d9b640 1 --2- 172.21.15.46:0/2199970 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff418155790 0x7ff418150f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:30:31.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.729+0000 7ff416d9d640 1 --2- 172.21.15.46:0/2199970 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff41815b5e0 0x7ff4181f0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:30:31.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.729+0000 7ff41659c640 1 --2- 172.21.15.46:0/2199970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff41806abe0 0x7ff418158400 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff410000c00 tx=0x7ff4100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:31.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.729+0000 7ff406ffd640 1 -- 172.21.15.46:0/2199970 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff410033880 con 0x7ff41806abe0 2023-12-02T00:30:31.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.729+0000 7ff406ffd640 1 -- 172.21.15.46:0/2199970 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff410033a00 con 0x7ff41806abe0 2023-12-02T00:30:31.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.729+0000 7ff41cde5640 1 -- 172.21.15.46:0/2199970 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4181e40b0 con 0x7ff41806abe0 2023-12-02T00:30:31.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.729+0000 7ff406ffd640 1 -- 172.21.15.46:0/2199970 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff410033db0 con 0x7ff41806abe0 2023-12-02T00:30:31.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.729+0000 7ff41cde5640 1 -- 172.21.15.46:0/2199970 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4181e4670 con 0x7ff41806abe0 2023-12-02T00:30:31.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.730+0000 7ff41cde5640 1 -- 172.21.15.46:0/2199970 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff418069400 con 0x7ff41806abe0 2023-12-02T00:30:31.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.731+0000 7ff406ffd640 1 -- 172.21.15.46:0/2199970 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff41003a050 con 0x7ff41806abe0 2023-12-02T00:30:31.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.731+0000 7ff406ffd640 1 --2- 172.21.15.46:0/2199970 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff3f806ae40 0x7ff3f806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:31.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.731+0000 7ff406ffd640 1 -- 172.21.15.46:0/2199970 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7ff4100ab260 con 0x7ff41806abe0 2023-12-02T00:30:31.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.732+0000 7ff415d9b640 1 --2- 172.21.15.46:0/2199970 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff3f806ae40 0x7ff3f806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:31.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.732+0000 7ff415d9b640 1 --2- 172.21.15.46:0/2199970 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff3f806ae40 0x7ff3f806d300 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff418158b70 tx=0x7ff40c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:31.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.736+0000 7ff406ffd640 1 -- 172.21.15.46:0/2199970 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff410076210 con 0x7ff41806abe0 2023-12-02T00:30:31.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.913+0000 7ff41cde5640 1 -- 172.21.15.46:0/2199970 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff4180b2c50 con 0x7ff41806abe0 2023-12-02T00:30:31.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.913+0000 7ff406ffd640 1 -- 172.21.15.46:0/2199970 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v38) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff410079830 con 0x7ff41806abe0 2023-12-02T00:30:31.914 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-12-02T00:30:31.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.915+0000 7ff41cde5640 1 -- 172.21.15.46:0/2199970 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff3f806ae40 msgr2=0x7ff3f806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:31.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.915+0000 7ff41cde5640 1 --2- 172.21.15.46:0/2199970 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff3f806ae40 0x7ff3f806d300 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff418158b70 tx=0x7ff40c009040 comp rx=0 tx=0).stop 2023-12-02T00:30:31.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.915+0000 7ff41cde5640 1 -- 172.21.15.46:0/2199970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff41806abe0 msgr2=0x7ff418158400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:31.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.915+0000 7ff41cde5640 1 --2- 172.21.15.46:0/2199970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff41806abe0 0x7ff418158400 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff410000c00 tx=0x7ff4100046c0 comp rx=0 tx=0).stop 2023-12-02T00:30:31.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.915+0000 7ff41cde5640 1 -- 172.21.15.46:0/2199970 shutdown_connections 2023-12-02T00:30:31.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.915+0000 7ff41cde5640 1 --2- 172.21.15.46:0/2199970 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff3f806ae40 0x7ff3f806d300 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:31.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.915+0000 7ff41cde5640 1 --2- 172.21.15.46:0/2199970 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff41815b5e0 0x7ff4181f0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:31.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.915+0000 7ff41cde5640 1 --2- 172.21.15.46:0/2199970 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff418155790 0x7ff418150f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:31.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.915+0000 7ff41cde5640 1 --2- 172.21.15.46:0/2199970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff41806abe0 0x7ff418158400 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:31.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.915+0000 7ff41cde5640 1 -- 172.21.15.46:0/2199970 >> 172.21.15.46:0/2199970 conn(0x7ff4180b1e50 msgr2=0x7ff41815e000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:31.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.915+0000 7ff41cde5640 1 -- 172.21.15.46:0/2199970 shutdown_connections 2023-12-02T00:30:31.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:31.915+0000 7ff41cde5640 1 -- 172.21.15.46:0/2199970 wait complete. 2023-12-02T00:30:31.927 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-02T00:30:32.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.035+0000 7f1f30583640 1 Processor -- start 2023-12-02T00:30:32.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.035+0000 7f1f30583640 1 -- start start 2023-12-02T00:30:32.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.036+0000 7f1f30583640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1f280709c0 0x7f1f280cd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:32.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.036+0000 7f1f30583640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f2806d580 con 0x7f1f280cdd10 2023-12-02T00:30:32.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.036+0000 7f1f30583640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f280d0c40 con 0x7f1f2815d270 2023-12-02T00:30:32.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.036+0000 7f1f30583640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f280ce1a0 con 0x7f1f280709c0 2023-12-02T00:30:32.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.036+0000 7f1f2db1e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1f280709c0 0x7f1f280cd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:32.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.036+0000 7f1f2db1e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1f280709c0 0x7f1f280cd7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40936/0 (socket says 172.21.15.46:40936) 2023-12-02T00:30:32.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.036+0000 7f1f2eb20640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1f280cdd10 0x7f1f280cc6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:44676/0 (socket says 172.21.15.46:44676) 2023-12-02T00:30:32.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.036+0000 7f1f2db1e640 1 -- 172.21.15.46:0/2762819670 learned_addr learned my addr 172.21.15.46:0/2762819670 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:32.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.037+0000 7f1f2db1e640 1 -- 172.21.15.46:0/2762819670 >> v1:172.21.15.162:6789/0 conn(0x7f1f2815d270 legacy=0x7f1f2806ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:32.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.037+0000 7f1f2db1e640 1 -- 172.21.15.46:0/2762819670 >> v1:172.21.15.46:6789/0 conn(0x7f1f280cdd10 legacy=0x7f1f280cc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:32.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.037+0000 7f1f2d31d640 1 -- 172.21.15.46:0/2762819670 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1260959919 0 0) 0x7f1f2806d580 con 0x7f1f280cdd10 2023-12-02T00:30:32.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.037+0000 7f1f2db1e640 1 -- 172.21.15.46:0/2762819670 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f280ced90 con 0x7f1f280709c0 2023-12-02T00:30:32.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.038+0000 7f1f2db1e640 1 --2- 172.21.15.46:0/2762819670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f280709c0 0x7f1f280cd7d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c009870 tx=0x7f1f1c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18a9a6af7156582e server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:32.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.038+0000 7f1f2d31d640 1 -- 172.21.15.46:0/2762819670 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c030860 con 0x7f1f280709c0 2023-12-02T00:30:32.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.038+0000 7f1f2d31d640 1 -- 172.21.15.46:0/2762819670 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f1c0309c0 con 0x7f1f280709c0 2023-12-02T00:30:32.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.038+0000 7f1f2d31d640 1 -- 172.21.15.46:0/2762819670 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c030d50 con 0x7f1f280709c0 2023-12-02T00:30:32.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.038+0000 7f1f30583640 1 -- 172.21.15.46:0/2762819670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f280709c0 msgr2=0x7f1f280cd7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:32.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.038+0000 7f1f30583640 1 --2- 172.21.15.46:0/2762819670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f280709c0 0x7f1f280cd7d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c009870 tx=0x7f1f1c02b9a0 comp rx=0 tx=0).stop 2023-12-02T00:30:32.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.038+0000 7f1f30583640 1 -- 172.21.15.46:0/2762819670 shutdown_connections 2023-12-02T00:30:32.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.038+0000 7f1f30583640 1 --2- 172.21.15.46:0/2762819670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f280709c0 0x7f1f280cd7d0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:32.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.038+0000 7f1f30583640 1 -- 172.21.15.46:0/2762819670 >> 172.21.15.46:0/2762819670 conn(0x7f1f280b2c10 msgr2=0x7f1f280b2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:32.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.038+0000 7f1f30583640 1 -- 172.21.15.46:0/2762819670 shutdown_connections 2023-12-02T00:30:32.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.038+0000 7f1f30583640 1 -- 172.21.15.46:0/2762819670 wait complete. 2023-12-02T00:30:32.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.039+0000 7f1f30583640 1 Processor -- start 2023-12-02T00:30:32.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.040+0000 7f1f30583640 1 -- start start 2023-12-02T00:30:32.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.040+0000 7f1f30583640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f280709c0 0x7f1f2815d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:32.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.040+0000 7f1f30583640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1f280cdd10 0x7f1f280ca260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:32.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.040+0000 7f1f30583640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f2815d270 0x7f1f2814de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:32.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.040+0000 7f1f30583640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f280c9400 con 0x7f1f280709c0 2023-12-02T00:30:32.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.040+0000 7f1f30583640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2815dce0 con 0x7f1f2815d270 2023-12-02T00:30:32.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.040+0000 7f1f30583640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2815dfc0 con 0x7f1f280cdd10 2023-12-02T00:30:32.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.040+0000 7f1f2e31f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f280709c0 0x7f1f2815d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:32.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.040+0000 7f1f2e31f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f280709c0 0x7f1f2815d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40952/0 (socket says 172.21.15.46:40952) 2023-12-02T00:30:32.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.040+0000 7f1f2e31f640 1 -- 172.21.15.46:0/3399564420 learned_addr learned my addr 172.21.15.46:0/3399564420 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:32.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.040+0000 7f1f2db1e640 1 --2- 172.21.15.46:0/3399564420 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1f280cdd10 0x7f1f280ca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:32.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.040+0000 7f1f2eb20640 1 --2- 172.21.15.46:0/3399564420 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f2815d270 0x7f1f2814de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:32.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.041+0000 7f1f2e31f640 1 -- 172.21.15.46:0/3399564420 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1f280cdd10 msgr2=0x7f1f280ca260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:32.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.041+0000 7f1f2e31f640 1 --2- 172.21.15.46:0/3399564420 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1f280cdd10 0x7f1f280ca260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:32.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.041+0000 7f1f2e31f640 1 -- 172.21.15.46:0/3399564420 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f2815d270 msgr2=0x7f1f2814de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:32.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.041+0000 7f1f2e31f640 1 --2- 172.21.15.46:0/3399564420 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f2815d270 0x7f1f2814de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:32.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.041+0000 7f1f2e31f640 1 -- 172.21.15.46:0/3399564420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f1c0094d0 con 0x7f1f280709c0 2023-12-02T00:30:32.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.041+0000 7f1f2e31f640 1 --2- 172.21.15.46:0/3399564420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f280709c0 0x7f1f2815d650 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f1f2000c450 tx=0x7f1f2000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:32.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.041+0000 7f1f26ffd640 1 -- 172.21.15.46:0/3399564420 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f200052d0 con 0x7f1f280709c0 2023-12-02T00:30:32.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.041+0000 7f1f26ffd640 1 -- 172.21.15.46:0/3399564420 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f20005430 con 0x7f1f280709c0 2023-12-02T00:30:32.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.041+0000 7f1f30583640 1 -- 172.21.15.46:0/3399564420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f2815e2a0 con 0x7f1f280709c0 2023-12-02T00:30:32.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.041+0000 7f1f26ffd640 1 -- 172.21.15.46:0/3399564420 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f20012870 con 0x7f1f280709c0 2023-12-02T00:30:32.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.041+0000 7f1f30583640 1 -- 172.21.15.46:0/3399564420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f2815e860 con 0x7f1f280709c0 2023-12-02T00:30:32.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.042+0000 7f1f30583640 1 -- 172.21.15.46:0/3399564420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f28067fc0 con 0x7f1f280709c0 2023-12-02T00:30:32.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.042+0000 7f1f26ffd640 1 -- 172.21.15.46:0/3399564420 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1f200129d0 con 0x7f1f280709c0 2023-12-02T00:30:32.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.043+0000 7f1f26ffd640 1 --2- 172.21.15.46:0/3399564420 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f1f1406ae40 0x7f1f1406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:32.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.043+0000 7f1f26ffd640 1 -- 172.21.15.46:0/3399564420 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f1f200892e0 con 0x7f1f280709c0 2023-12-02T00:30:32.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.043+0000 7f1f2db1e640 1 --2- 172.21.15.46:0/3399564420 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f1f1406ae40 0x7f1f1406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:32.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.044+0000 7f1f2db1e640 1 --2- 172.21.15.46:0/3399564420 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f1f1406ae40 0x7f1f1406d300 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c036000 tx=0x7f1f1c03b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:32.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.047+0000 7f1f26ffd640 1 -- 172.21.15.46:0/3399564420 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1f20054200 con 0x7f1f280709c0 2023-12-02T00:30:32.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.227+0000 7f1f30583640 1 -- 172.21.15.46:0/3399564420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1f28067420 con 0x7f1f280709c0 2023-12-02T00:30:32.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.227+0000 7f1f26ffd640 1 -- 172.21.15.46:0/3399564420 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v38) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1f20057820 con 0x7f1f280709c0 2023-12-02T00:30:32.229 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-12-02T00:30:32.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.229+0000 7f1f30583640 1 -- 172.21.15.46:0/3399564420 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f1f1406ae40 msgr2=0x7f1f1406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:32.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.229+0000 7f1f30583640 1 --2- 172.21.15.46:0/3399564420 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f1f1406ae40 0x7f1f1406d300 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c036000 tx=0x7f1f1c03b040 comp rx=0 tx=0).stop 2023-12-02T00:30:32.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.229+0000 7f1f30583640 1 -- 172.21.15.46:0/3399564420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f280709c0 msgr2=0x7f1f2815d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:32.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.229+0000 7f1f30583640 1 --2- 172.21.15.46:0/3399564420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f280709c0 0x7f1f2815d650 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f1f2000c450 tx=0x7f1f2000c910 comp rx=0 tx=0).stop 2023-12-02T00:30:32.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.229+0000 7f1f30583640 1 -- 172.21.15.46:0/3399564420 shutdown_connections 2023-12-02T00:30:32.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.229+0000 7f1f30583640 1 --2- 172.21.15.46:0/3399564420 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f1f1406ae40 0x7f1f1406d300 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:32.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.229+0000 7f1f30583640 1 --2- 172.21.15.46:0/3399564420 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f2815d270 0x7f1f2814de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:32.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.229+0000 7f1f30583640 1 --2- 172.21.15.46:0/3399564420 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1f280cdd10 0x7f1f280ca260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:32.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.229+0000 7f1f30583640 1 --2- 172.21.15.46:0/3399564420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f280709c0 0x7f1f2815d650 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:32.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.229+0000 7f1f30583640 1 -- 172.21.15.46:0/3399564420 >> 172.21.15.46:0/3399564420 conn(0x7f1f280b2c10 msgr2=0x7f1f2815b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:32.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.229+0000 7f1f30583640 1 -- 172.21.15.46:0/3399564420 shutdown_connections 2023-12-02T00:30:32.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.229+0000 7f1f30583640 1 -- 172.21.15.46:0/3399564420 wait complete. 2023-12-02T00:30:32.241 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2023-12-02T00:30:32.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.350+0000 7fe95529b640 1 Processor -- start 2023-12-02T00:30:32.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.350+0000 7fe95529b640 1 -- start start 2023-12-02T00:30:32.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.351+0000 7fe95529b640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe9500d1d50 0x7fe95006bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:32.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.351+0000 7fe95529b640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe95006ce00 con 0x7fe95015d2e0 2023-12-02T00:30:32.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.351+0000 7fe95529b640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe95006c1f0 con 0x7fe95015d6c0 2023-12-02T00:30:32.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.351+0000 7fe95529b640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe95006c4d0 con 0x7fe9500d1d50 2023-12-02T00:30:32.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.351+0000 7fe94effd640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe95015d2e0 0x7fe9500d1640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:44692/0 (socket says 172.21.15.46:44692) 2023-12-02T00:30:32.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.351+0000 7fe94effd640 1 -- 172.21.15.46:0/2159236861 learned_addr learned my addr 172.21.15.46:0/2159236861 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:30:32.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.351+0000 7fe94f7fe640 1 --2- 172.21.15.46:0/2159236861 >> v2:172.21.15.162:3300/0 conn(0x7fe9500d1d50 0x7fe95006bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:32.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.351+0000 7fe94dffb640 1 -- 172.21.15.46:0/2159236861 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 29194583 0 0) 0x7fe95006ce00 con 0x7fe95015d2e0 2023-12-02T00:30:32.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.351+0000 7fe94dffb640 1 -- 172.21.15.46:0/2159236861 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe93c003520 con 0x7fe95015d2e0 2023-12-02T00:30:32.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.352+0000 7fe94dffb640 1 -- 172.21.15.46:0/2159236861 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4276714886 0 0) 0x7fe93c003520 con 0x7fe95015d2e0 2023-12-02T00:30:32.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.352+0000 7fe94dffb640 1 -- 172.21.15.46:0/2159236861 >> v2:172.21.15.162:3300/0 conn(0x7fe9500d1d50 msgr2=0x7fe95006bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:32.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.352+0000 7fe94dffb640 1 --2- 172.21.15.46:0/2159236861 >> v2:172.21.15.162:3300/0 conn(0x7fe9500d1d50 0x7fe95006bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:32.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.352+0000 7fe94dffb640 1 -- 172.21.15.46:0/2159236861 >> v1:172.21.15.162:6789/0 conn(0x7fe95015d6c0 legacy=0x7fe95006b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:32.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.352+0000 7fe94dffb640 1 -- 172.21.15.46:0/2159236861 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe95006c7b0 con 0x7fe95015d2e0 2023-12-02T00:30:32.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.352+0000 7fe94dffb640 1 -- 172.21.15.46:0/2159236861 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7fe948001670 con 0x7fe95015d2e0 2023-12-02T00:30:32.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.352+0000 7fe94dffb640 1 -- 172.21.15.46:0/2159236861 >> v1:172.21.15.46:6789/0 conn(0x7fe95015d2e0 legacy=0x7fe9500d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:32.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.352+0000 7fe94dffb640 1 --2- 172.21.15.46:0/2159236861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe93c003970 0x7fe93c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:32.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.352+0000 7fe94dffb640 1 --2- 172.21.15.46:0/2159236861 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe93c0047c0 0x7fe93c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:32.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.352+0000 7fe94dffb640 1 --2- 172.21.15.46:0/2159236861 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe93c005180 0x7fe93c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:32.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.352+0000 7fe94dffb640 1 -- 172.21.15.46:0/2159236861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95006c4d0 con 0x7fe93c003970 2023-12-02T00:30:32.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.352+0000 7fe94dffb640 1 -- 172.21.15.46:0/2159236861 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe93c00c120 con 0x7fe93c005180 2023-12-02T00:30:32.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.352+0000 7fe94dffb640 1 -- 172.21.15.46:0/2159236861 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe93c00c2e0 con 0x7fe93c0047c0 2023-12-02T00:30:32.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.352+0000 7fe94effd640 1 --2- 172.21.15.46:0/2159236861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe93c003970 0x7fe93c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:32.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.352+0000 7fe94e7fc640 1 --2- 172.21.15.46:0/2159236861 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe93c0047c0 0x7fe93c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:32.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.352+0000 7fe94f7fe640 1 --2- 172.21.15.46:0/2159236861 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe93c005180 0x7fe93c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:32.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.352+0000 7fe94effd640 1 -- 172.21.15.46:0/2159236861 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe93c0047c0 msgr2=0x7fe93c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:32.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.352+0000 7fe94effd640 1 --2- 172.21.15.46:0/2159236861 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe93c0047c0 0x7fe93c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:32.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.352+0000 7fe94effd640 1 -- 172.21.15.46:0/2159236861 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe93c005180 msgr2=0x7fe93c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:32.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.352+0000 7fe94effd640 1 --2- 172.21.15.46:0/2159236861 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe93c005180 0x7fe93c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:32.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.352+0000 7fe94effd640 1 -- 172.21.15.46:0/2159236861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe93c003520 con 0x7fe93c003970 2023-12-02T00:30:32.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.352+0000 7fe94f7fe640 1 --2- 172.21.15.46:0/2159236861 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe93c005180 0x7fe93c00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:30:32.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.352+0000 7fe94f7fe640 1 -- 172.21.15.46:0/2159236861 reap_dead start 2023-12-02T00:30:32.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.353+0000 7fe94effd640 1 --2- 172.21.15.46:0/2159236861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe93c003970 0x7fe93c003d90 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fe94800c240 tx=0x7fe93c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=babfd5db25e5f0b3 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:32.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.353+0000 7fe94dffb640 1 -- 172.21.15.46:0/2159236861 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe948005ad0 con 0x7fe93c003970 2023-12-02T00:30:32.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.353+0000 7fe94dffb640 1 -- 172.21.15.46:0/2159236861 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe948005c30 con 0x7fe93c003970 2023-12-02T00:30:32.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.353+0000 7fe94dffb640 1 -- 172.21.15.46:0/2159236861 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94802cab0 con 0x7fe93c003970 2023-12-02T00:30:32.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.354+0000 7fe95529b640 1 -- 172.21.15.46:0/2159236861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe93c003970 msgr2=0x7fe93c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:32.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.354+0000 7fe95529b640 1 --2- 172.21.15.46:0/2159236861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe93c003970 0x7fe93c003d90 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fe94800c240 tx=0x7fe93c0057f0 comp rx=0 tx=0).stop 2023-12-02T00:30:32.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.354+0000 7fe95529b640 1 -- 172.21.15.46:0/2159236861 shutdown_connections 2023-12-02T00:30:32.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.354+0000 7fe95529b640 1 --2- 172.21.15.46:0/2159236861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe93c003970 0x7fe93c003d90 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:32.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.354+0000 7fe95529b640 1 -- 172.21.15.46:0/2159236861 >> 172.21.15.46:0/2159236861 conn(0x7fe95009c580 msgr2=0x7fe9500d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:32.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.354+0000 7fe95529b640 1 -- 172.21.15.46:0/2159236861 shutdown_connections 2023-12-02T00:30:32.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.354+0000 7fe95529b640 1 -- 172.21.15.46:0/2159236861 wait complete. 2023-12-02T00:30:32.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.355+0000 7fe95529b640 1 Processor -- start 2023-12-02T00:30:32.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.355+0000 7fe95529b640 1 -- start start 2023-12-02T00:30:32.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.356+0000 7fe95529b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe93c003970 0x7fe9501f2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:32.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.356+0000 7fe95529b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe95014ce80 0x7fe9500b3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:32.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.356+0000 7fe95529b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe9501472c0 0x7fe950147740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:32.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.356+0000 7fe95529b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9500b33b0 con 0x7fe95014ce80 2023-12-02T00:30:32.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.356+0000 7fe95529b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9500b24c0 con 0x7fe93c003970 2023-12-02T00:30:32.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.356+0000 7fe95529b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9500b27a0 con 0x7fe9501472c0 2023-12-02T00:30:32.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.356+0000 7fe94e7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe95014ce80 0x7fe9500b3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:32.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.356+0000 7fe94e7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe95014ce80 0x7fe9500b3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40972/0 (socket says 172.21.15.46:40972) 2023-12-02T00:30:32.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.356+0000 7fe94e7fc640 1 -- 172.21.15.46:0/1015378946 learned_addr learned my addr 172.21.15.46:0/1015378946 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:32.362 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.356+0000 7fe94effd640 1 --2- 172.21.15.46:0/1015378946 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe93c003970 0x7fe9501f2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:32.362 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.356+0000 7fe94f7fe640 1 --2- 172.21.15.46:0/1015378946 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe9501472c0 0x7fe950147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:32.362 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.356+0000 7fe94e7fc640 1 -- 172.21.15.46:0/1015378946 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe9501472c0 msgr2=0x7fe950147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:32.362 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.356+0000 7fe94e7fc640 1 --2- 172.21.15.46:0/1015378946 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe9501472c0 0x7fe950147740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:32.362 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.356+0000 7fe94e7fc640 1 -- 172.21.15.46:0/1015378946 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe93c003970 msgr2=0x7fe9501f2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:32.362 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.356+0000 7fe94e7fc640 1 --2- 172.21.15.46:0/1015378946 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe93c003970 0x7fe9501f2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:32.362 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.356+0000 7fe94e7fc640 1 -- 172.21.15.46:0/1015378946 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9500b2a80 con 0x7fe95014ce80 2023-12-02T00:30:32.362 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.356+0000 7fe94e7fc640 1 --2- 172.21.15.46:0/1015378946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe95014ce80 0x7fe9500b3840 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe93800f7f0 tx=0x7fe93800fcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:32.362 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.357+0000 7fe9477fe640 1 -- 172.21.15.46:0/1015378946 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9380050d0 con 0x7fe95014ce80 2023-12-02T00:30:32.362 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.357+0000 7fe9477fe640 1 -- 172.21.15.46:0/1015378946 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe938005230 con 0x7fe95014ce80 2023-12-02T00:30:32.362 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.357+0000 7fe95529b640 1 -- 172.21.15.46:0/1015378946 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9500b2d60 con 0x7fe95014ce80 2023-12-02T00:30:32.362 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.357+0000 7fe9477fe640 1 -- 172.21.15.46:0/1015378946 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93800d730 con 0x7fe95014ce80 2023-12-02T00:30:32.362 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.357+0000 7fe95529b640 1 -- 172.21.15.46:0/1015378946 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe950147c80 con 0x7fe95014ce80 2023-12-02T00:30:32.362 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.358+0000 7fe95529b640 1 -- 172.21.15.46:0/1015378946 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe950067420 con 0x7fe95014ce80 2023-12-02T00:30:32.362 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.358+0000 7fe9477fe640 1 -- 172.21.15.46:0/1015378946 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe93800d890 con 0x7fe95014ce80 2023-12-02T00:30:32.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.358+0000 7fe9477fe640 1 --2- 172.21.15.46:0/1015378946 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe93006ae40 0x7fe93006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:32.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.359+0000 7fe9477fe640 1 -- 172.21.15.46:0/1015378946 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fe938089510 con 0x7fe95014ce80 2023-12-02T00:30:32.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.359+0000 7fe94effd640 1 --2- 172.21.15.46:0/1015378946 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe93006ae40 0x7fe93006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:32.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.359+0000 7fe94effd640 1 --2- 172.21.15.46:0/1015378946 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe93006ae40 0x7fe93006d300 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe948034000 tx=0x7fe948009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:32.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.363+0000 7fe9477fe640 1 -- 172.21.15.46:0/1015378946 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe938054570 con 0x7fe95014ce80 2023-12-02T00:30:32.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.537+0000 7fe95529b640 1 -- 172.21.15.46:0/1015378946 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7fe9500d0950 con 0x7fe95014ce80 2023-12-02T00:30:32.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.538+0000 7fe9477fe640 1 -- 172.21.15.46:0/1015378946 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v38) v1 ==== 97+0+11 (secure 0 0 0) 0x7fe938057b90 con 0x7fe95014ce80 2023-12-02T00:30:32.539 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-12-02T00:30:32.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.539+0000 7fe95529b640 1 -- 172.21.15.46:0/1015378946 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe93006ae40 msgr2=0x7fe93006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:32.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.539+0000 7fe95529b640 1 --2- 172.21.15.46:0/1015378946 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe93006ae40 0x7fe93006d300 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe948034000 tx=0x7fe948009030 comp rx=0 tx=0).stop 2023-12-02T00:30:32.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.539+0000 7fe95529b640 1 -- 172.21.15.46:0/1015378946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe95014ce80 msgr2=0x7fe9500b3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:32.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.539+0000 7fe95529b640 1 --2- 172.21.15.46:0/1015378946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe95014ce80 0x7fe9500b3840 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe93800f7f0 tx=0x7fe93800fcb0 comp rx=0 tx=0).stop 2023-12-02T00:30:32.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.539+0000 7fe95529b640 1 -- 172.21.15.46:0/1015378946 shutdown_connections 2023-12-02T00:30:32.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.539+0000 7fe95529b640 1 --2- 172.21.15.46:0/1015378946 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe93006ae40 0x7fe93006d300 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:32.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.539+0000 7fe95529b640 1 --2- 172.21.15.46:0/1015378946 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe9501472c0 0x7fe950147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:32.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.539+0000 7fe95529b640 1 --2- 172.21.15.46:0/1015378946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe95014ce80 0x7fe9500b3840 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:32.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.539+0000 7fe95529b640 1 --2- 172.21.15.46:0/1015378946 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe93c003970 0x7fe9501f2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:32.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.539+0000 7fe95529b640 1 -- 172.21.15.46:0/1015378946 >> 172.21.15.46:0/1015378946 conn(0x7fe95009c580 msgr2=0x7fe9500b0630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:32.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.539+0000 7fe95529b640 1 -- 172.21.15.46:0/1015378946 shutdown_connections 2023-12-02T00:30:32.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.539+0000 7fe95529b640 1 -- 172.21.15.46:0/1015378946 wait complete. 2023-12-02T00:30:32.551 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-02T00:30:32.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.659+0000 7f7898210640 1 Processor -- start 2023-12-02T00:30:32.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.659+0000 7f7898210640 1 -- start start 2023-12-02T00:30:32.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.660+0000 7f7898210640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7890143490 0x7f78900b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:32.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.660+0000 7f7898210640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f789015abf0 0x7f78900b1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:32.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.660+0000 7f7898210640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f789015e2a0 con 0x7f789015e730 2023-12-02T00:30:32.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.660+0000 7f7898210640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f789015b520 con 0x7f789015abf0 2023-12-02T00:30:32.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.660+0000 7f7898210640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7890143870 con 0x7f7890143490 2023-12-02T00:30:32.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.660+0000 7f78967ad640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f789015abf0 0x7f78900b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:32.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.660+0000 7f78957ab640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f789015e730 0x7f789015a4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49234/0 (socket says 172.21.15.46:49234) 2023-12-02T00:30:32.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.660+0000 7f78957ab640 1 -- 172.21.15.46:0/1342646375 learned_addr learned my addr 172.21.15.46:0/1342646375 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:30:32.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.661+0000 7f7895fac640 1 --2- 172.21.15.46:0/1342646375 >> v2:172.21.15.162:3300/0 conn(0x7f7890143490 0x7f78900b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:32.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.661+0000 7f7894faa640 1 -- 172.21.15.46:0/1342646375 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3105539693 0 0) 0x7f789015e2a0 con 0x7f789015e730 2023-12-02T00:30:32.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.661+0000 7f7894faa640 1 -- 172.21.15.46:0/1342646375 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7878003520 con 0x7f789015e730 2023-12-02T00:30:32.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.661+0000 7f78967ad640 1 -- 172.21.15.46:0/1342646375 >> v2:172.21.15.162:3300/0 conn(0x7f7890143490 msgr2=0x7f78900b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:32.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.661+0000 7f78967ad640 1 --2- 172.21.15.46:0/1342646375 >> v2:172.21.15.162:3300/0 conn(0x7f7890143490 0x7f78900b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:32.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.661+0000 7f78967ad640 1 -- 172.21.15.46:0/1342646375 >> v1:172.21.15.46:6789/0 conn(0x7f789015e730 legacy=0x7f789015a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:32.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.661+0000 7f78967ad640 1 -- 172.21.15.46:0/1342646375 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f789015b240 con 0x7f789015abf0 2023-12-02T00:30:32.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.661+0000 7f78967ad640 1 --2- 172.21.15.46:0/1342646375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f789015abf0 0x7f78900b1230 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f789006e620 tx=0x7f788002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac82fdb4e5da2ac server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:32.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.662+0000 7f7894faa640 1 -- 172.21.15.46:0/1342646375 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7880031830 con 0x7f789015abf0 2023-12-02T00:30:32.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.662+0000 7f7894faa640 1 -- 172.21.15.46:0/1342646375 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7880031990 con 0x7f789015abf0 2023-12-02T00:30:32.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.662+0000 7f7894faa640 1 -- 172.21.15.46:0/1342646375 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7880031d00 con 0x7f789015abf0 2023-12-02T00:30:32.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.662+0000 7f7898210640 1 -- 172.21.15.46:0/1342646375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f789015abf0 msgr2=0x7f78900b1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:32.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.662+0000 7f7898210640 1 --2- 172.21.15.46:0/1342646375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f789015abf0 0x7f78900b1230 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f789006e620 tx=0x7f788002b8d0 comp rx=0 tx=0).stop 2023-12-02T00:30:32.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.662+0000 7f7898210640 1 -- 172.21.15.46:0/1342646375 shutdown_connections 2023-12-02T00:30:32.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.662+0000 7f7898210640 1 --2- 172.21.15.46:0/1342646375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f789015abf0 0x7f78900b1230 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:32.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.662+0000 7f7898210640 1 --2- 172.21.15.46:0/1342646375 >> v2:172.21.15.162:3300/0 conn(0x7f7890143490 0x7f78900b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:32.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.662+0000 7f7898210640 1 -- 172.21.15.46:0/1342646375 >> 172.21.15.46:0/1342646375 conn(0x7f789006a4d0 msgr2=0x7f789006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:32.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.662+0000 7f7898210640 1 -- 172.21.15.46:0/1342646375 shutdown_connections 2023-12-02T00:30:32.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.662+0000 7f7898210640 1 -- 172.21.15.46:0/1342646375 wait complete. 2023-12-02T00:30:32.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.663+0000 7f7898210640 1 Processor -- start 2023-12-02T00:30:32.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.663+0000 7f7898210640 1 -- start start 2023-12-02T00:30:32.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.664+0000 7f7898210640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7890143490 0x7f78900ca960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:32.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.664+0000 7f7898210640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f789015abf0 0x7f78900aa8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:32.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.664+0000 7f7898210640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f789015e730 0x7f7890132c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:32.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.664+0000 7f7898210640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7890131b70 con 0x7f789015e730 2023-12-02T00:30:32.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.664+0000 7f7898210640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7890131e50 con 0x7f7890143490 2023-12-02T00:30:32.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.664+0000 7f7898210640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7890132130 con 0x7f789015abf0 2023-12-02T00:30:32.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.664+0000 7f78967ad640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f789015e730 0x7f7890132c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:32.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.664+0000 7f78967ad640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f789015e730 0x7f7890132c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52802/0 (socket says 172.21.15.46:52802) 2023-12-02T00:30:32.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.664+0000 7f78967ad640 1 -- 172.21.15.46:0/2978949522 learned_addr learned my addr 172.21.15.46:0/2978949522 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:32.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.664+0000 7f7895fac640 1 --2- 172.21.15.46:0/2978949522 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7890143490 0x7f78900ca960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:32.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.664+0000 7f78957ab640 1 --2- 172.21.15.46:0/2978949522 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f789015abf0 0x7f78900aa8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:32.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.665+0000 7f78967ad640 1 -- 172.21.15.46:0/2978949522 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f789015abf0 msgr2=0x7f78900aa8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:32.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.665+0000 7f78967ad640 1 --2- 172.21.15.46:0/2978949522 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f789015abf0 0x7f78900aa8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:32.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.665+0000 7f78967ad640 1 -- 172.21.15.46:0/2978949522 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7890143490 msgr2=0x7f78900ca960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:32.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.665+0000 7f78967ad640 1 --2- 172.21.15.46:0/2978949522 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7890143490 0x7f78900ca960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:32.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.665+0000 7f78967ad640 1 -- 172.21.15.46:0/2978949522 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7880009400 con 0x7f789015e730 2023-12-02T00:30:32.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.665+0000 7f78967ad640 1 --2- 172.21.15.46:0/2978949522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f789015e730 0x7f7890132c10 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f7880009890 tx=0x7f78800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:32.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.665+0000 7f788e7fc640 1 -- 172.21.15.46:0/2978949522 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788003a630 con 0x7f789015e730 2023-12-02T00:30:32.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.665+0000 7f788e7fc640 1 -- 172.21.15.46:0/2978949522 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f788003a790 con 0x7f789015e730 2023-12-02T00:30:32.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.665+0000 7f788e7fc640 1 -- 172.21.15.46:0/2978949522 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788003ab20 con 0x7f789015e730 2023-12-02T00:30:32.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.665+0000 7f7898210640 1 -- 172.21.15.46:0/2978949522 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7890132410 con 0x7f789015e730 2023-12-02T00:30:32.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.665+0000 7f7898210640 1 -- 172.21.15.46:0/2978949522 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78900caea0 con 0x7f789015e730 2023-12-02T00:30:32.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.666+0000 7f7898210640 1 -- 172.21.15.46:0/2978949522 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7890132930 con 0x7f789015e730 2023-12-02T00:30:32.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.666+0000 7f788e7fc640 1 -- 172.21.15.46:0/2978949522 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f788003ac80 con 0x7f789015e730 2023-12-02T00:30:32.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.667+0000 7f788e7fc640 1 --2- 172.21.15.46:0/2978949522 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f787c06ae40 0x7f787c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:32.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.667+0000 7f788e7fc640 1 -- 172.21.15.46:0/2978949522 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f78800ab1f0 con 0x7f789015e730 2023-12-02T00:30:32.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.667+0000 7f7895fac640 1 --2- 172.21.15.46:0/2978949522 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f787c06ae40 0x7f787c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:32.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.668+0000 7f7895fac640 1 --2- 172.21.15.46:0/2978949522 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f787c06ae40 0x7f787c06d300 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7890070df0 tx=0x7f78880039b0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:32.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.671+0000 7f788e7fc640 1 -- 172.21.15.46:0/2978949522 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7890132930 con 0x7f789015e730 2023-12-02T00:30:32.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.860+0000 7f7898210640 1 -- 172.21.15.46:0/2978949522 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7890132930 con 0x7f789015e730 2023-12-02T00:30:32.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.860+0000 7f788e7fc640 1 -- 172.21.15.46:0/2978949522 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v38) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7890132930 con 0x7f789015e730 2023-12-02T00:30:32.861 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2023-12-02T00:30:32.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.862+0000 7f7898210640 1 -- 172.21.15.46:0/2978949522 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f787c06ae40 msgr2=0x7f787c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:32.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.862+0000 7f7898210640 1 --2- 172.21.15.46:0/2978949522 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f787c06ae40 0x7f787c06d300 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7890070df0 tx=0x7f78880039b0 comp rx=0 tx=0).stop 2023-12-02T00:30:32.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.862+0000 7f7898210640 1 -- 172.21.15.46:0/2978949522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f789015e730 msgr2=0x7f7890132c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:32.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.862+0000 7f7898210640 1 --2- 172.21.15.46:0/2978949522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f789015e730 0x7f7890132c10 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f7880009890 tx=0x7f78800046c0 comp rx=0 tx=0).stop 2023-12-02T00:30:32.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.862+0000 7f7898210640 1 -- 172.21.15.46:0/2978949522 shutdown_connections 2023-12-02T00:30:32.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.862+0000 7f7898210640 1 --2- 172.21.15.46:0/2978949522 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f787c06ae40 0x7f787c06d300 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:32.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.862+0000 7f7898210640 1 --2- 172.21.15.46:0/2978949522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f789015e730 0x7f7890132c10 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:32.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.862+0000 7f7898210640 1 --2- 172.21.15.46:0/2978949522 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f789015abf0 0x7f78900aa8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:32.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.862+0000 7f7898210640 1 --2- 172.21.15.46:0/2978949522 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7890143490 0x7f78900ca960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:32.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.862+0000 7f7898210640 1 -- 172.21.15.46:0/2978949522 >> 172.21.15.46:0/2978949522 conn(0x7f789006a4d0 msgr2=0x7f789006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:32.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.862+0000 7f7898210640 1 -- 172.21.15.46:0/2978949522 shutdown_connections 2023-12-02T00:30:32.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:32.862+0000 7f7898210640 1 -- 172.21.15.46:0/2978949522 wait complete. 2023-12-02T00:30:32.874 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi046.front.sepia.ceph.com... 2023-12-02T00:30:32.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:30:32.874 DEBUG:teuthology.orchestra.run.smithi046:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-12-02T00:30:32.903 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[106757]: fuse finished with error 0 and tester_r 0 2023-12-02T00:30:32.905 INFO:teuthology.orchestra.run:waiting for 300 2023-12-02T00:30:39.007 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2023-12-02T00:30:39.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:30:39.008 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-02T00:30:39.027 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2023-12-02T00:30:39.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.192+0000 7f92eee4f640 1 Processor -- start 2023-12-02T00:30:39.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.192+0000 7f92eee4f640 1 -- start start 2023-12-02T00:30:39.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.193+0000 7f92eee4f640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f92e80b2180 0x7f92e806f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:39.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.193+0000 7f92eee4f640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f92e80cf660 0x7f92e81610b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:39.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.193+0000 7f92eee4f640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92e8147150 con 0x7f92e806f7f0 2023-12-02T00:30:39.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.193+0000 7f92eee4f640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92e815e570 con 0x7f92e80b2180 2023-12-02T00:30:39.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.193+0000 7f92eee4f640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92e806fca0 con 0x7f92e80cf660 2023-12-02T00:30:39.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.193+0000 7f92ecbeb640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f92e80b2180 0x7f92e806f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:39.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.193+0000 7f92ecbeb640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f92e80b2180 0x7f92e806f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52818/0 (socket says 172.21.15.46:52818) 2023-12-02T00:30:39.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.193+0000 7f92ecbeb640 1 -- 172.21.15.46:0/2090326703 learned_addr learned my addr 172.21.15.46:0/2090326703 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:39.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.193+0000 7f92ed3ec640 1 --2- 172.21.15.46:0/2090326703 >> v2:172.21.15.162:3300/0 conn(0x7f92e80cf660 0x7f92e81610b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:39.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.194+0000 7f92ecbeb640 1 -- 172.21.15.46:0/2090326703 >> v2:172.21.15.162:3300/0 conn(0x7f92e80cf660 msgr2=0x7f92e81610b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:39.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.194+0000 7f92ecbeb640 1 --2- 172.21.15.46:0/2090326703 >> v2:172.21.15.162:3300/0 conn(0x7f92e80cf660 0x7f92e81610b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:39.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.194+0000 7f92ecbeb640 1 -- 172.21.15.46:0/2090326703 >> v1:172.21.15.162:6789/0 conn(0x7f92e806f7f0 legacy=0x7f92e80cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:39.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.194+0000 7f92ecbeb640 1 -- 172.21.15.46:0/2090326703 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e80cfad0 con 0x7f92e80b2180 2023-12-02T00:30:39.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.194+0000 7f92ecbeb640 1 --2- 172.21.15.46:0/2090326703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92e80b2180 0x7f92e806f2b0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f92e815dac0 tx=0x7f92e002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdf97c5a94c459f5 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:39.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.194+0000 7f92e77fe640 1 -- 172.21.15.46:0/2090326703 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e0031870 con 0x7f92e80b2180 2023-12-02T00:30:39.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.195+0000 7f92e77fe640 1 -- 172.21.15.46:0/2090326703 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92e00319d0 con 0x7f92e80b2180 2023-12-02T00:30:39.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.195+0000 7f92e77fe640 1 -- 172.21.15.46:0/2090326703 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e0031d60 con 0x7f92e80b2180 2023-12-02T00:30:39.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.195+0000 7f92eee4f640 1 -- 172.21.15.46:0/2090326703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92e80b2180 msgr2=0x7f92e806f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:39.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.195+0000 7f92eee4f640 1 --2- 172.21.15.46:0/2090326703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92e80b2180 0x7f92e806f2b0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f92e815dac0 tx=0x7f92e002b910 comp rx=0 tx=0).stop 2023-12-02T00:30:39.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.195+0000 7f92eee4f640 1 -- 172.21.15.46:0/2090326703 shutdown_connections 2023-12-02T00:30:39.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.195+0000 7f92eee4f640 1 --2- 172.21.15.46:0/2090326703 >> v2:172.21.15.162:3300/0 conn(0x7f92e80cf660 0x7f92e81610b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:39.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.195+0000 7f92eee4f640 1 --2- 172.21.15.46:0/2090326703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92e80b2180 0x7f92e806f2b0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:39.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.195+0000 7f92eee4f640 1 -- 172.21.15.46:0/2090326703 >> 172.21.15.46:0/2090326703 conn(0x7f92e806d690 msgr2=0x7f92e806da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:39.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.195+0000 7f92eee4f640 1 -- 172.21.15.46:0/2090326703 shutdown_connections 2023-12-02T00:30:39.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.195+0000 7f92eee4f640 1 -- 172.21.15.46:0/2090326703 wait complete. 2023-12-02T00:30:39.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.196+0000 7f92eee4f640 1 Processor -- start 2023-12-02T00:30:39.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.196+0000 7f92eee4f640 1 -- start start 2023-12-02T00:30:39.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.197+0000 7f92eee4f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92e806f7f0 0x7f92e81e4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:39.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.197+0000 7f92eee4f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92e80b2180 0x7f92e8158180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:39.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.197+0000 7f92eee4f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92e80cf660 0x7f92e81586c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:39.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.197+0000 7f92eee4f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e81e4780 con 0x7f92e80cf660 2023-12-02T00:30:39.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.197+0000 7f92eee4f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e81e3e50 con 0x7f92e80b2180 2023-12-02T00:30:39.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.197+0000 7f92eee4f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e8159470 con 0x7f92e806f7f0 2023-12-02T00:30:39.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.197+0000 7f92ed3ec640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92e80cf660 0x7f92e81586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:39.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.197+0000 7f92ed3ec640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92e80cf660 0x7f92e81586c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52820/0 (socket says 172.21.15.46:52820) 2023-12-02T00:30:39.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.197+0000 7f92ed3ec640 1 -- 172.21.15.46:0/1291792995 learned_addr learned my addr 172.21.15.46:0/1291792995 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:39.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.198+0000 7f92ecbeb640 1 --2- 172.21.15.46:0/1291792995 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92e806f7f0 0x7f92e81e4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:39.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.198+0000 7f92e7fff640 1 --2- 172.21.15.46:0/1291792995 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92e80b2180 0x7f92e8158180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:39.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.198+0000 7f92ed3ec640 1 -- 172.21.15.46:0/1291792995 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92e806f7f0 msgr2=0x7f92e81e4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:39.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.198+0000 7f92ed3ec640 1 --2- 172.21.15.46:0/1291792995 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92e806f7f0 0x7f92e81e4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:39.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.198+0000 7f92ed3ec640 1 -- 172.21.15.46:0/1291792995 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92e80b2180 msgr2=0x7f92e8158180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:39.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.198+0000 7f92ed3ec640 1 --2- 172.21.15.46:0/1291792995 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92e80b2180 0x7f92e8158180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:39.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.198+0000 7f92ed3ec640 1 -- 172.21.15.46:0/1291792995 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e00094d0 con 0x7f92e80cf660 2023-12-02T00:30:39.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.198+0000 7f92ed3ec640 1 --2- 172.21.15.46:0/1291792995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92e80cf660 0x7f92e81586c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f92d000d800 tx=0x7f92d000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:39.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.198+0000 7f92e57fa640 1 -- 172.21.15.46:0/1291792995 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d000b6c0 con 0x7f92e80cf660 2023-12-02T00:30:39.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.198+0000 7f92e57fa640 1 -- 172.21.15.46:0/1291792995 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92d000b840 con 0x7f92e80cf660 2023-12-02T00:30:39.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.198+0000 7f92eee4f640 1 -- 172.21.15.46:0/1291792995 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92e8159750 con 0x7f92e80cf660 2023-12-02T00:30:39.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.198+0000 7f92e57fa640 1 -- 172.21.15.46:0/1291792995 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d000bbf0 con 0x7f92e80cf660 2023-12-02T00:30:39.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.198+0000 7f92eee4f640 1 -- 172.21.15.46:0/1291792995 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92e81e38b0 con 0x7f92e80cf660 2023-12-02T00:30:39.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.199+0000 7f92eee4f640 1 -- 172.21.15.46:0/1291792995 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92e80cd150 con 0x7f92e80cf660 2023-12-02T00:30:39.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.200+0000 7f92e57fa640 1 -- 172.21.15.46:0/1291792995 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f92d000bd50 con 0x7f92e80cf660 2023-12-02T00:30:39.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.200+0000 7f92e57fa640 1 --2- 172.21.15.46:0/1291792995 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f92c806ae70 0x7f92c806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:39.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.200+0000 7f92e57fa640 1 -- 172.21.15.46:0/1291792995 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f92d0089ad0 con 0x7f92e80cf660 2023-12-02T00:30:39.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.200+0000 7f92ecbeb640 1 --2- 172.21.15.46:0/1291792995 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f92c806ae70 0x7f92c806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:39.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.201+0000 7f92ecbeb640 1 --2- 172.21.15.46:0/1291792995 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f92c806ae70 0x7f92c806d330 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f92e0036000 tx=0x7f92e003c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.204+0000 7f92e57fa640 1 -- 172.21.15.46:0/1291792995 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f92d0054b30 con 0x7f92e80cf660 2023-12-02T00:30:39.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.448+0000 7f92eee4f640 1 -- 172.21.15.46:0/1291792995 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f92e8069400 con 0x7f92e80cf660 2023-12-02T00:30:39.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.450+0000 7f92e57fa640 1 -- 172.21.15.46:0/1291792995 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v16) v1 ==== 184+0+141 (secure 0 0 0) 0x7f92d0058150 con 0x7f92e80cf660 2023-12-02T00:30:39.451 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2023-12-02T00:30:39.451 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAJempl5Lx1DhAAjpK6ajoQ5jQiUr/EMpFeLg== 2023-12-02T00:30:39.451 INFO:teuthology.orchestra.run.smithi046.stdout: caps mds = "allow" 2023-12-02T00:30:39.451 INFO:teuthology.orchestra.run.smithi046.stdout: caps mon = "allow r" 2023-12-02T00:30:39.451 INFO:teuthology.orchestra.run.smithi046.stdout: caps osd = "allow rw pool=cephfs_data" 2023-12-02T00:30:39.451 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2023-12-02T00:30:39.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.452+0000 7f92eee4f640 1 -- 172.21.15.46:0/1291792995 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f92c806ae70 msgr2=0x7f92c806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:39.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.452+0000 7f92eee4f640 1 --2- 172.21.15.46:0/1291792995 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f92c806ae70 0x7f92c806d330 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f92e0036000 tx=0x7f92e003c040 comp rx=0 tx=0).stop 2023-12-02T00:30:39.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.452+0000 7f92eee4f640 1 -- 172.21.15.46:0/1291792995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92e80cf660 msgr2=0x7f92e81586c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:39.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.452+0000 7f92eee4f640 1 --2- 172.21.15.46:0/1291792995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92e80cf660 0x7f92e81586c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f92d000d800 tx=0x7f92d000dcc0 comp rx=0 tx=0).stop 2023-12-02T00:30:39.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.452+0000 7f92eee4f640 1 -- 172.21.15.46:0/1291792995 shutdown_connections 2023-12-02T00:30:39.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.452+0000 7f92eee4f640 1 --2- 172.21.15.46:0/1291792995 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f92c806ae70 0x7f92c806d330 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:39.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.452+0000 7f92eee4f640 1 --2- 172.21.15.46:0/1291792995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92e80cf660 0x7f92e81586c0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:39.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.452+0000 7f92eee4f640 1 --2- 172.21.15.46:0/1291792995 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92e80b2180 0x7f92e8158180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:39.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.452+0000 7f92eee4f640 1 --2- 172.21.15.46:0/1291792995 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92e806f7f0 0x7f92e81e4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:39.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.452+0000 7f92eee4f640 1 -- 172.21.15.46:0/1291792995 >> 172.21.15.46:0/1291792995 conn(0x7f92e806d690 msgr2=0x7f92e80cffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:39.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.452+0000 7f92eee4f640 1 -- 172.21.15.46:0/1291792995 shutdown_connections 2023-12-02T00:30:39.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.452+0000 7f92eee4f640 1 -- 172.21.15.46:0/1291792995 wait complete. 2023-12-02T00:30:39.464 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-02T00:30:39.464 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-02T00:30:39.507 DEBUG:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-02T00:30:39.560 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-02T00:30:39.561 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-02T00:30:39.561 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-02T00:30:39.576 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057150,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498968k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19546 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249480k,nr_inodes=812370,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498968k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-02T00:30:39.576 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2023-12-02T00:30:39.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:30:39.577 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-02T00:30:39.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-02T00:30:39.632 INFO:teuthology.orchestra.run.smithi046.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-12-02T00:30:39.632 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-12-02T00:30:39.632 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-02T00:30:39.651 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056379,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497728k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15716 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248860k,nr_inodes=812215,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497728k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-02T00:30:39.651 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi162.front.sepia.ceph.com 2023-12-02T00:30:39.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:30:39.651 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-12-02T00:30:39.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-02T00:30:39.708 INFO:teuthology.orchestra.run.smithi162.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-12-02T00:30:39.709 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-02T00:30:39.709 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-02T00:30:39.726 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057150,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498968k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19546 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249480k,nr_inodes=812370,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498968k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-02T00:30:39.726 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2023-12-02T00:30:39.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:30:39.726 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-12-02T00:30:39.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-02T00:30:39.782 INFO:teuthology.orchestra.run.smithi046.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-12-02T00:30:39.782 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-12-02T00:30:39.782 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-02T00:30:39.800 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056379,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497728k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15716 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248860k,nr_inodes=812215,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497728k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-02T00:30:39.800 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi162.front.sepia.ceph.com 2023-12-02T00:30:39.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:30:39.801 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-12-02T00:30:39.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-02T00:30:39.858 INFO:teuthology.orchestra.run.smithi162.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-12-02T00:30:39.858 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-02T00:30:39.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.965+0000 7ff6c7bb0640 1 Processor -- start 2023-12-02T00:30:39.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.965+0000 7ff6c7bb0640 1 -- start start 2023-12-02T00:30:39.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.966+0000 7ff6c7bb0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6c015d330 0x7ff6c00b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:39.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.966+0000 7ff6c7bb0640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff6c00d2910 0x7ff6c00d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:39.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.966+0000 7ff6c7bb0640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff6c00d2d90 0x7ff6c00d3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:39.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.966+0000 7ff6c7bb0640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6c00b2b10 con 0x7ff6c00d2d90 2023-12-02T00:30:39.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.966+0000 7ff6c7bb0640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6c00d1590 con 0x7ff6c00d2910 2023-12-02T00:30:39.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.966+0000 7ff6c7bb0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c00d1870 con 0x7ff6c015d330 2023-12-02T00:30:39.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.966+0000 7ff6c614d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff6c00d2d90 0x7ff6c00d3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:39.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.966+0000 7ff6c614d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff6c00d2d90 0x7ff6c00d3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52824/0 (socket says 172.21.15.46:52824) 2023-12-02T00:30:39.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.966+0000 7ff6c614d640 1 -- 172.21.15.46:0/4114798508 learned_addr learned my addr 172.21.15.46:0/4114798508 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:39.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.967+0000 7ff6c514b640 1 --2- 172.21.15.46:0/4114798508 >> v2:172.21.15.162:3300/0 conn(0x7ff6c00d2910 0x7ff6c00d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:39.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.967+0000 7ff6c594c640 1 --2- 172.21.15.46:0/4114798508 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6c015d330 0x7ff6c00b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:39.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.967+0000 7ff6c614d640 1 -- 172.21.15.46:0/4114798508 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6c015d330 msgr2=0x7ff6c00b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:39.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.967+0000 7ff6c614d640 1 --2- 172.21.15.46:0/4114798508 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6c015d330 0x7ff6c00b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:39.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.967+0000 7ff6c614d640 1 -- 172.21.15.46:0/4114798508 >> v2:172.21.15.162:3300/0 conn(0x7ff6c00d2910 msgr2=0x7ff6c00d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:39.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.967+0000 7ff6c614d640 1 --2- 172.21.15.46:0/4114798508 >> v2:172.21.15.162:3300/0 conn(0x7ff6c00d2910 0x7ff6c00d31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:39.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.967+0000 7ff6c614d640 1 -- 172.21.15.46:0/4114798508 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6c00d1b50 con 0x7ff6c00d2d90 2023-12-02T00:30:39.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.967+0000 7ff6c614d640 1 --2- 172.21.15.46:0/4114798508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6c00d2d90 0x7ff6c00d3730 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff6c00d4360 tx=0x7ff6b802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2c5d00566a3b6a8 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:39.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.968+0000 7ff6c494a640 1 -- 172.21.15.46:0/4114798508 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b802c9e0 con 0x7ff6c00d2d90 2023-12-02T00:30:39.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.968+0000 7ff6c494a640 1 -- 172.21.15.46:0/4114798508 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6b802cb40 con 0x7ff6c00d2d90 2023-12-02T00:30:39.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.968+0000 7ff6c494a640 1 -- 172.21.15.46:0/4114798508 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b80343f0 con 0x7ff6c00d2d90 2023-12-02T00:30:39.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.968+0000 7ff6c7bb0640 1 -- 172.21.15.46:0/4114798508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6c00d2d90 msgr2=0x7ff6c00d3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:39.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.968+0000 7ff6c7bb0640 1 --2- 172.21.15.46:0/4114798508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6c00d2d90 0x7ff6c00d3730 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff6c00d4360 tx=0x7ff6b802be40 comp rx=0 tx=0).stop 2023-12-02T00:30:39.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.968+0000 7ff6c7bb0640 1 -- 172.21.15.46:0/4114798508 shutdown_connections 2023-12-02T00:30:39.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.968+0000 7ff6c7bb0640 1 --2- 172.21.15.46:0/4114798508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6c00d2d90 0x7ff6c00d3730 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:39.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.968+0000 7ff6c7bb0640 1 --2- 172.21.15.46:0/4114798508 >> v2:172.21.15.162:3300/0 conn(0x7ff6c00d2910 0x7ff6c00d31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:39.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.968+0000 7ff6c7bb0640 1 --2- 172.21.15.46:0/4114798508 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6c015d330 0x7ff6c00b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:39.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.968+0000 7ff6c7bb0640 1 -- 172.21.15.46:0/4114798508 >> 172.21.15.46:0/4114798508 conn(0x7ff6c0072020 msgr2=0x7ff6c0072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:39.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.968+0000 7ff6c7bb0640 1 -- 172.21.15.46:0/4114798508 shutdown_connections 2023-12-02T00:30:39.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.968+0000 7ff6c7bb0640 1 -- 172.21.15.46:0/4114798508 wait complete. 2023-12-02T00:30:39.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.969+0000 7ff6c7bb0640 1 Processor -- start 2023-12-02T00:30:39.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.970+0000 7ff6c7bb0640 1 -- start start 2023-12-02T00:30:39.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.970+0000 7ff6c7bb0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6c00d2910 0x7ff6c015e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:39.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.970+0000 7ff6c7bb0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6c00d2d90 0x7ff6c00b1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:39.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.970+0000 7ff6c7bb0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6c015d330 0x7ff6c00ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:39.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.970+0000 7ff6c7bb0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c015fcc0 con 0x7ff6c00d2910 2023-12-02T00:30:39.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.970+0000 7ff6c7bb0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c00cd420 con 0x7ff6c015d330 2023-12-02T00:30:39.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.970+0000 7ff6c7bb0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c015f040 con 0x7ff6c00d2d90 2023-12-02T00:30:39.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.970+0000 7ff6c594c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6c00d2910 0x7ff6c015e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:39.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.970+0000 7ff6c594c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6c00d2910 0x7ff6c015e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52830/0 (socket says 172.21.15.46:52830) 2023-12-02T00:30:39.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.970+0000 7ff6c594c640 1 -- 172.21.15.46:0/2920877188 learned_addr learned my addr 172.21.15.46:0/2920877188 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:39.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.970+0000 7ff6c514b640 1 --2- 172.21.15.46:0/2920877188 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6c00d2d90 0x7ff6c00b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:39.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.971+0000 7ff6c614d640 1 --2- 172.21.15.46:0/2920877188 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6c015d330 0x7ff6c00ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:39.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.971+0000 7ff6c594c640 1 -- 172.21.15.46:0/2920877188 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6c00d2d90 msgr2=0x7ff6c00b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:39.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.971+0000 7ff6c594c640 1 --2- 172.21.15.46:0/2920877188 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6c00d2d90 0x7ff6c00b1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:39.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.971+0000 7ff6c594c640 1 -- 172.21.15.46:0/2920877188 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6c015d330 msgr2=0x7ff6c00ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:39.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.971+0000 7ff6c594c640 1 --2- 172.21.15.46:0/2920877188 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6c015d330 0x7ff6c00ccee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:39.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.971+0000 7ff6c594c640 1 -- 172.21.15.46:0/2920877188 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b8009b30 con 0x7ff6c00d2910 2023-12-02T00:30:39.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.971+0000 7ff6c514b640 1 --2- 172.21.15.46:0/2920877188 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6c00d2d90 0x7ff6c00b1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:30:39.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.971+0000 7ff6c594c640 1 --2- 172.21.15.46:0/2920877188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6c00d2910 0x7ff6c015e9d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc00d5f0 tx=0x7ff6bc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:39.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.971+0000 7ff6b67fc640 1 -- 172.21.15.46:0/2920877188 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc0125f0 con 0x7ff6c00d2910 2023-12-02T00:30:39.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.971+0000 7ff6b67fc640 1 -- 172.21.15.46:0/2920877188 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6bc012750 con 0x7ff6c00d2910 2023-12-02T00:30:39.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.971+0000 7ff6c7bb0640 1 -- 172.21.15.46:0/2920877188 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6c015f320 con 0x7ff6c00d2910 2023-12-02T00:30:39.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.971+0000 7ff6b67fc640 1 -- 172.21.15.46:0/2920877188 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc012b00 con 0x7ff6c00d2910 2023-12-02T00:30:39.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.971+0000 7ff6c7bb0640 1 -- 172.21.15.46:0/2920877188 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6c015f8e0 con 0x7ff6c00d2910 2023-12-02T00:30:39.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.972+0000 7ff6c7bb0640 1 -- 172.21.15.46:0/2920877188 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6c015ff30 con 0x7ff6c00d2910 2023-12-02T00:30:39.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.972+0000 7ff6b67fc640 1 -- 172.21.15.46:0/2920877188 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff6bc012c60 con 0x7ff6c00d2910 2023-12-02T00:30:39.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.973+0000 7ff6b67fc640 1 --2- 172.21.15.46:0/2920877188 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff6a406ae70 0x7ff6a406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:39.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.973+0000 7ff6b67fc640 1 -- 172.21.15.46:0/2920877188 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7ff6bc089b80 con 0x7ff6c00d2910 2023-12-02T00:30:39.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.973+0000 7ff6c514b640 1 --2- 172.21.15.46:0/2920877188 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff6a406ae70 0x7ff6a406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:39.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.974+0000 7ff6c514b640 1 --2- 172.21.15.46:0/2920877188 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff6a406ae70 0x7ff6a406d330 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff6c015c4f0 tx=0x7ff6b0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:39.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:39.978+0000 7ff6b67fc640 1 -- 172.21.15.46:0/2920877188 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff6bc054b30 con 0x7ff6c00d2910 2023-12-02T00:30:40.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.190+0000 7ff6c7bb0640 1 -- 172.21.15.46:0/2920877188 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6c00d4080 con 0x7ff6c00d2910 2023-12-02T00:30:40.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.191+0000 7ff6b67fc640 1 -- 172.21.15.46:0/2920877188 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4919 (secure 0 0 0) 0x7ff6bc058150 con 0x7ff6c00d2910 2023-12-02T00:30:40.192 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:30:40.192 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4811,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/3097254664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3097254664},{"type":"v1","addr":"172.21.15.162:6835","nonce":3097254664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":4826,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/3489476943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3489476943},{"type":"v1","addr":"172.21.15.162:6837","nonce":3489476943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5274,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1283185308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":1283185308},{"type":"v1","addr":"172.21.15.46:6835","nonce":1283185308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-02T00:29:57.279169+0000","modified":"2023-12-02T00:30:09.302069+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5277},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5277":{"gid":5277,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":5,"addr":"172.21.15.46:6834/2036053042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2036053042},{"type":"v1","addr":"172.21.15.46:6834","nonce":2036053042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-02T00:30:40.192 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 14 2023-12-02T00:30:40.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.193+0000 7ff6c7bb0640 1 -- 172.21.15.46:0/2920877188 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff6a406ae70 msgr2=0x7ff6a406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:40.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.193+0000 7ff6c7bb0640 1 --2- 172.21.15.46:0/2920877188 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff6a406ae70 0x7ff6a406d330 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff6c015c4f0 tx=0x7ff6b0008040 comp rx=0 tx=0).stop 2023-12-02T00:30:40.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.193+0000 7ff6c7bb0640 1 -- 172.21.15.46:0/2920877188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6c00d2910 msgr2=0x7ff6c015e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:40.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.193+0000 7ff6c7bb0640 1 --2- 172.21.15.46:0/2920877188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6c00d2910 0x7ff6c015e9d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc00d5f0 tx=0x7ff6bc00dab0 comp rx=0 tx=0).stop 2023-12-02T00:30:40.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.193+0000 7ff6c7bb0640 1 -- 172.21.15.46:0/2920877188 shutdown_connections 2023-12-02T00:30:40.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.193+0000 7ff6c7bb0640 1 --2- 172.21.15.46:0/2920877188 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff6a406ae70 0x7ff6a406d330 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:40.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.193+0000 7ff6c7bb0640 1 --2- 172.21.15.46:0/2920877188 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6c015d330 0x7ff6c00ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:40.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.193+0000 7ff6c7bb0640 1 --2- 172.21.15.46:0/2920877188 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6c00d2d90 0x7ff6c00b1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:40.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.193+0000 7ff6c7bb0640 1 --2- 172.21.15.46:0/2920877188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6c00d2910 0x7ff6c015e9d0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:40.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.193+0000 7ff6c7bb0640 1 -- 172.21.15.46:0/2920877188 >> 172.21.15.46:0/2920877188 conn(0x7ff6c0072020 msgr2=0x7ff6c0072400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:40.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.193+0000 7ff6c7bb0640 1 -- 172.21.15.46:0/2920877188 shutdown_connections 2023-12-02T00:30:40.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.193+0000 7ff6c7bb0640 1 -- 172.21.15.46:0/2920877188 wait complete. 2023-12-02T00:30:40.205 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:30:40.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.310+0000 7f5549948640 1 Processor -- start 2023-12-02T00:30:40.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.310+0000 7f5549948640 1 -- start start 2023-12-02T00:30:40.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.311+0000 7f5549948640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5544143490 0x7f55440b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:40.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.311+0000 7f5549948640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f554415e730 0x7f55440b1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:40.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.311+0000 7f5549948640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f554415a4e0 0x7f554415a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:40.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.311+0000 7f5549948640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f554415b520 con 0x7f5544143490 2023-12-02T00:30:40.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.311+0000 7f5549948640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5544143870 con 0x7f554415a4e0 2023-12-02T00:30:40.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.311+0000 7f5549948640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f554415e2a0 con 0x7f554415e730 2023-12-02T00:30:40.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.311+0000 7f5542ffd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5544143490 0x7f55440b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:40.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.311+0000 7f5542ffd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5544143490 0x7f55440b0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52842/0 (socket says 172.21.15.46:52842) 2023-12-02T00:30:40.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.311+0000 7f5542ffd640 1 -- 172.21.15.46:0/3287455179 learned_addr learned my addr 172.21.15.46:0/3287455179 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:40.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.311+0000 7f55437fe640 1 --2- 172.21.15.46:0/3287455179 >> v2:172.21.15.162:3300/0 conn(0x7f554415a4e0 0x7f554415a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:40.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.311+0000 7f5542ffd640 1 -- 172.21.15.46:0/3287455179 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f554415e730 msgr2=0x7f55440b1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:40.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.311+0000 7f553bfff640 1 --2- 172.21.15.46:0/3287455179 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f554415e730 0x7f55440b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:40.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.311+0000 7f5542ffd640 1 --2- 172.21.15.46:0/3287455179 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f554415e730 0x7f55440b1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:40.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.311+0000 7f5542ffd640 1 -- 172.21.15.46:0/3287455179 >> v2:172.21.15.162:3300/0 conn(0x7f554415a4e0 msgr2=0x7f554415a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:40.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.311+0000 7f5542ffd640 1 --2- 172.21.15.46:0/3287455179 >> v2:172.21.15.162:3300/0 conn(0x7f554415a4e0 0x7f554415a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:40.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.311+0000 7f5542ffd640 1 -- 172.21.15.46:0/3287455179 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f554415b240 con 0x7f5544143490 2023-12-02T00:30:40.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.311+0000 7f55437fe640 1 --2- 172.21.15.46:0/3287455179 >> v2:172.21.15.162:3300/0 conn(0x7f554415a4e0 0x7f554415a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:30:40.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.311+0000 7f553bfff640 1 --2- 172.21.15.46:0/3287455179 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f554415e730 0x7f55440b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:30:40.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.312+0000 7f5542ffd640 1 --2- 172.21.15.46:0/3287455179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5544143490 0x7f55440b0cf0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f554415eb90 tx=0x7f553402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d349c375d27fb57 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:40.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.312+0000 7f55427fc640 1 -- 172.21.15.46:0/3287455179 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5534031870 con 0x7f5544143490 2023-12-02T00:30:40.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.312+0000 7f55427fc640 1 -- 172.21.15.46:0/3287455179 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55340319d0 con 0x7f5544143490 2023-12-02T00:30:40.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.312+0000 7f55427fc640 1 -- 172.21.15.46:0/3287455179 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5534031d60 con 0x7f5544143490 2023-12-02T00:30:40.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.313+0000 7f5549948640 1 -- 172.21.15.46:0/3287455179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5544143490 msgr2=0x7f55440b0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:40.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.313+0000 7f5549948640 1 --2- 172.21.15.46:0/3287455179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5544143490 0x7f55440b0cf0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f554415eb90 tx=0x7f553402b910 comp rx=0 tx=0).stop 2023-12-02T00:30:40.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.313+0000 7f5549948640 1 -- 172.21.15.46:0/3287455179 shutdown_connections 2023-12-02T00:30:40.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.313+0000 7f5549948640 1 --2- 172.21.15.46:0/3287455179 >> v2:172.21.15.162:3300/0 conn(0x7f554415a4e0 0x7f554415a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:40.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.313+0000 7f5549948640 1 --2- 172.21.15.46:0/3287455179 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f554415e730 0x7f55440b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:40.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.313+0000 7f5549948640 1 --2- 172.21.15.46:0/3287455179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5544143490 0x7f55440b0cf0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:40.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.313+0000 7f5549948640 1 -- 172.21.15.46:0/3287455179 >> 172.21.15.46:0/3287455179 conn(0x7f554406a4d0 msgr2=0x7f554406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:40.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.313+0000 7f5549948640 1 -- 172.21.15.46:0/3287455179 shutdown_connections 2023-12-02T00:30:40.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.313+0000 7f5549948640 1 -- 172.21.15.46:0/3287455179 wait complete. 2023-12-02T00:30:40.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.314+0000 7f5549948640 1 Processor -- start 2023-12-02T00:30:40.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.314+0000 7f5549948640 1 -- start start 2023-12-02T00:30:40.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.315+0000 7f5549948640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5544143490 0x7f55440ade30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:40.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.315+0000 7f5549948640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f554415a4e0 0x7f554413e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:40.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.315+0000 7f5549948640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f554415e730 0x7f554415c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:40.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.315+0000 7f5549948640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55441f1400 con 0x7f5544143490 2023-12-02T00:30:40.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.315+0000 7f5549948640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55441f1690 con 0x7f554415e730 2023-12-02T00:30:40.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.315+0000 7f5549948640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55441f1970 con 0x7f554415a4e0 2023-12-02T00:30:40.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.315+0000 7f5542ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5544143490 0x7f55440ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:40.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.315+0000 7f5542ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5544143490 0x7f55440ade30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52852/0 (socket says 172.21.15.46:52852) 2023-12-02T00:30:40.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.315+0000 7f5542ffd640 1 -- 172.21.15.46:0/994735194 learned_addr learned my addr 172.21.15.46:0/994735194 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:40.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.315+0000 7f553bfff640 1 --2- 172.21.15.46:0/994735194 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f554415a4e0 0x7f554413e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:40.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.315+0000 7f55437fe640 1 --2- 172.21.15.46:0/994735194 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f554415e730 0x7f554415c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:40.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.315+0000 7f5542ffd640 1 -- 172.21.15.46:0/994735194 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f554415a4e0 msgr2=0x7f554413e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:40.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.315+0000 7f5542ffd640 1 --2- 172.21.15.46:0/994735194 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f554415a4e0 0x7f554413e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:40.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.315+0000 7f5542ffd640 1 -- 172.21.15.46:0/994735194 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f554415e730 msgr2=0x7f554415c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:40.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.315+0000 7f5542ffd640 1 --2- 172.21.15.46:0/994735194 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f554415e730 0x7f554415c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:40.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.315+0000 7f5542ffd640 1 -- 172.21.15.46:0/994735194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55340094d0 con 0x7f5544143490 2023-12-02T00:30:40.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.315+0000 7f553bfff640 1 --2- 172.21.15.46:0/994735194 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f554415a4e0 0x7f554413e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:30:40.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.315+0000 7f55437fe640 1 --2- 172.21.15.46:0/994735194 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f554415e730 0x7f554415c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:30:40.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.315+0000 7f5542ffd640 1 --2- 172.21.15.46:0/994735194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5544143490 0x7f55440ade30 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f5534000c00 tx=0x7f55340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:40.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.315+0000 7f5540ff9640 1 -- 172.21.15.46:0/994735194 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5534031840 con 0x7f5544143490 2023-12-02T00:30:40.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.315+0000 7f5540ff9640 1 -- 172.21.15.46:0/994735194 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55340443f0 con 0x7f5544143490 2023-12-02T00:30:40.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.316+0000 7f5549948640 1 -- 172.21.15.46:0/994735194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55441f1c50 con 0x7f5544143490 2023-12-02T00:30:40.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.316+0000 7f5540ff9640 1 -- 172.21.15.46:0/994735194 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55340447a0 con 0x7f5544143490 2023-12-02T00:30:40.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.316+0000 7f5549948640 1 -- 172.21.15.46:0/994735194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f554415c750 con 0x7f5544143490 2023-12-02T00:30:40.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.316+0000 7f5549948640 1 -- 172.21.15.46:0/994735194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5544069400 con 0x7f5544143490 2023-12-02T00:30:40.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.317+0000 7f5540ff9640 1 -- 172.21.15.46:0/994735194 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f553404d450 con 0x7f5544143490 2023-12-02T00:30:40.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.317+0000 7f5540ff9640 1 --2- 172.21.15.46:0/994735194 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f552406ae40 0x7f552406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:40.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.318+0000 7f5540ff9640 1 -- 172.21.15.46:0/994735194 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f553402d040 con 0x7f5544143490 2023-12-02T00:30:40.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.318+0000 7f553bfff640 1 --2- 172.21.15.46:0/994735194 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f552406ae40 0x7f552406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:40.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.318+0000 7f553bfff640 1 --2- 172.21.15.46:0/994735194 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f552406ae40 0x7f552406d300 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f552c006fd0 tx=0x7f552c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:40.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.322+0000 7f5540ff9640 1 -- 172.21.15.46:0/994735194 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f55340af030 con 0x7f5544143490 2023-12-02T00:30:40.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.499+0000 7f5549948640 1 -- 172.21.15.46:0/994735194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f55440b2820 con 0x7f5544143490 2023-12-02T00:30:40.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.500+0000 7f5540ff9640 1 -- 172.21.15.46:0/994735194 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+18324 (secure 0 0 0) 0x7f5534076ec0 con 0x7f5544143490 2023-12-02T00:30:40.501 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:30:40.502 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":38,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:30:10.714057+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-02T00:29:55.676572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-02T00:29:56.244869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"data_new","create_time":"2023-12-02T00:30:08.897990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:30:40.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.502+0000 7f5549948640 1 -- 172.21.15.46:0/994735194 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f552406ae40 msgr2=0x7f552406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:40.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.502+0000 7f5549948640 1 --2- 172.21.15.46:0/994735194 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f552406ae40 0x7f552406d300 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f552c006fd0 tx=0x7f552c008040 comp rx=0 tx=0).stop 2023-12-02T00:30:40.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.502+0000 7f5549948640 1 -- 172.21.15.46:0/994735194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5544143490 msgr2=0x7f55440ade30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:40.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.502+0000 7f5549948640 1 --2- 172.21.15.46:0/994735194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5544143490 0x7f55440ade30 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f5534000c00 tx=0x7f55340046c0 comp rx=0 tx=0).stop 2023-12-02T00:30:40.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.502+0000 7f5549948640 1 -- 172.21.15.46:0/994735194 shutdown_connections 2023-12-02T00:30:40.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.502+0000 7f5549948640 1 --2- 172.21.15.46:0/994735194 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f552406ae40 0x7f552406d300 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:40.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.502+0000 7f5549948640 1 --2- 172.21.15.46:0/994735194 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f554415e730 0x7f554415c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:40.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.502+0000 7f5549948640 1 --2- 172.21.15.46:0/994735194 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f554415a4e0 0x7f554413e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:40.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.502+0000 7f5549948640 1 --2- 172.21.15.46:0/994735194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5544143490 0x7f55440ade30 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:40.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.502+0000 7f5549948640 1 -- 172.21.15.46:0/994735194 >> 172.21.15.46:0/994735194 conn(0x7f554406a4d0 msgr2=0x7f554415cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:40.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.502+0000 7f5549948640 1 -- 172.21.15.46:0/994735194 shutdown_connections 2023-12-02T00:30:40.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.502+0000 7f5549948640 1 -- 172.21.15.46:0/994735194 wait complete. 2023-12-02T00:30:40.515 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-02T00:28:37.213645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-02T00:29:55.676572+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-12-02T00:29:56.244869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'data_new', 'create_time': '2023-12-02T00:30:08.897990+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-02T00:30:40.515 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-02T00:30:40.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.623+0000 7f8b50bb0640 1 Processor -- start 2023-12-02T00:30:40.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.623+0000 7f8b50bb0640 1 -- start start 2023-12-02T00:30:40.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.623+0000 7f8b50bb0640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8b4c15d2d0 0x7f8b4c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:40.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.623+0000 7f8b50bb0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b4c0ce650 0x7f8b4c0d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:40.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.624+0000 7f8b50bb0640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b4c1614e0 con 0x7f8b4c160f70 2023-12-02T00:30:40.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.624+0000 7f8b50bb0640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b4c1617c0 con 0x7f8b4c15d2d0 2023-12-02T00:30:40.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.624+0000 7f8b50bb0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4c161aa0 con 0x7f8b4c0ce650 2023-12-02T00:30:40.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.624+0000 7f8b4a59c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8b4c15d2d0 0x7f8b4c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:40.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.624+0000 7f8b43fff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8b4c160f70 0x7f8b4c0cdf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49238/0 (socket says 172.21.15.46:49238) 2023-12-02T00:30:40.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.624+0000 7f8b43fff640 1 -- 172.21.15.46:0/2143619562 learned_addr learned my addr 172.21.15.46:0/2143619562 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:30:40.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.624+0000 7f8b4ad9d640 1 --2- 172.21.15.46:0/2143619562 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b4c0ce650 0x7f8b4c0d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:40.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.624+0000 7f8b4a59c640 1 -- 172.21.15.46:0/2143619562 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b4c0ce650 msgr2=0x7f8b4c0d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:40.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.624+0000 7f8b4a59c640 1 --2- 172.21.15.46:0/2143619562 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b4c0ce650 0x7f8b4c0d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:40.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.624+0000 7f8b4a59c640 1 -- 172.21.15.46:0/2143619562 >> v1:172.21.15.46:6789/0 conn(0x7f8b4c160f70 legacy=0x7f8b4c0cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:40.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.624+0000 7f8b4a59c640 1 -- 172.21.15.46:0/2143619562 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b4c161d80 con 0x7f8b4c15d2d0 2023-12-02T00:30:40.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.624+0000 7f8b49d9b640 1 -- 172.21.15.46:0/2143619562 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 729527340 0 0) 0x7f8b4c1614e0 con 0x7f8b4c160f70 2023-12-02T00:30:40.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.625+0000 7f8b4a59c640 1 --2- 172.21.15.46:0/2143619562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b4c15d2d0 0x7f8b4c0b8b90 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c0097a0 tx=0x7f8b3c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34b0dbba3b90acc2 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:40.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.625+0000 7f8b49d9b640 1 -- 172.21.15.46:0/2143619562 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c031830 con 0x7f8b4c15d2d0 2023-12-02T00:30:40.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.625+0000 7f8b49d9b640 1 -- 172.21.15.46:0/2143619562 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b3c031990 con 0x7f8b4c15d2d0 2023-12-02T00:30:40.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.625+0000 7f8b49d9b640 1 -- 172.21.15.46:0/2143619562 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c031d20 con 0x7f8b4c15d2d0 2023-12-02T00:30:40.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.625+0000 7f8b50bb0640 1 -- 172.21.15.46:0/2143619562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b4c15d2d0 msgr2=0x7f8b4c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:40.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.625+0000 7f8b50bb0640 1 --2- 172.21.15.46:0/2143619562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b4c15d2d0 0x7f8b4c0b8b90 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c0097a0 tx=0x7f8b3c02b8d0 comp rx=0 tx=0).stop 2023-12-02T00:30:40.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.625+0000 7f8b50bb0640 1 -- 172.21.15.46:0/2143619562 shutdown_connections 2023-12-02T00:30:40.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.625+0000 7f8b50bb0640 1 --2- 172.21.15.46:0/2143619562 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b4c0ce650 0x7f8b4c0d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:40.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.625+0000 7f8b50bb0640 1 --2- 172.21.15.46:0/2143619562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b4c15d2d0 0x7f8b4c0b8b90 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:40.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.625+0000 7f8b50bb0640 1 -- 172.21.15.46:0/2143619562 >> 172.21.15.46:0/2143619562 conn(0x7f8b4c072230 msgr2=0x7f8b4c0cfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:40.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.625+0000 7f8b50bb0640 1 -- 172.21.15.46:0/2143619562 shutdown_connections 2023-12-02T00:30:40.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.625+0000 7f8b50bb0640 1 -- 172.21.15.46:0/2143619562 wait complete. 2023-12-02T00:30:40.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.627+0000 7f8b50bb0640 1 Processor -- start 2023-12-02T00:30:40.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.627+0000 7f8b50bb0640 1 -- start start 2023-12-02T00:30:40.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.627+0000 7f8b50bb0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b4c0ce650 0x7f8b4c159b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:40.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.627+0000 7f8b50bb0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b4c15d2d0 0x7f8b4c154dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:40.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.627+0000 7f8b50bb0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b4c160f70 0x7f8b4c1f2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:40.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.627+0000 7f8b50bb0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4c0b2b60 con 0x7f8b4c160f70 2023-12-02T00:30:40.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.628+0000 7f8b50bb0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4c0b39c0 con 0x7f8b4c0ce650 2023-12-02T00:30:40.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.628+0000 7f8b50bb0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4c0b2db0 con 0x7f8b4c15d2d0 2023-12-02T00:30:40.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.628+0000 7f8b4ad9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b4c160f70 0x7f8b4c1f2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:40.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.628+0000 7f8b4ad9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b4c160f70 0x7f8b4c1f2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52880/0 (socket says 172.21.15.46:52880) 2023-12-02T00:30:40.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.628+0000 7f8b4ad9d640 1 -- 172.21.15.46:0/1163442313 learned_addr learned my addr 172.21.15.46:0/1163442313 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:40.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.628+0000 7f8b4a59c640 1 --2- 172.21.15.46:0/1163442313 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b4c0ce650 0x7f8b4c159b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:40.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.628+0000 7f8b43fff640 1 --2- 172.21.15.46:0/1163442313 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b4c15d2d0 0x7f8b4c154dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:40.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.628+0000 7f8b4ad9d640 1 -- 172.21.15.46:0/1163442313 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b4c15d2d0 msgr2=0x7f8b4c154dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:40.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.628+0000 7f8b4ad9d640 1 --2- 172.21.15.46:0/1163442313 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b4c15d2d0 0x7f8b4c154dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:40.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.628+0000 7f8b4ad9d640 1 -- 172.21.15.46:0/1163442313 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b4c0ce650 msgr2=0x7f8b4c159b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:40.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.628+0000 7f8b4ad9d640 1 --2- 172.21.15.46:0/1163442313 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b4c0ce650 0x7f8b4c159b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:40.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.628+0000 7f8b4ad9d640 1 -- 172.21.15.46:0/1163442313 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b3c009400 con 0x7f8b4c160f70 2023-12-02T00:30:40.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.628+0000 7f8b43fff640 1 --2- 172.21.15.46:0/1163442313 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b4c15d2d0 0x7f8b4c154dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:30:40.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.628+0000 7f8b4ad9d640 1 --2- 172.21.15.46:0/1163442313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b4c160f70 0x7f8b4c1f2ea0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f8b44007e40 tx=0x7f8b4400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:40.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.628+0000 7f8b42ffd640 1 -- 172.21.15.46:0/1163442313 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b44014620 con 0x7f8b4c160f70 2023-12-02T00:30:40.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.628+0000 7f8b42ffd640 1 -- 172.21.15.46:0/1163442313 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b4400ee90 con 0x7f8b4c160f70 2023-12-02T00:30:40.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.628+0000 7f8b42ffd640 1 -- 172.21.15.46:0/1163442313 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b440149e0 con 0x7f8b4c160f70 2023-12-02T00:30:40.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.628+0000 7f8b50bb0640 1 -- 172.21.15.46:0/1163442313 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b4c0b3090 con 0x7f8b4c160f70 2023-12-02T00:30:40.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.628+0000 7f8b50bb0640 1 -- 172.21.15.46:0/1163442313 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b4c0b3650 con 0x7f8b4c160f70 2023-12-02T00:30:40.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.629+0000 7f8b50bb0640 1 -- 172.21.15.46:0/1163442313 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b4c0b3b30 con 0x7f8b4c160f70 2023-12-02T00:30:40.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.630+0000 7f8b42ffd640 1 -- 172.21.15.46:0/1163442313 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8b44014b40 con 0x7f8b4c160f70 2023-12-02T00:30:40.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.630+0000 7f8b42ffd640 1 --2- 172.21.15.46:0/1163442313 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8b2c06ae40 0x7f8b2c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:40.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.630+0000 7f8b42ffd640 1 -- 172.21.15.46:0/1163442313 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f8b4408a010 con 0x7f8b4c160f70 2023-12-02T00:30:40.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.631+0000 7f8b4a59c640 1 --2- 172.21.15.46:0/1163442313 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8b2c06ae40 0x7f8b2c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:40.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.631+0000 7f8b4a59c640 1 --2- 172.21.15.46:0/1163442313 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8b2c06ae40 0x7f8b2c06d300 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c036000 tx=0x7f8b3c03c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:40.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.635+0000 7f8b42ffd640 1 -- 172.21.15.46:0/1163442313 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8b44054fc0 con 0x7f8b4c160f70 2023-12-02T00:30:40.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.810+0000 7f8b50bb0640 1 -- 172.21.15.46:0/1163442313 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8b4c069400 con 0x7f8b4c160f70 2023-12-02T00:30:40.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.810+0000 7f8b42ffd640 1 -- 172.21.15.46:0/1163442313 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8b440585e0 con 0x7f8b4c160f70 2023-12-02T00:30:40.812 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-12-02T00:30:40.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.812+0000 7f8b50bb0640 1 -- 172.21.15.46:0/1163442313 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8b2c06ae40 msgr2=0x7f8b2c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:40.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.812+0000 7f8b50bb0640 1 --2- 172.21.15.46:0/1163442313 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8b2c06ae40 0x7f8b2c06d300 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c036000 tx=0x7f8b3c03c040 comp rx=0 tx=0).stop 2023-12-02T00:30:40.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.812+0000 7f8b50bb0640 1 -- 172.21.15.46:0/1163442313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b4c160f70 msgr2=0x7f8b4c1f2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:40.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.812+0000 7f8b50bb0640 1 --2- 172.21.15.46:0/1163442313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b4c160f70 0x7f8b4c1f2ea0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f8b44007e40 tx=0x7f8b4400e4b0 comp rx=0 tx=0).stop 2023-12-02T00:30:40.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.812+0000 7f8b50bb0640 1 -- 172.21.15.46:0/1163442313 shutdown_connections 2023-12-02T00:30:40.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.812+0000 7f8b50bb0640 1 --2- 172.21.15.46:0/1163442313 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8b2c06ae40 0x7f8b2c06d300 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:40.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.812+0000 7f8b50bb0640 1 --2- 172.21.15.46:0/1163442313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b4c160f70 0x7f8b4c1f2ea0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:40.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.812+0000 7f8b50bb0640 1 --2- 172.21.15.46:0/1163442313 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b4c15d2d0 0x7f8b4c154dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:40.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.812+0000 7f8b50bb0640 1 --2- 172.21.15.46:0/1163442313 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b4c0ce650 0x7f8b4c159b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:40.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.812+0000 7f8b50bb0640 1 -- 172.21.15.46:0/1163442313 >> 172.21.15.46:0/1163442313 conn(0x7f8b4c072230 msgr2=0x7f8b4c15dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:40.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.812+0000 7f8b50bb0640 1 -- 172.21.15.46:0/1163442313 shutdown_connections 2023-12-02T00:30:40.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.812+0000 7f8b50bb0640 1 -- 172.21.15.46:0/1163442313 wait complete. 2023-12-02T00:30:40.825 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-02T00:30:40.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.933+0000 7f2fd4e56640 1 Processor -- start 2023-12-02T00:30:40.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.933+0000 7f2fd4e56640 1 -- start start 2023-12-02T00:30:40.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.934+0000 7f2fd4e56640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2fd006ea50 0x7f2fd006ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:40.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.934+0000 7f2fd4e56640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2fd0160fb0 0x7f2fd0161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:40.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.934+0000 7f2fd4e56640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fd0161d20 con 0x7f2fd00b3ff0 2023-12-02T00:30:40.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.934+0000 7f2fd4e56640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fd00d1ee0 con 0x7f2fd0160fb0 2023-12-02T00:30:40.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.934+0000 7f2fd4e56640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd006f370 con 0x7f2fd006ea50 2023-12-02T00:30:40.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.934+0000 7f2fc7fff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2fd0160fb0 0x7f2fd0161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:40.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.934+0000 7f2fc7fff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2fd0160fb0 0x7f2fd0161410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52896/0 (socket says 172.21.15.46:52896) 2023-12-02T00:30:40.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.934+0000 7f2fc7fff640 1 -- 172.21.15.46:0/2100589450 learned_addr learned my addr 172.21.15.46:0/2100589450 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:40.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.934+0000 7f2fce59c640 1 --2- 172.21.15.46:0/2100589450 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2fd006ea50 0x7f2fd006ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:40.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.934+0000 7f2fc7fff640 1 -- 172.21.15.46:0/2100589450 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2fd006ea50 msgr2=0x7f2fd006ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:40.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.934+0000 7f2fc7fff640 1 --2- 172.21.15.46:0/2100589450 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2fd006ea50 0x7f2fd006ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:40.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.934+0000 7f2fc7fff640 1 -- 172.21.15.46:0/2100589450 >> v1:172.21.15.162:6789/0 conn(0x7f2fd00b3ff0 legacy=0x7f2fd00b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:40.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.934+0000 7f2fc7fff640 1 -- 172.21.15.46:0/2100589450 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd015e760 con 0x7f2fd0160fb0 2023-12-02T00:30:40.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.935+0000 7f2fc7fff640 1 --2- 172.21.15.46:0/2100589450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fd0160fb0 0x7f2fd0161410 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8009870 tx=0x7f2fb802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de1c2c689eeb42 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:40.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.935+0000 7f2fcdd9b640 1 -- 172.21.15.46:0/2100589450 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb8031640 con 0x7f2fd0160fb0 2023-12-02T00:30:40.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.935+0000 7f2fcdd9b640 1 -- 172.21.15.46:0/2100589450 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fb80317a0 con 0x7f2fd0160fb0 2023-12-02T00:30:40.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.935+0000 7f2fcdd9b640 1 -- 172.21.15.46:0/2100589450 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb8031b30 con 0x7f2fd0160fb0 2023-12-02T00:30:40.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.936+0000 7f2fd4e56640 1 -- 172.21.15.46:0/2100589450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fd0160fb0 msgr2=0x7f2fd0161410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:40.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.936+0000 7f2fd4e56640 1 --2- 172.21.15.46:0/2100589450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fd0160fb0 0x7f2fd0161410 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8009870 tx=0x7f2fb802b8f0 comp rx=0 tx=0).stop 2023-12-02T00:30:40.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.936+0000 7f2fd4e56640 1 -- 172.21.15.46:0/2100589450 shutdown_connections 2023-12-02T00:30:40.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.936+0000 7f2fd4e56640 1 --2- 172.21.15.46:0/2100589450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fd0160fb0 0x7f2fd0161410 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:40.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.936+0000 7f2fd4e56640 1 --2- 172.21.15.46:0/2100589450 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2fd006ea50 0x7f2fd006ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:40.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.936+0000 7f2fd4e56640 1 -- 172.21.15.46:0/2100589450 >> 172.21.15.46:0/2100589450 conn(0x7f2fd006a6b0 msgr2=0x7f2fd006aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:40.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.936+0000 7f2fd4e56640 1 -- 172.21.15.46:0/2100589450 shutdown_connections 2023-12-02T00:30:40.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.936+0000 7f2fd4e56640 1 -- 172.21.15.46:0/2100589450 wait complete. 2023-12-02T00:30:40.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.937+0000 7f2fd4e56640 1 Processor -- start 2023-12-02T00:30:40.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.937+0000 7f2fd4e56640 1 -- start start 2023-12-02T00:30:40.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.938+0000 7f2fd4e56640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2fd006ea50 0x7f2fd006c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:40.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.938+0000 7f2fd4e56640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fd00b3ff0 0x7f2fd0156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:40.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.938+0000 7f2fd4e56640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2fd0160fb0 0x7f2fd015c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:40.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.938+0000 7f2fd4e56640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd006cc40 con 0x7f2fd00b3ff0 2023-12-02T00:30:40.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.938+0000 7f2fd4e56640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd006b680 con 0x7f2fd0160fb0 2023-12-02T00:30:40.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.938+0000 7f2fd4e56640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd006b960 con 0x7f2fd006ea50 2023-12-02T00:30:40.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.938+0000 7f2fc7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fd00b3ff0 0x7f2fd0156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:40.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.938+0000 7f2fc7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fd00b3ff0 0x7f2fd0156b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52910/0 (socket says 172.21.15.46:52910) 2023-12-02T00:30:40.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.938+0000 7f2fc7fff640 1 -- 172.21.15.46:0/1884417727 learned_addr learned my addr 172.21.15.46:0/1884417727 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:40.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.938+0000 7f2fced9d640 1 --2- 172.21.15.46:0/1884417727 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2fd0160fb0 0x7f2fd015c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:40.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.938+0000 7f2fce59c640 1 --2- 172.21.15.46:0/1884417727 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2fd006ea50 0x7f2fd006c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:40.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.938+0000 7f2fc7fff640 1 -- 172.21.15.46:0/1884417727 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2fd006ea50 msgr2=0x7f2fd006c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:40.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.938+0000 7f2fc7fff640 1 --2- 172.21.15.46:0/1884417727 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2fd006ea50 0x7f2fd006c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:40.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.938+0000 7f2fc7fff640 1 -- 172.21.15.46:0/1884417727 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2fd0160fb0 msgr2=0x7f2fd015c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:40.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.938+0000 7f2fc7fff640 1 --2- 172.21.15.46:0/1884417727 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2fd0160fb0 0x7f2fd015c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:40.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.938+0000 7f2fc7fff640 1 -- 172.21.15.46:0/1884417727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb80094d0 con 0x7f2fd00b3ff0 2023-12-02T00:30:40.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.939+0000 7f2fc7fff640 1 --2- 172.21.15.46:0/1884417727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fd00b3ff0 0x7f2fd0156b00 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f2fb802b930 tx=0x7f2fb80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:40.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.939+0000 7f2fc6ffd640 1 -- 172.21.15.46:0/1884417727 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb803bc70 con 0x7f2fd00b3ff0 2023-12-02T00:30:40.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.939+0000 7f2fc6ffd640 1 -- 172.21.15.46:0/1884417727 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fb803bdf0 con 0x7f2fd00b3ff0 2023-12-02T00:30:40.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.939+0000 7f2fd4e56640 1 -- 172.21.15.46:0/1884417727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fd006bc40 con 0x7f2fd00b3ff0 2023-12-02T00:30:40.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.939+0000 7f2fc6ffd640 1 -- 172.21.15.46:0/1884417727 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb8045630 con 0x7f2fd00b3ff0 2023-12-02T00:30:40.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.939+0000 7f2fd4e56640 1 -- 172.21.15.46:0/1884417727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fd006c200 con 0x7f2fd00b3ff0 2023-12-02T00:30:40.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.940+0000 7f2fd4e56640 1 -- 172.21.15.46:0/1884417727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fd0069400 con 0x7f2fd00b3ff0 2023-12-02T00:30:40.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.940+0000 7f2fc6ffd640 1 -- 172.21.15.46:0/1884417727 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2fb802f050 con 0x7f2fd00b3ff0 2023-12-02T00:30:40.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.941+0000 7f2fc6ffd640 1 --2- 172.21.15.46:0/1884417727 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2fb006ae40 0x7f2fb006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:40.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.941+0000 7f2fc6ffd640 1 -- 172.21.15.46:0/1884417727 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f2fb80ab530 con 0x7f2fd00b3ff0 2023-12-02T00:30:40.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.941+0000 7f2fce59c640 1 --2- 172.21.15.46:0/1884417727 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2fb006ae40 0x7f2fb006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:40.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.941+0000 7f2fce59c640 1 --2- 172.21.15.46:0/1884417727 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2fb006ae40 0x7f2fb006d300 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2fc00079a0 tx=0x7f2fc0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:40.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:40.945+0000 7f2fc6ffd640 1 -- 172.21.15.46:0/1884417727 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2fb80764f0 con 0x7f2fd00b3ff0 2023-12-02T00:30:41.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.121+0000 7f2fd4e56640 1 -- 172.21.15.46:0/1884417727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2fd006ab30 con 0x7f2fd00b3ff0 2023-12-02T00:30:41.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.122+0000 7f2fc6ffd640 1 -- 172.21.15.46:0/1884417727 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v38) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2fb8079b10 con 0x7f2fd00b3ff0 2023-12-02T00:30:41.123 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-12-02T00:30:41.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.123+0000 7f2fd4e56640 1 -- 172.21.15.46:0/1884417727 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2fb006ae40 msgr2=0x7f2fb006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:41.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.124+0000 7f2fd4e56640 1 --2- 172.21.15.46:0/1884417727 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2fb006ae40 0x7f2fb006d300 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2fc00079a0 tx=0x7f2fc0008040 comp rx=0 tx=0).stop 2023-12-02T00:30:41.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.124+0000 7f2fd4e56640 1 -- 172.21.15.46:0/1884417727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fd00b3ff0 msgr2=0x7f2fd0156b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:41.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.124+0000 7f2fd4e56640 1 --2- 172.21.15.46:0/1884417727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fd00b3ff0 0x7f2fd0156b00 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f2fb802b930 tx=0x7f2fb80046c0 comp rx=0 tx=0).stop 2023-12-02T00:30:41.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.124+0000 7f2fd4e56640 1 -- 172.21.15.46:0/1884417727 shutdown_connections 2023-12-02T00:30:41.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.124+0000 7f2fd4e56640 1 --2- 172.21.15.46:0/1884417727 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2fb006ae40 0x7f2fb006d300 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:41.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.124+0000 7f2fd4e56640 1 --2- 172.21.15.46:0/1884417727 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2fd0160fb0 0x7f2fd015c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:41.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.124+0000 7f2fd4e56640 1 --2- 172.21.15.46:0/1884417727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fd00b3ff0 0x7f2fd0156b00 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:41.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.124+0000 7f2fd4e56640 1 --2- 172.21.15.46:0/1884417727 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2fd006ea50 0x7f2fd006c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:41.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.124+0000 7f2fd4e56640 1 -- 172.21.15.46:0/1884417727 >> 172.21.15.46:0/1884417727 conn(0x7f2fd006a6b0 msgr2=0x7f2fd00b4440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:41.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.124+0000 7f2fd4e56640 1 -- 172.21.15.46:0/1884417727 shutdown_connections 2023-12-02T00:30:41.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.124+0000 7f2fd4e56640 1 -- 172.21.15.46:0/1884417727 wait complete. 2023-12-02T00:30:41.136 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-02T00:30:41.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.244+0000 7f146f30f640 1 Processor -- start 2023-12-02T00:30:41.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.244+0000 7f146f30f640 1 -- start start 2023-12-02T00:30:41.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.245+0000 7f146f30f640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1468143490 0x7f14680b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:41.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.245+0000 7f146f30f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f146815e730 0x7f14680b1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:41.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.245+0000 7f146f30f640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f146815e2a0 con 0x7f146815a4e0 2023-12-02T00:30:41.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.245+0000 7f146f30f640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f146815b520 con 0x7f1468143490 2023-12-02T00:30:41.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.245+0000 7f146f30f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1468143870 con 0x7f146815e730 2023-12-02T00:30:41.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.245+0000 7f146d8ac640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f146815a4e0 0x7f146815a970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49252/0 (socket says 172.21.15.46:49252) 2023-12-02T00:30:41.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.245+0000 7f146d8ac640 1 -- 172.21.15.46:0/1795887162 learned_addr learned my addr 172.21.15.46:0/1795887162 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:30:41.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.245+0000 7f146d0ab640 1 --2- 172.21.15.46:0/1795887162 >> v2:172.21.15.162:3300/0 conn(0x7f1468143490 0x7f14680b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:41.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.245+0000 7f146c8aa640 1 --2- 172.21.15.46:0/1795887162 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f146815e730 0x7f14680b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:41.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.245+0000 7f1467fff640 1 -- 172.21.15.46:0/1795887162 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3875430070 0 0) 0x7f146815e2a0 con 0x7f146815a4e0 2023-12-02T00:30:41.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.245+0000 7f1467fff640 1 -- 172.21.15.46:0/1795887162 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1450005730 con 0x7f146815a4e0 2023-12-02T00:30:41.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.246+0000 7f146d0ab640 1 -- 172.21.15.46:0/1795887162 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f146815e730 msgr2=0x7f14680b1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:41.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.246+0000 7f1467fff640 1 -- 172.21.15.46:0/1795887162 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3347720109 0 0) 0x7f1450005730 con 0x7f146815a4e0 2023-12-02T00:30:41.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.246+0000 7f146d0ab640 1 --2- 172.21.15.46:0/1795887162 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f146815e730 0x7f14680b1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:41.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.246+0000 7f146d0ab640 1 -- 172.21.15.46:0/1795887162 >> v1:172.21.15.46:6789/0 conn(0x7f146815a4e0 legacy=0x7f146815a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:41.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.246+0000 7f146d0ab640 1 -- 172.21.15.46:0/1795887162 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f146815b240 con 0x7f1468143490 2023-12-02T00:30:41.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.246+0000 7f146c8aa640 1 --2- 172.21.15.46:0/1795887162 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f146815e730 0x7f14680b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:30:41.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.246+0000 7f146d0ab640 1 --2- 172.21.15.46:0/1795887162 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1468143490 0x7f14680b0cf0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1460009d90 tx=0x7f146002bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c55d9e19f4fa5c74 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:41.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.247+0000 7f1467fff640 1 -- 172.21.15.46:0/1795887162 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1458001670 con 0x7f1468143490 2023-12-02T00:30:41.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.247+0000 7f1467fff640 1 -- 172.21.15.46:0/1795887162 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f146002c8a0 con 0x7f1468143490 2023-12-02T00:30:41.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.247+0000 7f1467fff640 1 -- 172.21.15.46:0/1795887162 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146002cc10 con 0x7f1468143490 2023-12-02T00:30:41.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.247+0000 7f146f30f640 1 -- 172.21.15.46:0/1795887162 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1468143490 msgr2=0x7f14680b0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:41.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.247+0000 7f146f30f640 1 --2- 172.21.15.46:0/1795887162 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1468143490 0x7f14680b0cf0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1460009d90 tx=0x7f146002bd70 comp rx=0 tx=0).stop 2023-12-02T00:30:41.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.247+0000 7f146f30f640 1 -- 172.21.15.46:0/1795887162 shutdown_connections 2023-12-02T00:30:41.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.247+0000 7f146f30f640 1 --2- 172.21.15.46:0/1795887162 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f146815e730 0x7f14680b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:41.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.247+0000 7f146f30f640 1 --2- 172.21.15.46:0/1795887162 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1468143490 0x7f14680b0cf0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:41.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.247+0000 7f146f30f640 1 -- 172.21.15.46:0/1795887162 >> 172.21.15.46:0/1795887162 conn(0x7f146806a4d0 msgr2=0x7f146806a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:41.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.247+0000 7f146f30f640 1 -- 172.21.15.46:0/1795887162 shutdown_connections 2023-12-02T00:30:41.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.247+0000 7f146f30f640 1 -- 172.21.15.46:0/1795887162 wait complete. 2023-12-02T00:30:41.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.248+0000 7f146f30f640 1 Processor -- start 2023-12-02T00:30:41.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.249+0000 7f146f30f640 1 -- start start 2023-12-02T00:30:41.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.249+0000 7f146f30f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1468143490 0x7f1468140d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:41.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.249+0000 7f146f30f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f146815a4e0 0x7f1468141510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:41.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.249+0000 7f146f30f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f146815e730 0x7f1468158d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:41.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.249+0000 7f146f30f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1468140030 con 0x7f146815e730 2023-12-02T00:30:41.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.249+0000 7f146f30f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146813d620 con 0x7f146815a4e0 2023-12-02T00:30:41.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.249+0000 7f146f30f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1468140280 con 0x7f1468143490 2023-12-02T00:30:41.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.249+0000 7f146d8ac640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f146815e730 0x7f1468158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:41.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.249+0000 7f146d8ac640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f146815e730 0x7f1468158d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52920/0 (socket says 172.21.15.46:52920) 2023-12-02T00:30:41.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.249+0000 7f146d8ac640 1 -- 172.21.15.46:0/727749187 learned_addr learned my addr 172.21.15.46:0/727749187 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:41.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.250+0000 7f146d0ab640 1 --2- 172.21.15.46:0/727749187 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1468143490 0x7f1468140d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:41.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.250+0000 7f146c8aa640 1 --2- 172.21.15.46:0/727749187 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f146815a4e0 0x7f1468141510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:41.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.250+0000 7f146d8ac640 1 -- 172.21.15.46:0/727749187 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1468143490 msgr2=0x7f1468140d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:41.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.250+0000 7f146d8ac640 1 --2- 172.21.15.46:0/727749187 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1468143490 0x7f1468140d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:41.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.250+0000 7f146d8ac640 1 -- 172.21.15.46:0/727749187 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f146815a4e0 msgr2=0x7f1468141510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:41.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.250+0000 7f146d8ac640 1 --2- 172.21.15.46:0/727749187 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f146815a4e0 0x7f1468141510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:41.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.250+0000 7f146d8ac640 1 -- 172.21.15.46:0/727749187 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f145c009400 con 0x7f146815e730 2023-12-02T00:30:41.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.250+0000 7f146d8ac640 1 --2- 172.21.15.46:0/727749187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f146815e730 0x7f1468158d00 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f14580017a0 tx=0x7f145800cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:41.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.250+0000 7f1465ffb640 1 -- 172.21.15.46:0/727749187 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1458005ad0 con 0x7f146815e730 2023-12-02T00:30:41.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.250+0000 7f1465ffb640 1 -- 172.21.15.46:0/727749187 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1458005c30 con 0x7f146815e730 2023-12-02T00:30:41.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.250+0000 7f146f30f640 1 -- 172.21.15.46:0/727749187 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1460009a70 con 0x7f146815e730 2023-12-02T00:30:41.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.250+0000 7f1465ffb640 1 -- 172.21.15.46:0/727749187 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1458007070 con 0x7f146815e730 2023-12-02T00:30:41.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.250+0000 7f146f30f640 1 -- 172.21.15.46:0/727749187 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1468140840 con 0x7f146815e730 2023-12-02T00:30:41.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.251+0000 7f146f30f640 1 -- 172.21.15.46:0/727749187 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1468069400 con 0x7f146815e730 2023-12-02T00:30:41.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.252+0000 7f1465ffb640 1 -- 172.21.15.46:0/727749187 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f14580071d0 con 0x7f146815e730 2023-12-02T00:30:41.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.252+0000 7f1465ffb640 1 --2- 172.21.15.46:0/727749187 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f145406ae40 0x7f145406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:41.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.252+0000 7f1465ffb640 1 -- 172.21.15.46:0/727749187 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f1458089480 con 0x7f146815e730 2023-12-02T00:30:41.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.252+0000 7f146d0ab640 1 --2- 172.21.15.46:0/727749187 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f145406ae40 0x7f145406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:41.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.253+0000 7f146d0ab640 1 --2- 172.21.15.46:0/727749187 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f145406ae40 0x7f145406d300 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1460036000 tx=0x7f1460030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:41.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.256+0000 7f1465ffb640 1 -- 172.21.15.46:0/727749187 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f14580543b0 con 0x7f146815e730 2023-12-02T00:30:41.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.430+0000 7f146f30f640 1 -- 172.21.15.46:0/727749187 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f14680b2820 con 0x7f146815e730 2023-12-02T00:30:41.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.431+0000 7f1465ffb640 1 -- 172.21.15.46:0/727749187 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v38) v1 ==== 100+0+11 (secure 0 0 0) 0x7f14580579d0 con 0x7f146815e730 2023-12-02T00:30:41.432 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-12-02T00:30:41.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.432+0000 7f146f30f640 1 -- 172.21.15.46:0/727749187 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f145406ae40 msgr2=0x7f145406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:41.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.432+0000 7f146f30f640 1 --2- 172.21.15.46:0/727749187 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f145406ae40 0x7f145406d300 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1460036000 tx=0x7f1460030000 comp rx=0 tx=0).stop 2023-12-02T00:30:41.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.432+0000 7f146f30f640 1 -- 172.21.15.46:0/727749187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f146815e730 msgr2=0x7f1468158d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:41.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.432+0000 7f146f30f640 1 --2- 172.21.15.46:0/727749187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f146815e730 0x7f1468158d00 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f14580017a0 tx=0x7f145800cbe0 comp rx=0 tx=0).stop 2023-12-02T00:30:41.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.432+0000 7f146f30f640 1 -- 172.21.15.46:0/727749187 shutdown_connections 2023-12-02T00:30:41.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.432+0000 7f146f30f640 1 --2- 172.21.15.46:0/727749187 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f145406ae40 0x7f145406d300 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:41.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.432+0000 7f146f30f640 1 --2- 172.21.15.46:0/727749187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f146815e730 0x7f1468158d00 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:41.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.432+0000 7f146f30f640 1 --2- 172.21.15.46:0/727749187 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f146815a4e0 0x7f1468141510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:41.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.432+0000 7f146f30f640 1 --2- 172.21.15.46:0/727749187 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1468143490 0x7f1468140d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:41.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.432+0000 7f146f30f640 1 -- 172.21.15.46:0/727749187 >> 172.21.15.46:0/727749187 conn(0x7f146806a4d0 msgr2=0x7f14680b3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:41.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.433+0000 7f146f30f640 1 -- 172.21.15.46:0/727749187 shutdown_connections 2023-12-02T00:30:41.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.433+0000 7f146f30f640 1 -- 172.21.15.46:0/727749187 wait complete. 2023-12-02T00:30:41.444 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2023-12-02T00:30:41.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.553+0000 7fc961fbb640 1 Processor -- start 2023-12-02T00:30:41.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.553+0000 7fc961fbb640 1 -- start start 2023-12-02T00:30:41.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.554+0000 7fc961fbb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc95c15c2f0 0x7fc95c06eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:41.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.554+0000 7fc961fbb640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc95c15ba40 con 0x7fc95c06f040 2023-12-02T00:30:41.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.554+0000 7fc961fbb640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc95c06f4a0 con 0x7fc95c140f10 2023-12-02T00:30:41.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.554+0000 7fc961fbb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc95c15c6d0 con 0x7fc95c15c2f0 2023-12-02T00:30:41.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.554+0000 7fc95affd640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc95c06f040 0x7fc95c140800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49254/0 (socket says 172.21.15.46:49254) 2023-12-02T00:30:41.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.554+0000 7fc95affd640 1 -- 172.21.15.46:0/922907415 learned_addr learned my addr 172.21.15.46:0/922907415 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:30:41.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.554+0000 7fc95b7fe640 1 --2- 172.21.15.46:0/922907415 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc95c15c2f0 0x7fc95c06eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:41.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.554+0000 7fc95a7fc640 1 -- 172.21.15.46:0/922907415 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 227412105 0 0) 0x7fc95c15ba40 con 0x7fc95c06f040 2023-12-02T00:30:41.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.554+0000 7fc95a7fc640 1 -- 172.21.15.46:0/922907415 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc944003520 con 0x7fc95c06f040 2023-12-02T00:30:41.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.554+0000 7fc95a7fc640 1 -- 172.21.15.46:0/922907415 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1866821440 0 0) 0x7fc95c06f4a0 con 0x7fc95c140f10 2023-12-02T00:30:41.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.554+0000 7fc95a7fc640 1 -- 172.21.15.46:0/922907415 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc95c15ba40 con 0x7fc95c140f10 2023-12-02T00:30:41.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.554+0000 7fc95a7fc640 1 -- 172.21.15.46:0/922907415 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2485895154 0 0) 0x7fc944003520 con 0x7fc95c06f040 2023-12-02T00:30:41.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.554+0000 7fc95a7fc640 1 -- 172.21.15.46:0/922907415 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc95c15c2f0 msgr2=0x7fc95c06eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:41.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.554+0000 7fc95a7fc640 1 --2- 172.21.15.46:0/922907415 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc95c15c2f0 0x7fc95c06eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:41.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.554+0000 7fc95a7fc640 1 -- 172.21.15.46:0/922907415 >> v1:172.21.15.162:6789/0 conn(0x7fc95c140f10 legacy=0x7fc95c13f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:41.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.554+0000 7fc95a7fc640 1 -- 172.21.15.46:0/922907415 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc95c1416e0 con 0x7fc95c06f040 2023-12-02T00:30:41.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.554+0000 7fc95a7fc640 1 -- 172.21.15.46:0/922907415 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7fc950001670 con 0x7fc95c06f040 2023-12-02T00:30:41.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.554+0000 7fc95b7fe640 1 --2- 172.21.15.46:0/922907415 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc95c15c2f0 0x7fc95c06eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:30:41.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.554+0000 7fc95a7fc640 1 -- 172.21.15.46:0/922907415 >> v1:172.21.15.46:6789/0 conn(0x7fc95c06f040 legacy=0x7fc95c140800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:41.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.554+0000 7fc95a7fc640 1 --2- 172.21.15.46:0/922907415 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc95c15c2f0 0x7fc944003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:41.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.554+0000 7fc95a7fc640 1 --2- 172.21.15.46:0/922907415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc944004070 0x7fc9440047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:41.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.554+0000 7fc95a7fc640 1 --2- 172.21.15.46:0/922907415 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc944004d00 0x7fc944005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:41.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.554+0000 7fc95a7fc640 1 -- 172.21.15.46:0/922907415 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94400ba30 con 0x7fc944004070 2023-12-02T00:30:41.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.554+0000 7fc95a7fc640 1 -- 172.21.15.46:0/922907415 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94400bb70 con 0x7fc944004d00 2023-12-02T00:30:41.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.554+0000 7fc95a7fc640 1 -- 172.21.15.46:0/922907415 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc94400bd30 con 0x7fc95c15c2f0 2023-12-02T00:30:41.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.555+0000 7fc95affd640 1 --2- 172.21.15.46:0/922907415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc944004070 0x7fc9440047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:41.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.555+0000 7fc95b7fe640 1 --2- 172.21.15.46:0/922907415 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc95c15c2f0 0x7fc944003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:41.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.555+0000 7fc95bfff640 1 --2- 172.21.15.46:0/922907415 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc944004d00 0x7fc944005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:41.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.555+0000 7fc95affd640 1 -- 172.21.15.46:0/922907415 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc95c15c2f0 msgr2=0x7fc944003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:41.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.555+0000 7fc95affd640 1 --2- 172.21.15.46:0/922907415 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc95c15c2f0 0x7fc944003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:41.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.555+0000 7fc95affd640 1 -- 172.21.15.46:0/922907415 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc944004d00 msgr2=0x7fc944005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:41.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.555+0000 7fc95affd640 1 --2- 172.21.15.46:0/922907415 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc944004d00 0x7fc944005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:41.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.555+0000 7fc95affd640 1 -- 172.21.15.46:0/922907415 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc94c009400 con 0x7fc944004070 2023-12-02T00:30:41.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.556+0000 7fc95affd640 1 --2- 172.21.15.46:0/922907415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc944004070 0x7fc9440047c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fc950007fd0 tx=0x7fc9440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbae97829803a92b server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:41.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.556+0000 7fc95a7fc640 1 -- 172.21.15.46:0/922907415 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc950005ad0 con 0x7fc944004070 2023-12-02T00:30:41.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.556+0000 7fc95a7fc640 1 -- 172.21.15.46:0/922907415 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc950005c50 con 0x7fc944004070 2023-12-02T00:30:41.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.556+0000 7fc95a7fc640 1 -- 172.21.15.46:0/922907415 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95002c830 con 0x7fc944004070 2023-12-02T00:30:41.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.556+0000 7fc961fbb640 1 -- 172.21.15.46:0/922907415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc944004070 msgr2=0x7fc9440047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:41.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.556+0000 7fc961fbb640 1 --2- 172.21.15.46:0/922907415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc944004070 0x7fc9440047c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fc950007fd0 tx=0x7fc9440057f0 comp rx=0 tx=0).stop 2023-12-02T00:30:41.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.556+0000 7fc95bfff640 1 -- 172.21.15.46:0/922907415 reap_dead start 2023-12-02T00:30:41.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.556+0000 7fc961fbb640 1 -- 172.21.15.46:0/922907415 shutdown_connections 2023-12-02T00:30:41.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.556+0000 7fc961fbb640 1 -- 172.21.15.46:0/922907415 >> 172.21.15.46:0/922907415 conn(0x7fc95c06a6b0 msgr2=0x7fc95c06aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:41.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.556+0000 7fc961fbb640 1 -- 172.21.15.46:0/922907415 shutdown_connections 2023-12-02T00:30:41.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.556+0000 7fc961fbb640 1 -- 172.21.15.46:0/922907415 wait complete. 2023-12-02T00:30:41.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.557+0000 7fc961fbb640 1 Processor -- start 2023-12-02T00:30:41.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.558+0000 7fc961fbb640 1 -- start start 2023-12-02T00:30:41.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.558+0000 7fc961fbb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc944004070 0x7fc95c153160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:41.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.558+0000 7fc961fbb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc95c06c710 0x7fc95c1f5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:41.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.558+0000 7fc961fbb640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc95c06cb90 0x7fc95c1550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:41.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.558+0000 7fc961fbb640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95c152710 con 0x7fc95c06c710 2023-12-02T00:30:41.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.558+0000 7fc961fbb640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95c1f6a20 con 0x7fc95c06cb90 2023-12-02T00:30:41.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.558+0000 7fc961fbb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc95c1f6d00 con 0x7fc944004070 2023-12-02T00:30:41.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.558+0000 7fc95affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc95c06c710 0x7fc95c1f5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:41.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.559+0000 7fc95affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc95c06c710 0x7fc95c1f5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52944/0 (socket says 172.21.15.46:52944) 2023-12-02T00:30:41.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.559+0000 7fc95affd640 1 -- 172.21.15.46:0/2434448269 learned_addr learned my addr 172.21.15.46:0/2434448269 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:41.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.559+0000 7fc95b7fe640 1 --2- 172.21.15.46:0/2434448269 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc944004070 0x7fc95c153160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:41.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.559+0000 7fc95bfff640 1 --2- 172.21.15.46:0/2434448269 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc95c06cb90 0x7fc95c1550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:41.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.559+0000 7fc95affd640 1 -- 172.21.15.46:0/2434448269 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc944004070 msgr2=0x7fc95c153160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:41.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.559+0000 7fc95affd640 1 --2- 172.21.15.46:0/2434448269 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc944004070 0x7fc95c153160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:41.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.559+0000 7fc95affd640 1 -- 172.21.15.46:0/2434448269 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc95c06cb90 msgr2=0x7fc95c1550f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:41.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.559+0000 7fc95affd640 1 --2- 172.21.15.46:0/2434448269 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc95c06cb90 0x7fc95c1550f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:41.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.559+0000 7fc95affd640 1 -- 172.21.15.46:0/2434448269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc95c152960 con 0x7fc95c06c710 2023-12-02T00:30:41.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.559+0000 7fc95affd640 1 --2- 172.21.15.46:0/2434448269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc95c06c710 0x7fc95c1f5fd0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fc9500018b0 tx=0x7fc950001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:41.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.559+0000 7fc943fff640 1 -- 172.21.15.46:0/2434448269 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc950039070 con 0x7fc95c06c710 2023-12-02T00:30:41.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.559+0000 7fc943fff640 1 -- 172.21.15.46:0/2434448269 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc95002cc60 con 0x7fc95c06c710 2023-12-02T00:30:41.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.559+0000 7fc943fff640 1 -- 172.21.15.46:0/2434448269 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95003c040 con 0x7fc95c06c710 2023-12-02T00:30:41.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.559+0000 7fc961fbb640 1 -- 172.21.15.46:0/2434448269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc95c152c40 con 0x7fc95c06c710 2023-12-02T00:30:41.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.559+0000 7fc961fbb640 1 -- 172.21.15.46:0/2434448269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc95c134c90 con 0x7fc95c06c710 2023-12-02T00:30:41.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.560+0000 7fc961fbb640 1 -- 172.21.15.46:0/2434448269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc95c152e00 con 0x7fc95c06c710 2023-12-02T00:30:41.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.561+0000 7fc943fff640 1 -- 172.21.15.46:0/2434448269 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc950032070 con 0x7fc95c06c710 2023-12-02T00:30:41.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.561+0000 7fc943fff640 1 --2- 172.21.15.46:0/2434448269 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc94806ae70 0x7fc94806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:41.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.561+0000 7fc943fff640 1 -- 172.21.15.46:0/2434448269 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fc9500aa940 con 0x7fc95c06c710 2023-12-02T00:30:41.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.561+0000 7fc95b7fe640 1 --2- 172.21.15.46:0/2434448269 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc94806ae70 0x7fc94806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:41.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.562+0000 7fc95b7fe640 1 --2- 172.21.15.46:0/2434448269 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc94806ae70 0x7fc94806d330 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc94c009c60 tx=0x7fc94c009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:41.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.565+0000 7fc943fff640 1 -- 172.21.15.46:0/2434448269 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc9500759a0 con 0x7fc95c06c710 2023-12-02T00:30:41.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.743+0000 7fc961fbb640 1 -- 172.21.15.46:0/2434448269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7fc95c069400 con 0x7fc95c06c710 2023-12-02T00:30:41.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.743+0000 7fc943fff640 1 -- 172.21.15.46:0/2434448269 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v38) v1 ==== 97+0+11 (secure 0 0 0) 0x7fc950078fc0 con 0x7fc95c06c710 2023-12-02T00:30:41.745 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-12-02T00:30:41.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.745+0000 7fc961fbb640 1 -- 172.21.15.46:0/2434448269 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc94806ae70 msgr2=0x7fc94806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:41.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.745+0000 7fc961fbb640 1 --2- 172.21.15.46:0/2434448269 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc94806ae70 0x7fc94806d330 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc94c009c60 tx=0x7fc94c009190 comp rx=0 tx=0).stop 2023-12-02T00:30:41.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.745+0000 7fc961fbb640 1 -- 172.21.15.46:0/2434448269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc95c06c710 msgr2=0x7fc95c1f5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:41.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.745+0000 7fc961fbb640 1 --2- 172.21.15.46:0/2434448269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc95c06c710 0x7fc95c1f5fd0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fc9500018b0 tx=0x7fc950001980 comp rx=0 tx=0).stop 2023-12-02T00:30:41.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.745+0000 7fc961fbb640 1 -- 172.21.15.46:0/2434448269 shutdown_connections 2023-12-02T00:30:41.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.745+0000 7fc961fbb640 1 --2- 172.21.15.46:0/2434448269 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc94806ae70 0x7fc94806d330 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:41.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.745+0000 7fc961fbb640 1 --2- 172.21.15.46:0/2434448269 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc95c06cb90 0x7fc95c1550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:41.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.746+0000 7fc961fbb640 1 --2- 172.21.15.46:0/2434448269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc95c06c710 0x7fc95c1f5fd0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:41.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.746+0000 7fc961fbb640 1 --2- 172.21.15.46:0/2434448269 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc944004070 0x7fc95c153160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:41.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.746+0000 7fc961fbb640 1 -- 172.21.15.46:0/2434448269 >> 172.21.15.46:0/2434448269 conn(0x7fc95c06a6b0 msgr2=0x7fc95c06aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:41.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.746+0000 7fc961fbb640 1 -- 172.21.15.46:0/2434448269 shutdown_connections 2023-12-02T00:30:41.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.746+0000 7fc961fbb640 1 -- 172.21.15.46:0/2434448269 wait complete. 2023-12-02T00:30:41.758 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-02T00:30:41.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.863+0000 7f14b417c640 1 Processor -- start 2023-12-02T00:30:41.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.863+0000 7f14b417c640 1 -- start start 2023-12-02T00:30:41.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.863+0000 7f14b417c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14ac06ea50 0x7f14ac06ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:41.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.863+0000 7f14b417c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f14ac160fb0 0x7f14ac161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:41.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.864+0000 7f14b417c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f14ac0b3ff0 0x7f14ac0b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:41.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.864+0000 7f14b417c640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14ac161d20 con 0x7f14ac160fb0 2023-12-02T00:30:41.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.864+0000 7f14b417c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14ac06f370 con 0x7f14ac0b3ff0 2023-12-02T00:30:41.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.864+0000 7f14b417c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac15e760 con 0x7f14ac06ea50 2023-12-02T00:30:41.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.864+0000 7f14b1717640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f14ac160fb0 0x7f14ac161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:41.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.864+0000 7f14b1717640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f14ac160fb0 0x7f14ac161410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52954/0 (socket says 172.21.15.46:52954) 2023-12-02T00:30:41.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.864+0000 7f14b1717640 1 -- 172.21.15.46:0/3135225870 learned_addr learned my addr 172.21.15.46:0/3135225870 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:41.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.864+0000 7f14b1f18640 1 --2- 172.21.15.46:0/3135225870 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14ac06ea50 0x7f14ac06ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:41.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.864+0000 7f14b2719640 1 --2- 172.21.15.46:0/3135225870 >> v2:172.21.15.162:3300/0 conn(0x7f14ac0b3ff0 0x7f14ac0b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:41.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.864+0000 7f14b1717640 1 -- 172.21.15.46:0/3135225870 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14ac06ea50 msgr2=0x7f14ac06ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:41.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.864+0000 7f14b1717640 1 --2- 172.21.15.46:0/3135225870 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14ac06ea50 0x7f14ac06ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:41.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.864+0000 7f14b1717640 1 -- 172.21.15.46:0/3135225870 >> v2:172.21.15.162:3300/0 conn(0x7f14ac0b3ff0 msgr2=0x7f14ac0b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:41.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.864+0000 7f14b1717640 1 --2- 172.21.15.46:0/3135225870 >> v2:172.21.15.162:3300/0 conn(0x7f14ac0b3ff0 0x7f14ac0b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:41.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.864+0000 7f14b1717640 1 -- 172.21.15.46:0/3135225870 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14ac15d9a0 con 0x7f14ac160fb0 2023-12-02T00:30:41.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.864+0000 7f14b2719640 1 --2- 172.21.15.46:0/3135225870 >> v2:172.21.15.162:3300/0 conn(0x7f14ac0b3ff0 0x7f14ac0b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:30:41.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.864+0000 7f14b1f18640 1 --2- 172.21.15.46:0/3135225870 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14ac06ea50 0x7f14ac06ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:30:41.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.865+0000 7f14b1717640 1 --2- 172.21.15.46:0/3135225870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14ac160fb0 0x7f14ac161410 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f14ac0b66c0 tx=0x7f149c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3f0441b56513cbf server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:41.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.865+0000 7f14b0f16640 1 -- 172.21.15.46:0/3135225870 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149c0317e0 con 0x7f14ac160fb0 2023-12-02T00:30:41.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.865+0000 7f14b0f16640 1 -- 172.21.15.46:0/3135225870 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f149c031940 con 0x7f14ac160fb0 2023-12-02T00:30:41.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.865+0000 7f14b0f16640 1 -- 172.21.15.46:0/3135225870 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149c031cd0 con 0x7f14ac160fb0 2023-12-02T00:30:41.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.865+0000 7f14b417c640 1 -- 172.21.15.46:0/3135225870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14ac160fb0 msgr2=0x7f14ac161410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:41.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.865+0000 7f14b417c640 1 --2- 172.21.15.46:0/3135225870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14ac160fb0 0x7f14ac161410 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f14ac0b66c0 tx=0x7f149c02b880 comp rx=0 tx=0).stop 2023-12-02T00:30:41.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.866+0000 7f14b417c640 1 -- 172.21.15.46:0/3135225870 shutdown_connections 2023-12-02T00:30:41.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.866+0000 7f14b417c640 1 --2- 172.21.15.46:0/3135225870 >> v2:172.21.15.162:3300/0 conn(0x7f14ac0b3ff0 0x7f14ac0b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:41.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.866+0000 7f14b417c640 1 --2- 172.21.15.46:0/3135225870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14ac160fb0 0x7f14ac161410 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:41.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.866+0000 7f14b417c640 1 --2- 172.21.15.46:0/3135225870 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14ac06ea50 0x7f14ac06ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:41.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.866+0000 7f14b417c640 1 -- 172.21.15.46:0/3135225870 >> 172.21.15.46:0/3135225870 conn(0x7f14ac06a6b0 msgr2=0x7f14ac06aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:41.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.866+0000 7f14b417c640 1 -- 172.21.15.46:0/3135225870 shutdown_connections 2023-12-02T00:30:41.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.866+0000 7f14b417c640 1 -- 172.21.15.46:0/3135225870 wait complete. 2023-12-02T00:30:41.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.867+0000 7f14b417c640 1 Processor -- start 2023-12-02T00:30:41.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.867+0000 7f14b417c640 1 -- start start 2023-12-02T00:30:41.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.868+0000 7f14b417c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14ac06ea50 0x7f14ac15ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:41.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.868+0000 7f14b417c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14ac0b3ff0 0x7f14ac1f61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:41.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.868+0000 7f14b417c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14ac160fb0 0x7f14ac1f6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:41.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.868+0000 7f14b417c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac06cfe0 con 0x7f14ac160fb0 2023-12-02T00:30:41.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.868+0000 7f14b417c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac142520 con 0x7f14ac06ea50 2023-12-02T00:30:41.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.868+0000 7f14b417c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac158160 con 0x7f14ac0b3ff0 2023-12-02T00:30:41.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.868+0000 7f14b2719640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14ac160fb0 0x7f14ac1f6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:41.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.868+0000 7f14b2719640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14ac160fb0 0x7f14ac1f6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52958/0 (socket says 172.21.15.46:52958) 2023-12-02T00:30:41.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.868+0000 7f14b2719640 1 -- 172.21.15.46:0/309973633 learned_addr learned my addr 172.21.15.46:0/309973633 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:41.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.868+0000 7f14b1f18640 1 --2- 172.21.15.46:0/309973633 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14ac06ea50 0x7f14ac15ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:41.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.868+0000 7f14b2719640 1 -- 172.21.15.46:0/309973633 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14ac0b3ff0 msgr2=0x7f14ac1f61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:41.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.868+0000 7f14b1717640 1 --2- 172.21.15.46:0/309973633 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14ac0b3ff0 0x7f14ac1f61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:41.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.868+0000 7f14b2719640 1 --2- 172.21.15.46:0/309973633 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14ac0b3ff0 0x7f14ac1f61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:41.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.868+0000 7f14b2719640 1 -- 172.21.15.46:0/309973633 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14ac06ea50 msgr2=0x7f14ac15ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:41.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.868+0000 7f14b2719640 1 --2- 172.21.15.46:0/309973633 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14ac06ea50 0x7f14ac15ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:41.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.868+0000 7f14b2719640 1 -- 172.21.15.46:0/309973633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f149c0094d0 con 0x7f14ac160fb0 2023-12-02T00:30:41.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.868+0000 7f14b1f18640 1 --2- 172.21.15.46:0/309973633 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14ac06ea50 0x7f14ac15ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:30:41.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.868+0000 7f14b1717640 1 --2- 172.21.15.46:0/309973633 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14ac0b3ff0 0x7f14ac1f61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:30:41.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.868+0000 7f14b2719640 1 --2- 172.21.15.46:0/309973633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14ac160fb0 0x7f14ac1f6700 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f14a400ce70 tx=0x7f14a400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:41.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.868+0000 7f14a27fc640 1 -- 172.21.15.46:0/309973633 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a400d840 con 0x7f14ac160fb0 2023-12-02T00:30:41.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.868+0000 7f14a27fc640 1 -- 172.21.15.46:0/309973633 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a400d9a0 con 0x7f14ac160fb0 2023-12-02T00:30:41.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.868+0000 7f14b417c640 1 -- 172.21.15.46:0/309973633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14ac06cd00 con 0x7f14ac160fb0 2023-12-02T00:30:41.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.869+0000 7f14a27fc640 1 -- 172.21.15.46:0/309973633 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a400dd50 con 0x7f14ac160fb0 2023-12-02T00:30:41.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.869+0000 7f14b417c640 1 -- 172.21.15.46:0/309973633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14ac12da50 con 0x7f14ac160fb0 2023-12-02T00:30:41.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.870+0000 7f14b417c640 1 -- 172.21.15.46:0/309973633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14ac12d5d0 con 0x7f14ac160fb0 2023-12-02T00:30:41.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.870+0000 7f14a27fc640 1 -- 172.21.15.46:0/309973633 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f14a401e3b0 con 0x7f14ac160fb0 2023-12-02T00:30:41.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.870+0000 7f14a27fc640 1 --2- 172.21.15.46:0/309973633 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f149006ae40 0x7f149006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:41.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.871+0000 7f14a27fc640 1 -- 172.21.15.46:0/309973633 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f14a408a130 con 0x7f14ac160fb0 2023-12-02T00:30:41.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.871+0000 7f14b1f18640 1 --2- 172.21.15.46:0/309973633 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f149006ae40 0x7f149006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:41.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.871+0000 7f14b1f18640 1 --2- 172.21.15.46:0/309973633 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f149006ae40 0x7f149006d300 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f14a8006fd0 tx=0x7f14a8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:41.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:41.875+0000 7f14a27fc640 1 -- 172.21.15.46:0/309973633 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f14ac12d5d0 con 0x7f14ac160fb0 2023-12-02T00:30:42.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.087+0000 7f14b417c640 1 -- 172.21.15.46:0/309973633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14ac12d5d0 con 0x7f14ac160fb0 2023-12-02T00:30:42.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.087+0000 7f14a27fc640 1 -- 172.21.15.46:0/309973633 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f14ac12d5d0 con 0x7f14ac160fb0 2023-12-02T00:30:42.089 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:30:42.089 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4811,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/3097254664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3097254664},{"type":"v1","addr":"172.21.15.162:6835","nonce":3097254664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":4826,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/3489476943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3489476943},{"type":"v1","addr":"172.21.15.162:6837","nonce":3489476943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5274,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1283185308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":1283185308},{"type":"v1","addr":"172.21.15.46:6835","nonce":1283185308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-02T00:29:57.279169+0000","modified":"2023-12-02T00:30:09.302069+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5277},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5277":{"gid":5277,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":5,"addr":"172.21.15.46:6834/2036053042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2036053042},{"type":"v1","addr":"172.21.15.46:6834","nonce":2036053042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-02T00:30:42.089 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 14 2023-12-02T00:30:42.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.089+0000 7f14b417c640 1 -- 172.21.15.46:0/309973633 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f149006ae40 msgr2=0x7f149006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:42.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.089+0000 7f14b417c640 1 --2- 172.21.15.46:0/309973633 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f149006ae40 0x7f149006d300 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f14a8006fd0 tx=0x7f14a8008040 comp rx=0 tx=0).stop 2023-12-02T00:30:42.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.089+0000 7f14b417c640 1 -- 172.21.15.46:0/309973633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14ac160fb0 msgr2=0x7f14ac1f6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:42.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.089+0000 7f14b417c640 1 --2- 172.21.15.46:0/309973633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14ac160fb0 0x7f14ac1f6700 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f14a400ce70 tx=0x7f14a400a710 comp rx=0 tx=0).stop 2023-12-02T00:30:42.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.089+0000 7f14b417c640 1 -- 172.21.15.46:0/309973633 shutdown_connections 2023-12-02T00:30:42.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.089+0000 7f14b417c640 1 --2- 172.21.15.46:0/309973633 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f149006ae40 0x7f149006d300 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:42.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.089+0000 7f14b417c640 1 --2- 172.21.15.46:0/309973633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14ac160fb0 0x7f14ac1f6700 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:42.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.089+0000 7f14b417c640 1 --2- 172.21.15.46:0/309973633 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14ac0b3ff0 0x7f14ac1f61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:42.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.089+0000 7f14b417c640 1 --2- 172.21.15.46:0/309973633 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14ac06ea50 0x7f14ac15ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:42.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.089+0000 7f14b417c640 1 -- 172.21.15.46:0/309973633 >> 172.21.15.46:0/309973633 conn(0x7f14ac06a6b0 msgr2=0x7f14ac06aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:42.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.089+0000 7f14b417c640 1 -- 172.21.15.46:0/309973633 shutdown_connections 2023-12-02T00:30:42.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.089+0000 7f14b417c640 1 -- 172.21.15.46:0/309973633 wait complete. 2023-12-02T00:30:42.102 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:30:42.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.208+0000 7f0c5202b640 1 Processor -- start 2023-12-02T00:30:42.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.208+0000 7f0c5202b640 1 -- start start 2023-12-02T00:30:42.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.208+0000 7f0c5202b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0c4c144980 0x7f0c4c15c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:42.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.209+0000 7f0c5202b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0c4c0b4370 0x7f0c4c0b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:42.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.209+0000 7f0c5202b640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c4c071770 con 0x7f0c4c0ceb70 2023-12-02T00:30:42.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.209+0000 7f0c5202b640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c4c0b4d20 con 0x7f0c4c0b4370 2023-12-02T00:30:42.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.209+0000 7f0c5202b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4c0ccb60 con 0x7f0c4c144980 2023-12-02T00:30:42.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.209+0000 7f0c4bfff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0c4c0b4370 0x7f0c4c0b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:42.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.209+0000 7f0c4bfff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0c4c0b4370 0x7f0c4c0b47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52964/0 (socket says 172.21.15.46:52964) 2023-12-02T00:30:42.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.209+0000 7f0c4bfff640 1 -- 172.21.15.46:0/159124002 learned_addr learned my addr 172.21.15.46:0/159124002 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:42.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.209+0000 7f0c4b7fe640 1 --2- 172.21.15.46:0/159124002 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0c4c144980 0x7f0c4c15c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:42.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.209+0000 7f0c4bfff640 1 -- 172.21.15.46:0/159124002 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0c4c144980 msgr2=0x7f0c4c15c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:42.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.209+0000 7f0c4bfff640 1 --2- 172.21.15.46:0/159124002 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0c4c144980 0x7f0c4c15c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:42.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.209+0000 7f0c4bfff640 1 -- 172.21.15.46:0/159124002 >> v1:172.21.15.162:6789/0 conn(0x7f0c4c0ceb70 legacy=0x7f0c4c0cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:42.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.209+0000 7f0c4bfff640 1 -- 172.21.15.46:0/159124002 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c4c0ccdc0 con 0x7f0c4c0b4370 2023-12-02T00:30:42.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.210+0000 7f0c4bfff640 1 --2- 172.21.15.46:0/159124002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c4c0b4370 0x7f0c4c0b47e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f0c44009e50 tx=0x7f0c4402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93aa1a51baa5f4b7 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:42.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.210+0000 7f0c4a7fc640 1 -- 172.21.15.46:0/159124002 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4402cb00 con 0x7f0c4c0b4370 2023-12-02T00:30:42.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.210+0000 7f0c4a7fc640 1 -- 172.21.15.46:0/159124002 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c4402cc60 con 0x7f0c4c0b4370 2023-12-02T00:30:42.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.210+0000 7f0c4a7fc640 1 -- 172.21.15.46:0/159124002 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c440343f0 con 0x7f0c4c0b4370 2023-12-02T00:30:42.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.211+0000 7f0c5202b640 1 -- 172.21.15.46:0/159124002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c4c0b4370 msgr2=0x7f0c4c0b47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:42.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.211+0000 7f0c5202b640 1 --2- 172.21.15.46:0/159124002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c4c0b4370 0x7f0c4c0b47e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f0c44009e50 tx=0x7f0c4402bf60 comp rx=0 tx=0).stop 2023-12-02T00:30:42.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.211+0000 7f0c5202b640 1 -- 172.21.15.46:0/159124002 shutdown_connections 2023-12-02T00:30:42.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.211+0000 7f0c5202b640 1 --2- 172.21.15.46:0/159124002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c4c0b4370 0x7f0c4c0b47e0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:42.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.211+0000 7f0c5202b640 1 --2- 172.21.15.46:0/159124002 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0c4c144980 0x7f0c4c15c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:42.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.211+0000 7f0c5202b640 1 -- 172.21.15.46:0/159124002 >> 172.21.15.46:0/159124002 conn(0x7f0c4c06a4d0 msgr2=0x7f0c4c06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:42.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.211+0000 7f0c5202b640 1 -- 172.21.15.46:0/159124002 shutdown_connections 2023-12-02T00:30:42.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.211+0000 7f0c5202b640 1 -- 172.21.15.46:0/159124002 wait complete. 2023-12-02T00:30:42.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.212+0000 7f0c5202b640 1 Processor -- start 2023-12-02T00:30:42.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.212+0000 7f0c5202b640 1 -- start start 2023-12-02T00:30:42.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.213+0000 7f0c5202b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c4c0b4370 0x7f0c4c13ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:42.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.213+0000 7f0c5202b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0c4c0ceb70 0x7f0c4c1f3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:42.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.213+0000 7f0c5202b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c4c144980 0x7f0c4c1566a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:42.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.213+0000 7f0c5202b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4c13d3f0 con 0x7f0c4c0b4370 2023-12-02T00:30:42.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.213+0000 7f0c5202b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4c1f41e0 con 0x7f0c4c144980 2023-12-02T00:30:42.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.213+0000 7f0c5202b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4c0aecd0 con 0x7f0c4c0ceb70 2023-12-02T00:30:42.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.213+0000 7f0c4b7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c4c0b4370 0x7f0c4c13ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:42.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.213+0000 7f0c4b7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c4c0b4370 0x7f0c4c13ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52980/0 (socket says 172.21.15.46:52980) 2023-12-02T00:30:42.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.213+0000 7f0c4b7fe640 1 -- 172.21.15.46:0/4294377194 learned_addr learned my addr 172.21.15.46:0/4294377194 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:42.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.213+0000 7f0c4affd640 1 --2- 172.21.15.46:0/4294377194 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0c4c0ceb70 0x7f0c4c1f3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:42.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.213+0000 7f0c4bfff640 1 --2- 172.21.15.46:0/4294377194 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c4c144980 0x7f0c4c1566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:42.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.213+0000 7f0c4b7fe640 1 -- 172.21.15.46:0/4294377194 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0c4c0ceb70 msgr2=0x7f0c4c1f3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:42.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.213+0000 7f0c4b7fe640 1 --2- 172.21.15.46:0/4294377194 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0c4c0ceb70 0x7f0c4c1f3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:42.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.213+0000 7f0c4b7fe640 1 -- 172.21.15.46:0/4294377194 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c4c144980 msgr2=0x7f0c4c1566a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:42.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.213+0000 7f0c4b7fe640 1 --2- 172.21.15.46:0/4294377194 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c4c144980 0x7f0c4c1566a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:42.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.213+0000 7f0c4b7fe640 1 -- 172.21.15.46:0/4294377194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c44009b30 con 0x7f0c4c0b4370 2023-12-02T00:30:42.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.213+0000 7f0c4bfff640 1 --2- 172.21.15.46:0/4294377194 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c4c144980 0x7f0c4c1566a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:30:42.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.213+0000 7f0c4b7fe640 1 --2- 172.21.15.46:0/4294377194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c4c0b4370 0x7f0c4c13ceb0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c00e8a0 tx=0x7f0c3c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:42.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.213+0000 7f0c33fff640 1 -- 172.21.15.46:0/4294377194 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c0094f0 con 0x7f0c4c0b4370 2023-12-02T00:30:42.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.214+0000 7f0c33fff640 1 -- 172.21.15.46:0/4294377194 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c3c00c940 con 0x7f0c4c0b4370 2023-12-02T00:30:42.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.214+0000 7f0c33fff640 1 -- 172.21.15.46:0/4294377194 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c00ccf0 con 0x7f0c4c0b4370 2023-12-02T00:30:42.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.214+0000 7f0c5202b640 1 -- 172.21.15.46:0/4294377194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c4c156f20 con 0x7f0c4c0b4370 2023-12-02T00:30:42.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.214+0000 7f0c5202b640 1 -- 172.21.15.46:0/4294377194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c4c0af250 con 0x7f0c4c0b4370 2023-12-02T00:30:42.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.215+0000 7f0c5202b640 1 -- 172.21.15.46:0/4294377194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c4c15cb80 con 0x7f0c4c0b4370 2023-12-02T00:30:42.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.215+0000 7f0c33fff640 1 -- 172.21.15.46:0/4294377194 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0c3c013470 con 0x7f0c4c0b4370 2023-12-02T00:30:42.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.215+0000 7f0c33fff640 1 --2- 172.21.15.46:0/4294377194 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f0c3806ae40 0x7f0c3806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:42.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.216+0000 7f0c33fff640 1 -- 172.21.15.46:0/4294377194 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f0c3c0897b0 con 0x7f0c4c0b4370 2023-12-02T00:30:42.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.216+0000 7f0c4affd640 1 --2- 172.21.15.46:0/4294377194 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f0c3806ae40 0x7f0c3806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:42.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.216+0000 7f0c4affd640 1 --2- 172.21.15.46:0/4294377194 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f0c3806ae40 0x7f0c3806d300 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0c40000f60 tx=0x7f0c40009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:42.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.220+0000 7f0c33fff640 1 -- 172.21.15.46:0/4294377194 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0c3c054760 con 0x7f0c4c0b4370 2023-12-02T00:30:42.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.398+0000 7f0c5202b640 1 -- 172.21.15.46:0/4294377194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0c4c069400 con 0x7f0c4c0b4370 2023-12-02T00:30:42.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.399+0000 7f0c33fff640 1 -- 172.21.15.46:0/4294377194 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+18324 (secure 0 0 0) 0x7f0c3c057d80 con 0x7f0c4c0b4370 2023-12-02T00:30:42.400 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:30:42.400 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":38,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:30:10.714057+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-02T00:29:55.676572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-02T00:29:56.244869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"data_new","create_time":"2023-12-02T00:30:08.897990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:30:42.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.401+0000 7f0c5202b640 1 -- 172.21.15.46:0/4294377194 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f0c3806ae40 msgr2=0x7f0c3806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:42.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.401+0000 7f0c5202b640 1 --2- 172.21.15.46:0/4294377194 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f0c3806ae40 0x7f0c3806d300 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0c40000f60 tx=0x7f0c40009040 comp rx=0 tx=0).stop 2023-12-02T00:30:42.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.401+0000 7f0c5202b640 1 -- 172.21.15.46:0/4294377194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c4c0b4370 msgr2=0x7f0c4c13ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:42.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.401+0000 7f0c5202b640 1 --2- 172.21.15.46:0/4294377194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c4c0b4370 0x7f0c4c13ceb0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c00e8a0 tx=0x7f0c3c00ed60 comp rx=0 tx=0).stop 2023-12-02T00:30:42.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.401+0000 7f0c5202b640 1 -- 172.21.15.46:0/4294377194 shutdown_connections 2023-12-02T00:30:42.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.401+0000 7f0c5202b640 1 --2- 172.21.15.46:0/4294377194 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f0c3806ae40 0x7f0c3806d300 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:42.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.401+0000 7f0c5202b640 1 --2- 172.21.15.46:0/4294377194 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c4c144980 0x7f0c4c1566a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:42.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.401+0000 7f0c5202b640 1 --2- 172.21.15.46:0/4294377194 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0c4c0ceb70 0x7f0c4c1f3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:42.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.401+0000 7f0c5202b640 1 --2- 172.21.15.46:0/4294377194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c4c0b4370 0x7f0c4c13ceb0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:42.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.401+0000 7f0c5202b640 1 -- 172.21.15.46:0/4294377194 >> 172.21.15.46:0/4294377194 conn(0x7f0c4c06a4d0 msgr2=0x7f0c4c0cbcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:42.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.401+0000 7f0c5202b640 1 -- 172.21.15.46:0/4294377194 shutdown_connections 2023-12-02T00:30:42.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.401+0000 7f0c5202b640 1 -- 172.21.15.46:0/4294377194 wait complete. 2023-12-02T00:30:42.415 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-02T00:30:42.416 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-02T00:30:42.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.522+0000 7efce3ae5640 1 Processor -- start 2023-12-02T00:30:42.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.522+0000 7efce3ae5640 1 -- start start 2023-12-02T00:30:42.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.523+0000 7efce3ae5640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7efcdc150560 0x7efcdc0b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:42.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.523+0000 7efce3ae5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcdc150940 0x7efcdc0ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:42.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.523+0000 7efce3ae5640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcdc0b49f0 con 0x7efcdc0b3390 2023-12-02T00:30:42.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.523+0000 7efce3ae5640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcdc0b3de0 con 0x7efcdc150560 2023-12-02T00:30:42.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.523+0000 7efce3ae5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcdc0b40c0 con 0x7efcdc150940 2023-12-02T00:30:42.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.523+0000 7efce1080640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7efcdc0b3390 0x7efcdc15d200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49268/0 (socket says 172.21.15.46:49268) 2023-12-02T00:30:42.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.523+0000 7efce1080640 1 -- 172.21.15.46:0/906987520 learned_addr learned my addr 172.21.15.46:0/906987520 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:30:42.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.523+0000 7efce1881640 1 --2- 172.21.15.46:0/906987520 >> v2:172.21.15.162:3300/0 conn(0x7efcdc150560 0x7efcdc0b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:42.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.523+0000 7efce2082640 1 --2- 172.21.15.46:0/906987520 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcdc150940 0x7efcdc0ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:42.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.523+0000 7efce087f640 1 -- 172.21.15.46:0/906987520 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 665085021 0 0) 0x7efcdc0b49f0 con 0x7efcdc0b3390 2023-12-02T00:30:42.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.523+0000 7efce087f640 1 -- 172.21.15.46:0/906987520 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcc8003520 con 0x7efcdc0b3390 2023-12-02T00:30:42.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.524+0000 7efce087f640 1 -- 172.21.15.46:0/906987520 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4188634589 0 0) 0x7efcc8003520 con 0x7efcdc0b3390 2023-12-02T00:30:42.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.524+0000 7efce087f640 1 -- 172.21.15.46:0/906987520 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcdc150940 msgr2=0x7efcdc0ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:42.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.524+0000 7efce087f640 1 --2- 172.21.15.46:0/906987520 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcdc150940 0x7efcdc0ce0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:42.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.524+0000 7efce087f640 1 -- 172.21.15.46:0/906987520 >> v2:172.21.15.162:3300/0 conn(0x7efcdc150560 msgr2=0x7efcdc0b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:42.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.524+0000 7efce087f640 1 --2- 172.21.15.46:0/906987520 >> v2:172.21.15.162:3300/0 conn(0x7efcdc150560 0x7efcdc0b2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:42.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.524+0000 7efce087f640 1 -- 172.21.15.46:0/906987520 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcdc0b43a0 con 0x7efcdc0b3390 2023-12-02T00:30:42.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.524+0000 7efce087f640 1 -- 172.21.15.46:0/906987520 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7efccc001670 con 0x7efcdc0b3390 2023-12-02T00:30:42.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.524+0000 7efce087f640 1 -- 172.21.15.46:0/906987520 >> v1:172.21.15.46:6789/0 conn(0x7efcdc0b3390 legacy=0x7efcdc15d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:42.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.524+0000 7efce087f640 1 --2- 172.21.15.46:0/906987520 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcc8003970 0x7efcc8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:42.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.524+0000 7efce087f640 1 --2- 172.21.15.46:0/906987520 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcc80047c0 0x7efcc8004c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:42.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.524+0000 7efce087f640 1 --2- 172.21.15.46:0/906987520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efcc8005140 0x7efcc800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:42.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.524+0000 7efce087f640 1 -- 172.21.15.46:0/906987520 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcdc0b3de0 con 0x7efcc8005140 2023-12-02T00:30:42.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.524+0000 7efce087f640 1 -- 172.21.15.46:0/906987520 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcdc0b40c0 con 0x7efcc8003970 2023-12-02T00:30:42.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.524+0000 7efce087f640 1 -- 172.21.15.46:0/906987520 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcc80045c0 con 0x7efcc80047c0 2023-12-02T00:30:42.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.524+0000 7efce2082640 1 --2- 172.21.15.46:0/906987520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efcc8005140 0x7efcc800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:42.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.524+0000 7efce1881640 1 --2- 172.21.15.46:0/906987520 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcc8003970 0x7efcc8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:42.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.524+0000 7efce1080640 1 --2- 172.21.15.46:0/906987520 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcc80047c0 0x7efcc8004c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:42.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.524+0000 7efce2082640 1 -- 172.21.15.46:0/906987520 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcc80047c0 msgr2=0x7efcc8004c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:42.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.524+0000 7efce2082640 1 --2- 172.21.15.46:0/906987520 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcc80047c0 0x7efcc8004c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:42.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.524+0000 7efce2082640 1 -- 172.21.15.46:0/906987520 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcc8003970 msgr2=0x7efcc8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:42.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.524+0000 7efce2082640 1 --2- 172.21.15.46:0/906987520 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcc8003970 0x7efcc8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:42.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.524+0000 7efce2082640 1 -- 172.21.15.46:0/906987520 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcc8003520 con 0x7efcc8005140 2023-12-02T00:30:42.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.524+0000 7efce1080640 1 --2- 172.21.15.46:0/906987520 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcc80047c0 0x7efcc8004c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:30:42.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.525+0000 7efce2082640 1 --2- 172.21.15.46:0/906987520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efcc8005140 0x7efcc800b980 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7efcdc0b49a0 tx=0x7efcc80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b8cb9d159e8a2b server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:42.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.525+0000 7efce087f640 1 -- 172.21.15.46:0/906987520 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd402cb90 con 0x7efcc8005140 2023-12-02T00:30:42.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.525+0000 7efce087f640 1 -- 172.21.15.46:0/906987520 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd400f040 con 0x7efcc8005140 2023-12-02T00:30:42.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.525+0000 7efce087f640 1 -- 172.21.15.46:0/906987520 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd4034440 con 0x7efcc8005140 2023-12-02T00:30:42.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.526+0000 7efce3ae5640 1 -- 172.21.15.46:0/906987520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efcc8005140 msgr2=0x7efcc800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:42.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.526+0000 7efce3ae5640 1 --2- 172.21.15.46:0/906987520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efcc8005140 0x7efcc800b980 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7efcdc0b49a0 tx=0x7efcc80057f0 comp rx=0 tx=0).stop 2023-12-02T00:30:42.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.526+0000 7efce2082640 1 -- 172.21.15.46:0/906987520 reap_dead start 2023-12-02T00:30:42.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.526+0000 7efce3ae5640 1 -- 172.21.15.46:0/906987520 shutdown_connections 2023-12-02T00:30:42.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.526+0000 7efce3ae5640 1 -- 172.21.15.46:0/906987520 >> 172.21.15.46:0/906987520 conn(0x7efcdc06a4d0 msgr2=0x7efcdc06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:42.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.526+0000 7efce3ae5640 1 -- 172.21.15.46:0/906987520 shutdown_connections 2023-12-02T00:30:42.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.526+0000 7efce3ae5640 1 -- 172.21.15.46:0/906987520 wait complete. 2023-12-02T00:30:42.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.527+0000 7efce3ae5640 1 Processor -- start 2023-12-02T00:30:42.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.527+0000 7efce3ae5640 1 -- start start 2023-12-02T00:30:42.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.528+0000 7efce3ae5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcdc159850 0x7efcdc146cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:42.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.528+0000 7efce3ae5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efcdc159c30 0x7efcdc14e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:42.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.528+0000 7efce3ae5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcdc12f6c0 0x7efcdc12fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:42.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.528+0000 7efce3ae5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcdc14eec0 con 0x7efcdc159c30 2023-12-02T00:30:42.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.528+0000 7efce3ae5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcdc14f1a0 con 0x7efcdc12f6c0 2023-12-02T00:30:42.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.528+0000 7efce3ae5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcdc14f480 con 0x7efcdc159850 2023-12-02T00:30:42.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.528+0000 7efce1080640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efcdc159c30 0x7efcdc14e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:42.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.528+0000 7efce1080640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efcdc159c30 0x7efcdc14e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52994/0 (socket says 172.21.15.46:52994) 2023-12-02T00:30:42.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.528+0000 7efce1080640 1 -- 172.21.15.46:0/1595241501 learned_addr learned my addr 172.21.15.46:0/1595241501 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:42.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.528+0000 7efce1881640 1 --2- 172.21.15.46:0/1595241501 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcdc159850 0x7efcdc146cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:42.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.528+0000 7efce2082640 1 --2- 172.21.15.46:0/1595241501 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcdc12f6c0 0x7efcdc12fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:42.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.528+0000 7efce1080640 1 -- 172.21.15.46:0/1595241501 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcdc159850 msgr2=0x7efcdc146cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:42.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.528+0000 7efce1080640 1 --2- 172.21.15.46:0/1595241501 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcdc159850 0x7efcdc146cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:42.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.528+0000 7efce1080640 1 -- 172.21.15.46:0/1595241501 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcdc12f6c0 msgr2=0x7efcdc12fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:42.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.528+0000 7efce1080640 1 --2- 172.21.15.46:0/1595241501 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcdc12f6c0 0x7efcdc12fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:42.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.528+0000 7efce1080640 1 -- 172.21.15.46:0/1595241501 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcdc14f760 con 0x7efcdc159c30 2023-12-02T00:30:42.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.528+0000 7efce1080640 1 --2- 172.21.15.46:0/1595241501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efcdc159c30 0x7efcdc14e5e0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7efccc000fb0 tx=0x7efccc00eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:42.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.529+0000 7efcd27fc640 1 -- 172.21.15.46:0/1595241501 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efccc00c8d0 con 0x7efcdc159c30 2023-12-02T00:30:42.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.529+0000 7efcd27fc640 1 -- 172.21.15.46:0/1595241501 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efccc00ca30 con 0x7efcdc159c30 2023-12-02T00:30:42.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.529+0000 7efce3ae5640 1 -- 172.21.15.46:0/1595241501 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcdc14fa40 con 0x7efcdc159c30 2023-12-02T00:30:42.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.529+0000 7efcd27fc640 1 -- 172.21.15.46:0/1595241501 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efccc00cdc0 con 0x7efcdc159c30 2023-12-02T00:30:42.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.529+0000 7efce3ae5640 1 -- 172.21.15.46:0/1595241501 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcdc14ff60 con 0x7efcdc159c30 2023-12-02T00:30:42.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.530+0000 7efce3ae5640 1 -- 172.21.15.46:0/1595241501 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcdc14fc60 con 0x7efcdc159c30 2023-12-02T00:30:42.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.530+0000 7efcd27fc640 1 -- 172.21.15.46:0/1595241501 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efccc01d460 con 0x7efcdc159c30 2023-12-02T00:30:42.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.530+0000 7efcd27fc640 1 --2- 172.21.15.46:0/1595241501 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7efcc006ae70 0x7efcc006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:42.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.531+0000 7efcd27fc640 1 -- 172.21.15.46:0/1595241501 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7efccc089720 con 0x7efcdc159c30 2023-12-02T00:30:42.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.531+0000 7efce1881640 1 --2- 172.21.15.46:0/1595241501 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7efcc006ae70 0x7efcc006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:42.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.532+0000 7efce1881640 1 --2- 172.21.15.46:0/1595241501 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7efcc006ae70 0x7efcc006d330 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7efcd8009bc0 tx=0x7efcd8009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:42.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.535+0000 7efcd27fc640 1 -- 172.21.15.46:0/1595241501 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efccc054640 con 0x7efcdc159c30 2023-12-02T00:30:42.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.748+0000 7efce3ae5640 1 -- 172.21.15.46:0/1595241501 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efcdc15de90 con 0x7efcdc159c30 2023-12-02T00:30:42.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.748+0000 7efcd27fc640 1 -- 172.21.15.46:0/1595241501 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4919 (secure 0 0 0) 0x7efccc057c60 con 0x7efcdc159c30 2023-12-02T00:30:42.750 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:30:42.750 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4811,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/3097254664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3097254664},{"type":"v1","addr":"172.21.15.162:6835","nonce":3097254664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":4826,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/3489476943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3489476943},{"type":"v1","addr":"172.21.15.162:6837","nonce":3489476943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5274,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1283185308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":1283185308},{"type":"v1","addr":"172.21.15.46:6835","nonce":1283185308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-02T00:29:57.279169+0000","modified":"2023-12-02T00:30:09.302069+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5277},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5277":{"gid":5277,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":5,"addr":"172.21.15.46:6834/2036053042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2036053042},{"type":"v1","addr":"172.21.15.46:6834","nonce":2036053042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-02T00:30:42.750 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 14 2023-12-02T00:30:42.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.750+0000 7efce3ae5640 1 -- 172.21.15.46:0/1595241501 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7efcc006ae70 msgr2=0x7efcc006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:42.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.750+0000 7efce3ae5640 1 --2- 172.21.15.46:0/1595241501 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7efcc006ae70 0x7efcc006d330 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7efcd8009bc0 tx=0x7efcd8009190 comp rx=0 tx=0).stop 2023-12-02T00:30:42.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.750+0000 7efce3ae5640 1 -- 172.21.15.46:0/1595241501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efcdc159c30 msgr2=0x7efcdc14e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:42.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.750+0000 7efce3ae5640 1 --2- 172.21.15.46:0/1595241501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efcdc159c30 0x7efcdc14e5e0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7efccc000fb0 tx=0x7efccc00eec0 comp rx=0 tx=0).stop 2023-12-02T00:30:42.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.750+0000 7efce3ae5640 1 -- 172.21.15.46:0/1595241501 shutdown_connections 2023-12-02T00:30:42.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.750+0000 7efce3ae5640 1 --2- 172.21.15.46:0/1595241501 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7efcc006ae70 0x7efcc006d330 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:42.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.750+0000 7efce3ae5640 1 --2- 172.21.15.46:0/1595241501 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcdc12f6c0 0x7efcdc12fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:42.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.750+0000 7efce3ae5640 1 --2- 172.21.15.46:0/1595241501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efcdc159c30 0x7efcdc14e5e0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:42.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.750+0000 7efce3ae5640 1 --2- 172.21.15.46:0/1595241501 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcdc159850 0x7efcdc146cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:42.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.750+0000 7efce3ae5640 1 -- 172.21.15.46:0/1595241501 >> 172.21.15.46:0/1595241501 conn(0x7efcdc06a4d0 msgr2=0x7efcdc0cd900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:42.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.750+0000 7efce3ae5640 1 -- 172.21.15.46:0/1595241501 shutdown_connections 2023-12-02T00:30:42.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.750+0000 7efce3ae5640 1 -- 172.21.15.46:0/1595241501 wait complete. 2023-12-02T00:30:42.762 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-02T00:30:42.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.868+0000 7fe5eb0d0640 1 Processor -- start 2023-12-02T00:30:42.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.868+0000 7fe5eb0d0640 1 -- start start 2023-12-02T00:30:42.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.868+0000 7fe5eb0d0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5e415d330 0x7fe5e40b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:42.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.868+0000 7fe5eb0d0640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe5e40d2d90 0x7fe5e40b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:42.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.868+0000 7fe5eb0d0640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5e40b2b10 con 0x7fe5e40d2910 2023-12-02T00:30:42.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.868+0000 7fe5eb0d0640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5e40d1590 con 0x7fe5e40d2d90 2023-12-02T00:30:42.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.868+0000 7fe5eb0d0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e40d1870 con 0x7fe5e415d330 2023-12-02T00:30:42.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.869+0000 7fe5e966d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe5e40d2d90 0x7fe5e40b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:42.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.869+0000 7fe5e966d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe5e40d2d90 0x7fe5e40b6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33252/0 (socket says 172.21.15.46:33252) 2023-12-02T00:30:42.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.869+0000 7fe5e966d640 1 -- 172.21.15.46:0/3876145522 learned_addr learned my addr 172.21.15.46:0/3876145522 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:42.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.869+0000 7fe5e8e6c640 1 --2- 172.21.15.46:0/3876145522 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5e415d330 0x7fe5e40b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:42.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.869+0000 7fe5e966d640 1 -- 172.21.15.46:0/3876145522 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5e415d330 msgr2=0x7fe5e40b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:42.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.869+0000 7fe5e966d640 1 --2- 172.21.15.46:0/3876145522 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5e415d330 0x7fe5e40b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:42.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.869+0000 7fe5e966d640 1 -- 172.21.15.46:0/3876145522 >> v1:172.21.15.46:6789/0 conn(0x7fe5e40d2910 legacy=0x7fe5e40d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:42.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.869+0000 7fe5e37fe640 1 -- 172.21.15.46:0/3876145522 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2921201972 0 0) 0x7fe5e40b2b10 con 0x7fe5e40d2910 2023-12-02T00:30:42.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.869+0000 7fe5e966d640 1 -- 172.21.15.46:0/3876145522 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e40d1b50 con 0x7fe5e40d2d90 2023-12-02T00:30:42.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.869+0000 7fe5e8e6c640 1 --2- 172.21.15.46:0/3876145522 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5e415d330 0x7fe5e40b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:30:42.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.870+0000 7fe5e966d640 1 --2- 172.21.15.46:0/3876145522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5e40d2d90 0x7fe5e40b6390 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc009e50 tx=0x7fe5dc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e2d79d85e815f77 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:42.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.870+0000 7fe5e37fe640 1 -- 172.21.15.46:0/3876145522 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc02ca70 con 0x7fe5e40d2d90 2023-12-02T00:30:42.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.870+0000 7fe5e37fe640 1 -- 172.21.15.46:0/3876145522 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5dc02cbd0 con 0x7fe5e40d2d90 2023-12-02T00:30:42.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.870+0000 7fe5e37fe640 1 -- 172.21.15.46:0/3876145522 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc0343f0 con 0x7fe5e40d2d90 2023-12-02T00:30:42.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.870+0000 7fe5eb0d0640 1 -- 172.21.15.46:0/3876145522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5e40d2d90 msgr2=0x7fe5e40b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:42.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.870+0000 7fe5eb0d0640 1 --2- 172.21.15.46:0/3876145522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5e40d2d90 0x7fe5e40b6390 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc009e50 tx=0x7fe5dc02bed0 comp rx=0 tx=0).stop 2023-12-02T00:30:42.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.870+0000 7fe5eb0d0640 1 -- 172.21.15.46:0/3876145522 shutdown_connections 2023-12-02T00:30:42.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.870+0000 7fe5eb0d0640 1 --2- 172.21.15.46:0/3876145522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5e40d2d90 0x7fe5e40b6390 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:42.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.870+0000 7fe5eb0d0640 1 --2- 172.21.15.46:0/3876145522 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5e415d330 0x7fe5e40b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:42.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.870+0000 7fe5eb0d0640 1 -- 172.21.15.46:0/3876145522 >> 172.21.15.46:0/3876145522 conn(0x7fe5e4072020 msgr2=0x7fe5e4072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:42.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.870+0000 7fe5eb0d0640 1 -- 172.21.15.46:0/3876145522 shutdown_connections 2023-12-02T00:30:42.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.870+0000 7fe5eb0d0640 1 -- 172.21.15.46:0/3876145522 wait complete. 2023-12-02T00:30:42.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.872+0000 7fe5eb0d0640 1 Processor -- start 2023-12-02T00:30:42.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.872+0000 7fe5eb0d0640 1 -- start start 2023-12-02T00:30:42.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.872+0000 7fe5eb0d0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5e40d2910 0x7fe5e40b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:42.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.873+0000 7fe5eb0d0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5e40d2d90 0x7fe5e41f2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:42.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.873+0000 7fe5eb0d0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5e415d330 0x7fe5e415fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:42.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.873+0000 7fe5eb0d0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e41f3c30 con 0x7fe5e40d2d90 2023-12-02T00:30:42.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.873+0000 7fe5eb0d0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e40b8320 con 0x7fe5e415d330 2023-12-02T00:30:42.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.873+0000 7fe5eb0d0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e41f3390 con 0x7fe5e40d2910 2023-12-02T00:30:42.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.873+0000 7fe5e3fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5e40d2d90 0x7fe5e41f2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:42.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.873+0000 7fe5e3fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5e40d2d90 0x7fe5e41f2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33254/0 (socket says 172.21.15.46:33254) 2023-12-02T00:30:42.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.873+0000 7fe5e3fff640 1 -- 172.21.15.46:0/3632609083 learned_addr learned my addr 172.21.15.46:0/3632609083 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:42.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.873+0000 7fe5e8e6c640 1 --2- 172.21.15.46:0/3632609083 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5e40d2910 0x7fe5e40b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:42.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.873+0000 7fe5e966d640 1 --2- 172.21.15.46:0/3632609083 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5e415d330 0x7fe5e415fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:42.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.873+0000 7fe5e3fff640 1 -- 172.21.15.46:0/3632609083 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5e40d2910 msgr2=0x7fe5e40b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:42.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.873+0000 7fe5e3fff640 1 --2- 172.21.15.46:0/3632609083 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5e40d2910 0x7fe5e40b45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:42.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.873+0000 7fe5e3fff640 1 -- 172.21.15.46:0/3632609083 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5e415d330 msgr2=0x7fe5e415fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:42.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.873+0000 7fe5e3fff640 1 --2- 172.21.15.46:0/3632609083 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5e415d330 0x7fe5e415fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:42.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.873+0000 7fe5e3fff640 1 -- 172.21.15.46:0/3632609083 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5dc009b30 con 0x7fe5e40d2d90 2023-12-02T00:30:42.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.873+0000 7fe5e8e6c640 1 --2- 172.21.15.46:0/3632609083 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5e40d2910 0x7fe5e40b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:30:42.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.873+0000 7fe5e966d640 1 --2- 172.21.15.46:0/3632609083 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5e415d330 0x7fe5e415fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:30:42.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.873+0000 7fe5e3fff640 1 --2- 172.21.15.46:0/3632609083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5e40d2d90 0x7fe5e41f2c80 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe5d000c4e0 tx=0x7fe5d000c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:42.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.873+0000 7fe5e17fa640 1 -- 172.21.15.46:0/3632609083 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d0005480 con 0x7fe5e40d2d90 2023-12-02T00:30:42.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.873+0000 7fe5e17fa640 1 -- 172.21.15.46:0/3632609083 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d00035f0 con 0x7fe5e40d2d90 2023-12-02T00:30:42.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.873+0000 7fe5e17fa640 1 -- 172.21.15.46:0/3632609083 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d00039c0 con 0x7fe5e40d2d90 2023-12-02T00:30:42.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.873+0000 7fe5eb0d0640 1 -- 172.21.15.46:0/3632609083 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5e41f3670 con 0x7fe5e40d2d90 2023-12-02T00:30:42.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.873+0000 7fe5eb0d0640 1 -- 172.21.15.46:0/3632609083 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5e41f63f0 con 0x7fe5e40d2d90 2023-12-02T00:30:42.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.874+0000 7fe5eb0d0640 1 -- 172.21.15.46:0/3632609083 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5e41f6250 con 0x7fe5e40d2d90 2023-12-02T00:30:42.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.875+0000 7fe5e17fa640 1 -- 172.21.15.46:0/3632609083 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe5d0003b20 con 0x7fe5e40d2d90 2023-12-02T00:30:42.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.875+0000 7fe5e17fa640 1 --2- 172.21.15.46:0/3632609083 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe5c806ae40 0x7fe5c806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:42.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.875+0000 7fe5e17fa640 1 -- 172.21.15.46:0/3632609083 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fe5d0088be0 con 0x7fe5e40d2d90 2023-12-02T00:30:42.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.875+0000 7fe5e8e6c640 1 --2- 172.21.15.46:0/3632609083 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe5c806ae40 0x7fe5c806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:42.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.876+0000 7fe5e8e6c640 1 --2- 172.21.15.46:0/3632609083 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe5c806ae40 0x7fe5c806d300 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe5d800a990 tx=0x7fe5d8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:42.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:42.879+0000 7fe5e17fa640 1 -- 172.21.15.46:0/3632609083 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe5e41f6250 con 0x7fe5e40d2d90 2023-12-02T00:30:43.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.094+0000 7fe5eb0d0640 1 -- 172.21.15.46:0/3632609083 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe5e41f6250 con 0x7fe5e40d2d90 2023-12-02T00:30:43.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.094+0000 7fe5e17fa640 1 -- 172.21.15.46:0/3632609083 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4919 (secure 0 0 0) 0x7fe5e41f6250 con 0x7fe5e40d2d90 2023-12-02T00:30:43.096 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:30:43.096 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4811,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/3097254664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3097254664},{"type":"v1","addr":"172.21.15.162:6835","nonce":3097254664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":4826,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/3489476943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3489476943},{"type":"v1","addr":"172.21.15.162:6837","nonce":3489476943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5274,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1283185308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":1283185308},{"type":"v1","addr":"172.21.15.46:6835","nonce":1283185308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-02T00:29:57.279169+0000","modified":"2023-12-02T00:30:09.302069+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5277},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5277":{"gid":5277,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":5,"addr":"172.21.15.46:6834/2036053042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2036053042},{"type":"v1","addr":"172.21.15.46:6834","nonce":2036053042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-02T00:30:43.096 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 14 2023-12-02T00:30:43.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.096+0000 7fe5eb0d0640 1 -- 172.21.15.46:0/3632609083 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe5c806ae40 msgr2=0x7fe5c806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:43.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.096+0000 7fe5eb0d0640 1 --2- 172.21.15.46:0/3632609083 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe5c806ae40 0x7fe5c806d300 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe5d800a990 tx=0x7fe5d8008040 comp rx=0 tx=0).stop 2023-12-02T00:30:43.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.096+0000 7fe5eb0d0640 1 -- 172.21.15.46:0/3632609083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5e40d2d90 msgr2=0x7fe5e41f2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:43.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.096+0000 7fe5eb0d0640 1 --2- 172.21.15.46:0/3632609083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5e40d2d90 0x7fe5e41f2c80 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe5d000c4e0 tx=0x7fe5d000c9a0 comp rx=0 tx=0).stop 2023-12-02T00:30:43.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.096+0000 7fe5eb0d0640 1 -- 172.21.15.46:0/3632609083 shutdown_connections 2023-12-02T00:30:43.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.096+0000 7fe5eb0d0640 1 --2- 172.21.15.46:0/3632609083 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe5c806ae40 0x7fe5c806d300 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:43.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.096+0000 7fe5eb0d0640 1 --2- 172.21.15.46:0/3632609083 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5e415d330 0x7fe5e415fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:43.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.096+0000 7fe5eb0d0640 1 --2- 172.21.15.46:0/3632609083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5e40d2d90 0x7fe5e41f2c80 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:43.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.096+0000 7fe5eb0d0640 1 --2- 172.21.15.46:0/3632609083 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5e40d2910 0x7fe5e40b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:43.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.096+0000 7fe5eb0d0640 1 -- 172.21.15.46:0/3632609083 >> 172.21.15.46:0/3632609083 conn(0x7fe5e4072020 msgr2=0x7fe5e40d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:43.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.096+0000 7fe5eb0d0640 1 -- 172.21.15.46:0/3632609083 shutdown_connections 2023-12-02T00:30:43.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.096+0000 7fe5eb0d0640 1 -- 172.21.15.46:0/3632609083 wait complete. 2023-12-02T00:30:43.109 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:30:43.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.215+0000 7fc4efd83640 1 Processor -- start 2023-12-02T00:30:43.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.215+0000 7fc4efd83640 1 -- start start 2023-12-02T00:30:43.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.216+0000 7fc4efd83640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc4e815d2d0 0x7fc4e80b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:43.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.216+0000 7fc4efd83640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc4e80ce150 0x7fc4e80b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:43.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.216+0000 7fc4efd83640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4e80d1bb0 con 0x7fc4e80d2040 2023-12-02T00:30:43.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.216+0000 7fc4efd83640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4e815d6b0 con 0x7fc4e80ce150 2023-12-02T00:30:43.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.216+0000 7fc4efd83640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4e80ce700 con 0x7fc4e815d2d0 2023-12-02T00:30:43.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.216+0000 7fc4ee320640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc4e80ce150 0x7fc4e80b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:43.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.216+0000 7fc4ee320640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc4e80ce150 0x7fc4e80b51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33264/0 (socket says 172.21.15.46:33264) 2023-12-02T00:30:43.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.216+0000 7fc4ee320640 1 -- 172.21.15.46:0/3184246650 learned_addr learned my addr 172.21.15.46:0/3184246650 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:43.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.216+0000 7fc4edb1f640 1 --2- 172.21.15.46:0/3184246650 >> v2:172.21.15.162:3300/0 conn(0x7fc4e815d2d0 0x7fc4e80b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:43.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.216+0000 7fc4ee320640 1 -- 172.21.15.46:0/3184246650 >> v2:172.21.15.162:3300/0 conn(0x7fc4e815d2d0 msgr2=0x7fc4e80b4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:43.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.216+0000 7fc4ee320640 1 --2- 172.21.15.46:0/3184246650 >> v2:172.21.15.162:3300/0 conn(0x7fc4e815d2d0 0x7fc4e80b4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:43.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.216+0000 7fc4ee320640 1 -- 172.21.15.46:0/3184246650 >> v1:172.21.15.162:6789/0 conn(0x7fc4e80d2040 legacy=0x7fc4e80cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:43.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.216+0000 7fc4ee320640 1 -- 172.21.15.46:0/3184246650 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e80d1280 con 0x7fc4e80ce150 2023-12-02T00:30:43.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.216+0000 7fc4edb1f640 1 --2- 172.21.15.46:0/3184246650 >> v2:172.21.15.162:3300/0 conn(0x7fc4e815d2d0 0x7fc4e80b4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:30:43.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.217+0000 7fc4ee320640 1 --2- 172.21.15.46:0/3184246650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4e80ce150 0x7fc4e80b51a0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fc4e806c5d0 tx=0x7fc4e002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c503c78fe58b92 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:43.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.217+0000 7fc4ecb1d640 1 -- 172.21.15.46:0/3184246650 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e002ca80 con 0x7fc4e80ce150 2023-12-02T00:30:43.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.217+0000 7fc4ecb1d640 1 -- 172.21.15.46:0/3184246650 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4e002cbe0 con 0x7fc4e80ce150 2023-12-02T00:30:43.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.217+0000 7fc4ecb1d640 1 -- 172.21.15.46:0/3184246650 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e00343f0 con 0x7fc4e80ce150 2023-12-02T00:30:43.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.217+0000 7fc4efd83640 1 -- 172.21.15.46:0/3184246650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4e80ce150 msgr2=0x7fc4e80b51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:43.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.217+0000 7fc4efd83640 1 --2- 172.21.15.46:0/3184246650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4e80ce150 0x7fc4e80b51a0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fc4e806c5d0 tx=0x7fc4e002bee0 comp rx=0 tx=0).stop 2023-12-02T00:30:43.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.218+0000 7fc4efd83640 1 -- 172.21.15.46:0/3184246650 shutdown_connections 2023-12-02T00:30:43.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.218+0000 7fc4efd83640 1 --2- 172.21.15.46:0/3184246650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4e80ce150 0x7fc4e80b51a0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:43.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.218+0000 7fc4efd83640 1 --2- 172.21.15.46:0/3184246650 >> v2:172.21.15.162:3300/0 conn(0x7fc4e815d2d0 0x7fc4e80b4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:43.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.218+0000 7fc4efd83640 1 -- 172.21.15.46:0/3184246650 >> 172.21.15.46:0/3184246650 conn(0x7fc4e8072050 msgr2=0x7fc4e8072460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:43.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.218+0000 7fc4efd83640 1 -- 172.21.15.46:0/3184246650 shutdown_connections 2023-12-02T00:30:43.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.218+0000 7fc4efd83640 1 -- 172.21.15.46:0/3184246650 wait complete. 2023-12-02T00:30:43.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.219+0000 7fc4efd83640 1 Processor -- start 2023-12-02T00:30:43.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.219+0000 7fc4efd83640 1 -- start start 2023-12-02T00:30:43.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.219+0000 7fc4efd83640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4e80ce150 0x7fc4e80c8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:43.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.219+0000 7fc4efd83640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4e80d2040 0x7fc4e80c9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:43.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.219+0000 7fc4efd83640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc4e815d2d0 0x7fc4e80cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:43.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.219+0000 7fc4efd83640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e80cd7e0 con 0x7fc4e80ce150 2023-12-02T00:30:43.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.219+0000 7fc4efd83640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e80ccb60 con 0x7fc4e80d2040 2023-12-02T00:30:43.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.219+0000 7fc4efd83640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e80cce40 con 0x7fc4e815d2d0 2023-12-02T00:30:43.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.220+0000 7fc4edb1f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4e80ce150 0x7fc4e80c8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:43.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.220+0000 7fc4edb1f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4e80ce150 0x7fc4e80c8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33280/0 (socket says 172.21.15.46:33280) 2023-12-02T00:30:43.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.220+0000 7fc4edb1f640 1 -- 172.21.15.46:0/737082303 learned_addr learned my addr 172.21.15.46:0/737082303 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:43.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.220+0000 7fc4ed31e640 1 --2- 172.21.15.46:0/737082303 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4e80d2040 0x7fc4e80c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:43.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.220+0000 7fc4ee320640 1 --2- 172.21.15.46:0/737082303 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc4e815d2d0 0x7fc4e80cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:43.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.220+0000 7fc4edb1f640 1 -- 172.21.15.46:0/737082303 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc4e815d2d0 msgr2=0x7fc4e80cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:43.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.220+0000 7fc4edb1f640 1 --2- 172.21.15.46:0/737082303 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc4e815d2d0 0x7fc4e80cbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:43.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.220+0000 7fc4edb1f640 1 -- 172.21.15.46:0/737082303 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4e80d2040 msgr2=0x7fc4e80c9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:43.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.220+0000 7fc4edb1f640 1 --2- 172.21.15.46:0/737082303 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4e80d2040 0x7fc4e80c9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:43.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.220+0000 7fc4edb1f640 1 -- 172.21.15.46:0/737082303 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e0009b30 con 0x7fc4e80ce150 2023-12-02T00:30:43.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.220+0000 7fc4ee320640 1 --2- 172.21.15.46:0/737082303 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc4e815d2d0 0x7fc4e80cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:30:43.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.220+0000 7fc4edb1f640 1 --2- 172.21.15.46:0/737082303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4e80ce150 0x7fc4e80c8ec0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fc4e400c7f0 tx=0x7fc4e400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:43.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.220+0000 7fc4de7fc640 1 -- 172.21.15.46:0/737082303 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e4007820 con 0x7fc4e80ce150 2023-12-02T00:30:43.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.220+0000 7fc4de7fc640 1 -- 172.21.15.46:0/737082303 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4e4007980 con 0x7fc4e80ce150 2023-12-02T00:30:43.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.220+0000 7fc4de7fc640 1 -- 172.21.15.46:0/737082303 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e4007cf0 con 0x7fc4e80ce150 2023-12-02T00:30:43.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.220+0000 7fc4efd83640 1 -- 172.21.15.46:0/737082303 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4e80cd120 con 0x7fc4e80ce150 2023-12-02T00:30:43.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.220+0000 7fc4efd83640 1 -- 172.21.15.46:0/737082303 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4e80cdc00 con 0x7fc4e80ce150 2023-12-02T00:30:43.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.221+0000 7fc4efd83640 1 -- 172.21.15.46:0/737082303 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4e80cda50 con 0x7fc4e80ce150 2023-12-02T00:30:43.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.222+0000 7fc4de7fc640 1 -- 172.21.15.46:0/737082303 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc4e40113f0 con 0x7fc4e80ce150 2023-12-02T00:30:43.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.222+0000 7fc4de7fc640 1 --2- 172.21.15.46:0/737082303 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc4cc06ae40 0x7fc4cc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:43.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.222+0000 7fc4de7fc640 1 -- 172.21.15.46:0/737082303 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fc4e4089a10 con 0x7fc4e80ce150 2023-12-02T00:30:43.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.223+0000 7fc4ed31e640 1 --2- 172.21.15.46:0/737082303 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc4cc06ae40 0x7fc4cc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:43.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.223+0000 7fc4ed31e640 1 --2- 172.21.15.46:0/737082303 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc4cc06ae40 0x7fc4cc06d300 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc4e80ce120 tx=0x7fc4d8007e30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:43.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.227+0000 7fc4de7fc640 1 -- 172.21.15.46:0/737082303 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc4e40549d0 con 0x7fc4e80ce150 2023-12-02T00:30:43.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.403+0000 7fc4efd83640 1 -- 172.21.15.46:0/737082303 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc4e8069400 con 0x7fc4e80ce150 2023-12-02T00:30:43.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.405+0000 7fc4de7fc640 1 -- 172.21.15.46:0/737082303 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+18324 (secure 0 0 0) 0x7fc4e401f090 con 0x7fc4e80ce150 2023-12-02T00:30:43.406 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:30:43.406 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":38,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:30:10.714057+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-02T00:29:55.676572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-02T00:29:56.244869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"data_new","create_time":"2023-12-02T00:30:08.897990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:30:43.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.407+0000 7fc4efd83640 1 -- 172.21.15.46:0/737082303 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc4cc06ae40 msgr2=0x7fc4cc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:43.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.407+0000 7fc4efd83640 1 --2- 172.21.15.46:0/737082303 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc4cc06ae40 0x7fc4cc06d300 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc4e80ce120 tx=0x7fc4d8007e30 comp rx=0 tx=0).stop 2023-12-02T00:30:43.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.407+0000 7fc4efd83640 1 -- 172.21.15.46:0/737082303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4e80ce150 msgr2=0x7fc4e80c8ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:43.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.407+0000 7fc4efd83640 1 --2- 172.21.15.46:0/737082303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4e80ce150 0x7fc4e80c8ec0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fc4e400c7f0 tx=0x7fc4e400ccb0 comp rx=0 tx=0).stop 2023-12-02T00:30:43.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.407+0000 7fc4efd83640 1 -- 172.21.15.46:0/737082303 shutdown_connections 2023-12-02T00:30:43.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.407+0000 7fc4efd83640 1 --2- 172.21.15.46:0/737082303 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc4cc06ae40 0x7fc4cc06d300 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:43.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.407+0000 7fc4efd83640 1 --2- 172.21.15.46:0/737082303 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc4e815d2d0 0x7fc4e80cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:43.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.407+0000 7fc4efd83640 1 --2- 172.21.15.46:0/737082303 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4e80d2040 0x7fc4e80c9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:43.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.407+0000 7fc4efd83640 1 --2- 172.21.15.46:0/737082303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4e80ce150 0x7fc4e80c8ec0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:43.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.407+0000 7fc4efd83640 1 -- 172.21.15.46:0/737082303 >> 172.21.15.46:0/737082303 conn(0x7fc4e8072050 msgr2=0x7fc4e80b3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:43.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.407+0000 7fc4efd83640 1 -- 172.21.15.46:0/737082303 shutdown_connections 2023-12-02T00:30:43.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.407+0000 7fc4efd83640 1 -- 172.21.15.46:0/737082303 wait complete. 2023-12-02T00:30:43.419 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-02T00:30:43.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.525+0000 7f8767a90640 1 Processor -- start 2023-12-02T00:30:43.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.526+0000 7f8767a90640 1 -- start start 2023-12-02T00:30:43.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.526+0000 7f8767a90640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f876015d330 0x7f87600b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:43.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.526+0000 7f8767a90640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f87600d2910 0x7f87600d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:43.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.526+0000 7f8767a90640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f87600d3730 0x7f87600b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:43.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.526+0000 7f8767a90640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87600b2b10 con 0x7f87600d2910 2023-12-02T00:30:43.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.526+0000 7f8767a90640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87600d3b90 con 0x7f876015d330 2023-12-02T00:30:43.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.526+0000 7f8767a90640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87600d1590 con 0x7f87600d3730 2023-12-02T00:30:43.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.527+0000 7f876502b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f87600d2910 0x7f87600d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:43.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.527+0000 7f876502b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f87600d2910 0x7f87600d31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33282/0 (socket says 172.21.15.46:33282) 2023-12-02T00:30:43.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.527+0000 7f876502b640 1 -- 172.21.15.46:0/3159465986 learned_addr learned my addr 172.21.15.46:0/3159465986 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:43.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.527+0000 7f876582c640 1 --2- 172.21.15.46:0/3159465986 >> v2:172.21.15.162:3300/0 conn(0x7f876015d330 0x7f87600b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:43.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.527+0000 7f876602d640 1 --2- 172.21.15.46:0/3159465986 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f87600d3730 0x7f87600b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:43.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.527+0000 7f876502b640 1 -- 172.21.15.46:0/3159465986 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f87600d3730 msgr2=0x7f87600b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:43.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.527+0000 7f876502b640 1 --2- 172.21.15.46:0/3159465986 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f87600d3730 0x7f87600b6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:43.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.527+0000 7f876502b640 1 -- 172.21.15.46:0/3159465986 >> v2:172.21.15.162:3300/0 conn(0x7f876015d330 msgr2=0x7f87600b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:43.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.527+0000 7f876502b640 1 --2- 172.21.15.46:0/3159465986 >> v2:172.21.15.162:3300/0 conn(0x7f876015d330 0x7f87600b8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:43.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.527+0000 7f876502b640 1 -- 172.21.15.46:0/3159465986 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87600d1870 con 0x7f87600d2910 2023-12-02T00:30:43.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.528+0000 7f876502b640 1 --2- 172.21.15.46:0/3159465986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87600d2910 0x7f87600d31f0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f87600d0d10 tx=0x7f875002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=815b8a24bc28ec48 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:43.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.528+0000 7f876482a640 1 -- 172.21.15.46:0/3159465986 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87500317e0 con 0x7f87600d2910 2023-12-02T00:30:43.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.528+0000 7f876482a640 1 -- 172.21.15.46:0/3159465986 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8750031940 con 0x7f87600d2910 2023-12-02T00:30:43.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.528+0000 7f876482a640 1 -- 172.21.15.46:0/3159465986 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8750031cd0 con 0x7f87600d2910 2023-12-02T00:30:43.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.528+0000 7f8767a90640 1 -- 172.21.15.46:0/3159465986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87600d2910 msgr2=0x7f87600d31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:43.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.528+0000 7f8767a90640 1 --2- 172.21.15.46:0/3159465986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87600d2910 0x7f87600d31f0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f87600d0d10 tx=0x7f875002b880 comp rx=0 tx=0).stop 2023-12-02T00:30:43.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.528+0000 7f8767a90640 1 -- 172.21.15.46:0/3159465986 shutdown_connections 2023-12-02T00:30:43.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.528+0000 7f8767a90640 1 --2- 172.21.15.46:0/3159465986 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f87600d3730 0x7f87600b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:43.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.528+0000 7f8767a90640 1 --2- 172.21.15.46:0/3159465986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87600d2910 0x7f87600d31f0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:43.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.528+0000 7f8767a90640 1 --2- 172.21.15.46:0/3159465986 >> v2:172.21.15.162:3300/0 conn(0x7f876015d330 0x7f87600b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:43.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.528+0000 7f8767a90640 1 -- 172.21.15.46:0/3159465986 >> 172.21.15.46:0/3159465986 conn(0x7f8760072020 msgr2=0x7f8760072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:43.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.529+0000 7f8767a90640 1 -- 172.21.15.46:0/3159465986 shutdown_connections 2023-12-02T00:30:43.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.529+0000 7f8767a90640 1 -- 172.21.15.46:0/3159465986 wait complete. 2023-12-02T00:30:43.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.530+0000 7f8767a90640 1 Processor -- start 2023-12-02T00:30:43.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.530+0000 7f8767a90640 1 -- start start 2023-12-02T00:30:43.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.530+0000 7f8767a90640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87600d2910 0x7f87601f3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:43.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.530+0000 7f8767a90640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f87600d3730 0x7f8760149430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:43.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.530+0000 7f8767a90640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f876015d330 0x7f8760153580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:43.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.530+0000 7f8767a90640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87600c9b40 con 0x7f876015d330 2023-12-02T00:30:43.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.530+0000 7f8767a90640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f876015e490 con 0x7f87600d2910 2023-12-02T00:30:43.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.530+0000 7f8767a90640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f876015e770 con 0x7f87600d3730 2023-12-02T00:30:43.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.531+0000 7f876602d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f876015d330 0x7f8760153580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:43.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.531+0000 7f876602d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f876015d330 0x7f8760153580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33288/0 (socket says 172.21.15.46:33288) 2023-12-02T00:30:43.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.531+0000 7f876602d640 1 -- 172.21.15.46:0/1631497268 learned_addr learned my addr 172.21.15.46:0/1631497268 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:43.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.531+0000 7f876582c640 1 --2- 172.21.15.46:0/1631497268 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87600d2910 0x7f87601f3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:43.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.531+0000 7f876502b640 1 --2- 172.21.15.46:0/1631497268 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f87600d3730 0x7f8760149430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:43.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.531+0000 7f876602d640 1 -- 172.21.15.46:0/1631497268 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f87600d3730 msgr2=0x7f8760149430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:43.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.531+0000 7f876602d640 1 --2- 172.21.15.46:0/1631497268 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f87600d3730 0x7f8760149430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:43.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.531+0000 7f876602d640 1 -- 172.21.15.46:0/1631497268 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87600d2910 msgr2=0x7f87601f3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:43.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.531+0000 7f876602d640 1 --2- 172.21.15.46:0/1631497268 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87600d2910 0x7f87601f3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:43.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.531+0000 7f876602d640 1 -- 172.21.15.46:0/1631497268 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87500094d0 con 0x7f876015d330 2023-12-02T00:30:43.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.531+0000 7f876602d640 1 --2- 172.21.15.46:0/1631497268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f876015d330 0x7f8760153580 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f875800ccf0 tx=0x7f875800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:43.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.531+0000 7f87567fc640 1 -- 172.21.15.46:0/1631497268 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8758003b90 con 0x7f876015d330 2023-12-02T00:30:43.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.531+0000 7f87567fc640 1 -- 172.21.15.46:0/1631497268 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8758003cf0 con 0x7f876015d330 2023-12-02T00:30:43.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.531+0000 7f87567fc640 1 -- 172.21.15.46:0/1631497268 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8758004240 con 0x7f876015d330 2023-12-02T00:30:43.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.531+0000 7f8767a90640 1 -- 172.21.15.46:0/1631497268 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f876015ea50 con 0x7f876015d330 2023-12-02T00:30:43.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.531+0000 7f8767a90640 1 -- 172.21.15.46:0/1631497268 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8760149970 con 0x7f876015d330 2023-12-02T00:30:43.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.532+0000 7f8767a90640 1 -- 172.21.15.46:0/1631497268 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8760069400 con 0x7f876015d330 2023-12-02T00:30:43.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.533+0000 7f87567fc640 1 -- 172.21.15.46:0/1631497268 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f87580043a0 con 0x7f876015d330 2023-12-02T00:30:43.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.533+0000 7f87567fc640 1 --2- 172.21.15.46:0/1631497268 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f874406ae40 0x7f874406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:43.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.533+0000 7f87567fc640 1 -- 172.21.15.46:0/1631497268 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f8758012070 con 0x7f876015d330 2023-12-02T00:30:43.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.534+0000 7f876582c640 1 --2- 172.21.15.46:0/1631497268 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f874406ae40 0x7f874406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:43.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.534+0000 7f876582c640 1 --2- 172.21.15.46:0/1631497268 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f874406ae40 0x7f874406d300 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f875c00aac0 tx=0x7f875c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:43.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.538+0000 7f87567fc640 1 -- 172.21.15.46:0/1631497268 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8758054f80 con 0x7f876015d330 2023-12-02T00:30:43.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.750+0000 7f8767a90640 1 -- 172.21.15.46:0/1631497268 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f8760072780 con 0x7f876015d330 2023-12-02T00:30:43.752 INFO:tasks.ceph.mon.a.smithi046.stderr:2023-12-02T00:30:43.751+0000 7f59c5f43640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-02T00:30:43.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.752+0000 7f87567fc640 1 -- 172.21.15.46:0/1631497268 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v15) v1 ==== 161+0+0 (secure 0 0 0) 0x7f87580585a0 con 0x7f876015d330 2023-12-02T00:30:43.754 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-02T00:30:43.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.756+0000 7f8767a90640 1 -- 172.21.15.46:0/1631497268 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f874406ae40 msgr2=0x7f874406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:43.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.756+0000 7f8767a90640 1 --2- 172.21.15.46:0/1631497268 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f874406ae40 0x7f874406d300 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f875c00aac0 tx=0x7f875c008040 comp rx=0 tx=0).stop 2023-12-02T00:30:43.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.756+0000 7f8767a90640 1 -- 172.21.15.46:0/1631497268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f876015d330 msgr2=0x7f8760153580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:43.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.756+0000 7f8767a90640 1 --2- 172.21.15.46:0/1631497268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f876015d330 0x7f8760153580 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f875800ccf0 tx=0x7f875800a660 comp rx=0 tx=0).stop 2023-12-02T00:30:43.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.756+0000 7f8767a90640 1 -- 172.21.15.46:0/1631497268 shutdown_connections 2023-12-02T00:30:43.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.756+0000 7f8767a90640 1 --2- 172.21.15.46:0/1631497268 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f874406ae40 0x7f874406d300 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:43.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.756+0000 7f8767a90640 1 --2- 172.21.15.46:0/1631497268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f876015d330 0x7f8760153580 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:43.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.756+0000 7f8767a90640 1 --2- 172.21.15.46:0/1631497268 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f87600d3730 0x7f8760149430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:43.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.756+0000 7f8767a90640 1 --2- 172.21.15.46:0/1631497268 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87600d2910 0x7f87601f3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:43.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.756+0000 7f8767a90640 1 -- 172.21.15.46:0/1631497268 >> 172.21.15.46:0/1631497268 conn(0x7f8760072020 msgr2=0x7f87600b6a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:43.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.756+0000 7f8767a90640 1 -- 172.21.15.46:0/1631497268 shutdown_connections 2023-12-02T00:30:43.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.756+0000 7f8767a90640 1 -- 172.21.15.46:0/1631497268 wait complete. 2023-12-02T00:30:43.759 INFO:tasks.ceph.mds.a.smithi046.stderr: -3538> 2023-12-02T00:30:00.467+0000 7fc9c4aea640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-02T00:30:43.771 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-02T00:30:43.792 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2023-12-02T00:30:43.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.881+0000 7f61473de640 1 Processor -- start 2023-12-02T00:30:43.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.881+0000 7f61473de640 1 -- start start 2023-12-02T00:30:43.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.882+0000 7f61473de640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f61400b1400 0x7f614015a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:43.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.882+0000 7f61473de640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f614015e730 0x7f614015aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:43.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.882+0000 7f61473de640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f614015e2a0 con 0x7f6140143490 2023-12-02T00:30:43.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.882+0000 7f61473de640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f614015b520 con 0x7f61400b1400 2023-12-02T00:30:43.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.882+0000 7f61473de640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6140143870 con 0x7f614015e730 2023-12-02T00:30:43.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.882+0000 7f6144979640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f61400b1400 0x7f614015a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:43.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.882+0000 7f6144979640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f61400b1400 0x7f614015a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33304/0 (socket says 172.21.15.46:33304) 2023-12-02T00:30:43.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.882+0000 7f6144979640 1 -- 172.21.15.46:0/1131539249 learned_addr learned my addr 172.21.15.46:0/1131539249 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:43.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.882+0000 7f614597b640 1 --2- 172.21.15.46:0/1131539249 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f614015e730 0x7f614015aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:43.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.882+0000 7f6144979640 1 -- 172.21.15.46:0/1131539249 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f614015e730 msgr2=0x7f614015aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:43.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.882+0000 7f6144979640 1 --2- 172.21.15.46:0/1131539249 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f614015e730 0x7f614015aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:43.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.882+0000 7f6144979640 1 -- 172.21.15.46:0/1131539249 >> v1:172.21.15.162:6789/0 conn(0x7f6140143490 legacy=0x7f61400b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:43.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.882+0000 7f6144979640 1 -- 172.21.15.46:0/1131539249 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f614015b240 con 0x7f61400b1400 2023-12-02T00:30:43.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.883+0000 7f6144979640 1 --2- 172.21.15.46:0/1131539249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61400b1400 0x7f614015a4e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f613c009870 tx=0x7f613c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9389951f0d165ad7 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:43.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.883+0000 7f6133fff640 1 -- 172.21.15.46:0/1131539249 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613c030840 con 0x7f61400b1400 2023-12-02T00:30:43.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.883+0000 7f6133fff640 1 -- 172.21.15.46:0/1131539249 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f613c0309a0 con 0x7f61400b1400 2023-12-02T00:30:43.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.883+0000 7f6133fff640 1 -- 172.21.15.46:0/1131539249 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613c030d30 con 0x7f61400b1400 2023-12-02T00:30:43.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.884+0000 7f61473de640 1 -- 172.21.15.46:0/1131539249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61400b1400 msgr2=0x7f614015a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:43.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.884+0000 7f61473de640 1 --2- 172.21.15.46:0/1131539249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61400b1400 0x7f614015a4e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f613c009870 tx=0x7f613c02b980 comp rx=0 tx=0).stop 2023-12-02T00:30:43.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.884+0000 7f61473de640 1 -- 172.21.15.46:0/1131539249 shutdown_connections 2023-12-02T00:30:43.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.884+0000 7f61473de640 1 --2- 172.21.15.46:0/1131539249 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f614015e730 0x7f614015aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:43.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.884+0000 7f61473de640 1 --2- 172.21.15.46:0/1131539249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61400b1400 0x7f614015a4e0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:43.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.884+0000 7f61473de640 1 -- 172.21.15.46:0/1131539249 >> 172.21.15.46:0/1131539249 conn(0x7f614006a4d0 msgr2=0x7f614006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:43.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.884+0000 7f61473de640 1 -- 172.21.15.46:0/1131539249 shutdown_connections 2023-12-02T00:30:43.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.884+0000 7f61473de640 1 -- 172.21.15.46:0/1131539249 wait complete. 2023-12-02T00:30:43.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.885+0000 7f61473de640 1 Processor -- start 2023-12-02T00:30:43.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.885+0000 7f61473de640 1 -- start start 2023-12-02T00:30:43.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.886+0000 7f61473de640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61400b1400 0x7f61400ac910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:43.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.886+0000 7f61473de640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6140143490 0x7f614015cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:43.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.886+0000 7f61473de640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f614015e730 0x7f614015d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:43.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.886+0000 7f61473de640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61400ad3d0 con 0x7f614015e730 2023-12-02T00:30:43.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.886+0000 7f61473de640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61400ad6b0 con 0x7f6140143490 2023-12-02T00:30:43.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.886+0000 7f61473de640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61400ad990 con 0x7f61400b1400 2023-12-02T00:30:43.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.886+0000 7f614597b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f614015e730 0x7f614015d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:43.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.886+0000 7f614597b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f614015e730 0x7f614015d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33306/0 (socket says 172.21.15.46:33306) 2023-12-02T00:30:43.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.886+0000 7f614597b640 1 -- 172.21.15.46:0/608465863 learned_addr learned my addr 172.21.15.46:0/608465863 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:43.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.886+0000 7f6144979640 1 --2- 172.21.15.46:0/608465863 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6140143490 0x7f614015cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:43.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.886+0000 7f614517a640 1 --2- 172.21.15.46:0/608465863 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61400b1400 0x7f61400ac910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:43.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.886+0000 7f614597b640 1 -- 172.21.15.46:0/608465863 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61400b1400 msgr2=0x7f61400ac910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:43.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.886+0000 7f614597b640 1 --2- 172.21.15.46:0/608465863 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61400b1400 0x7f61400ac910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:43.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.886+0000 7f614597b640 1 -- 172.21.15.46:0/608465863 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6140143490 msgr2=0x7f614015cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:43.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.886+0000 7f614597b640 1 --2- 172.21.15.46:0/608465863 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6140143490 0x7f614015cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:43.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.886+0000 7f614597b640 1 -- 172.21.15.46:0/608465863 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f613c0094d0 con 0x7f614015e730 2023-12-02T00:30:43.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.886+0000 7f614517a640 1 --2- 172.21.15.46:0/608465863 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61400b1400 0x7f61400ac910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:30:43.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.886+0000 7f614597b640 1 --2- 172.21.15.46:0/608465863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f614015e730 0x7f614015d190 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f6138007e60 tx=0x7f613800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:43.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.886+0000 7f6131ffb640 1 -- 172.21.15.46:0/608465863 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6138014420 con 0x7f614015e730 2023-12-02T00:30:43.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.886+0000 7f6131ffb640 1 -- 172.21.15.46:0/608465863 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61380145a0 con 0x7f614015e730 2023-12-02T00:30:43.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.886+0000 7f6131ffb640 1 -- 172.21.15.46:0/608465863 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6138014950 con 0x7f614015e730 2023-12-02T00:30:43.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.886+0000 7f61473de640 1 -- 172.21.15.46:0/608465863 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61400adc70 con 0x7f614015e730 2023-12-02T00:30:43.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.887+0000 7f61473de640 1 -- 172.21.15.46:0/608465863 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61400cb400 con 0x7f614015e730 2023-12-02T00:30:43.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.887+0000 7f61473de640 1 -- 172.21.15.46:0/608465863 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61400ae190 con 0x7f614015e730 2023-12-02T00:30:43.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.888+0000 7f6131ffb640 1 -- 172.21.15.46:0/608465863 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6138014ab0 con 0x7f614015e730 2023-12-02T00:30:43.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.888+0000 7f6131ffb640 1 --2- 172.21.15.46:0/608465863 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f612406ae40 0x7f612406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:43.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.888+0000 7f6131ffb640 1 -- 172.21.15.46:0/608465863 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6190+0+0 (secure 0 0 0) 0x7f613808a1e0 con 0x7f614015e730 2023-12-02T00:30:43.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.888+0000 7f614517a640 1 --2- 172.21.15.46:0/608465863 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f612406ae40 0x7f612406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:43.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.889+0000 7f614517a640 1 --2- 172.21.15.46:0/608465863 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f612406ae40 0x7f612406d300 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6134000f60 tx=0x7f6134009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:43.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:43.893+0000 7f6131ffb640 1 -- 172.21.15.46:0/608465863 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6138055010 con 0x7f614015e730 2023-12-02T00:30:44.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.106+0000 7f61473de640 1 -- 172.21.15.46:0/608465863 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f61400cb7a0 con 0x7f614015e730 2023-12-02T00:30:44.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.754+0000 7f6131ffb640 1 -- 172.21.15.46:0/608465863 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v16) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6138058630 con 0x7f614015e730 2023-12-02T00:30:44.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.757+0000 7f61473de640 1 -- 172.21.15.46:0/608465863 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f612406ae40 msgr2=0x7f612406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:44.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.757+0000 7f61473de640 1 --2- 172.21.15.46:0/608465863 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f612406ae40 0x7f612406d300 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6134000f60 tx=0x7f6134009040 comp rx=0 tx=0).stop 2023-12-02T00:30:44.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.757+0000 7f61473de640 1 -- 172.21.15.46:0/608465863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f614015e730 msgr2=0x7f614015d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:44.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.757+0000 7f61473de640 1 --2- 172.21.15.46:0/608465863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f614015e730 0x7f614015d190 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f6138007e60 tx=0x7f613800e4b0 comp rx=0 tx=0).stop 2023-12-02T00:30:44.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.757+0000 7f61473de640 1 -- 172.21.15.46:0/608465863 shutdown_connections 2023-12-02T00:30:44.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.757+0000 7f61473de640 1 --2- 172.21.15.46:0/608465863 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f612406ae40 0x7f612406d300 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:44.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.757+0000 7f61473de640 1 --2- 172.21.15.46:0/608465863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f614015e730 0x7f614015d190 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:44.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.757+0000 7f61473de640 1 --2- 172.21.15.46:0/608465863 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6140143490 0x7f614015cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:44.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.757+0000 7f61473de640 1 --2- 172.21.15.46:0/608465863 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61400b1400 0x7f61400ac910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:44.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.757+0000 7f61473de640 1 -- 172.21.15.46:0/608465863 >> 172.21.15.46:0/608465863 conn(0x7f614006a4d0 msgr2=0x7f614006c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:44.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.757+0000 7f61473de640 1 -- 172.21.15.46:0/608465863 shutdown_connections 2023-12-02T00:30:44.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.757+0000 7f61473de640 1 -- 172.21.15.46:0/608465863 wait complete. 2023-12-02T00:30:44.772 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-02T00:30:44.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.931+0000 7f36750d0640 1 Processor -- start 2023-12-02T00:30:44.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.931+0000 7f36750d0640 1 -- start start 2023-12-02T00:30:44.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.932+0000 7f36750d0640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3670160f70 0x7f36700cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:44.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.932+0000 7f36750d0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f36700ce480 0x7f36700d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:44.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.932+0000 7f36750d0640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36701614e0 con 0x7f367015d2d0 2023-12-02T00:30:44.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.932+0000 7f36750d0640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36701617c0 con 0x7f3670160f70 2023-12-02T00:30:44.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.932+0000 7f36750d0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3670161aa0 con 0x7f36700ce480 2023-12-02T00:30:44.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.932+0000 7f3667fff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3670160f70 0x7f36700cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:44.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.932+0000 7f3667fff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3670160f70 0x7f36700cdf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33322/0 (socket says 172.21.15.46:33322) 2023-12-02T00:30:44.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.932+0000 7f3667fff640 1 -- 172.21.15.46:0/1887248247 learned_addr learned my addr 172.21.15.46:0/1887248247 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:44.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.932+0000 7f366f59e640 1 --2- 172.21.15.46:0/1887248247 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f36700ce480 0x7f36700d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:44.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.933+0000 7f3667fff640 1 -- 172.21.15.46:0/1887248247 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f36700ce480 msgr2=0x7f36700d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:44.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.933+0000 7f3667fff640 1 --2- 172.21.15.46:0/1887248247 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f36700ce480 0x7f36700d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:44.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.933+0000 7f3667fff640 1 -- 172.21.15.46:0/1887248247 >> v1:172.21.15.162:6789/0 conn(0x7f367015d2d0 legacy=0x7f36700b8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:44.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.933+0000 7f3667fff640 1 -- 172.21.15.46:0/1887248247 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3670161d80 con 0x7f3670160f70 2023-12-02T00:30:44.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.933+0000 7f3667fff640 1 --2- 172.21.15.46:0/1887248247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3670160f70 0x7f36700cdf40 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f3658009870 tx=0x7f365802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b318c5f572789e5d server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:44.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.933+0000 7f366e59c640 1 -- 172.21.15.46:0/1887248247 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3658030840 con 0x7f3670160f70 2023-12-02T00:30:44.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.934+0000 7f366e59c640 1 -- 172.21.15.46:0/1887248247 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36580309a0 con 0x7f3670160f70 2023-12-02T00:30:44.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.934+0000 7f366e59c640 1 -- 172.21.15.46:0/1887248247 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3658030d30 con 0x7f3670160f70 2023-12-02T00:30:44.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.934+0000 7f36750d0640 1 -- 172.21.15.46:0/1887248247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3670160f70 msgr2=0x7f36700cdf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:44.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.934+0000 7f36750d0640 1 --2- 172.21.15.46:0/1887248247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3670160f70 0x7f36700cdf40 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f3658009870 tx=0x7f365802b980 comp rx=0 tx=0).stop 2023-12-02T00:30:44.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.934+0000 7f36750d0640 1 -- 172.21.15.46:0/1887248247 shutdown_connections 2023-12-02T00:30:44.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.934+0000 7f36750d0640 1 --2- 172.21.15.46:0/1887248247 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f36700ce480 0x7f36700d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:44.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.934+0000 7f36750d0640 1 --2- 172.21.15.46:0/1887248247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3670160f70 0x7f36700cdf40 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:44.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.934+0000 7f36750d0640 1 -- 172.21.15.46:0/1887248247 >> 172.21.15.46:0/1887248247 conn(0x7f3670072230 msgr2=0x7f36700cfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:44.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.934+0000 7f36750d0640 1 -- 172.21.15.46:0/1887248247 shutdown_connections 2023-12-02T00:30:44.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.934+0000 7f36750d0640 1 -- 172.21.15.46:0/1887248247 wait complete. 2023-12-02T00:30:44.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.935+0000 7f36750d0640 1 Processor -- start 2023-12-02T00:30:44.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.936+0000 7f36750d0640 1 -- start start 2023-12-02T00:30:44.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.936+0000 7f36750d0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36700ce480 0x7f36700b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:44.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.936+0000 7f36750d0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f367015d2d0 0x7f36700ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:44.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.936+0000 7f36750d0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3670160f70 0x7f36700ca810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:44.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.936+0000 7f36750d0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36700b2bf0 con 0x7f36700ce480 2023-12-02T00:30:44.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.936+0000 7f36750d0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36700b5050 con 0x7f3670160f70 2023-12-02T00:30:44.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.936+0000 7f36750d0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36700c8ce0 con 0x7f367015d2d0 2023-12-02T00:30:44.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.936+0000 7f366ed9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36700ce480 0x7f36700b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:44.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.936+0000 7f366ed9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36700ce480 0x7f36700b4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33326/0 (socket says 172.21.15.46:33326) 2023-12-02T00:30:44.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.936+0000 7f366ed9d640 1 -- 172.21.15.46:0/483690640 learned_addr learned my addr 172.21.15.46:0/483690640 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:44.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.936+0000 7f366f59e640 1 --2- 172.21.15.46:0/483690640 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3670160f70 0x7f36700ca810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:44.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.937+0000 7f3667fff640 1 --2- 172.21.15.46:0/483690640 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f367015d2d0 0x7f36700ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:44.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.937+0000 7f366ed9d640 1 -- 172.21.15.46:0/483690640 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f367015d2d0 msgr2=0x7f36700ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:44.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.937+0000 7f366ed9d640 1 --2- 172.21.15.46:0/483690640 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f367015d2d0 0x7f36700ca2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:44.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.937+0000 7f366ed9d640 1 -- 172.21.15.46:0/483690640 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3670160f70 msgr2=0x7f36700ca810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:44.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.937+0000 7f366ed9d640 1 --2- 172.21.15.46:0/483690640 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3670160f70 0x7f36700ca810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:44.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.937+0000 7f366ed9d640 1 -- 172.21.15.46:0/483690640 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36580094d0 con 0x7f36700ce480 2023-12-02T00:30:44.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.937+0000 7f366ed9d640 1 --2- 172.21.15.46:0/483690640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36700ce480 0x7f36700b4b10 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f366000c480 tx=0x7f366000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:44.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.937+0000 7f36677fe640 1 -- 172.21.15.46:0/483690640 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36600052d0 con 0x7f36700ce480 2023-12-02T00:30:44.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.937+0000 7f36677fe640 1 -- 172.21.15.46:0/483690640 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3660005430 con 0x7f36700ce480 2023-12-02T00:30:44.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.937+0000 7f36677fe640 1 -- 172.21.15.46:0/483690640 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3660012870 con 0x7f36700ce480 2023-12-02T00:30:44.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.937+0000 7f36750d0640 1 -- 172.21.15.46:0/483690640 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36700c8fc0 con 0x7f36700ce480 2023-12-02T00:30:44.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.937+0000 7f36750d0640 1 -- 172.21.15.46:0/483690640 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36700c9580 con 0x7f36700ce480 2023-12-02T00:30:44.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.938+0000 7f36750d0640 1 -- 172.21.15.46:0/483690640 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3670069400 con 0x7f36700ce480 2023-12-02T00:30:44.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.939+0000 7f36677fe640 1 -- 172.21.15.46:0/483690640 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f36600129d0 con 0x7f36700ce480 2023-12-02T00:30:44.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.939+0000 7f36677fe640 1 --2- 172.21.15.46:0/483690640 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f365006ae70 0x7f365006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:44.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.939+0000 7f36677fe640 1 -- 172.21.15.46:0/483690640 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6190+0+0 (secure 0 0 0) 0x7f3660089210 con 0x7f36700ce480 2023-12-02T00:30:44.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.939+0000 7f3667fff640 1 --2- 172.21.15.46:0/483690640 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f365006ae70 0x7f365006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:44.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.940+0000 7f3667fff640 1 --2- 172.21.15.46:0/483690640 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f365006ae70 0x7f365006d330 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3658036000 tx=0x7f365803b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:44.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:44.944+0000 7f36677fe640 1 -- 172.21.15.46:0/483690640 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3660057cb0 con 0x7f36700ce480 2023-12-02T00:30:45.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.123+0000 7f36750d0640 1 -- 172.21.15.46:0/483690640 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f36700c9710 con 0x7f36700ce480 2023-12-02T00:30:45.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.759+0000 7f36677fe640 1 -- 172.21.15.46:0/483690640 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v40) v1 ==== 181+0+0 (secure 0 0 0) 0x7f36600011c0 con 0x7f36700ce480 2023-12-02T00:30:45.761 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2023-12-02T00:30:45.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.762+0000 7f36750d0640 1 -- 172.21.15.46:0/483690640 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f365006ae70 msgr2=0x7f365006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:45.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.762+0000 7f36750d0640 1 --2- 172.21.15.46:0/483690640 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f365006ae70 0x7f365006d330 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3658036000 tx=0x7f365803b040 comp rx=0 tx=0).stop 2023-12-02T00:30:45.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.762+0000 7f36750d0640 1 -- 172.21.15.46:0/483690640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36700ce480 msgr2=0x7f36700b4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:45.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.762+0000 7f36750d0640 1 --2- 172.21.15.46:0/483690640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36700ce480 0x7f36700b4b10 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f366000c480 tx=0x7f366000c940 comp rx=0 tx=0).stop 2023-12-02T00:30:45.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.763+0000 7f36750d0640 1 -- 172.21.15.46:0/483690640 shutdown_connections 2023-12-02T00:30:45.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.763+0000 7f36750d0640 1 --2- 172.21.15.46:0/483690640 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f365006ae70 0x7f365006d330 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:45.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.763+0000 7f36750d0640 1 --2- 172.21.15.46:0/483690640 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3670160f70 0x7f36700ca810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:45.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.763+0000 7f36750d0640 1 --2- 172.21.15.46:0/483690640 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f367015d2d0 0x7f36700ca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:45.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.763+0000 7f36750d0640 1 --2- 172.21.15.46:0/483690640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36700ce480 0x7f36700b4b10 secure :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f366000c480 tx=0x7f366000c940 comp rx=0 tx=0).stop 2023-12-02T00:30:45.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.763+0000 7f36750d0640 1 -- 172.21.15.46:0/483690640 >> 172.21.15.46:0/483690640 conn(0x7f3670072230 msgr2=0x7f36700d31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:45.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.763+0000 7f36750d0640 1 -- 172.21.15.46:0/483690640 shutdown_connections 2023-12-02T00:30:45.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.763+0000 7f36750d0640 1 -- 172.21.15.46:0/483690640 wait complete. 2023-12-02T00:30:45.779 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-02T00:30:45.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.899+0000 7f155ed53640 1 Processor -- start 2023-12-02T00:30:45.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.899+0000 7f155ed53640 1 -- start start 2023-12-02T00:30:45.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.899+0000 7f155ed53640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f155815d330 0x7f15580b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:45.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.899+0000 7f155ed53640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15580b2b10 con 0x7f15580d3900 2023-12-02T00:30:45.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.899+0000 7f155ed53640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15580d1590 con 0x7f15580d2910 2023-12-02T00:30:45.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.899+0000 7f155ed53640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15580d1870 con 0x7f155815d330 2023-12-02T00:30:45.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.900+0000 7f155caef640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f155815d330 0x7f15580b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:45.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.900+0000 7f155caef640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f155815d330 0x7f15580b8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33332/0 (socket says 172.21.15.46:33332) 2023-12-02T00:30:45.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.900+0000 7f155d2f0640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f15580d3900 0x7f15580b6390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51446/0 (socket says 172.21.15.46:51446) 2023-12-02T00:30:45.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.900+0000 7f155caef640 1 -- 172.21.15.46:0/109158977 learned_addr learned my addr 172.21.15.46:0/109158977 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:45.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.900+0000 7f155caef640 1 -- 172.21.15.46:0/109158977 >> v1:172.21.15.162:6789/0 conn(0x7f15580d2910 legacy=0x7f15580d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:45.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.900+0000 7f155caef640 1 -- 172.21.15.46:0/109158977 >> v1:172.21.15.46:6789/0 conn(0x7f15580d3900 legacy=0x7f15580b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:45.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.900+0000 7f15577fe640 1 -- 172.21.15.46:0/109158977 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2321338580 0 0) 0x7f15580b2b10 con 0x7f15580d3900 2023-12-02T00:30:45.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.900+0000 7f155caef640 1 -- 172.21.15.46:0/109158977 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15580d1b50 con 0x7f155815d330 2023-12-02T00:30:45.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.901+0000 7f155caef640 1 --2- 172.21.15.46:0/109158977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f155815d330 0x7f15580b8b80 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f1548009870 tx=0x7f154802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c98c93b73250347 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:45.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.901+0000 7f15577fe640 1 -- 172.21.15.46:0/109158977 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1548031870 con 0x7f155815d330 2023-12-02T00:30:45.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.901+0000 7f15577fe640 1 -- 172.21.15.46:0/109158977 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15480319d0 con 0x7f155815d330 2023-12-02T00:30:45.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.901+0000 7f15577fe640 1 -- 172.21.15.46:0/109158977 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1548031d60 con 0x7f155815d330 2023-12-02T00:30:45.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.901+0000 7f155ed53640 1 -- 172.21.15.46:0/109158977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f155815d330 msgr2=0x7f15580b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:45.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.901+0000 7f155ed53640 1 --2- 172.21.15.46:0/109158977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f155815d330 0x7f15580b8b80 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f1548009870 tx=0x7f154802b910 comp rx=0 tx=0).stop 2023-12-02T00:30:45.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.901+0000 7f155ed53640 1 -- 172.21.15.46:0/109158977 shutdown_connections 2023-12-02T00:30:45.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.901+0000 7f155ed53640 1 --2- 172.21.15.46:0/109158977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f155815d330 0x7f15580b8b80 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:45.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.901+0000 7f155ed53640 1 -- 172.21.15.46:0/109158977 >> 172.21.15.46:0/109158977 conn(0x7f1558072020 msgr2=0x7f1558072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:45.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.901+0000 7f155ed53640 1 -- 172.21.15.46:0/109158977 shutdown_connections 2023-12-02T00:30:45.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.901+0000 7f155ed53640 1 -- 172.21.15.46:0/109158977 wait complete. 2023-12-02T00:30:45.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.903+0000 7f155ed53640 1 Processor -- start 2023-12-02T00:30:45.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.903+0000 7f155ed53640 1 -- start start 2023-12-02T00:30:45.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.903+0000 7f155ed53640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15580d2910 0x7f15580c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:45.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.904+0000 7f155ed53640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f15580d3900 0x7f1558153890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:45.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.904+0000 7f155ed53640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f155815d330 0x7f155815efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:45.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.904+0000 7f155ed53640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1558160820 con 0x7f15580d2910 2023-12-02T00:30:45.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.904+0000 7f155ed53640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1558160390 con 0x7f155815d330 2023-12-02T00:30:45.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.904+0000 7f155ed53640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1558153dd0 con 0x7f15580d3900 2023-12-02T00:30:45.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.904+0000 7f155caef640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15580d2910 0x7f15580c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:45.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.904+0000 7f155caef640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15580d2910 0x7f15580c7ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33336/0 (socket says 172.21.15.46:33336) 2023-12-02T00:30:45.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.904+0000 7f155caef640 1 -- 172.21.15.46:0/2547167559 learned_addr learned my addr 172.21.15.46:0/2547167559 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:45.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.904+0000 7f155d2f0640 1 --2- 172.21.15.46:0/2547167559 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f155815d330 0x7f155815efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:45.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.904+0000 7f155caef640 1 -- 172.21.15.46:0/2547167559 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f15580d3900 msgr2=0x7f1558153890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:45.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.904+0000 7f1557fff640 1 --2- 172.21.15.46:0/2547167559 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f15580d3900 0x7f1558153890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:45.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.904+0000 7f155caef640 1 --2- 172.21.15.46:0/2547167559 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f15580d3900 0x7f1558153890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:45.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.904+0000 7f155caef640 1 -- 172.21.15.46:0/2547167559 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f155815d330 msgr2=0x7f155815efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:45.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.904+0000 7f155caef640 1 --2- 172.21.15.46:0/2547167559 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f155815d330 0x7f155815efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:45.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.904+0000 7f155caef640 1 -- 172.21.15.46:0/2547167559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15480094d0 con 0x7f15580d2910 2023-12-02T00:30:45.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.904+0000 7f155d2f0640 1 --2- 172.21.15.46:0/2547167559 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f155815d330 0x7f155815efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:30:45.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.904+0000 7f155caef640 1 --2- 172.21.15.46:0/2547167559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15580d2910 0x7f15580c7ce0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f1548009870 tx=0x7f15480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:45.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.904+0000 7f15557fa640 1 -- 172.21.15.46:0/2547167559 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1548031820 con 0x7f15580d2910 2023-12-02T00:30:45.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.904+0000 7f15557fa640 1 -- 172.21.15.46:0/2547167559 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15480443f0 con 0x7f15580d2910 2023-12-02T00:30:45.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.904+0000 7f15557fa640 1 -- 172.21.15.46:0/2547167559 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15480447a0 con 0x7f15580d2910 2023-12-02T00:30:45.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.904+0000 7f155ed53640 1 -- 172.21.15.46:0/2547167559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f155815f4e0 con 0x7f15580d2910 2023-12-02T00:30:45.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.905+0000 7f155ed53640 1 -- 172.21.15.46:0/2547167559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f155815fa60 con 0x7f15580d2910 2023-12-02T00:30:45.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.905+0000 7f155ed53640 1 -- 172.21.15.46:0/2547167559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1558069400 con 0x7f15580d2910 2023-12-02T00:30:45.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.906+0000 7f15557fa640 1 -- 172.21.15.46:0/2547167559 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f154804d450 con 0x7f15580d2910 2023-12-02T00:30:45.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.906+0000 7f15557fa640 1 --2- 172.21.15.46:0/2547167559 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f153806ae70 0x7f153806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:45.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.906+0000 7f15557fa640 1 -- 172.21.15.46:0/2547167559 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5752+0+0 (secure 0 0 0) 0x7f154802d040 con 0x7f15580d2910 2023-12-02T00:30:45.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.906+0000 7f1557fff640 1 --2- 172.21.15.46:0/2547167559 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f153806ae70 0x7f153806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:45.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.907+0000 7f1557fff640 1 --2- 172.21.15.46:0/2547167559 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f153806ae70 0x7f153806d330 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f15580cc530 tx=0x7f1540009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:45.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:45.910+0000 7f15557fa640 1 -- 172.21.15.46:0/2547167559 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1548076dd0 con 0x7f15580d2910 2023-12-02T00:30:46.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.092+0000 7f155ed53640 1 -- 172.21.15.46:0/2547167559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f15580b1d80 con 0x7f15580d2910 2023-12-02T00:30:46.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.764+0000 7f15557fa640 1 -- 172.21.15.46:0/2547167559 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v41) v1 ==== 169+0+0 (secure 0 0 0) 0x7f154807a3f0 con 0x7f15580d2910 2023-12-02T00:30:46.765 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2023-12-02T00:30:46.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.768+0000 7f155ed53640 1 -- 172.21.15.46:0/2547167559 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f153806ae70 msgr2=0x7f153806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:46.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.768+0000 7f155ed53640 1 --2- 172.21.15.46:0/2547167559 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f153806ae70 0x7f153806d330 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f15580cc530 tx=0x7f1540009040 comp rx=0 tx=0).stop 2023-12-02T00:30:46.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.768+0000 7f155ed53640 1 -- 172.21.15.46:0/2547167559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15580d2910 msgr2=0x7f15580c7ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:46.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.768+0000 7f155ed53640 1 --2- 172.21.15.46:0/2547167559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15580d2910 0x7f15580c7ce0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f1548009870 tx=0x7f15480046c0 comp rx=0 tx=0).stop 2023-12-02T00:30:46.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.768+0000 7f155ed53640 1 -- 172.21.15.46:0/2547167559 shutdown_connections 2023-12-02T00:30:46.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.768+0000 7f155ed53640 1 --2- 172.21.15.46:0/2547167559 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f153806ae70 0x7f153806d330 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:46.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.768+0000 7f155ed53640 1 --2- 172.21.15.46:0/2547167559 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f155815d330 0x7f155815efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:46.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.768+0000 7f155ed53640 1 --2- 172.21.15.46:0/2547167559 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f15580d3900 0x7f1558153890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:46.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.768+0000 7f155ed53640 1 --2- 172.21.15.46:0/2547167559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15580d2910 0x7f15580c7ce0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:46.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.768+0000 7f155ed53640 1 -- 172.21.15.46:0/2547167559 >> 172.21.15.46:0/2547167559 conn(0x7f1558072020 msgr2=0x7f15580d0050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:46.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.769+0000 7f155ed53640 1 -- 172.21.15.46:0/2547167559 shutdown_connections 2023-12-02T00:30:46.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.769+0000 7f155ed53640 1 -- 172.21.15.46:0/2547167559 wait complete. 2023-12-02T00:30:46.784 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm data_new data_new --yes-i-really-really-mean-it 2023-12-02T00:30:46.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.904+0000 7f94378ee640 1 Processor -- start 2023-12-02T00:30:46.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.905+0000 7f94378ee640 1 -- start start 2023-12-02T00:30:46.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.905+0000 7f94378ee640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9430160bb0 0x7f94300d3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:46.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.905+0000 7f94378ee640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f94300b3fc0 0x7f94300cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:46.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.905+0000 7f94378ee640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f94300ce270 0x7f94300d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:46.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.905+0000 7f94378ee640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94300b3450 con 0x7f94300ce270 2023-12-02T00:30:46.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.905+0000 7f94378ee640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94300b45b0 con 0x7f94300b3fc0 2023-12-02T00:30:46.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.905+0000 7f94378ee640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94300b4890 con 0x7f9430160bb0 2023-12-02T00:30:46.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.906+0000 7f9435e8b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f94300ce270 0x7f94300d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:46.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.906+0000 7f9435e8b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f94300ce270 0x7f94300d1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33340/0 (socket says 172.21.15.46:33340) 2023-12-02T00:30:46.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.906+0000 7f9435e8b640 1 -- 172.21.15.46:0/870524806 learned_addr learned my addr 172.21.15.46:0/870524806 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:46.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.906+0000 7f9434e89640 1 --2- 172.21.15.46:0/870524806 >> v2:172.21.15.162:3300/0 conn(0x7f94300b3fc0 0x7f94300cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:46.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.906+0000 7f9435e8b640 1 -- 172.21.15.46:0/870524806 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9430160bb0 msgr2=0x7f94300d3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:46.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.906+0000 7f943568a640 1 --2- 172.21.15.46:0/870524806 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9430160bb0 0x7f94300d3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:46.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.906+0000 7f9435e8b640 1 --2- 172.21.15.46:0/870524806 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9430160bb0 0x7f94300d3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:46.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.906+0000 7f9435e8b640 1 -- 172.21.15.46:0/870524806 >> v2:172.21.15.162:3300/0 conn(0x7f94300b3fc0 msgr2=0x7f94300cdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:46.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.906+0000 7f9435e8b640 1 --2- 172.21.15.46:0/870524806 >> v2:172.21.15.162:3300/0 conn(0x7f94300b3fc0 0x7f94300cdd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:46.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.906+0000 7f9435e8b640 1 -- 172.21.15.46:0/870524806 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94300b36a0 con 0x7f94300ce270 2023-12-02T00:30:46.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.906+0000 7f9434e89640 1 --2- 172.21.15.46:0/870524806 >> v2:172.21.15.162:3300/0 conn(0x7f94300b3fc0 0x7f94300cdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:30:46.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.906+0000 7f943568a640 1 --2- 172.21.15.46:0/870524806 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9430160bb0 0x7f94300d3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:30:46.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.907+0000 7f9435e8b640 1 --2- 172.21.15.46:0/870524806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94300ce270 0x7f94300d1120 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f943008cc00 tx=0x7f942002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cb4614fb9f1873a server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:46.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.907+0000 7f942ffff640 1 -- 172.21.15.46:0/870524806 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942002ca70 con 0x7f94300ce270 2023-12-02T00:30:46.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.907+0000 7f942ffff640 1 -- 172.21.15.46:0/870524806 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f942002cbd0 con 0x7f94300ce270 2023-12-02T00:30:46.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.907+0000 7f942ffff640 1 -- 172.21.15.46:0/870524806 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94200343f0 con 0x7f94300ce270 2023-12-02T00:30:46.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.907+0000 7f94378ee640 1 -- 172.21.15.46:0/870524806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94300ce270 msgr2=0x7f94300d1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:46.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.907+0000 7f94378ee640 1 --2- 172.21.15.46:0/870524806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94300ce270 0x7f94300d1120 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f943008cc00 tx=0x7f942002bed0 comp rx=0 tx=0).stop 2023-12-02T00:30:46.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.907+0000 7f94378ee640 1 -- 172.21.15.46:0/870524806 shutdown_connections 2023-12-02T00:30:46.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.907+0000 7f94378ee640 1 --2- 172.21.15.46:0/870524806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94300ce270 0x7f94300d1120 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:46.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.907+0000 7f94378ee640 1 --2- 172.21.15.46:0/870524806 >> v2:172.21.15.162:3300/0 conn(0x7f94300b3fc0 0x7f94300cdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:46.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.907+0000 7f94378ee640 1 --2- 172.21.15.46:0/870524806 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9430160bb0 0x7f94300d3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:46.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.907+0000 7f94378ee640 1 -- 172.21.15.46:0/870524806 >> 172.21.15.46:0/870524806 conn(0x7f943009c590 msgr2=0x7f94300d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:46.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.908+0000 7f94378ee640 1 -- 172.21.15.46:0/870524806 shutdown_connections 2023-12-02T00:30:46.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.908+0000 7f94378ee640 1 -- 172.21.15.46:0/870524806 wait complete. 2023-12-02T00:30:46.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.909+0000 7f94378ee640 1 Processor -- start 2023-12-02T00:30:46.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.909+0000 7f94378ee640 1 -- start start 2023-12-02T00:30:46.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.909+0000 7f94378ee640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f94300b3fc0 0x7f94300b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:46.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.909+0000 7f94378ee640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94300ce270 0x7f94300b6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:46.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.909+0000 7f94378ee640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9430160bb0 0x7f94300b7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:46.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.909+0000 7f94378ee640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94301524b0 con 0x7f94300ce270 2023-12-02T00:30:46.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.909+0000 7f94378ee640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943014dfa0 con 0x7f9430160bb0 2023-12-02T00:30:46.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.909+0000 7f94378ee640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94301540e0 con 0x7f94300b3fc0 2023-12-02T00:30:46.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.910+0000 7f9434e89640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94300ce270 0x7f94300b6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:46.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.910+0000 7f9434e89640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94300ce270 0x7f94300b6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33350/0 (socket says 172.21.15.46:33350) 2023-12-02T00:30:46.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.910+0000 7f9434e89640 1 -- 172.21.15.46:0/1537492852 learned_addr learned my addr 172.21.15.46:0/1537492852 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:46.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.910+0000 7f9435e8b640 1 --2- 172.21.15.46:0/1537492852 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9430160bb0 0x7f94300b7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:46.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.910+0000 7f943568a640 1 --2- 172.21.15.46:0/1537492852 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f94300b3fc0 0x7f94300b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:46.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.910+0000 7f9434e89640 1 -- 172.21.15.46:0/1537492852 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f94300b3fc0 msgr2=0x7f94300b5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:46.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.910+0000 7f9434e89640 1 --2- 172.21.15.46:0/1537492852 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f94300b3fc0 0x7f94300b5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:46.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.910+0000 7f9434e89640 1 -- 172.21.15.46:0/1537492852 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9430160bb0 msgr2=0x7f94300b7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:46.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.910+0000 7f9434e89640 1 --2- 172.21.15.46:0/1537492852 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9430160bb0 0x7f94300b7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:46.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.910+0000 7f9434e89640 1 -- 172.21.15.46:0/1537492852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9420009b30 con 0x7f94300ce270 2023-12-02T00:30:46.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.910+0000 7f943568a640 1 --2- 172.21.15.46:0/1537492852 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f94300b3fc0 0x7f94300b5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:30:46.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.910+0000 7f9434e89640 1 --2- 172.21.15.46:0/1537492852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94300ce270 0x7f94300b6380 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f942400c7c0 tx=0x7f942400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:46.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.910+0000 7f942dffb640 1 -- 172.21.15.46:0/1537492852 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9424007a30 con 0x7f94300ce270 2023-12-02T00:30:46.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.910+0000 7f942dffb640 1 -- 172.21.15.46:0/1537492852 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9424007b90 con 0x7f94300ce270 2023-12-02T00:30:46.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.910+0000 7f942dffb640 1 -- 172.21.15.46:0/1537492852 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942400ce70 con 0x7f94300ce270 2023-12-02T00:30:46.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.910+0000 7f94378ee640 1 -- 172.21.15.46:0/1537492852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94300d40f0 con 0x7f94300ce270 2023-12-02T00:30:46.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.911+0000 7f94378ee640 1 -- 172.21.15.46:0/1537492852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9430151b10 con 0x7f94300ce270 2023-12-02T00:30:46.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.911+0000 7f94378ee640 1 -- 172.21.15.46:0/1537492852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94301525f0 con 0x7f94300ce270 2023-12-02T00:30:46.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.912+0000 7f942dffb640 1 -- 172.21.15.46:0/1537492852 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f94240113f0 con 0x7f94300ce270 2023-12-02T00:30:46.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.912+0000 7f942dffb640 1 --2- 172.21.15.46:0/1537492852 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f941c06ae40 0x7f941c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:46.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.913+0000 7f942dffb640 1 -- 172.21.15.46:0/1537492852 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5338+0+0 (secure 0 0 0) 0x7f9424089460 con 0x7f94300ce270 2023-12-02T00:30:46.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.913+0000 7f943568a640 1 --2- 172.21.15.46:0/1537492852 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f941c06ae40 0x7f941c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:46.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.913+0000 7f943568a640 1 --2- 172.21.15.46:0/1537492852 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f941c06ae40 0x7f941c06d300 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f942800a990 tx=0x7f9428008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:46.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:46.917+0000 7f942dffb640 1 -- 172.21.15.46:0/1537492852 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f94301525f0 con 0x7f94300ce270 2023-12-02T00:30:47.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.094+0000 7f94378ee640 1 -- 172.21.15.46:0/1537492852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "data_new", "pool2": "data_new", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f94301525f0 con 0x7f94300ce270 2023-12-02T00:30:47.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.769+0000 7f942dffb640 1 -- 172.21.15.46:0/1537492852 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "data_new", "pool2": "data_new", "yes_i_really_really_mean_it": true}]=0 pool 'data_new' removed v42) v1 ==== 160+0+0 (secure 0 0 0) 0x7f94301525f0 con 0x7f94300ce270 2023-12-02T00:30:47.770 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'data_new' removed 2023-12-02T00:30:47.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.773+0000 7f94378ee640 1 -- 172.21.15.46:0/1537492852 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f941c06ae40 msgr2=0x7f941c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:47.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.773+0000 7f94378ee640 1 --2- 172.21.15.46:0/1537492852 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f941c06ae40 0x7f941c06d300 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f942800a990 tx=0x7f9428008040 comp rx=0 tx=0).stop 2023-12-02T00:30:47.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.773+0000 7f94378ee640 1 -- 172.21.15.46:0/1537492852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94300ce270 msgr2=0x7f94300b6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:47.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.773+0000 7f94378ee640 1 --2- 172.21.15.46:0/1537492852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94300ce270 0x7f94300b6380 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f942400c7c0 tx=0x7f942400cc80 comp rx=0 tx=0).stop 2023-12-02T00:30:47.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.773+0000 7f94378ee640 1 -- 172.21.15.46:0/1537492852 shutdown_connections 2023-12-02T00:30:47.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.773+0000 7f94378ee640 1 --2- 172.21.15.46:0/1537492852 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f941c06ae40 0x7f941c06d300 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:47.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.773+0000 7f94378ee640 1 --2- 172.21.15.46:0/1537492852 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9430160bb0 0x7f94300b7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:47.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.773+0000 7f94378ee640 1 --2- 172.21.15.46:0/1537492852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94300ce270 0x7f94300b6380 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:47.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.773+0000 7f94378ee640 1 --2- 172.21.15.46:0/1537492852 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f94300b3fc0 0x7f94300b5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:47.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.774+0000 7f94378ee640 1 -- 172.21.15.46:0/1537492852 >> 172.21.15.46:0/1537492852 conn(0x7f943009c590 msgr2=0x7f94300d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:47.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.774+0000 7f94378ee640 1 -- 172.21.15.46:0/1537492852 shutdown_connections 2023-12-02T00:30:47.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.774+0000 7f94378ee640 1 -- 172.21.15.46:0/1537492852 wait complete. 2023-12-02T00:30:47.789 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-02T00:30:47.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.902+0000 7fdbad87c640 1 Processor -- start 2023-12-02T00:30:47.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.903+0000 7fdbad87c640 1 -- start start 2023-12-02T00:30:47.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.903+0000 7fdbad87c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdba8143490 0x7fdba80b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:47.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.903+0000 7fdbad87c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fdba815e730 0x7fdba80b1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:47.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.903+0000 7fdbad87c640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdba815e2a0 con 0x7fdba815a4e0 2023-12-02T00:30:47.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.903+0000 7fdbad87c640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdba815b520 con 0x7fdba8143490 2023-12-02T00:30:47.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.903+0000 7fdbad87c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdba8143870 con 0x7fdba815e730 2023-12-02T00:30:47.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.904+0000 7fdba6ffd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdba8143490 0x7fdba80b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:47.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.904+0000 7fdba6ffd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdba8143490 0x7fdba80b0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33356/0 (socket says 172.21.15.46:33356) 2023-12-02T00:30:47.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.904+0000 7fdba6ffd640 1 -- 172.21.15.46:0/2576434714 learned_addr learned my addr 172.21.15.46:0/2576434714 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:47.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.904+0000 7fdba67fc640 1 --2- 172.21.15.46:0/2576434714 >> v2:172.21.15.162:3300/0 conn(0x7fdba815e730 0x7fdba80b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:47.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.904+0000 7fdba6ffd640 1 -- 172.21.15.46:0/2576434714 >> v2:172.21.15.162:3300/0 conn(0x7fdba815e730 msgr2=0x7fdba80b1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:47.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.904+0000 7fdba6ffd640 1 --2- 172.21.15.46:0/2576434714 >> v2:172.21.15.162:3300/0 conn(0x7fdba815e730 0x7fdba80b1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:47.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.904+0000 7fdba6ffd640 1 -- 172.21.15.46:0/2576434714 >> v1:172.21.15.162:6789/0 conn(0x7fdba815a4e0 legacy=0x7fdba815a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:47.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.904+0000 7fdba6ffd640 1 -- 172.21.15.46:0/2576434714 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba815b240 con 0x7fdba8143490 2023-12-02T00:30:47.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.905+0000 7fdba6ffd640 1 --2- 172.21.15.46:0/2576434714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdba8143490 0x7fdba80b0cf0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fdba80aeca0 tx=0x7fdba002e170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a42fd2e7eedb02f server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:47.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.905+0000 7fdba5ffb640 1 -- 172.21.15.46:0/2576434714 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba0036440 con 0x7fdba8143490 2023-12-02T00:30:47.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.905+0000 7fdba5ffb640 1 -- 172.21.15.46:0/2576434714 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba00365a0 con 0x7fdba8143490 2023-12-02T00:30:47.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.905+0000 7fdba5ffb640 1 -- 172.21.15.46:0/2576434714 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba0036930 con 0x7fdba8143490 2023-12-02T00:30:47.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.905+0000 7fdbad87c640 1 -- 172.21.15.46:0/2576434714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdba8143490 msgr2=0x7fdba80b0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:47.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.905+0000 7fdbad87c640 1 --2- 172.21.15.46:0/2576434714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdba8143490 0x7fdba80b0cf0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fdba80aeca0 tx=0x7fdba002e170 comp rx=0 tx=0).stop 2023-12-02T00:30:47.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.905+0000 7fdbad87c640 1 -- 172.21.15.46:0/2576434714 shutdown_connections 2023-12-02T00:30:47.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.905+0000 7fdbad87c640 1 --2- 172.21.15.46:0/2576434714 >> v2:172.21.15.162:3300/0 conn(0x7fdba815e730 0x7fdba80b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:47.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.905+0000 7fdbad87c640 1 --2- 172.21.15.46:0/2576434714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdba8143490 0x7fdba80b0cf0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:47.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.905+0000 7fdbad87c640 1 -- 172.21.15.46:0/2576434714 >> 172.21.15.46:0/2576434714 conn(0x7fdba806a4d0 msgr2=0x7fdba806a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:47.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.905+0000 7fdbad87c640 1 -- 172.21.15.46:0/2576434714 shutdown_connections 2023-12-02T00:30:47.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.905+0000 7fdbad87c640 1 -- 172.21.15.46:0/2576434714 wait complete. 2023-12-02T00:30:47.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.907+0000 7fdbad87c640 1 Processor -- start 2023-12-02T00:30:47.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.907+0000 7fdbad87c640 1 -- start start 2023-12-02T00:30:47.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.907+0000 7fdbad87c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdba8143490 0x7fdba80a9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:47.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.907+0000 7fdbad87c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdba815a4e0 0x7fdba80ac6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:47.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.907+0000 7fdbad87c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdba815e730 0x7fdba8140b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:47.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.907+0000 7fdbad87c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba80acc00 con 0x7fdba815e730 2023-12-02T00:30:47.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.907+0000 7fdbad87c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba80a9d50 con 0x7fdba815a4e0 2023-12-02T00:30:47.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.907+0000 7fdbad87c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdba80aa030 con 0x7fdba8143490 2023-12-02T00:30:47.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.908+0000 7fdba77fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdba815e730 0x7fdba8140b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:47.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.908+0000 7fdba77fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdba815e730 0x7fdba8140b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33368/0 (socket says 172.21.15.46:33368) 2023-12-02T00:30:47.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.908+0000 7fdba77fe640 1 -- 172.21.15.46:0/1353741623 learned_addr learned my addr 172.21.15.46:0/1353741623 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:47.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.908+0000 7fdba67fc640 1 --2- 172.21.15.46:0/1353741623 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdba815a4e0 0x7fdba80ac6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:47.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.908+0000 7fdba6ffd640 1 --2- 172.21.15.46:0/1353741623 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdba8143490 0x7fdba80a9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:47.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.908+0000 7fdba77fe640 1 -- 172.21.15.46:0/1353741623 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdba8143490 msgr2=0x7fdba80a9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:47.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.908+0000 7fdba77fe640 1 --2- 172.21.15.46:0/1353741623 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdba8143490 0x7fdba80a9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:47.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.908+0000 7fdba77fe640 1 -- 172.21.15.46:0/1353741623 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdba815a4e0 msgr2=0x7fdba80ac6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:47.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.908+0000 7fdba77fe640 1 --2- 172.21.15.46:0/1353741623 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdba815a4e0 0x7fdba80ac6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:47.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.908+0000 7fdba77fe640 1 -- 172.21.15.46:0/1353741623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba000b770 con 0x7fdba815e730 2023-12-02T00:30:47.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.908+0000 7fdba77fe640 1 --2- 172.21.15.46:0/1353741623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdba815e730 0x7fdba8140b60 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fdb9800c7a0 tx=0x7fdb9800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:47.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.908+0000 7fdb9ffff640 1 -- 172.21.15.46:0/1353741623 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb98003b70 con 0x7fdba815e730 2023-12-02T00:30:47.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.908+0000 7fdb9ffff640 1 -- 172.21.15.46:0/1353741623 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb98003cf0 con 0x7fdba815e730 2023-12-02T00:30:47.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.908+0000 7fdbad87c640 1 -- 172.21.15.46:0/1353741623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdba80aa310 con 0x7fdba815e730 2023-12-02T00:30:47.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.908+0000 7fdb9ffff640 1 -- 172.21.15.46:0/1353741623 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb98003ea0 con 0x7fdba815e730 2023-12-02T00:30:47.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.908+0000 7fdbad87c640 1 -- 172.21.15.46:0/1353741623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdba80aa8d0 con 0x7fdba815e730 2023-12-02T00:30:47.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.909+0000 7fdbad87c640 1 -- 172.21.15.46:0/1353741623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdba80b4380 con 0x7fdba815e730 2023-12-02T00:30:47.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.910+0000 7fdb9ffff640 1 -- 172.21.15.46:0/1353741623 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdb9800aa90 con 0x7fdba815e730 2023-12-02T00:30:47.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.910+0000 7fdb9ffff640 1 --2- 172.21.15.46:0/1353741623 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fdb8806ae40 0x7fdb8806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:47.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.910+0000 7fdb9ffff640 1 -- 172.21.15.46:0/1353741623 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fdb98089000 con 0x7fdba815e730 2023-12-02T00:30:47.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.910+0000 7fdba6ffd640 1 --2- 172.21.15.46:0/1353741623 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fdb8806ae40 0x7fdb8806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:47.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.911+0000 7fdba6ffd640 1 --2- 172.21.15.46:0/1353741623 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fdb8806ae40 0x7fdb8806d300 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fdba0038000 tx=0x7fdba0032000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:47.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:47.914+0000 7fdb9ffff640 1 -- 172.21.15.46:0/1353741623 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdb98054440 con 0x7fdba815e730 2023-12-02T00:30:48.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.093+0000 7fdbad87c640 1 -- 172.21.15.46:0/1353741623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fdba8069400 con 0x7fdba815e730 2023-12-02T00:30:48.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.096+0000 7fdb9ffff640 1 -- 172.21.15.46:0/1353741623 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fdb98057a60 con 0x7fdba815e730 2023-12-02T00:30:48.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.098+0000 7fdbad87c640 1 -- 172.21.15.46:0/1353741623 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fdb8806ae40 msgr2=0x7fdb8806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:48.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.098+0000 7fdbad87c640 1 --2- 172.21.15.46:0/1353741623 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fdb8806ae40 0x7fdb8806d300 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fdba0038000 tx=0x7fdba0032000 comp rx=0 tx=0).stop 2023-12-02T00:30:48.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.098+0000 7fdbad87c640 1 -- 172.21.15.46:0/1353741623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdba815e730 msgr2=0x7fdba8140b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:48.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.098+0000 7fdbad87c640 1 --2- 172.21.15.46:0/1353741623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdba815e730 0x7fdba8140b60 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fdb9800c7a0 tx=0x7fdb9800cc60 comp rx=0 tx=0).stop 2023-12-02T00:30:48.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.098+0000 7fdbad87c640 1 -- 172.21.15.46:0/1353741623 shutdown_connections 2023-12-02T00:30:48.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.098+0000 7fdbad87c640 1 --2- 172.21.15.46:0/1353741623 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fdb8806ae40 0x7fdb8806d300 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:48.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.098+0000 7fdbad87c640 1 --2- 172.21.15.46:0/1353741623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdba815e730 0x7fdba8140b60 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:48.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.098+0000 7fdbad87c640 1 --2- 172.21.15.46:0/1353741623 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdba815a4e0 0x7fdba80ac6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:48.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.098+0000 7fdbad87c640 1 --2- 172.21.15.46:0/1353741623 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdba8143490 0x7fdba80a9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:48.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.098+0000 7fdbad87c640 1 -- 172.21.15.46:0/1353741623 >> 172.21.15.46:0/1353741623 conn(0x7fdba806a4d0 msgr2=0x7fdba8070290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:48.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.098+0000 7fdbad87c640 1 -- 172.21.15.46:0/1353741623 shutdown_connections 2023-12-02T00:30:48.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.098+0000 7fdbad87c640 1 -- 172.21.15.46:0/1353741623 wait complete. 2023-12-02T00:30:48.110 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification' 2023-12-02T00:30:48.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.216+0000 7f2be2e36640 1 Processor -- start 2023-12-02T00:30:48.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.216+0000 7f2be2e36640 1 -- start start 2023-12-02T00:30:48.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.217+0000 7f2be2e36640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2bdc140310 0x7f2bdc1618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:48.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.217+0000 7f2be2e36640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2bdc1409d0 0x7f2bdc06a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:48.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.217+0000 7f2be2e36640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bdc1406f0 con 0x7f2bdc1428d0 2023-12-02T00:30:48.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.217+0000 7f2be2e36640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bdc161e20 con 0x7f2bdc140310 2023-12-02T00:30:48.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.217+0000 7f2be2e36640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bdc143960 con 0x7f2bdc1409d0 2023-12-02T00:30:48.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.217+0000 7f2be0bd2640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2bdc140310 0x7f2bdc1618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:48.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.217+0000 7f2bdbfff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2bdc1409d0 0x7f2bdc06a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:48.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.217+0000 7f2be13d3640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f2bdc1428d0 0x7f2bdc0714a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.46:36820/0 (socket says 172.21.15.46:36820) 2023-12-02T00:30:48.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.217+0000 7f2be13d3640 1 -- 172.21.15.46:0/3129478346 learned_addr learned my addr 172.21.15.46:0/3129478346 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:30:48.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.218+0000 7f2be0bd2640 1 -- 172.21.15.46:0/3129478346 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2bdc1409d0 msgr2=0x7f2bdc06a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:48.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.218+0000 7f2be0bd2640 1 --2- 172.21.15.46:0/3129478346 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2bdc1409d0 0x7f2bdc06a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:48.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.218+0000 7f2be0bd2640 1 -- 172.21.15.46:0/3129478346 >> v1:172.21.15.162:6789/0 conn(0x7f2bdc1428d0 legacy=0x7f2bdc0714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:48.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.218+0000 7f2be0bd2640 1 -- 172.21.15.46:0/3129478346 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bdc142de0 con 0x7f2bdc140310 2023-12-02T00:30:48.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.219+0000 7f2be0bd2640 1 --2- 172.21.15.46:0/3129478346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2bdc140310 0x7f2bdc1618e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4009d90 tx=0x7f2bd402be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f42c11d921e3c55 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:48.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.219+0000 7f2bdb7fe640 1 -- 172.21.15.46:0/3129478346 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd402cbc0 con 0x7f2bdc140310 2023-12-02T00:30:48.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.219+0000 7f2bdb7fe640 1 -- 172.21.15.46:0/3129478346 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bd402cd20 con 0x7f2bdc140310 2023-12-02T00:30:48.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.219+0000 7f2bdb7fe640 1 -- 172.21.15.46:0/3129478346 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4034660 con 0x7f2bdc140310 2023-12-02T00:30:48.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.219+0000 7f2be2e36640 1 -- 172.21.15.46:0/3129478346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2bdc140310 msgr2=0x7f2bdc1618e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:48.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.219+0000 7f2be2e36640 1 --2- 172.21.15.46:0/3129478346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2bdc140310 0x7f2bdc1618e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4009d90 tx=0x7f2bd402be10 comp rx=0 tx=0).stop 2023-12-02T00:30:48.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.219+0000 7f2be2e36640 1 -- 172.21.15.46:0/3129478346 shutdown_connections 2023-12-02T00:30:48.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.219+0000 7f2be2e36640 1 --2- 172.21.15.46:0/3129478346 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2bdc1409d0 0x7f2bdc06a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:48.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.219+0000 7f2be2e36640 1 --2- 172.21.15.46:0/3129478346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2bdc140310 0x7f2bdc1618e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:48.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.219+0000 7f2be2e36640 1 -- 172.21.15.46:0/3129478346 >> 172.21.15.46:0/3129478346 conn(0x7f2bdc0b1e50 msgr2=0x7f2bdc0b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:48.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.219+0000 7f2be2e36640 1 -- 172.21.15.46:0/3129478346 shutdown_connections 2023-12-02T00:30:48.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.219+0000 7f2be2e36640 1 -- 172.21.15.46:0/3129478346 wait complete. 2023-12-02T00:30:48.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.220+0000 7f2be2e36640 1 Processor -- start 2023-12-02T00:30:48.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.220+0000 7f2be2e36640 1 -- start start 2023-12-02T00:30:48.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.221+0000 7f2be2e36640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2bdc140310 0x7f2bdc15f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:48.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.221+0000 7f2be2e36640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2bdc1409d0 0x7f2bdc1f3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:48.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.221+0000 7f2be2e36640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bdc1428d0 0x7f2bdc1f6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:48.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.221+0000 7f2be2e36640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bdc1421b0 con 0x7f2bdc1428d0 2023-12-02T00:30:48.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.221+0000 7f2be2e36640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bdc1f73b0 con 0x7f2bdc140310 2023-12-02T00:30:48.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.221+0000 7f2be2e36640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bdc1f70d0 con 0x7f2bdc1409d0 2023-12-02T00:30:48.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.221+0000 7f2be13d3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bdc1428d0 0x7f2bdc1f6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:48.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.221+0000 7f2be13d3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bdc1428d0 0x7f2bdc1f6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33374/0 (socket says 172.21.15.46:33374) 2023-12-02T00:30:48.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.221+0000 7f2be13d3640 1 -- 172.21.15.46:0/3575332017 learned_addr learned my addr 172.21.15.46:0/3575332017 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:48.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.222+0000 7f2bdbfff640 1 --2- 172.21.15.46:0/3575332017 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2bdc1409d0 0x7f2bdc1f3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:48.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.222+0000 7f2be0bd2640 1 --2- 172.21.15.46:0/3575332017 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2bdc140310 0x7f2bdc15f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:48.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.222+0000 7f2be13d3640 1 -- 172.21.15.46:0/3575332017 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2bdc1409d0 msgr2=0x7f2bdc1f3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:48.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.222+0000 7f2be13d3640 1 --2- 172.21.15.46:0/3575332017 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2bdc1409d0 0x7f2bdc1f3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:48.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.222+0000 7f2be13d3640 1 -- 172.21.15.46:0/3575332017 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2bdc140310 msgr2=0x7f2bdc15f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:48.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.222+0000 7f2be13d3640 1 --2- 172.21.15.46:0/3575332017 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2bdc140310 0x7f2bdc15f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:48.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.222+0000 7f2be13d3640 1 -- 172.21.15.46:0/3575332017 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd4009a70 con 0x7f2bdc1428d0 2023-12-02T00:30:48.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.222+0000 7f2be13d3640 1 --2- 172.21.15.46:0/3575332017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bdc1428d0 0x7f2bdc1f6510 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc00c420 tx=0x7f2bcc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:48.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.222+0000 7f2bd97fa640 1 -- 172.21.15.46:0/3575332017 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc005ce0 con 0x7f2bdc1428d0 2023-12-02T00:30:48.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.222+0000 7f2bd97fa640 1 -- 172.21.15.46:0/3575332017 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bcc005e40 con 0x7f2bdc1428d0 2023-12-02T00:30:48.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.222+0000 7f2be2e36640 1 -- 172.21.15.46:0/3575332017 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bdc143bd0 con 0x7f2bdc1428d0 2023-12-02T00:30:48.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.222+0000 7f2bd97fa640 1 -- 172.21.15.46:0/3575332017 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc0072d0 con 0x7f2bdc1428d0 2023-12-02T00:30:48.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.222+0000 7f2be2e36640 1 -- 172.21.15.46:0/3575332017 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bdc144190 con 0x7f2bdc1428d0 2023-12-02T00:30:48.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.223+0000 7f2be2e36640 1 -- 172.21.15.46:0/3575332017 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bdc069400 con 0x7f2bdc1428d0 2023-12-02T00:30:48.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.224+0000 7f2bd97fa640 1 -- 172.21.15.46:0/3575332017 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2bcc0074f0 con 0x7f2bdc1428d0 2023-12-02T00:30:48.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.224+0000 7f2bd97fa640 1 --2- 172.21.15.46:0/3575332017 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2bbc06ae40 0x7f2bbc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:48.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.224+0000 7f2bd97fa640 1 -- 172.21.15.46:0/3575332017 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2bcc088730 con 0x7f2bdc1428d0 2023-12-02T00:30:48.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.224+0000 7f2be0bd2640 1 --2- 172.21.15.46:0/3575332017 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2bbc06ae40 0x7f2bbc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:48.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.225+0000 7f2be0bd2640 1 --2- 172.21.15.46:0/3575332017 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2bbc06ae40 0x7f2bbc06d300 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4036000 tx=0x7f2bd4030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:48.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.228+0000 7f2bd97fa640 1 -- 172.21.15.46:0/3575332017 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2bcc053c20 con 0x7f2bdc1428d0 2023-12-02T00:30:48.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.474+0000 7f2be2e36640 1 -- 172.21.15.46:0/3575332017 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]} v 0) v1 -- 0x7f2bdc06ea60 con 0x7f2bdc1428d0 2023-12-02T00:30:48.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.770+0000 7f2bd97fa640 1 -- 172.21.15.46:0/3575332017 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]}]=0 v132) v1 ==== 147+0+0 (secure 0 0 0) 0x7f2bcc0170d0 con 0x7f2bdc1428d0 2023-12-02T00:30:48.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.772+0000 7f2be2e36640 1 -- 172.21.15.46:0/3575332017 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2bbc06ae40 msgr2=0x7f2bbc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:48.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.772+0000 7f2be2e36640 1 --2- 172.21.15.46:0/3575332017 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2bbc06ae40 0x7f2bbc06d300 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4036000 tx=0x7f2bd4030000 comp rx=0 tx=0).stop 2023-12-02T00:30:48.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.772+0000 7f2be2e36640 1 -- 172.21.15.46:0/3575332017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bdc1428d0 msgr2=0x7f2bdc1f6510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:48.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.772+0000 7f2be2e36640 1 --2- 172.21.15.46:0/3575332017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bdc1428d0 0x7f2bdc1f6510 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc00c420 tx=0x7f2bcc00c8e0 comp rx=0 tx=0).stop 2023-12-02T00:30:48.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.773+0000 7f2be2e36640 1 -- 172.21.15.46:0/3575332017 shutdown_connections 2023-12-02T00:30:48.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.773+0000 7f2be2e36640 1 --2- 172.21.15.46:0/3575332017 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2bbc06ae40 0x7f2bbc06d300 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:48.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.773+0000 7f2be2e36640 1 --2- 172.21.15.46:0/3575332017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bdc1428d0 0x7f2bdc1f6510 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:48.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.773+0000 7f2be2e36640 1 --2- 172.21.15.46:0/3575332017 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2bdc1409d0 0x7f2bdc1f3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:48.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.773+0000 7f2be2e36640 1 --2- 172.21.15.46:0/3575332017 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2bdc140310 0x7f2bdc15f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:48.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.773+0000 7f2be2e36640 1 -- 172.21.15.46:0/3575332017 >> 172.21.15.46:0/3575332017 conn(0x7f2bdc0b1e50 msgr2=0x7f2bdc06f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:48.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.773+0000 7f2be2e36640 1 -- 172.21.15.46:0/3575332017 shutdown_connections 2023-12-02T00:30:48.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.773+0000 7f2be2e36640 1 -- 172.21.15.46:0/3575332017 wait complete. 2023-12-02T00:30:48.785 INFO:tasks.cephfs_test_runner:That a client who does not have the capability for setting ... ok 2023-12-02T00:30:48.785 INFO:tasks.cephfs_test_runner:Starting test: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2023-12-02T00:30:48.786 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:30:48.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.892+0000 7f28b3036640 1 Processor -- start 2023-12-02T00:30:48.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.893+0000 7f28b3036640 1 -- start start 2023-12-02T00:30:48.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.893+0000 7f28b3036640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f28ac15d330 0x7f28ac0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:48.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.893+0000 7f28b3036640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f28ac0d2910 0x7f28ac0d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:48.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.893+0000 7f28b3036640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28ac0b2b10 con 0x7f28ac0d3730 2023-12-02T00:30:48.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.893+0000 7f28b3036640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28ac0d3b50 con 0x7f28ac15d330 2023-12-02T00:30:48.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.893+0000 7f28b3036640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28ac0d1590 con 0x7f28ac0d2910 2023-12-02T00:30:48.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.893+0000 7f28b0dd2640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f28ac15d330 0x7f28ac0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:48.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.893+0000 7f28b0dd2640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f28ac15d330 0x7f28ac0b8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33390/0 (socket says 172.21.15.46:33390) 2023-12-02T00:30:48.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.893+0000 7f28b15d3640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f28ac0d3730 0x7f28ac0b6390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51458/0 (socket says 172.21.15.46:51458) 2023-12-02T00:30:48.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.893+0000 7f28b0dd2640 1 -- 172.21.15.46:0/1606802960 learned_addr learned my addr 172.21.15.46:0/1606802960 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:48.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.894+0000 7f28abfff640 1 --2- 172.21.15.46:0/1606802960 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f28ac0d2910 0x7f28ac0d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:48.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.894+0000 7f28b0dd2640 1 -- 172.21.15.46:0/1606802960 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f28ac0d2910 msgr2=0x7f28ac0d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:48.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.894+0000 7f28b0dd2640 1 --2- 172.21.15.46:0/1606802960 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f28ac0d2910 0x7f28ac0d31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:48.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.894+0000 7f28b0dd2640 1 -- 172.21.15.46:0/1606802960 >> v1:172.21.15.46:6789/0 conn(0x7f28ac0d3730 legacy=0x7f28ac0b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:48.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.894+0000 7f28b0dd2640 1 -- 172.21.15.46:0/1606802960 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28ac0d1870 con 0x7f28ac15d330 2023-12-02T00:30:48.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.894+0000 7f28ab7fe640 1 -- 172.21.15.46:0/1606802960 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2584904858 0 0) 0x7f28ac0b2b10 con 0x7f28ac0d3730 2023-12-02T00:30:48.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.894+0000 7f28abfff640 1 --2- 172.21.15.46:0/1606802960 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f28ac0d2910 0x7f28ac0d31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:30:48.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.895+0000 7f28b0dd2640 1 --2- 172.21.15.46:0/1606802960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28ac15d330 0x7f28ac0b8b80 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f289c009870 tx=0x7f289c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8515913b818fc1c1 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:48.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.895+0000 7f28ab7fe640 1 -- 172.21.15.46:0/1606802960 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289c031640 con 0x7f28ac15d330 2023-12-02T00:30:48.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.895+0000 7f28ab7fe640 1 -- 172.21.15.46:0/1606802960 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f289c0317a0 con 0x7f28ac15d330 2023-12-02T00:30:48.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.895+0000 7f28ab7fe640 1 -- 172.21.15.46:0/1606802960 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289c031b30 con 0x7f28ac15d330 2023-12-02T00:30:48.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.895+0000 7f28b3036640 1 -- 172.21.15.46:0/1606802960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28ac15d330 msgr2=0x7f28ac0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:48.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.895+0000 7f28b3036640 1 --2- 172.21.15.46:0/1606802960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28ac15d330 0x7f28ac0b8b80 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f289c009870 tx=0x7f289c02b8f0 comp rx=0 tx=0).stop 2023-12-02T00:30:48.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.895+0000 7f28b3036640 1 -- 172.21.15.46:0/1606802960 shutdown_connections 2023-12-02T00:30:48.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.895+0000 7f28b3036640 1 --2- 172.21.15.46:0/1606802960 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f28ac0d2910 0x7f28ac0d31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:48.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.895+0000 7f28b3036640 1 --2- 172.21.15.46:0/1606802960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28ac15d330 0x7f28ac0b8b80 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:48.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.895+0000 7f28b3036640 1 -- 172.21.15.46:0/1606802960 >> 172.21.15.46:0/1606802960 conn(0x7f28ac072020 msgr2=0x7f28ac072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:48.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.895+0000 7f28b3036640 1 -- 172.21.15.46:0/1606802960 shutdown_connections 2023-12-02T00:30:48.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.895+0000 7f28b3036640 1 -- 172.21.15.46:0/1606802960 wait complete. 2023-12-02T00:30:48.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.896+0000 7f28b3036640 1 Processor -- start 2023-12-02T00:30:48.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.897+0000 7f28b3036640 1 -- start start 2023-12-02T00:30:48.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.897+0000 7f28b3036640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28ac0d2910 0x7f28ac0c7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:48.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.897+0000 7f28b3036640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f28ac0d3730 0x7f28ac0b8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:48.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.897+0000 7f28b3036640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28ac15d330 0x7f28ac0c9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:48.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.897+0000 7f28b3036640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ac0c8de0 con 0x7f28ac15d330 2023-12-02T00:30:48.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.897+0000 7f28b3036640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ac0c81d0 con 0x7f28ac0d2910 2023-12-02T00:30:48.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.897+0000 7f28b3036640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28ac0c84b0 con 0x7f28ac0d3730 2023-12-02T00:30:48.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.898+0000 7f28b15d3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28ac15d330 0x7f28ac0c9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:48.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.898+0000 7f28b15d3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28ac15d330 0x7f28ac0c9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33402/0 (socket says 172.21.15.46:33402) 2023-12-02T00:30:48.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.898+0000 7f28b15d3640 1 -- 172.21.15.46:0/3733786589 learned_addr learned my addr 172.21.15.46:0/3733786589 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:48.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.898+0000 7f28b0dd2640 1 --2- 172.21.15.46:0/3733786589 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28ac0d2910 0x7f28ac0c7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:48.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.898+0000 7f28abfff640 1 --2- 172.21.15.46:0/3733786589 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f28ac0d3730 0x7f28ac0b8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:48.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.898+0000 7f28b15d3640 1 -- 172.21.15.46:0/3733786589 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f28ac0d3730 msgr2=0x7f28ac0b8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:48.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.898+0000 7f28b15d3640 1 --2- 172.21.15.46:0/3733786589 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f28ac0d3730 0x7f28ac0b8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:48.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.898+0000 7f28b15d3640 1 -- 172.21.15.46:0/3733786589 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28ac0d2910 msgr2=0x7f28ac0c7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:48.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.898+0000 7f28b15d3640 1 --2- 172.21.15.46:0/3733786589 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28ac0d2910 0x7f28ac0c7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:48.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.898+0000 7f28b15d3640 1 -- 172.21.15.46:0/3733786589 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f289c0094d0 con 0x7f28ac15d330 2023-12-02T00:30:48.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.898+0000 7f28b15d3640 1 --2- 172.21.15.46:0/3733786589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28ac15d330 0x7f28ac0c9270 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f28a400c7a0 tx=0x7f28a400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:48.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.898+0000 7f28a97fa640 1 -- 172.21.15.46:0/3733786589 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a400a640 con 0x7f28ac15d330 2023-12-02T00:30:48.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.898+0000 7f28a97fa640 1 -- 172.21.15.46:0/3733786589 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28a400a7a0 con 0x7f28ac15d330 2023-12-02T00:30:48.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.898+0000 7f28a97fa640 1 -- 172.21.15.46:0/3733786589 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a400abd0 con 0x7f28ac15d330 2023-12-02T00:30:48.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.898+0000 7f28b3036640 1 -- 172.21.15.46:0/3733786589 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28ac0c8790 con 0x7f28ac15d330 2023-12-02T00:30:48.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.898+0000 7f28b3036640 1 -- 172.21.15.46:0/3733786589 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28ac0cd600 con 0x7f28ac15d330 2023-12-02T00:30:48.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.899+0000 7f28b3036640 1 -- 172.21.15.46:0/3733786589 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28ac0c8f50 con 0x7f28ac15d330 2023-12-02T00:30:48.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.900+0000 7f28a97fa640 1 -- 172.21.15.46:0/3733786589 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f28a400ad30 con 0x7f28ac15d330 2023-12-02T00:30:48.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.900+0000 7f28a97fa640 1 --2- 172.21.15.46:0/3733786589 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f289806ae40 0x7f289806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:48.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.900+0000 7f28a97fa640 1 -- 172.21.15.46:0/3733786589 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f28a4088880 con 0x7f28ac15d330 2023-12-02T00:30:48.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.900+0000 7f28b0dd2640 1 --2- 172.21.15.46:0/3733786589 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f289806ae40 0x7f289806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:48.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.901+0000 7f28b0dd2640 1 --2- 172.21.15.46:0/3733786589 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f289806ae40 0x7f289806d300 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f289c036000 tx=0x7f289c03d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:48.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:48.905+0000 7f28a97fa640 1 -- 172.21.15.46:0/3733786589 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f28a4053c50 con 0x7f28ac15d330 2023-12-02T00:30:49.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.080+0000 7f28b3036640 1 -- 172.21.15.46:0/3733786589 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28ac069400 con 0x7f28ac15d330 2023-12-02T00:30:49.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.081+0000 7f28a97fa640 1 -- 172.21.15.46:0/3733786589 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+12962 (secure 0 0 0) 0x7f28a4018030 con 0x7f28ac15d330 2023-12-02T00:30:49.082 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:30:49.082 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":42,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:30:47.765690+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6834/2036053042":"2023-12-03T00:30:43.752015+0000","172.21.15.46:6832/2036053042":"2023-12-03T00:30:43.752015+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:30:49.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.083+0000 7f28b3036640 1 -- 172.21.15.46:0/3733786589 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f289806ae40 msgr2=0x7f289806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:49.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.083+0000 7f28b3036640 1 --2- 172.21.15.46:0/3733786589 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f289806ae40 0x7f289806d300 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f289c036000 tx=0x7f289c03d040 comp rx=0 tx=0).stop 2023-12-02T00:30:49.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.083+0000 7f28b3036640 1 -- 172.21.15.46:0/3733786589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28ac15d330 msgr2=0x7f28ac0c9270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:49.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.083+0000 7f28b3036640 1 --2- 172.21.15.46:0/3733786589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28ac15d330 0x7f28ac0c9270 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f28a400c7a0 tx=0x7f28a400cc60 comp rx=0 tx=0).stop 2023-12-02T00:30:49.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.083+0000 7f28b3036640 1 -- 172.21.15.46:0/3733786589 shutdown_connections 2023-12-02T00:30:49.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.083+0000 7f28b3036640 1 --2- 172.21.15.46:0/3733786589 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f289806ae40 0x7f289806d300 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:49.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.083+0000 7f28b3036640 1 --2- 172.21.15.46:0/3733786589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28ac15d330 0x7f28ac0c9270 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:49.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.083+0000 7f28b3036640 1 --2- 172.21.15.46:0/3733786589 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f28ac0d3730 0x7f28ac0b8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:49.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.083+0000 7f28b3036640 1 --2- 172.21.15.46:0/3733786589 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28ac0d2910 0x7f28ac0c7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:49.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.083+0000 7f28b3036640 1 -- 172.21.15.46:0/3733786589 >> 172.21.15.46:0/3733786589 conn(0x7f28ac072020 msgr2=0x7f28ac072400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:49.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.083+0000 7f28b3036640 1 -- 172.21.15.46:0/3733786589 shutdown_connections 2023-12-02T00:30:49.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.083+0000 7f28b3036640 1 -- 172.21.15.46:0/3733786589 wait complete. 2023-12-02T00:30:49.095 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-02T00:28:37.213645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-02T00:30:49.096 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-02T00:30:49.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.204+0000 7f46c652c640 1 Processor -- start 2023-12-02T00:30:49.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.204+0000 7f46c652c640 1 -- start start 2023-12-02T00:30:49.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.205+0000 7f46c652c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f46c01415f0 0x7f46c0141e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:49.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.205+0000 7f46c652c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f46c006bbf0 0x7f46c00b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:49.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.205+0000 7f46c652c640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46c00b2eb0 con 0x7f46c0144a00 2023-12-02T00:30:49.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.205+0000 7f46c652c640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46c00b3190 con 0x7f46c01415f0 2023-12-02T00:30:49.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.205+0000 7f46c652c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46c00b3470 con 0x7f46c006bbf0 2023-12-02T00:30:49.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.205+0000 7f46bffff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f46c01415f0 0x7f46c0141e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:49.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.205+0000 7f46bffff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f46c01415f0 0x7f46c0141e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33418/0 (socket says 172.21.15.46:33418) 2023-12-02T00:30:49.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.205+0000 7f46bffff640 1 -- 172.21.15.46:0/4128114207 learned_addr learned my addr 172.21.15.46:0/4128114207 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:49.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.205+0000 7f46c4ac9640 1 --2- 172.21.15.46:0/4128114207 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f46c006bbf0 0x7f46c00b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:49.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.205+0000 7f46bffff640 1 -- 172.21.15.46:0/4128114207 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f46c006bbf0 msgr2=0x7f46c00b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:49.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.205+0000 7f46bffff640 1 --2- 172.21.15.46:0/4128114207 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f46c006bbf0 0x7f46c00b3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:49.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.205+0000 7f46bffff640 1 -- 172.21.15.46:0/4128114207 >> v1:172.21.15.162:6789/0 conn(0x7f46c0144a00 legacy=0x7f46c006b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:49.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.205+0000 7f46bffff640 1 -- 172.21.15.46:0/4128114207 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c00b3750 con 0x7f46c01415f0 2023-12-02T00:30:49.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.206+0000 7f46bffff640 1 --2- 172.21.15.46:0/4128114207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46c01415f0 0x7f46c0141e00 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f46a8009870 tx=0x7f46a802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=302be19c612449c server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:49.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.206+0000 7f46beffd640 1 -- 172.21.15.46:0/4128114207 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a8030840 con 0x7f46c01415f0 2023-12-02T00:30:49.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.206+0000 7f46beffd640 1 -- 172.21.15.46:0/4128114207 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46a80309a0 con 0x7f46c01415f0 2023-12-02T00:30:49.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.206+0000 7f46beffd640 1 -- 172.21.15.46:0/4128114207 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a8030d30 con 0x7f46c01415f0 2023-12-02T00:30:49.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.207+0000 7f46c652c640 1 -- 172.21.15.46:0/4128114207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46c01415f0 msgr2=0x7f46c0141e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:49.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.207+0000 7f46c652c640 1 --2- 172.21.15.46:0/4128114207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46c01415f0 0x7f46c0141e00 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f46a8009870 tx=0x7f46a802b980 comp rx=0 tx=0).stop 2023-12-02T00:30:49.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.207+0000 7f46c652c640 1 -- 172.21.15.46:0/4128114207 shutdown_connections 2023-12-02T00:30:49.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.207+0000 7f46c652c640 1 --2- 172.21.15.46:0/4128114207 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f46c006bbf0 0x7f46c00b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:49.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.207+0000 7f46c652c640 1 --2- 172.21.15.46:0/4128114207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46c01415f0 0x7f46c0141e00 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:49.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.207+0000 7f46c652c640 1 -- 172.21.15.46:0/4128114207 >> 172.21.15.46:0/4128114207 conn(0x7f46c0070e70 msgr2=0x7f46c0071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:49.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.207+0000 7f46c652c640 1 -- 172.21.15.46:0/4128114207 shutdown_connections 2023-12-02T00:30:49.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.207+0000 7f46c652c640 1 -- 172.21.15.46:0/4128114207 wait complete. 2023-12-02T00:30:49.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.208+0000 7f46c652c640 1 Processor -- start 2023-12-02T00:30:49.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.208+0000 7f46c652c640 1 -- start start 2023-12-02T00:30:49.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.209+0000 7f46c652c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46c006bbf0 0x7f46c012fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:49.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.209+0000 7f46c652c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f46c01415f0 0x7f46c0156f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:49.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.209+0000 7f46c652c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46c0144a00 0x7f46c0131bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:49.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.209+0000 7f46c652c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c0130340 con 0x7f46c006bbf0 2023-12-02T00:30:49.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.209+0000 7f46c652c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c0130b20 con 0x7f46c0144a00 2023-12-02T00:30:49.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.209+0000 7f46c652c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46c0130e00 con 0x7f46c01415f0 2023-12-02T00:30:49.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.209+0000 7f46bffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46c006bbf0 0x7f46c012fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:49.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.209+0000 7f46bffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46c006bbf0 0x7f46c012fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33422/0 (socket says 172.21.15.46:33422) 2023-12-02T00:30:49.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.209+0000 7f46bffff640 1 -- 172.21.15.46:0/2515375794 learned_addr learned my addr 172.21.15.46:0/2515375794 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:49.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.209+0000 7f46c4ac9640 1 --2- 172.21.15.46:0/2515375794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46c0144a00 0x7f46c0131bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:49.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.209+0000 7f46bf7fe640 1 --2- 172.21.15.46:0/2515375794 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f46c01415f0 0x7f46c0156f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:49.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.209+0000 7f46bffff640 1 -- 172.21.15.46:0/2515375794 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f46c01415f0 msgr2=0x7f46c0156f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:49.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.209+0000 7f46bffff640 1 --2- 172.21.15.46:0/2515375794 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f46c01415f0 0x7f46c0156f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:49.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.209+0000 7f46bffff640 1 -- 172.21.15.46:0/2515375794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46c0144a00 msgr2=0x7f46c0131bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:49.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.209+0000 7f46bffff640 1 --2- 172.21.15.46:0/2515375794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46c0144a00 0x7f46c0131bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:49.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.209+0000 7f46bffff640 1 -- 172.21.15.46:0/2515375794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a80094d0 con 0x7f46c006bbf0 2023-12-02T00:30:49.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.209+0000 7f46bffff640 1 --2- 172.21.15.46:0/2515375794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46c006bbf0 0x7f46c012fe00 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f46a8000c00 tx=0x7f46a8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:49.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.210+0000 7f46bcff9640 1 -- 172.21.15.46:0/2515375794 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a8039c90 con 0x7f46c006bbf0 2023-12-02T00:30:49.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.210+0000 7f46bcff9640 1 -- 172.21.15.46:0/2515375794 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46a8039df0 con 0x7f46c006bbf0 2023-12-02T00:30:49.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.210+0000 7f46c652c640 1 -- 172.21.15.46:0/2515375794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46c01310e0 con 0x7f46c006bbf0 2023-12-02T00:30:49.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.210+0000 7f46bcff9640 1 -- 172.21.15.46:0/2515375794 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a8043640 con 0x7f46c006bbf0 2023-12-02T00:30:49.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.210+0000 7f46c652c640 1 -- 172.21.15.46:0/2515375794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46c01316a0 con 0x7f46c006bbf0 2023-12-02T00:30:49.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.211+0000 7f46c652c640 1 -- 172.21.15.46:0/2515375794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46c006f5b0 con 0x7f46c006bbf0 2023-12-02T00:30:49.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.211+0000 7f46bcff9640 1 -- 172.21.15.46:0/2515375794 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f46a80437a0 con 0x7f46c006bbf0 2023-12-02T00:30:49.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.211+0000 7f46bcff9640 1 --2- 172.21.15.46:0/2515375794 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f46a006ae40 0x7f46a006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:49.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.212+0000 7f46bcff9640 1 -- 172.21.15.46:0/2515375794 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f46a803e070 con 0x7f46c006bbf0 2023-12-02T00:30:49.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.212+0000 7f46bf7fe640 1 --2- 172.21.15.46:0/2515375794 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f46a006ae40 0x7f46a006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:49.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.212+0000 7f46bf7fe640 1 --2- 172.21.15.46:0/2515375794 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f46a006ae40 0x7f46a006d300 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f46c0157880 tx=0x7f46b0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:49.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.216+0000 7f46bcff9640 1 -- 172.21.15.46:0/2515375794 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f46a8077760 con 0x7f46c006bbf0 2023-12-02T00:30:49.392 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.391+0000 7f46c652c640 1 -- 172.21.15.46:0/2515375794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f46c0069400 con 0x7f46c006bbf0 2023-12-02T00:30:49.393 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.392+0000 7f46bcff9640 1 -- 172.21.15.46:0/2515375794 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7f46a807ad80 con 0x7f46c006bbf0 2023-12-02T00:30:49.393 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-12-02T00:30:49.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.393+0000 7f46c652c640 1 -- 172.21.15.46:0/2515375794 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f46a006ae40 msgr2=0x7f46a006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:49.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.393+0000 7f46c652c640 1 --2- 172.21.15.46:0/2515375794 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f46a006ae40 0x7f46a006d300 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f46c0157880 tx=0x7f46b0009040 comp rx=0 tx=0).stop 2023-12-02T00:30:49.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.393+0000 7f46c652c640 1 -- 172.21.15.46:0/2515375794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46c006bbf0 msgr2=0x7f46c012fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:49.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.393+0000 7f46c652c640 1 --2- 172.21.15.46:0/2515375794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46c006bbf0 0x7f46c012fe00 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f46a8000c00 tx=0x7f46a8004730 comp rx=0 tx=0).stop 2023-12-02T00:30:49.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.393+0000 7f46c652c640 1 -- 172.21.15.46:0/2515375794 shutdown_connections 2023-12-02T00:30:49.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.393+0000 7f46c652c640 1 --2- 172.21.15.46:0/2515375794 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f46a006ae40 0x7f46a006d300 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:49.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.393+0000 7f46c652c640 1 --2- 172.21.15.46:0/2515375794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46c0144a00 0x7f46c0131bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:49.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.393+0000 7f46c652c640 1 --2- 172.21.15.46:0/2515375794 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f46c01415f0 0x7f46c0156f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:49.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.393+0000 7f46c652c640 1 --2- 172.21.15.46:0/2515375794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46c006bbf0 0x7f46c012fe00 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:49.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.393+0000 7f46c652c640 1 -- 172.21.15.46:0/2515375794 >> 172.21.15.46:0/2515375794 conn(0x7f46c0070e70 msgr2=0x7f46c00b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:49.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.394+0000 7f46c652c640 1 -- 172.21.15.46:0/2515375794 shutdown_connections 2023-12-02T00:30:49.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.394+0000 7f46c652c640 1 -- 172.21.15.46:0/2515375794 wait complete. 2023-12-02T00:30:49.406 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm' 2023-12-02T00:30:49.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.512+0000 7fb2162e1640 1 Processor -- start 2023-12-02T00:30:49.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.512+0000 7fb2162e1640 1 -- start start 2023-12-02T00:30:49.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.513+0000 7fb2162e1640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb2100ad640 0x7fb21006d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:49.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.513+0000 7fb2162e1640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2100adae0 con 0x7fb21015d1d0 2023-12-02T00:30:49.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.513+0000 7fb2162e1640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2100addc0 con 0x7fb21006bbf0 2023-12-02T00:30:49.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.513+0000 7fb2162e1640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2100ae0a0 con 0x7fb2100ad640 2023-12-02T00:30:49.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.513+0000 7fb20ffff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb21015d1d0 0x7fb21006b4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51468/0 (socket says 172.21.15.46:51468) 2023-12-02T00:30:49.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.513+0000 7fb20ffff640 1 -- 172.21.15.46:0/2933422984 learned_addr learned my addr 172.21.15.46:0/2933422984 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:30:49.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.513+0000 7fb20f7fe640 1 --2- 172.21.15.46:0/2933422984 >> v2:172.21.15.162:3300/0 conn(0x7fb2100ad640 0x7fb21006d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:49.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.513+0000 7fb20effd640 1 -- 172.21.15.46:0/2933422984 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 827174044 0 0) 0x7fb2100adae0 con 0x7fb21015d1d0 2023-12-02T00:30:49.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.513+0000 7fb20effd640 1 -- 172.21.15.46:0/2933422984 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1f8003520 con 0x7fb21015d1d0 2023-12-02T00:30:49.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.513+0000 7fb20effd640 1 -- 172.21.15.46:0/2933422984 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3782854044 0 0) 0x7fb1f8003520 con 0x7fb21015d1d0 2023-12-02T00:30:49.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.513+0000 7fb20f7fe640 1 -- 172.21.15.46:0/2933422984 >> v1:172.21.15.162:6789/0 conn(0x7fb21006bbf0 legacy=0x7fb21006d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:49.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.514+0000 7fb20f7fe640 1 -- 172.21.15.46:0/2933422984 >> v1:172.21.15.46:6789/0 conn(0x7fb21015d1d0 legacy=0x7fb21006b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:49.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.514+0000 7fb20f7fe640 1 -- 172.21.15.46:0/2933422984 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2100ae380 con 0x7fb2100ad640 2023-12-02T00:30:49.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.514+0000 7fb20f7fe640 1 --2- 172.21.15.46:0/2933422984 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2100ad640 0x7fb21006d0b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb2000097a0 tx=0x7fb20002b680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da952194ff4e64c5 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:49.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.514+0000 7fb20effd640 1 -- 172.21.15.46:0/2933422984 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb204001670 con 0x7fb2100ad640 2023-12-02T00:30:49.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.515+0000 7fb20effd640 1 -- 172.21.15.46:0/2933422984 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb20002e050 con 0x7fb2100ad640 2023-12-02T00:30:49.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.515+0000 7fb20effd640 1 -- 172.21.15.46:0/2933422984 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb200030710 con 0x7fb2100ad640 2023-12-02T00:30:49.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.515+0000 7fb2162e1640 1 -- 172.21.15.46:0/2933422984 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2100ad640 msgr2=0x7fb21006d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:49.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.515+0000 7fb2162e1640 1 --2- 172.21.15.46:0/2933422984 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2100ad640 0x7fb21006d0b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb2000097a0 tx=0x7fb20002b680 comp rx=0 tx=0).stop 2023-12-02T00:30:49.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.515+0000 7fb2162e1640 1 -- 172.21.15.46:0/2933422984 shutdown_connections 2023-12-02T00:30:49.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.515+0000 7fb2162e1640 1 --2- 172.21.15.46:0/2933422984 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2100ad640 0x7fb21006d0b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:49.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.515+0000 7fb2162e1640 1 -- 172.21.15.46:0/2933422984 >> 172.21.15.46:0/2933422984 conn(0x7fb210070e70 msgr2=0x7fb210071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:49.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.515+0000 7fb2162e1640 1 -- 172.21.15.46:0/2933422984 shutdown_connections 2023-12-02T00:30:49.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.515+0000 7fb2162e1640 1 -- 172.21.15.46:0/2933422984 wait complete. 2023-12-02T00:30:49.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.516+0000 7fb2162e1640 1 Processor -- start 2023-12-02T00:30:49.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.516+0000 7fb2162e1640 1 -- start start 2023-12-02T00:30:49.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.517+0000 7fb2162e1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb21006bbf0 0x7fb2101f2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:49.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.517+0000 7fb2162e1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb2100ad640 0x7fb21015d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:49.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.517+0000 7fb2162e1640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb21015d1d0 0x7fb21015de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:49.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.517+0000 7fb2162e1640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2101f29c0 con 0x7fb21006bbf0 2023-12-02T00:30:49.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.517+0000 7fb2162e1640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb210159150 con 0x7fb21015d1d0 2023-12-02T00:30:49.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.517+0000 7fb2162e1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb210159430 con 0x7fb2100ad640 2023-12-02T00:30:49.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.517+0000 7fb20ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb21006bbf0 0x7fb2101f2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:49.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.517+0000 7fb20ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb21006bbf0 0x7fb2101f2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33430/0 (socket says 172.21.15.46:33430) 2023-12-02T00:30:49.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.517+0000 7fb20ffff640 1 -- 172.21.15.46:0/1439669917 learned_addr learned my addr 172.21.15.46:0/1439669917 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:49.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.517+0000 7fb20f7fe640 1 --2- 172.21.15.46:0/1439669917 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb2100ad640 0x7fb21015d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:49.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.517+0000 7fb21487e640 1 --2- 172.21.15.46:0/1439669917 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb21015d1d0 0x7fb21015de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:49.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.518+0000 7fb20ffff640 1 -- 172.21.15.46:0/1439669917 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb2100ad640 msgr2=0x7fb21015d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:49.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.518+0000 7fb20ffff640 1 --2- 172.21.15.46:0/1439669917 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb2100ad640 0x7fb21015d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:49.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.518+0000 7fb20ffff640 1 -- 172.21.15.46:0/1439669917 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb21015d1d0 msgr2=0x7fb21015de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:49.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.518+0000 7fb20ffff640 1 --2- 172.21.15.46:0/1439669917 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb21015d1d0 0x7fb21015de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:49.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.518+0000 7fb20ffff640 1 -- 172.21.15.46:0/1439669917 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb200009400 con 0x7fb21006bbf0 2023-12-02T00:30:49.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.518+0000 7fb21487e640 1 --2- 172.21.15.46:0/1439669917 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb21015d1d0 0x7fb21015de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:30:49.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.518+0000 7fb20ffff640 1 --2- 172.21.15.46:0/1439669917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb21006bbf0 0x7fb2101f2480 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fb20400c780 tx=0x7fb20400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:49.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.518+0000 7fb20cff9640 1 -- 172.21.15.46:0/1439669917 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2040072d0 con 0x7fb21006bbf0 2023-12-02T00:30:49.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.518+0000 7fb20cff9640 1 -- 172.21.15.46:0/1439669917 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb204007430 con 0x7fb21006bbf0 2023-12-02T00:30:49.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.518+0000 7fb2162e1640 1 -- 172.21.15.46:0/1439669917 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb210159710 con 0x7fb21006bbf0 2023-12-02T00:30:49.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.518+0000 7fb20cff9640 1 -- 172.21.15.46:0/1439669917 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2040047e0 con 0x7fb21006bbf0 2023-12-02T00:30:49.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.518+0000 7fb2162e1640 1 -- 172.21.15.46:0/1439669917 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2101583b0 con 0x7fb21006bbf0 2023-12-02T00:30:49.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.519+0000 7fb2162e1640 1 -- 172.21.15.46:0/1439669917 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb210159c30 con 0x7fb21006bbf0 2023-12-02T00:30:49.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.519+0000 7fb20cff9640 1 -- 172.21.15.46:0/1439669917 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb204004940 con 0x7fb21006bbf0 2023-12-02T00:30:49.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.520+0000 7fb20cff9640 1 --2- 172.21.15.46:0/1439669917 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb1fc06ae40 0x7fb1fc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:49.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.520+0000 7fb20cff9640 1 -- 172.21.15.46:0/1439669917 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb204088600 con 0x7fb21006bbf0 2023-12-02T00:30:49.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.520+0000 7fb20f7fe640 1 --2- 172.21.15.46:0/1439669917 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb1fc06ae40 0x7fb1fc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:49.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.521+0000 7fb20f7fe640 1 --2- 172.21.15.46:0/1439669917 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb1fc06ae40 0x7fb1fc06d300 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb200034000 tx=0x7fb200008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:49.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.524+0000 7fb20cff9640 1 -- 172.21.15.46:0/1439669917 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb210159c30 con 0x7fb21006bbf0 2023-12-02T00:30:49.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.767+0000 7fb2162e1640 1 -- 172.21.15.46:0/1439669917 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]} v 0) v1 -- 0x7fb210159c30 con 0x7fb21006bbf0 2023-12-02T00:30:49.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.774+0000 7fb20cff9640 1 -- 172.21.15.46:0/1439669917 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]}]=0 v133) v1 ==== 137+0+0 (secure 0 0 0) 0x7fb210159c30 con 0x7fb21006bbf0 2023-12-02T00:30:49.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.776+0000 7fb2162e1640 1 -- 172.21.15.46:0/1439669917 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb1fc06ae40 msgr2=0x7fb1fc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:49.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.776+0000 7fb2162e1640 1 --2- 172.21.15.46:0/1439669917 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb1fc06ae40 0x7fb1fc06d300 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb200034000 tx=0x7fb200008030 comp rx=0 tx=0).stop 2023-12-02T00:30:49.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.776+0000 7fb2162e1640 1 -- 172.21.15.46:0/1439669917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb21006bbf0 msgr2=0x7fb2101f2480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:49.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.776+0000 7fb2162e1640 1 --2- 172.21.15.46:0/1439669917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb21006bbf0 0x7fb2101f2480 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fb20400c780 tx=0x7fb20400cc40 comp rx=0 tx=0).stop 2023-12-02T00:30:49.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.776+0000 7fb2162e1640 1 -- 172.21.15.46:0/1439669917 shutdown_connections 2023-12-02T00:30:49.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.776+0000 7fb2162e1640 1 --2- 172.21.15.46:0/1439669917 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb1fc06ae40 0x7fb1fc06d300 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:49.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.776+0000 7fb2162e1640 1 --2- 172.21.15.46:0/1439669917 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb21015d1d0 0x7fb21015de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:49.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.776+0000 7fb2162e1640 1 --2- 172.21.15.46:0/1439669917 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb2100ad640 0x7fb21015d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:49.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.776+0000 7fb2162e1640 1 --2- 172.21.15.46:0/1439669917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb21006bbf0 0x7fb2101f2480 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:49.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.776+0000 7fb2162e1640 1 -- 172.21.15.46:0/1439669917 >> 172.21.15.46:0/1439669917 conn(0x7fb210070e70 msgr2=0x7fb21015a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:49.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.776+0000 7fb2162e1640 1 -- 172.21.15.46:0/1439669917 shutdown_connections 2023-12-02T00:30:49.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.776+0000 7fb2162e1640 1 -- 172.21.15.46:0/1439669917 wait complete. 2023-12-02T00:30:49.788 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-02T00:30:49.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.896+0000 7f07068c6640 1 Processor -- start 2023-12-02T00:30:49.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.896+0000 7f07068c6640 1 -- start start 2023-12-02T00:30:49.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.897+0000 7f07068c6640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f07000d2910 0x7f07000d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:49.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.897+0000 7f07068c6640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07000b2b10 con 0x7f07000d3730 2023-12-02T00:30:49.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.897+0000 7f07068c6640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07000d1590 con 0x7f070015d330 2023-12-02T00:30:49.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.897+0000 7f07068c6640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07000d3bc0 con 0x7f07000d2910 2023-12-02T00:30:49.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.897+0000 7f06ff7fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f07000d2910 0x7f07000d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:49.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.897+0000 7f06ff7fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f07000d2910 0x7f07000d31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33442/0 (socket says 172.21.15.46:33442) 2023-12-02T00:30:49.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.897+0000 7f0704e63640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f07000d3730 0x7f07000b6390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51480/0 (socket says 172.21.15.46:51480) 2023-12-02T00:30:49.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.897+0000 7f06ff7fe640 1 -- 172.21.15.46:0/2956762308 learned_addr learned my addr 172.21.15.46:0/2956762308 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:49.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.898+0000 7f06ff7fe640 1 -- 172.21.15.46:0/2956762308 >> v1:172.21.15.162:6789/0 conn(0x7f070015d330 legacy=0x7f07000b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:49.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.898+0000 7f06ff7fe640 1 -- 172.21.15.46:0/2956762308 >> v1:172.21.15.46:6789/0 conn(0x7f07000d3730 legacy=0x7f07000b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:49.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.898+0000 7f06ff7fe640 1 -- 172.21.15.46:0/2956762308 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07000d1870 con 0x7f07000d2910 2023-12-02T00:30:49.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.898+0000 7f06ff7fe640 1 --2- 172.21.15.46:0/2956762308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07000d2910 0x7f07000d31f0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f06f4009870 tx=0x7f06f402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecd28648af95af83 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:49.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.899+0000 7f06feffd640 1 -- 172.21.15.46:0/2956762308 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f4030860 con 0x7f07000d2910 2023-12-02T00:30:49.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.899+0000 7f06feffd640 1 -- 172.21.15.46:0/2956762308 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06f40309c0 con 0x7f07000d2910 2023-12-02T00:30:49.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.899+0000 7f06feffd640 1 -- 172.21.15.46:0/2956762308 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f4030d50 con 0x7f07000d2910 2023-12-02T00:30:49.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.899+0000 7f07068c6640 1 -- 172.21.15.46:0/2956762308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07000d2910 msgr2=0x7f07000d31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:49.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.899+0000 7f07068c6640 1 --2- 172.21.15.46:0/2956762308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07000d2910 0x7f07000d31f0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f06f4009870 tx=0x7f06f402b9a0 comp rx=0 tx=0).stop 2023-12-02T00:30:49.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.899+0000 7f07068c6640 1 -- 172.21.15.46:0/2956762308 shutdown_connections 2023-12-02T00:30:49.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.899+0000 7f07068c6640 1 --2- 172.21.15.46:0/2956762308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07000d2910 0x7f07000d31f0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:49.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.899+0000 7f07068c6640 1 -- 172.21.15.46:0/2956762308 >> 172.21.15.46:0/2956762308 conn(0x7f0700072020 msgr2=0x7f0700072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:49.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.899+0000 7f07068c6640 1 -- 172.21.15.46:0/2956762308 shutdown_connections 2023-12-02T00:30:49.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.899+0000 7f07068c6640 1 -- 172.21.15.46:0/2956762308 wait complete. 2023-12-02T00:30:49.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.900+0000 7f07068c6640 1 Processor -- start 2023-12-02T00:30:49.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.901+0000 7f07068c6640 1 -- start start 2023-12-02T00:30:49.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.901+0000 7f07068c6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07000d2910 0x7f07001f3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:49.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.901+0000 7f07068c6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07000d3730 0x7f07000cce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:49.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.901+0000 7f07068c6640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f070015d330 0x7f07000b7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:49.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.901+0000 7f07068c6640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0700159c00 con 0x7f07000d2910 2023-12-02T00:30:49.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.901+0000 7f07068c6640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07000cd6d0 con 0x7f070015d330 2023-12-02T00:30:49.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.901+0000 7f07068c6640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07000c8920 con 0x7f07000d3730 2023-12-02T00:30:49.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.901+0000 7f06fffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07000d2910 0x7f07001f3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:49.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.901+0000 7f06fffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07000d2910 0x7f07001f3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33446/0 (socket says 172.21.15.46:33446) 2023-12-02T00:30:49.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.901+0000 7f06fffff640 1 -- 172.21.15.46:0/121097115 learned_addr learned my addr 172.21.15.46:0/121097115 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:49.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.901+0000 7f0704e63640 1 --2- 172.21.15.46:0/121097115 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f070015d330 0x7f07000b7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:49.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.901+0000 7f06ff7fe640 1 --2- 172.21.15.46:0/121097115 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07000d3730 0x7f07000cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:49.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.902+0000 7f06fffff640 1 -- 172.21.15.46:0/121097115 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07000d3730 msgr2=0x7f07000cce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:49.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.902+0000 7f06fffff640 1 --2- 172.21.15.46:0/121097115 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07000d3730 0x7f07000cce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:49.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.902+0000 7f06fffff640 1 -- 172.21.15.46:0/121097115 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f070015d330 msgr2=0x7f07000b7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:49.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.902+0000 7f06fffff640 1 --2- 172.21.15.46:0/121097115 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f070015d330 0x7f07000b7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:49.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.902+0000 7f06fffff640 1 -- 172.21.15.46:0/121097115 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06f40094d0 con 0x7f07000d2910 2023-12-02T00:30:49.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.902+0000 7f06fffff640 1 --2- 172.21.15.46:0/121097115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07000d2910 0x7f07001f3010 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f06f000c400 tx=0x7f06f000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:49.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.902+0000 7f06fcff9640 1 -- 172.21.15.46:0/121097115 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f00052d0 con 0x7f07000d2910 2023-12-02T00:30:49.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.902+0000 7f06fcff9640 1 -- 172.21.15.46:0/121097115 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06f0005430 con 0x7f07000d2910 2023-12-02T00:30:49.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.902+0000 7f06fcff9640 1 -- 172.21.15.46:0/121097115 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f0003830 con 0x7f07000d2910 2023-12-02T00:30:49.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.902+0000 7f07068c6640 1 -- 172.21.15.46:0/121097115 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07000c7ff0 con 0x7f07000d2910 2023-12-02T00:30:49.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.902+0000 7f07068c6640 1 -- 172.21.15.46:0/121097115 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07000c85b0 con 0x7f07000d2910 2023-12-02T00:30:49.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.903+0000 7f07068c6640 1 -- 172.21.15.46:0/121097115 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0700066da0 con 0x7f07000d2910 2023-12-02T00:30:49.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.904+0000 7f06fcff9640 1 -- 172.21.15.46:0/121097115 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f06f0003990 con 0x7f07000d2910 2023-12-02T00:30:49.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.904+0000 7f06fcff9640 1 --2- 172.21.15.46:0/121097115 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f06ec06ae70 0x7f06ec06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:49.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.904+0000 7f06fcff9640 1 -- 172.21.15.46:0/121097115 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f06f0088730 con 0x7f07000d2910 2023-12-02T00:30:49.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.905+0000 7f06ff7fe640 1 --2- 172.21.15.46:0/121097115 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f06ec06ae70 0x7f06ec06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:49.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.905+0000 7f06ff7fe640 1 --2- 172.21.15.46:0/121097115 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f06ec06ae70 0x7f06ec06d330 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f06f4036000 tx=0x7f06f4008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:49.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:49.909+0000 7f06fcff9640 1 -- 172.21.15.46:0/121097115 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f06f0053c20 con 0x7f07000d2910 2023-12-02T00:30:50.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.088+0000 7f07068c6640 1 -- 172.21.15.46:0/121097115 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0700069400 con 0x7f07000d2910 2023-12-02T00:30:50.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.090+0000 7f06fcff9640 1 -- 172.21.15.46:0/121097115 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f06f0057240 con 0x7f07000d2910 2023-12-02T00:30:50.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.092+0000 7f07068c6640 1 -- 172.21.15.46:0/121097115 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f06ec06ae70 msgr2=0x7f06ec06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:50.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.092+0000 7f07068c6640 1 --2- 172.21.15.46:0/121097115 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f06ec06ae70 0x7f06ec06d330 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f06f4036000 tx=0x7f06f4008030 comp rx=0 tx=0).stop 2023-12-02T00:30:50.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.092+0000 7f07068c6640 1 -- 172.21.15.46:0/121097115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07000d2910 msgr2=0x7f07001f3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:50.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.092+0000 7f07068c6640 1 --2- 172.21.15.46:0/121097115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07000d2910 0x7f07001f3010 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f06f000c400 tx=0x7f06f000c8c0 comp rx=0 tx=0).stop 2023-12-02T00:30:50.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.092+0000 7f07068c6640 1 -- 172.21.15.46:0/121097115 shutdown_connections 2023-12-02T00:30:50.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.092+0000 7f07068c6640 1 --2- 172.21.15.46:0/121097115 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f06ec06ae70 0x7f06ec06d330 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:50.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.092+0000 7f07068c6640 1 --2- 172.21.15.46:0/121097115 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f070015d330 0x7f07000b7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:50.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.092+0000 7f07068c6640 1 --2- 172.21.15.46:0/121097115 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07000d3730 0x7f07000cce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:50.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.092+0000 7f07068c6640 1 --2- 172.21.15.46:0/121097115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07000d2910 0x7f07001f3010 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:50.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.092+0000 7f07068c6640 1 -- 172.21.15.46:0/121097115 >> 172.21.15.46:0/121097115 conn(0x7f0700072020 msgr2=0x7f0700072400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:50.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.093+0000 7f07068c6640 1 -- 172.21.15.46:0/121097115 shutdown_connections 2023-12-02T00:30:50.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.093+0000 7f07068c6640 1 -- 172.21.15.46:0/121097115 wait complete. 2023-12-02T00:30:50.104 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-02T00:30:50.105 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-02T00:30:50.145 DEBUG:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-02T00:30:50.191 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-02T00:30:50.191 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-02T00:30:50.192 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-02T00:30:50.207 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057150,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498968k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19546 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249480k,nr_inodes=812370,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498968k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-02T00:30:50.207 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-12-02T00:30:50.207 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-02T00:30:50.224 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056379,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497728k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15716 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248860k,nr_inodes=812215,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497728k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-02T00:30:50.224 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-02T00:30:50.224 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-02T00:30:50.262 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057150,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498968k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19546 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249480k,nr_inodes=812370,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498968k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-02T00:30:50.263 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-12-02T00:30:50.263 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-02T00:30:50.281 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056379,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497728k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15716 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248860k,nr_inodes=812215,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497728k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-02T00:30:50.281 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-02T00:30:50.281 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-02T00:30:50.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.407+0000 7f9c05790640 1 Processor -- start 2023-12-02T00:30:50.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.407+0000 7f9c05790640 1 -- start start 2023-12-02T00:30:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.408+0000 7f9c05790640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9c000d1d50 0x7f9c0006bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.408+0000 7f9c05790640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c0006ce00 con 0x7f9c0015d2e0 2023-12-02T00:30:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.408+0000 7f9c05790640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c0006c1f0 con 0x7f9c0015d6c0 2023-12-02T00:30:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.408+0000 7f9c05790640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c0006c4d0 con 0x7f9c000d1d50 2023-12-02T00:30:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.408+0000 7f9bff7fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9c000d1d50 0x7f9c0006bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.408+0000 7f9bfeffd640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9c0015d2e0 0x7f9c000d1640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51488/0 (socket says 172.21.15.46:51488) 2023-12-02T00:30:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.408+0000 7f9bfeffd640 1 -- 172.21.15.46:0/845278611 learned_addr learned my addr 172.21.15.46:0/845278611 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:30:50.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.409+0000 7f9bff7fe640 1 -- 172.21.15.46:0/845278611 >> v1:172.21.15.162:6789/0 conn(0x7f9c0015d6c0 legacy=0x7f9c0006b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:50.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.409+0000 7f9bff7fe640 1 -- 172.21.15.46:0/845278611 >> v1:172.21.15.46:6789/0 conn(0x7f9c0015d2e0 legacy=0x7f9c000d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:50.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.409+0000 7f9bff7fe640 1 -- 172.21.15.46:0/845278611 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c0006c7b0 con 0x7f9c000d1d50 2023-12-02T00:30:50.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.409+0000 7f9bfdffb640 1 -- 172.21.15.46:0/845278611 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2655881227 0 0) 0x7f9c0006ce00 con 0x7f9c0015d2e0 2023-12-02T00:30:50.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.409+0000 7f9bff7fe640 1 --2- 172.21.15.46:0/845278611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c000d1d50 0x7f9c0006bbd0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0009d90 tx=0x7f9bf002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9afe3383e1d9daaf server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:50.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.409+0000 7f9bfdffb640 1 -- 172.21.15.46:0/845278611 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf002cbd0 con 0x7f9c000d1d50 2023-12-02T00:30:50.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.410+0000 7f9bfdffb640 1 -- 172.21.15.46:0/845278611 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf002cd30 con 0x7f9c000d1d50 2023-12-02T00:30:50.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.410+0000 7f9bfdffb640 1 -- 172.21.15.46:0/845278611 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf002cea0 con 0x7f9c000d1d50 2023-12-02T00:30:50.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.410+0000 7f9c05790640 1 -- 172.21.15.46:0/845278611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c000d1d50 msgr2=0x7f9c0006bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:50.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.410+0000 7f9c05790640 1 --2- 172.21.15.46:0/845278611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c000d1d50 0x7f9c0006bbd0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0009d90 tx=0x7f9bf002be20 comp rx=0 tx=0).stop 2023-12-02T00:30:50.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.410+0000 7f9c05790640 1 -- 172.21.15.46:0/845278611 shutdown_connections 2023-12-02T00:30:50.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.410+0000 7f9c05790640 1 --2- 172.21.15.46:0/845278611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c000d1d50 0x7f9c0006bbd0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:50.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.410+0000 7f9c05790640 1 -- 172.21.15.46:0/845278611 >> 172.21.15.46:0/845278611 conn(0x7f9c0009c580 msgr2=0x7f9c000d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:50.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.410+0000 7f9c05790640 1 -- 172.21.15.46:0/845278611 shutdown_connections 2023-12-02T00:30:50.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.410+0000 7f9c05790640 1 -- 172.21.15.46:0/845278611 wait complete. 2023-12-02T00:30:50.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.411+0000 7f9c05790640 1 Processor -- start 2023-12-02T00:30:50.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.411+0000 7f9c05790640 1 -- start start 2023-12-02T00:30:50.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.412+0000 7f9c05790640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9c000d1d50 0x7f9c000b8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:50.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.412+0000 7f9c05790640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c0015d2e0 0x7f9c001545d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:50.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.412+0000 7f9c05790640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9c0015d6c0 0x7f9c00160120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:50.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.412+0000 7f9c05790640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c000b8170 con 0x7f9c0015d2e0 2023-12-02T00:30:50.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.412+0000 7f9c05790640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c000b7560 con 0x7f9c0015d6c0 2023-12-02T00:30:50.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.412+0000 7f9c05790640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c000b7840 con 0x7f9c000d1d50 2023-12-02T00:30:50.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.412+0000 7f9bfe7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c0015d2e0 0x7f9c001545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:50.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.412+0000 7f9bfe7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c0015d2e0 0x7f9c001545d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33466/0 (socket says 172.21.15.46:33466) 2023-12-02T00:30:50.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.412+0000 7f9bfe7fc640 1 -- 172.21.15.46:0/1450825415 learned_addr learned my addr 172.21.15.46:0/1450825415 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:50.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.413+0000 7f9bff7fe640 1 --2- 172.21.15.46:0/1450825415 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9c0015d6c0 0x7f9c00160120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:50.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.413+0000 7f9bfeffd640 1 --2- 172.21.15.46:0/1450825415 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9c000d1d50 0x7f9c000b8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:50.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.413+0000 7f9bfe7fc640 1 -- 172.21.15.46:0/1450825415 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9c000d1d50 msgr2=0x7f9c000b8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:50.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.413+0000 7f9bfe7fc640 1 --2- 172.21.15.46:0/1450825415 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9c000d1d50 0x7f9c000b8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:50.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.413+0000 7f9bfe7fc640 1 -- 172.21.15.46:0/1450825415 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9c0015d6c0 msgr2=0x7f9c00160120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:50.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.413+0000 7f9bfe7fc640 1 --2- 172.21.15.46:0/1450825415 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9c0015d6c0 0x7f9c00160120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:50.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.413+0000 7f9bfe7fc640 1 -- 172.21.15.46:0/1450825415 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf0009a70 con 0x7f9c0015d2e0 2023-12-02T00:30:50.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.413+0000 7f9bff7fe640 1 --2- 172.21.15.46:0/1450825415 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9c0015d6c0 0x7f9c00160120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:30:50.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.413+0000 7f9bfeffd640 1 --2- 172.21.15.46:0/1450825415 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9c000d1d50 0x7f9c000b8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:30:50.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.413+0000 7f9bfe7fc640 1 --2- 172.21.15.46:0/1450825415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c0015d2e0 0x7f9c001545d0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f9be800b3c0 tx=0x7f9be800b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:50.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.413+0000 7f9bf77fe640 1 -- 172.21.15.46:0/1450825415 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be8011830 con 0x7f9c0015d2e0 2023-12-02T00:30:50.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.413+0000 7f9bf77fe640 1 -- 172.21.15.46:0/1450825415 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9be8011990 con 0x7f9c0015d2e0 2023-12-02T00:30:50.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.413+0000 7f9bf77fe640 1 -- 172.21.15.46:0/1450825415 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be8011d80 con 0x7f9c0015d2e0 2023-12-02T00:30:50.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.413+0000 7f9c05790640 1 -- 172.21.15.46:0/1450825415 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c000b7b20 con 0x7f9c0015d2e0 2023-12-02T00:30:50.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.413+0000 7f9c05790640 1 -- 172.21.15.46:0/1450825415 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c00144140 con 0x7f9c0015d2e0 2023-12-02T00:30:50.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.414+0000 7f9c05790640 1 -- 172.21.15.46:0/1450825415 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c00066760 con 0x7f9c0015d2e0 2023-12-02T00:30:50.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.414+0000 7f9bf77fe640 1 -- 172.21.15.46:0/1450825415 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9be801b460 con 0x7f9c0015d2e0 2023-12-02T00:30:50.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.415+0000 7f9bf77fe640 1 --2- 172.21.15.46:0/1450825415 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9be006ae70 0x7f9be006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:50.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.415+0000 7f9bf77fe640 1 -- 172.21.15.46:0/1450825415 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9be8088dc0 con 0x7f9c0015d2e0 2023-12-02T00:30:50.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.415+0000 7f9bfeffd640 1 --2- 172.21.15.46:0/1450825415 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9be006ae70 0x7f9be006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:50.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.416+0000 7f9bfeffd640 1 --2- 172.21.15.46:0/1450825415 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9be006ae70 0x7f9be006d330 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8000f60 tx=0x7f9bf8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:50.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.419+0000 7f9bf77fe640 1 -- 172.21.15.46:0/1450825415 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9be8054200 con 0x7f9c0015d2e0 2023-12-02T00:30:50.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.632+0000 7f9c05790640 1 -- 172.21.15.46:0/1450825415 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9c00067420 con 0x7f9c0015d2e0 2023-12-02T00:30:50.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.632+0000 7f9bf77fe640 1 -- 172.21.15.46:0/1450825415 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+3703 (secure 0 0 0) 0x7f9be8057820 con 0x7f9c0015d2e0 2023-12-02T00:30:50.634 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:30:50.634 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":16,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4811,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/3097254664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3097254664},{"type":"v1","addr":"172.21.15.162:6835","nonce":3097254664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":4826,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/3489476943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3489476943},{"type":"v1","addr":"172.21.15.162:6837","nonce":3489476943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5274,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1283185308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":1283185308},{"type":"v1","addr":"172.21.15.46:6835","nonce":1283185308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":5898,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/2624310627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2624310627},{"type":"v1","addr":"172.21.15.46:6834","nonce":2624310627}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":16}],"filesystems":[]} 2023-12-02T00:30:50.634 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 16 2023-12-02T00:30:50.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.635+0000 7f9c05790640 1 -- 172.21.15.46:0/1450825415 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9be006ae70 msgr2=0x7f9be006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:50.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.635+0000 7f9c05790640 1 --2- 172.21.15.46:0/1450825415 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9be006ae70 0x7f9be006d330 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8000f60 tx=0x7f9bf8009040 comp rx=0 tx=0).stop 2023-12-02T00:30:50.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.635+0000 7f9c05790640 1 -- 172.21.15.46:0/1450825415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c0015d2e0 msgr2=0x7f9c001545d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:50.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.635+0000 7f9c05790640 1 --2- 172.21.15.46:0/1450825415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c0015d2e0 0x7f9c001545d0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f9be800b3c0 tx=0x7f9be800b880 comp rx=0 tx=0).stop 2023-12-02T00:30:50.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.635+0000 7f9c05790640 1 -- 172.21.15.46:0/1450825415 shutdown_connections 2023-12-02T00:30:50.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.635+0000 7f9c05790640 1 --2- 172.21.15.46:0/1450825415 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9be006ae70 0x7f9be006d330 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:50.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.635+0000 7f9c05790640 1 --2- 172.21.15.46:0/1450825415 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9c0015d6c0 0x7f9c00160120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:50.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.635+0000 7f9c05790640 1 --2- 172.21.15.46:0/1450825415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c0015d2e0 0x7f9c001545d0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:50.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.635+0000 7f9c05790640 1 --2- 172.21.15.46:0/1450825415 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9c000d1d50 0x7f9c000b8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:50.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.635+0000 7f9c05790640 1 -- 172.21.15.46:0/1450825415 >> 172.21.15.46:0/1450825415 conn(0x7f9c0009c580 msgr2=0x7f9c000ce2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:50.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.635+0000 7f9c05790640 1 -- 172.21.15.46:0/1450825415 shutdown_connections 2023-12-02T00:30:50.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:50.635+0000 7f9c05790640 1 -- 172.21.15.46:0/1450825415 wait complete. 2023-12-02T00:30:50.647 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-02T00:30:50.647 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-02T00:30:50.647 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-02T00:30:50.647 INFO:teuthology.orchestra.run:waiting for 300 2023-12-02T00:30:50.648 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-02T00:30:50.648 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-02T00:30:50.648 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-02T00:30:50.648 INFO:teuthology.orchestra.run:waiting for 300 2023-12-02T00:30:50.648 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-02T00:30:50.648 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-02T00:30:50.648 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-02T00:30:50.649 INFO:teuthology.orchestra.run:waiting for 300 2023-12-02T00:30:50.649 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-02T00:30:50.649 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-02T00:30:50.649 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-02T00:30:50.649 INFO:teuthology.orchestra.run:waiting for 300 2023-12-02T00:30:50.658 INFO:tasks.ceph.mds.c:Stopped 2023-12-02T00:30:50.659 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-02T00:30:50.660 INFO:tasks.ceph.mds.b:Stopped 2023-12-02T00:30:50.660 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-02T00:30:50.661 INFO:tasks.ceph.mds.d:Stopped 2023-12-02T00:30:50.661 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-02T00:30:50.688 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-02T00:30:50.701 INFO:tasks.ceph.mds.c:Started 2023-12-02T00:30:50.701 DEBUG:teuthology.parallel:result is None 2023-12-02T00:30:50.703 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-02T00:30:50.704 INFO:tasks.ceph.mds.b:Started 2023-12-02T00:30:50.704 DEBUG:teuthology.parallel:result is None 2023-12-02T00:30:50.705 INFO:tasks.ceph.mds.d:Started 2023-12-02T00:30:50.705 DEBUG:teuthology.parallel:result is None 2023-12-02T00:30:50.781 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2023-12-02T00:30:50.787 INFO:tasks.ceph.mds.b.smithi162.stdout:starting mds.b at 2023-12-02T00:30:50.787 INFO:tasks.ceph.mds.d.smithi162.stdout:starting mds.d at 2023-12-02T00:30:53.150 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2023-12-02T00:30:56.659 INFO:tasks.ceph.mds.a:Stopped 2023-12-02T00:30:56.659 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-02T00:30:56.662 INFO:tasks.ceph.mds.a:Started 2023-12-02T00:30:56.662 DEBUG:teuthology.parallel:result is None 2023-12-02T00:30:56.663 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-02T00:30:56.754 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2023-12-02T00:30:56.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.781+0000 7f6e5ea43640 1 Processor -- start 2023-12-02T00:30:56.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.781+0000 7f6e5ea43640 1 -- start start 2023-12-02T00:30:56.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.781+0000 7f6e5ea43640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6e5815d330 0x7f6e580b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:56.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.781+0000 7f6e5ea43640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e580b2b10 con 0x7f6e580d2910 2023-12-02T00:30:56.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.781+0000 7f6e5ea43640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e580d1590 con 0x7f6e580d3900 2023-12-02T00:30:56.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.781+0000 7f6e5ea43640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e580d1870 con 0x7f6e5815d330 2023-12-02T00:30:56.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.782+0000 7f6e577fe640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6e580d2910 0x7f6e580d31f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53760/0 (socket says 172.21.15.46:53760) 2023-12-02T00:30:56.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.782+0000 7f6e577fe640 1 -- 172.21.15.46:0/3444642814 learned_addr learned my addr 172.21.15.46:0/3444642814 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:30:56.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.782+0000 7f6e57fff640 1 --2- 172.21.15.46:0/3444642814 >> v2:172.21.15.162:3300/0 conn(0x7f6e5815d330 0x7f6e580b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:56.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.782+0000 7f6e56ffd640 1 -- 172.21.15.46:0/3444642814 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1988722900 0 0) 0x7f6e580b2b10 con 0x7f6e580d2910 2023-12-02T00:30:56.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.782+0000 7f6e56ffd640 1 -- 172.21.15.46:0/3444642814 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e40003520 con 0x7f6e580d2910 2023-12-02T00:30:56.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.782+0000 7f6e57fff640 1 -- 172.21.15.46:0/3444642814 >> v1:172.21.15.162:6789/0 conn(0x7f6e580d3900 legacy=0x7f6e580b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:56.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.782+0000 7f6e57fff640 1 -- 172.21.15.46:0/3444642814 >> v1:172.21.15.46:6789/0 conn(0x7f6e580d2910 legacy=0x7f6e580d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:56.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.782+0000 7f6e56ffd640 1 -- 172.21.15.46:0/3444642814 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1669218935 0 0) 0x7f6e580d1590 con 0x7f6e580d3900 2023-12-02T00:30:56.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.782+0000 7f6e57fff640 1 -- 172.21.15.46:0/3444642814 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e580d1b50 con 0x7f6e5815d330 2023-12-02T00:30:56.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.783+0000 7f6e57fff640 1 --2- 172.21.15.46:0/3444642814 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e5815d330 0x7f6e580b8b80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6e480097a0 tx=0x7f6e4802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40a89a9b77db4ab9 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:56.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.783+0000 7f6e56ffd640 1 -- 172.21.15.46:0/3444642814 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e48031830 con 0x7f6e5815d330 2023-12-02T00:30:56.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.783+0000 7f6e56ffd640 1 -- 172.21.15.46:0/3444642814 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e48031990 con 0x7f6e5815d330 2023-12-02T00:30:56.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.783+0000 7f6e56ffd640 1 -- 172.21.15.46:0/3444642814 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e48031d20 con 0x7f6e5815d330 2023-12-02T00:30:56.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.784+0000 7f6e5ea43640 1 -- 172.21.15.46:0/3444642814 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e5815d330 msgr2=0x7f6e580b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:56.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.784+0000 7f6e5ea43640 1 --2- 172.21.15.46:0/3444642814 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e5815d330 0x7f6e580b8b80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6e480097a0 tx=0x7f6e4802b8d0 comp rx=0 tx=0).stop 2023-12-02T00:30:56.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.784+0000 7f6e5ea43640 1 -- 172.21.15.46:0/3444642814 shutdown_connections 2023-12-02T00:30:56.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.784+0000 7f6e5ea43640 1 --2- 172.21.15.46:0/3444642814 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e5815d330 0x7f6e580b8b80 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:56.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.784+0000 7f6e5ea43640 1 -- 172.21.15.46:0/3444642814 >> 172.21.15.46:0/3444642814 conn(0x7f6e58072020 msgr2=0x7f6e58072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:56.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.784+0000 7f6e5ea43640 1 -- 172.21.15.46:0/3444642814 shutdown_connections 2023-12-02T00:30:56.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.784+0000 7f6e5ea43640 1 -- 172.21.15.46:0/3444642814 wait complete. 2023-12-02T00:30:56.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.785+0000 7f6e5ea43640 1 Processor -- start 2023-12-02T00:30:56.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.785+0000 7f6e5ea43640 1 -- start start 2023-12-02T00:30:56.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.786+0000 7f6e5ea43640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e580d2910 0x7f6e581f2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:56.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.786+0000 7f6e5ea43640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e580d3900 0x7f6e58159af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:56.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.786+0000 7f6e5ea43640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e5815d330 0x7f6e580ca650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:56.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.786+0000 7f6e5ea43640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e581f3a20 con 0x7f6e580d3900 2023-12-02T00:30:56.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.786+0000 7f6e5ea43640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e581f3090 con 0x7f6e5815d330 2023-12-02T00:30:56.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.786+0000 7f6e5ea43640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e581f3370 con 0x7f6e580d2910 2023-12-02T00:30:56.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.786+0000 7f6e577fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e580d3900 0x7f6e58159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:56.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.786+0000 7f6e577fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e580d3900 0x7f6e58159af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37352/0 (socket says 172.21.15.46:37352) 2023-12-02T00:30:56.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.786+0000 7f6e577fe640 1 -- 172.21.15.46:0/3834641257 learned_addr learned my addr 172.21.15.46:0/3834641257 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:56.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.786+0000 7f6e57fff640 1 --2- 172.21.15.46:0/3834641257 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e580d2910 0x7f6e581f2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:56.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.786+0000 7f6e5cfe0640 1 --2- 172.21.15.46:0/3834641257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e5815d330 0x7f6e580ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:56.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.786+0000 7f6e577fe640 1 -- 172.21.15.46:0/3834641257 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e580d2910 msgr2=0x7f6e581f2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:56.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.786+0000 7f6e577fe640 1 --2- 172.21.15.46:0/3834641257 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e580d2910 0x7f6e581f2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:56.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.786+0000 7f6e577fe640 1 -- 172.21.15.46:0/3834641257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e5815d330 msgr2=0x7f6e580ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:56.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.786+0000 7f6e577fe640 1 --2- 172.21.15.46:0/3834641257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e5815d330 0x7f6e580ca650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:56.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.786+0000 7f6e577fe640 1 -- 172.21.15.46:0/3834641257 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e48009400 con 0x7f6e580d3900 2023-12-02T00:30:56.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.786+0000 7f6e57fff640 1 --2- 172.21.15.46:0/3834641257 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e580d2910 0x7f6e581f2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:30:56.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.787+0000 7f6e577fe640 1 --2- 172.21.15.46:0/3834641257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e580d3900 0x7f6e58159af0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c00c430 tx=0x7f6e4c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:56.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.787+0000 7f6e54ff9640 1 -- 172.21.15.46:0/3834641257 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c005ce0 con 0x7f6e580d3900 2023-12-02T00:30:56.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.787+0000 7f6e54ff9640 1 -- 172.21.15.46:0/3834641257 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e4c005e40 con 0x7f6e580d3900 2023-12-02T00:30:56.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.787+0000 7f6e5ea43640 1 -- 172.21.15.46:0/3834641257 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e581f3650 con 0x7f6e580d3900 2023-12-02T00:30:56.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.787+0000 7f6e54ff9640 1 -- 172.21.15.46:0/3834641257 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c0072f0 con 0x7f6e580d3900 2023-12-02T00:30:56.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.787+0000 7f6e5ea43640 1 -- 172.21.15.46:0/3834641257 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e581f2da0 con 0x7f6e580d3900 2023-12-02T00:30:56.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.788+0000 7f6e5ea43640 1 -- 172.21.15.46:0/3834641257 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e581f3870 con 0x7f6e580d3900 2023-12-02T00:30:56.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.788+0000 7f6e54ff9640 1 -- 172.21.15.46:0/3834641257 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6e4c007450 con 0x7f6e580d3900 2023-12-02T00:30:56.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.789+0000 7f6e54ff9640 1 --2- 172.21.15.46:0/3834641257 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f6e4406ae40 0x7f6e4406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:56.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.789+0000 7f6e54ff9640 1 -- 172.21.15.46:0/3834641257 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6e4c088920 con 0x7f6e580d3900 2023-12-02T00:30:56.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.789+0000 7f6e57fff640 1 --2- 172.21.15.46:0/3834641257 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f6e4406ae40 0x7f6e4406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:56.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.790+0000 7f6e57fff640 1 --2- 172.21.15.46:0/3834641257 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f6e4406ae40 0x7f6e4406d300 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6e48036000 tx=0x7f6e4803c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:56.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.793+0000 7f6e54ff9640 1 -- 172.21.15.46:0/3834641257 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6e4c053d60 con 0x7f6e580d3900 2023-12-02T00:30:56.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:56.989+0000 7f6e5ea43640 1 -- 172.21.15.46:0/3834641257 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f6e581f3e70 con 0x7f6e580d3900 2023-12-02T00:30:57.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.805+0000 7f6e54ff9640 1 -- 172.21.15.46:0/3834641257 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v47) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6e581f3e70 con 0x7f6e580d3900 2023-12-02T00:30:57.808 INFO:teuthology.orchestra.run.smithi046.stderr: removed all blocklist entries 2023-12-02T00:30:57.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.808+0000 7f6e5ea43640 1 -- 172.21.15.46:0/3834641257 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f6e4406ae40 msgr2=0x7f6e4406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:57.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.808+0000 7f6e5ea43640 1 --2- 172.21.15.46:0/3834641257 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f6e4406ae40 0x7f6e4406d300 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6e48036000 tx=0x7f6e4803c040 comp rx=0 tx=0).stop 2023-12-02T00:30:57.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.808+0000 7f6e5ea43640 1 -- 172.21.15.46:0/3834641257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e580d3900 msgr2=0x7f6e58159af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:57.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.808+0000 7f6e5ea43640 1 --2- 172.21.15.46:0/3834641257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e580d3900 0x7f6e58159af0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c00c430 tx=0x7f6e4c00c8f0 comp rx=0 tx=0).stop 2023-12-02T00:30:57.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.808+0000 7f6e5ea43640 1 -- 172.21.15.46:0/3834641257 shutdown_connections 2023-12-02T00:30:57.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.808+0000 7f6e5ea43640 1 --2- 172.21.15.46:0/3834641257 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f6e4406ae40 0x7f6e4406d300 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:57.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.808+0000 7f6e5ea43640 1 --2- 172.21.15.46:0/3834641257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e5815d330 0x7f6e580ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:57.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.808+0000 7f6e5ea43640 1 --2- 172.21.15.46:0/3834641257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e580d3900 0x7f6e58159af0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:57.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.808+0000 7f6e5ea43640 1 --2- 172.21.15.46:0/3834641257 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e580d2910 0x7f6e581f2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:57.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.808+0000 7f6e5ea43640 1 -- 172.21.15.46:0/3834641257 >> 172.21.15.46:0/3834641257 conn(0x7f6e58072020 msgr2=0x7f6e580d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:57.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.808+0000 7f6e5ea43640 1 -- 172.21.15.46:0/3834641257 shutdown_connections 2023-12-02T00:30:57.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.808+0000 7f6e5ea43640 1 -- 172.21.15.46:0/3834641257 wait complete. 2023-12-02T00:30:57.823 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-02T00:30:57.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.929+0000 7f8c355ce640 1 Processor -- start 2023-12-02T00:30:57.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.929+0000 7f8c355ce640 1 -- start start 2023-12-02T00:30:57.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.930+0000 7f8c355ce640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8c30144980 0x7f8c3015c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:57.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.930+0000 7f8c355ce640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8c300b4370 0x7f8c300b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:57.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.930+0000 7f8c355ce640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c30071770 con 0x7f8c300ceb70 2023-12-02T00:30:57.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.930+0000 7f8c355ce640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c300ccb60 con 0x7f8c30144980 2023-12-02T00:30:57.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.930+0000 7f8c355ce640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c300b4ce0 con 0x7f8c300b4370 2023-12-02T00:30:57.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.930+0000 7f8c2effd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8c30144980 0x7f8c3015c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:57.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.930+0000 7f8c2e7fc640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8c300ceb70 0x7f8c300cc450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53768/0 (socket says 172.21.15.46:53768) 2023-12-02T00:30:57.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.930+0000 7f8c2e7fc640 1 -- 172.21.15.46:0/656497270 learned_addr learned my addr 172.21.15.46:0/656497270 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:30:57.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.930+0000 7f8c2effd640 1 -- 172.21.15.46:0/656497270 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8c300b4370 msgr2=0x7f8c300b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:57.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.930+0000 7f8c2f7fe640 1 --2- 172.21.15.46:0/656497270 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8c300b4370 0x7f8c300b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:57.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.930+0000 7f8c2effd640 1 --2- 172.21.15.46:0/656497270 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8c300b4370 0x7f8c300b47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:57.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.930+0000 7f8c2dffb640 1 -- 172.21.15.46:0/656497270 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1566833106 0 0) 0x7f8c30071770 con 0x7f8c300ceb70 2023-12-02T00:30:57.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.930+0000 7f8c2effd640 1 -- 172.21.15.46:0/656497270 >> v1:172.21.15.46:6789/0 conn(0x7f8c300ceb70 legacy=0x7f8c300cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:57.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.930+0000 7f8c2effd640 1 -- 172.21.15.46:0/656497270 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c300ccdc0 con 0x7f8c30144980 2023-12-02T00:30:57.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.930+0000 7f8c2f7fe640 1 --2- 172.21.15.46:0/656497270 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8c300b4370 0x7f8c300b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:30:57.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.931+0000 7f8c2effd640 1 --2- 172.21.15.46:0/656497270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c30144980 0x7f8c3015c7c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f8c280097a0 tx=0x7f8c2802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e804f50f9adc15a server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:57.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.931+0000 7f8c2dffb640 1 -- 172.21.15.46:0/656497270 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c28031830 con 0x7f8c30144980 2023-12-02T00:30:57.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.931+0000 7f8c2dffb640 1 -- 172.21.15.46:0/656497270 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c28031990 con 0x7f8c30144980 2023-12-02T00:30:57.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.931+0000 7f8c2dffb640 1 -- 172.21.15.46:0/656497270 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c28031d20 con 0x7f8c30144980 2023-12-02T00:30:57.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.932+0000 7f8c355ce640 1 -- 172.21.15.46:0/656497270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c30144980 msgr2=0x7f8c3015c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:57.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.932+0000 7f8c355ce640 1 --2- 172.21.15.46:0/656497270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c30144980 0x7f8c3015c7c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f8c280097a0 tx=0x7f8c2802b8d0 comp rx=0 tx=0).stop 2023-12-02T00:30:57.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.932+0000 7f8c355ce640 1 -- 172.21.15.46:0/656497270 shutdown_connections 2023-12-02T00:30:57.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.932+0000 7f8c355ce640 1 --2- 172.21.15.46:0/656497270 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8c300b4370 0x7f8c300b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:57.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.932+0000 7f8c355ce640 1 --2- 172.21.15.46:0/656497270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c30144980 0x7f8c3015c7c0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:57.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.932+0000 7f8c355ce640 1 -- 172.21.15.46:0/656497270 >> 172.21.15.46:0/656497270 conn(0x7f8c3006a4d0 msgr2=0x7f8c3006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:57.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.932+0000 7f8c355ce640 1 -- 172.21.15.46:0/656497270 shutdown_connections 2023-12-02T00:30:57.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.932+0000 7f8c355ce640 1 -- 172.21.15.46:0/656497270 wait complete. 2023-12-02T00:30:57.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.933+0000 7f8c355ce640 1 Processor -- start 2023-12-02T00:30:57.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.933+0000 7f8c355ce640 1 -- start start 2023-12-02T00:30:57.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.934+0000 7f8c355ce640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c300b4370 0x7f8c300adee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:57.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.934+0000 7f8c355ce640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c300ceb70 0x7f8c300a8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:57.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.934+0000 7f8c355ce640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8c30144980 0x7f8c301eff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:57.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.934+0000 7f8c355ce640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c300b0820 con 0x7f8c300b4370 2023-12-02T00:30:57.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.934+0000 7f8c355ce640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c300afef0 con 0x7f8c300ceb70 2023-12-02T00:30:57.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.934+0000 7f8c355ce640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c300b01d0 con 0x7f8c30144980 2023-12-02T00:30:57.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.934+0000 7f8c2effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c300b4370 0x7f8c300adee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.934+0000 7f8c2effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c300b4370 0x7f8c300adee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37364/0 (socket says 172.21.15.46:37364) 2023-12-02T00:30:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.934+0000 7f8c2effd640 1 -- 172.21.15.46:0/1518913206 learned_addr learned my addr 172.21.15.46:0/1518913206 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.934+0000 7f8c2effd640 1 -- 172.21.15.46:0/1518913206 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8c30144980 msgr2=0x7f8c301eff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.934+0000 7f8c2effd640 1 --2- 172.21.15.46:0/1518913206 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8c30144980 0x7f8c301eff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.934+0000 7f8c2effd640 1 -- 172.21.15.46:0/1518913206 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c300ceb70 msgr2=0x7f8c300a8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.934+0000 7f8c2effd640 1 --2- 172.21.15.46:0/1518913206 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c300ceb70 0x7f8c300a8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.934+0000 7f8c2effd640 1 -- 172.21.15.46:0/1518913206 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c28009400 con 0x7f8c300b4370 2023-12-02T00:30:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.935+0000 7f8c2effd640 1 --2- 172.21.15.46:0/1518913206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c300b4370 0x7f8c300adee0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f8c28000c00 tx=0x7f8c280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.935+0000 7f8c277fe640 1 -- 172.21.15.46:0/1518913206 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c28004230 con 0x7f8c300b4370 2023-12-02T00:30:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.935+0000 7f8c277fe640 1 -- 172.21.15.46:0/1518913206 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c28004390 con 0x7f8c300b4370 2023-12-02T00:30:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.935+0000 7f8c355ce640 1 -- 172.21.15.46:0/1518913206 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c300b04b0 con 0x7f8c300b4370 2023-12-02T00:30:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.935+0000 7f8c277fe640 1 -- 172.21.15.46:0/1518913206 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c2803a920 con 0x7f8c300b4370 2023-12-02T00:30:57.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.935+0000 7f8c355ce640 1 -- 172.21.15.46:0/1518913206 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c300ae420 con 0x7f8c300b4370 2023-12-02T00:30:57.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.936+0000 7f8c355ce640 1 -- 172.21.15.46:0/1518913206 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c30069400 con 0x7f8c300b4370 2023-12-02T00:30:57.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.936+0000 7f8c277fe640 1 -- 172.21.15.46:0/1518913206 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8c2803aa80 con 0x7f8c300b4370 2023-12-02T00:30:57.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.937+0000 7f8c277fe640 1 --2- 172.21.15.46:0/1518913206 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8c1006ae40 0x7f8c1006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:57.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.937+0000 7f8c277fe640 1 -- 172.21.15.46:0/1518913206 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8c280ab970 con 0x7f8c300b4370 2023-12-02T00:30:57.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.937+0000 7f8c2e7fc640 1 --2- 172.21.15.46:0/1518913206 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8c1006ae40 0x7f8c1006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:57.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.938+0000 7f8c2e7fc640 1 --2- 172.21.15.46:0/1518913206 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8c1006ae40 0x7f8c1006d300 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8c18007fd0 tx=0x7f8c18007960 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:57.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:57.941+0000 7f8c277fe640 1 -- 172.21.15.46:0/1518913206 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8c28076e00 con 0x7f8c300b4370 2023-12-02T00:30:58.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.182+0000 7f8c355ce640 1 -- 172.21.15.46:0/1518913206 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f8c300cc5d0 con 0x7f8c300b4370 2023-12-02T00:30:58.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.183+0000 7f8c277fe640 1 -- 172.21.15.46:0/1518913206 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5298 (secure 0 0 0) 0x7f8c28035020 con 0x7f8c300b4370 2023-12-02T00:30:58.185 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:30:58.185 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-12-02T00:30:58.185 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_dump": [ 2023-12-02T00:30:58.185 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-02T00:30:58.185 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.a", 2023-12-02T00:30:58.185 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAIemplYQrZMBAAqrkez6r//VUITan4SKdf4w==", 2023-12-02T00:30:58.185 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-02T00:30:58.185 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-12-02T00:30:58.185 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-12-02T00:30:58.185 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-12-02T00:30:58.185 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-12-02T00:30:58.185 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:30:58.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:30:58.186 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-02T00:30:58.186 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.b", 2023-12-02T00:30:58.186 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAJemplKoSRAhAAuQHpGMNCUSFEtCtWMvY8yA==", 2023-12-02T00:30:58.186 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-02T00:30:58.186 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-12-02T00:30:58.186 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-12-02T00:30:58.186 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-12-02T00:30:58.186 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-12-02T00:30:58.186 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:30:58.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:30:58.186 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-02T00:30:58.186 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.c", 2023-12-02T00:30:58.186 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAIempl69PfOBAAQOgkrli+yM4wpJabPb3QxQ==", 2023-12-02T00:30:58.186 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-02T00:30:58.187 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-12-02T00:30:58.187 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-12-02T00:30:58.187 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-12-02T00:30:58.187 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-12-02T00:30:58.187 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:30:58.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:30:58.187 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-02T00:30:58.187 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.d", 2023-12-02T00:30:58.187 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAJemplsTHqCRAAi7WJBcwYYDl0/qZkMy7GsQ==", 2023-12-02T00:30:58.187 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-02T00:30:58.187 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-12-02T00:30:58.187 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-12-02T00:30:58.187 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-12-02T00:30:58.187 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-12-02T00:30:58.187 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:30:58.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:30:58.188 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-02T00:30:58.188 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.0", 2023-12-02T00:30:58.188 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAKempluFfMMhAAmbD1GpzugHbxobJBl3RtkQ==", 2023-12-02T00:30:58.188 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-02T00:30:58.188 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-12-02T00:30:58.188 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-12-02T00:30:58.188 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-12-02T00:30:58.188 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:30:58.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:30:58.188 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-02T00:30:58.188 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.1", 2023-12-02T00:30:58.188 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQANempleNzmMRAALK5ExII/uG0A8pXBsc86pg==", 2023-12-02T00:30:58.188 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-02T00:30:58.188 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-12-02T00:30:58.188 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-12-02T00:30:58.188 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-12-02T00:30:58.189 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:30:58.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:30:58.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-02T00:30:58.189 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.2", 2023-12-02T00:30:58.189 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAQemplKDzyMBAAfPwOg3WNbFSBUZUqpG1l7w==", 2023-12-02T00:30:58.189 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-02T00:30:58.189 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-12-02T00:30:58.189 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-12-02T00:30:58.189 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-12-02T00:30:58.189 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:30:58.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:30:58.189 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-02T00:30:58.189 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.3", 2023-12-02T00:30:58.190 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQATemplIbgAMBAAbynDehkgSKMaPHXYgwK2cA==", 2023-12-02T00:30:58.190 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-02T00:30:58.190 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-12-02T00:30:58.190 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-12-02T00:30:58.190 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-12-02T00:30:58.190 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:30:58.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:30:58.190 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-02T00:30:58.190 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.4", 2023-12-02T00:30:58.190 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAaempl/vxEABAAThzqmuy54tDx5dYyTilSzg==", 2023-12-02T00:30:58.190 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-02T00:30:58.191 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-12-02T00:30:58.191 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-12-02T00:30:58.191 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-12-02T00:30:58.191 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:30:58.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:30:58.192 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-02T00:30:58.192 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.5", 2023-12-02T00:30:58.192 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAcemplWd7NOhAAO0eGoj+K+P0F4QvV4SMUxg==", 2023-12-02T00:30:58.193 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-02T00:30:58.193 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-12-02T00:30:58.193 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-12-02T00:30:58.193 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-12-02T00:30:58.193 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:30:58.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:30:58.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-02T00:30:58.193 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.6", 2023-12-02T00:30:58.193 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAfemplz38cORAAmb09RTXtiWf2J4q0jbYEug==", 2023-12-02T00:30:58.193 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-02T00:30:58.193 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-12-02T00:30:58.194 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-12-02T00:30:58.194 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-12-02T00:30:58.194 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:30:58.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:30:58.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-02T00:30:58.194 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.7", 2023-12-02T00:30:58.194 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAiempl+Md9NxAA5bPhqhkvQPRZ59k3OE9R9w==", 2023-12-02T00:30:58.194 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-02T00:30:58.194 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-12-02T00:30:58.194 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-12-02T00:30:58.194 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-12-02T00:30:58.194 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:30:58.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:30:58.194 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-02T00:30:58.195 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.0", 2023-12-02T00:30:58.195 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAJempl5Lx1DhAAjpK6ajoQ5jQiUr/EMpFeLg==", 2023-12-02T00:30:58.195 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-02T00:30:58.195 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-12-02T00:30:58.195 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow r", 2023-12-02T00:30:58.195 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rw pool=cephfs_data" 2023-12-02T00:30:58.195 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:30:58.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:30:58.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-02T00:30:58.195 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.1", 2023-12-02T00:30:58.195 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAJempl0gj4FRAAHZtOS7DxQQOSmYsu54uPZg==", 2023-12-02T00:30:58.195 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-02T00:30:58.195 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-12-02T00:30:58.195 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2023-12-02T00:30:58.195 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2023-12-02T00:30:58.195 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2023-12-02T00:30:58.196 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:30:58.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:30:58.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-02T00:30:58.196 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.2", 2023-12-02T00:30:58.196 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAJempl6t9OEhAACGdJ4lIgFImVMyS+vbiyrw==", 2023-12-02T00:30:58.196 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-02T00:30:58.196 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-12-02T00:30:58.196 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2023-12-02T00:30:58.196 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2023-12-02T00:30:58.196 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2023-12-02T00:30:58.196 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:30:58.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:30:58.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-02T00:30:58.196 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.3", 2023-12-02T00:30:58.196 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAJemplapXEGxAAz74ppCDNM8zW2+ozYOiifQ==", 2023-12-02T00:30:58.196 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-02T00:30:58.197 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-12-02T00:30:58.197 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2023-12-02T00:30:58.197 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2023-12-02T00:30:58.197 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2023-12-02T00:30:58.197 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:30:58.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:30:58.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-02T00:30:58.197 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.admin", 2023-12-02T00:30:58.197 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAIemplkFiEEhAAmwW5ztv7dNWpU/S9WKtyww==", 2023-12-02T00:30:58.197 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-02T00:30:58.197 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2023-12-02T00:30:58.197 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-12-02T00:30:58.197 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-12-02T00:30:58.197 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-12-02T00:30:58.197 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:30:58.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:30:58.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-02T00:30:58.198 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.x", 2023-12-02T00:30:58.198 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAIemplbBH1LBAAxs6g6VawCdt6IxZJn2yrQA==", 2023-12-02T00:30:58.198 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-02T00:30:58.198 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2023-12-02T00:30:58.198 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2023-12-02T00:30:58.198 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-12-02T00:30:58.198 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:30:58.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:30:58.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-02T00:30:58.198 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.y", 2023-12-02T00:30:58.198 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAIemplE8PaKBAAwleK4eGvEOffn5EzY+drMQ==", 2023-12-02T00:30:58.198 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-02T00:30:58.198 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2023-12-02T00:30:58.199 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2023-12-02T00:30:58.199 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-12-02T00:30:58.199 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:30:58.199 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:30:58.199 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2023-12-02T00:30:58.199 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-12-02T00:30:58.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.184+0000 7f8c355ce640 1 -- 172.21.15.46:0/1518913206 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8c1006ae40 msgr2=0x7f8c1006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:58.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.184+0000 7f8c355ce640 1 --2- 172.21.15.46:0/1518913206 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8c1006ae40 0x7f8c1006d300 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8c18007fd0 tx=0x7f8c18007960 comp rx=0 tx=0).stop 2023-12-02T00:30:58.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.184+0000 7f8c355ce640 1 -- 172.21.15.46:0/1518913206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c300b4370 msgr2=0x7f8c300adee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:58.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.184+0000 7f8c355ce640 1 --2- 172.21.15.46:0/1518913206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c300b4370 0x7f8c300adee0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f8c28000c00 tx=0x7f8c280046c0 comp rx=0 tx=0).stop 2023-12-02T00:30:58.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.184+0000 7f8c355ce640 1 -- 172.21.15.46:0/1518913206 shutdown_connections 2023-12-02T00:30:58.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.184+0000 7f8c355ce640 1 --2- 172.21.15.46:0/1518913206 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8c1006ae40 0x7f8c1006d300 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:58.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.184+0000 7f8c355ce640 1 --2- 172.21.15.46:0/1518913206 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8c30144980 0x7f8c301eff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:58.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.184+0000 7f8c355ce640 1 --2- 172.21.15.46:0/1518913206 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c300ceb70 0x7f8c300a8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:58.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.184+0000 7f8c355ce640 1 --2- 172.21.15.46:0/1518913206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c300b4370 0x7f8c300adee0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:58.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.184+0000 7f8c355ce640 1 -- 172.21.15.46:0/1518913206 >> 172.21.15.46:0/1518913206 conn(0x7f8c3006a4d0 msgr2=0x7f8c300cba40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:58.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.185+0000 7f8c355ce640 1 -- 172.21.15.46:0/1518913206 shutdown_connections 2023-12-02T00:30:58.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.185+0000 7f8c355ce640 1 -- 172.21.15.46:0/1518913206 wait complete. 2023-12-02T00:30:58.202 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:30:58.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.307+0000 7f7b9fc17640 1 Processor -- start 2023-12-02T00:30:58.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.307+0000 7f7b9fc17640 1 -- start start 2023-12-02T00:30:58.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.308+0000 7f7b9fc17640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b98160fb0 0x7f7b98161440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:58.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.308+0000 7f7b9fc17640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7b980d0ed0 0x7f7b980d1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:58.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.308+0000 7f7b9fc17640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b98161d20 con 0x7f7b9806ea50 2023-12-02T00:30:58.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.308+0000 7f7b9fc17640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b9815e760 con 0x7f7b980d0ed0 2023-12-02T00:30:58.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.308+0000 7f7b9fc17640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9815d9a0 con 0x7f7b98160fb0 2023-12-02T00:30:58.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.308+0000 7f7b9e1b4640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7b980d0ed0 0x7f7b980d1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:58.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.308+0000 7f7b9e1b4640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7b980d0ed0 0x7f7b980d1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37370/0 (socket says 172.21.15.46:37370) 2023-12-02T00:30:58.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.308+0000 7f7b9e1b4640 1 -- 172.21.15.46:0/1728969844 learned_addr learned my addr 172.21.15.46:0/1728969844 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:58.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.308+0000 7f7b9d1b2640 1 --2- 172.21.15.46:0/1728969844 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b98160fb0 0x7f7b98161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:58.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.308+0000 7f7b9e1b4640 1 -- 172.21.15.46:0/1728969844 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b98160fb0 msgr2=0x7f7b98161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:58.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.308+0000 7f7b9e1b4640 1 --2- 172.21.15.46:0/1728969844 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b98160fb0 0x7f7b98161440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:58.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.308+0000 7f7b9e1b4640 1 -- 172.21.15.46:0/1728969844 >> v1:172.21.15.162:6789/0 conn(0x7f7b9806ea50 legacy=0x7f7b9806ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:58.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.308+0000 7f7b9e1b4640 1 -- 172.21.15.46:0/1728969844 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b9815dc80 con 0x7f7b980d0ed0 2023-12-02T00:30:58.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.308+0000 7f7b9d1b2640 1 --2- 172.21.15.46:0/1728969844 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b98160fb0 0x7f7b98161440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:30:58.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.309+0000 7f7b9e1b4640 1 --2- 172.21.15.46:0/1728969844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b980d0ed0 0x7f7b980d1360 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f7b94009870 tx=0x7f7b9402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f81f3cb7b52f37a server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:58.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.309+0000 7f7b9c9b1640 1 -- 172.21.15.46:0/1728969844 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b94031640 con 0x7f7b980d0ed0 2023-12-02T00:30:58.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.309+0000 7f7b9c9b1640 1 -- 172.21.15.46:0/1728969844 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b940317a0 con 0x7f7b980d0ed0 2023-12-02T00:30:58.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.309+0000 7f7b9c9b1640 1 -- 172.21.15.46:0/1728969844 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b94031b30 con 0x7f7b980d0ed0 2023-12-02T00:30:58.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.310+0000 7f7b9fc17640 1 -- 172.21.15.46:0/1728969844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b980d0ed0 msgr2=0x7f7b980d1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:58.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.310+0000 7f7b9fc17640 1 --2- 172.21.15.46:0/1728969844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b980d0ed0 0x7f7b980d1360 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f7b94009870 tx=0x7f7b9402b8f0 comp rx=0 tx=0).stop 2023-12-02T00:30:58.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.310+0000 7f7b9fc17640 1 -- 172.21.15.46:0/1728969844 shutdown_connections 2023-12-02T00:30:58.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.310+0000 7f7b9fc17640 1 --2- 172.21.15.46:0/1728969844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b980d0ed0 0x7f7b980d1360 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:58.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.310+0000 7f7b9fc17640 1 --2- 172.21.15.46:0/1728969844 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b98160fb0 0x7f7b98161440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:58.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.310+0000 7f7b9fc17640 1 -- 172.21.15.46:0/1728969844 >> 172.21.15.46:0/1728969844 conn(0x7f7b9806a6b0 msgr2=0x7f7b9806aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:58.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.310+0000 7f7b9fc17640 1 -- 172.21.15.46:0/1728969844 shutdown_connections 2023-12-02T00:30:58.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.310+0000 7f7b9fc17640 1 -- 172.21.15.46:0/1728969844 wait complete. 2023-12-02T00:30:58.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.311+0000 7f7b9fc17640 1 Processor -- start 2023-12-02T00:30:58.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.311+0000 7f7b9fc17640 1 -- start start 2023-12-02T00:30:58.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.312+0000 7f7b9fc17640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b9806ea50 0x7f7b9813fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:58.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.312+0000 7f7b9fc17640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b980d0ed0 0x7f7b981f6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:58.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.312+0000 7f7b9fc17640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b98160fb0 0x7f7b981f2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:58.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.312+0000 7f7b9fc17640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b981400a0 con 0x7f7b98160fb0 2023-12-02T00:30:58.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.312+0000 7f7b9fc17640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9815a7e0 con 0x7f7b980d0ed0 2023-12-02T00:30:58.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.312+0000 7f7b9fc17640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9815ac70 con 0x7f7b9806ea50 2023-12-02T00:30:58.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.312+0000 7f7b9e1b4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b98160fb0 0x7f7b981f2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:58.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.312+0000 7f7b9e1b4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b98160fb0 0x7f7b981f2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37372/0 (socket says 172.21.15.46:37372) 2023-12-02T00:30:58.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.312+0000 7f7b9e1b4640 1 -- 172.21.15.46:0/3810281041 learned_addr learned my addr 172.21.15.46:0/3810281041 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:58.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.312+0000 7f7b9d9b3640 1 --2- 172.21.15.46:0/3810281041 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b9806ea50 0x7f7b9813fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:58.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.312+0000 7f7b9d1b2640 1 --2- 172.21.15.46:0/3810281041 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b980d0ed0 0x7f7b981f6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:58.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.312+0000 7f7b9e1b4640 1 -- 172.21.15.46:0/3810281041 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b9806ea50 msgr2=0x7f7b9813fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:58.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.312+0000 7f7b9e1b4640 1 --2- 172.21.15.46:0/3810281041 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b9806ea50 0x7f7b9813fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:58.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.312+0000 7f7b9e1b4640 1 -- 172.21.15.46:0/3810281041 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b980d0ed0 msgr2=0x7f7b981f6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:58.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.312+0000 7f7b9e1b4640 1 --2- 172.21.15.46:0/3810281041 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b980d0ed0 0x7f7b981f6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:58.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.312+0000 7f7b9e1b4640 1 -- 172.21.15.46:0/3810281041 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b940094d0 con 0x7f7b98160fb0 2023-12-02T00:30:58.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.312+0000 7f7b9e1b4640 1 --2- 172.21.15.46:0/3810281041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b98160fb0 0x7f7b981f2690 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f7b9402b930 tx=0x7f7b940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:58.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.313+0000 7f7b8e7fc640 1 -- 172.21.15.46:0/3810281041 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9403ab10 con 0x7f7b98160fb0 2023-12-02T00:30:58.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.313+0000 7f7b8e7fc640 1 -- 172.21.15.46:0/3810281041 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b9403ac90 con 0x7f7b98160fb0 2023-12-02T00:30:58.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.313+0000 7f7b8e7fc640 1 -- 172.21.15.46:0/3810281041 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b940443f0 con 0x7f7b98160fb0 2023-12-02T00:30:58.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.313+0000 7f7b9fc17640 1 -- 172.21.15.46:0/3810281041 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b981f7250 con 0x7f7b98160fb0 2023-12-02T00:30:58.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.313+0000 7f7b9fc17640 1 -- 172.21.15.46:0/3810281041 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b98159eb0 con 0x7f7b98160fb0 2023-12-02T00:30:58.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.314+0000 7f7b9fc17640 1 -- 172.21.15.46:0/3810281041 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b9815a950 con 0x7f7b98160fb0 2023-12-02T00:30:58.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.314+0000 7f7b8e7fc640 1 -- 172.21.15.46:0/3810281041 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7b9402f050 con 0x7f7b98160fb0 2023-12-02T00:30:58.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.315+0000 7f7b8e7fc640 1 --2- 172.21.15.46:0/3810281041 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7b7c06ae40 0x7f7b7c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:58.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.315+0000 7f7b8e7fc640 1 -- 172.21.15.46:0/3810281041 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7b940ab310 con 0x7f7b98160fb0 2023-12-02T00:30:58.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.315+0000 7f7b9d9b3640 1 --2- 172.21.15.46:0/3810281041 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7b7c06ae40 0x7f7b7c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:58.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.316+0000 7f7b9d9b3640 1 --2- 172.21.15.46:0/3810281041 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7b7c06ae40 0x7f7b7c06d300 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7b90009f20 tx=0x7f7b900094d0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:58.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.319+0000 7f7b8e7fc640 1 -- 172.21.15.46:0/3810281041 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7b94076670 con 0x7f7b98160fb0 2023-12-02T00:30:58.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.495+0000 7f7b9fc17640 1 -- 172.21.15.46:0/3810281041 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7b98069400 con 0x7f7b98160fb0 2023-12-02T00:30:58.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.496+0000 7f7b8e7fc640 1 -- 172.21.15.46:0/3810281041 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+12833 (secure 0 0 0) 0x7f7b94079c90 con 0x7f7b98160fb0 2023-12-02T00:30:58.498 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:30:58.498 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":47,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:30:57.802868+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:30:58.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.498+0000 7f7b9fc17640 1 -- 172.21.15.46:0/3810281041 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7b7c06ae40 msgr2=0x7f7b7c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:58.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.498+0000 7f7b9fc17640 1 --2- 172.21.15.46:0/3810281041 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7b7c06ae40 0x7f7b7c06d300 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7b90009f20 tx=0x7f7b900094d0 comp rx=0 tx=0).stop 2023-12-02T00:30:58.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.498+0000 7f7b9fc17640 1 -- 172.21.15.46:0/3810281041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b98160fb0 msgr2=0x7f7b981f2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:58.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.498+0000 7f7b9fc17640 1 --2- 172.21.15.46:0/3810281041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b98160fb0 0x7f7b981f2690 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f7b9402b930 tx=0x7f7b940046c0 comp rx=0 tx=0).stop 2023-12-02T00:30:58.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.498+0000 7f7b9fc17640 1 -- 172.21.15.46:0/3810281041 shutdown_connections 2023-12-02T00:30:58.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.498+0000 7f7b9fc17640 1 --2- 172.21.15.46:0/3810281041 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f7b7c06ae40 0x7f7b7c06d300 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:58.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.498+0000 7f7b9fc17640 1 --2- 172.21.15.46:0/3810281041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b98160fb0 0x7f7b981f2690 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:58.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.498+0000 7f7b9fc17640 1 --2- 172.21.15.46:0/3810281041 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b980d0ed0 0x7f7b981f6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:58.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.498+0000 7f7b9fc17640 1 --2- 172.21.15.46:0/3810281041 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b9806ea50 0x7f7b9813fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:58.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.498+0000 7f7b9fc17640 1 -- 172.21.15.46:0/3810281041 >> 172.21.15.46:0/3810281041 conn(0x7f7b9806a6b0 msgr2=0x7f7b9806ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:58.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.498+0000 7f7b9fc17640 1 -- 172.21.15.46:0/3810281041 shutdown_connections 2023-12-02T00:30:58.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.498+0000 7f7b9fc17640 1 -- 172.21.15.46:0/3810281041 wait complete. 2023-12-02T00:30:58.511 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-02T00:28:37.213645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-02T00:30:58.512 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-02T00:30:58.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.619+0000 7fecf4210640 1 Processor -- start 2023-12-02T00:30:58.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.619+0000 7fecf4210640 1 -- start start 2023-12-02T00:30:58.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.620+0000 7fecf4210640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fecec157b00 0x7fecec0714c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:58.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.620+0000 7fecf4210640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fecec071a00 0x7fecec15a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:58.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.620+0000 7fecf4210640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecec071e60 con 0x7fecec15ad60 2023-12-02T00:30:58.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.620+0000 7fecf4210640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecec06f510 con 0x7fecec157b00 2023-12-02T00:30:58.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.620+0000 7fecf4210640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecec06e960 con 0x7fecec071a00 2023-12-02T00:30:58.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.620+0000 7fecf1fac640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fecec157b00 0x7fecec0714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:58.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.620+0000 7fecf27ad640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fecec15ad60 0x7fecec15da90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53770/0 (socket says 172.21.15.46:53770) 2023-12-02T00:30:58.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.620+0000 7fecf27ad640 1 -- 172.21.15.46:0/1716153971 learned_addr learned my addr 172.21.15.46:0/1716153971 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:30:58.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.621+0000 7fecf1fac640 1 -- 172.21.15.46:0/1716153971 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fecec071a00 msgr2=0x7fecec15a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:58.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.621+0000 7fecf17ab640 1 --2- 172.21.15.46:0/1716153971 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fecec071a00 0x7fecec15a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:58.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.621+0000 7fecf0faa640 1 -- 172.21.15.46:0/1716153971 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 956530410 0 0) 0x7fecec071e60 con 0x7fecec15ad60 2023-12-02T00:30:58.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.621+0000 7fecf1fac640 1 --2- 172.21.15.46:0/1716153971 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fecec071a00 0x7fecec15a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:58.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.621+0000 7fecf1fac640 1 -- 172.21.15.46:0/1716153971 >> v1:172.21.15.46:6789/0 conn(0x7fecec15ad60 legacy=0x7fecec15da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:58.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.621+0000 7fecf1fac640 1 -- 172.21.15.46:0/1716153971 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecec157ee0 con 0x7fecec157b00 2023-12-02T00:30:58.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.622+0000 7fecf1fac640 1 --2- 172.21.15.46:0/1716153971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecec157b00 0x7fecec0714c0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fecdc009d90 tx=0x7fecdc02be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2350c194889a1ec server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:58.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.622+0000 7fecf0faa640 1 -- 172.21.15.46:0/1716153971 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecdc02cbc0 con 0x7fecec157b00 2023-12-02T00:30:58.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.622+0000 7fecf0faa640 1 -- 172.21.15.46:0/1716153971 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecdc02cd20 con 0x7fecec157b00 2023-12-02T00:30:58.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.622+0000 7fecf0faa640 1 -- 172.21.15.46:0/1716153971 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecdc034660 con 0x7fecec157b00 2023-12-02T00:30:58.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.622+0000 7fecf4210640 1 -- 172.21.15.46:0/1716153971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecec157b00 msgr2=0x7fecec0714c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:58.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.622+0000 7fecf4210640 1 --2- 172.21.15.46:0/1716153971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecec157b00 0x7fecec0714c0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fecdc009d90 tx=0x7fecdc02be10 comp rx=0 tx=0).stop 2023-12-02T00:30:58.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.622+0000 7fecf4210640 1 -- 172.21.15.46:0/1716153971 shutdown_connections 2023-12-02T00:30:58.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.622+0000 7fecf4210640 1 --2- 172.21.15.46:0/1716153971 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fecec071a00 0x7fecec15a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:58.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.622+0000 7fecf4210640 1 --2- 172.21.15.46:0/1716153971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecec157b00 0x7fecec0714c0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:58.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.622+0000 7fecf4210640 1 -- 172.21.15.46:0/1716153971 >> 172.21.15.46:0/1716153971 conn(0x7fecec0b1de0 msgr2=0x7fecec0b21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:58.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.622+0000 7fecf4210640 1 -- 172.21.15.46:0/1716153971 shutdown_connections 2023-12-02T00:30:58.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.622+0000 7fecf4210640 1 -- 172.21.15.46:0/1716153971 wait complete. 2023-12-02T00:30:58.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.623+0000 7fecf4210640 1 Processor -- start 2023-12-02T00:30:58.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.624+0000 7fecf4210640 1 -- start start 2023-12-02T00:30:58.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.624+0000 7fecf4210640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fecec071a00 0x7fecec143ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:58.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.624+0000 7fecf4210640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecec157b00 0x7fecec159770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:58.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.624+0000 7fecf4210640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fecec15ad60 0x7fecec144250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:58.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.624+0000 7fecf4210640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecec144000 con 0x7fecec157b00 2023-12-02T00:30:58.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.624+0000 7fecf4210640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecec147d70 con 0x7fecec071a00 2023-12-02T00:30:58.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.624+0000 7fecf4210640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecec148050 con 0x7fecec15ad60 2023-12-02T00:30:58.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.624+0000 7fecf17ab640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecec157b00 0x7fecec159770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:58.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.624+0000 7fecf17ab640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecec157b00 0x7fecec159770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37388/0 (socket says 172.21.15.46:37388) 2023-12-02T00:30:58.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.624+0000 7fecf17ab640 1 -- 172.21.15.46:0/1321274102 learned_addr learned my addr 172.21.15.46:0/1321274102 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:58.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.625+0000 7fecf27ad640 1 --2- 172.21.15.46:0/1321274102 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fecec15ad60 0x7fecec144250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:58.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.625+0000 7fecf1fac640 1 --2- 172.21.15.46:0/1321274102 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fecec071a00 0x7fecec143ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:58.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.625+0000 7fecf17ab640 1 -- 172.21.15.46:0/1321274102 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fecec15ad60 msgr2=0x7fecec144250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:58.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.625+0000 7fecf17ab640 1 --2- 172.21.15.46:0/1321274102 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fecec15ad60 0x7fecec144250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:58.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.625+0000 7fecf17ab640 1 -- 172.21.15.46:0/1321274102 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fecec071a00 msgr2=0x7fecec143ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:58.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.625+0000 7fecf17ab640 1 --2- 172.21.15.46:0/1321274102 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fecec071a00 0x7fecec143ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:58.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.625+0000 7fecf17ab640 1 -- 172.21.15.46:0/1321274102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecdc009a70 con 0x7fecec157b00 2023-12-02T00:30:58.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.625+0000 7fecf27ad640 1 --2- 172.21.15.46:0/1321274102 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fecec15ad60 0x7fecec144250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:30:58.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.625+0000 7fecf1fac640 1 --2- 172.21.15.46:0/1321274102 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fecec071a00 0x7fecec143ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:30:58.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.625+0000 7fecf17ab640 1 --2- 172.21.15.46:0/1321274102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecec157b00 0x7fecec159770 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fece000c7f0 tx=0x7fece000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:58.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.625+0000 7fecea7fc640 1 -- 172.21.15.46:0/1321274102 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece0004220 con 0x7fecec157b00 2023-12-02T00:30:58.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.625+0000 7fecea7fc640 1 -- 172.21.15.46:0/1321274102 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fece00043a0 con 0x7fecec157b00 2023-12-02T00:30:58.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.625+0000 7fecf4210640 1 -- 172.21.15.46:0/1321274102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecec148330 con 0x7fecec157b00 2023-12-02T00:30:58.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.625+0000 7fecea7fc640 1 -- 172.21.15.46:0/1321274102 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece0007860 con 0x7fecec157b00 2023-12-02T00:30:58.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.625+0000 7fecf4210640 1 -- 172.21.15.46:0/1321274102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecec1488f0 con 0x7fecec157b00 2023-12-02T00:30:58.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.626+0000 7fecf4210640 1 -- 172.21.15.46:0/1321274102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecec148ae0 con 0x7fecec157b00 2023-12-02T00:30:58.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.627+0000 7fecea7fc640 1 -- 172.21.15.46:0/1321274102 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fece0007bd0 con 0x7fecec157b00 2023-12-02T00:30:58.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.627+0000 7fecea7fc640 1 --2- 172.21.15.46:0/1321274102 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fecd806ae40 0x7fecd806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:58.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.627+0000 7fecea7fc640 1 -- 172.21.15.46:0/1321274102 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fece0089240 con 0x7fecec157b00 2023-12-02T00:30:58.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.627+0000 7fecf1fac640 1 --2- 172.21.15.46:0/1321274102 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fecd806ae40 0x7fecd806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:58.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.628+0000 7fecf1fac640 1 --2- 172.21.15.46:0/1321274102 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fecd806ae40 0x7fecd806d300 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fecdc036000 tx=0x7fecdc030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:58.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.631+0000 7fecea7fc640 1 -- 172.21.15.46:0/1321274102 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fece0054640 con 0x7fecec157b00 2023-12-02T00:30:58.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.805+0000 7fecf4210640 1 -- 172.21.15.46:0/1321274102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fecec067420 con 0x7fecec157b00 2023-12-02T00:30:58.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.806+0000 7fecea7fc640 1 -- 172.21.15.46:0/1321274102 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7fece001e020 con 0x7fecec157b00 2023-12-02T00:30:58.808 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-12-02T00:30:58.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.808+0000 7fecf4210640 1 -- 172.21.15.46:0/1321274102 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fecd806ae40 msgr2=0x7fecd806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:58.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.808+0000 7fecf4210640 1 --2- 172.21.15.46:0/1321274102 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fecd806ae40 0x7fecd806d300 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fecdc036000 tx=0x7fecdc030000 comp rx=0 tx=0).stop 2023-12-02T00:30:58.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.808+0000 7fecf4210640 1 -- 172.21.15.46:0/1321274102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecec157b00 msgr2=0x7fecec159770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:58.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.808+0000 7fecf4210640 1 --2- 172.21.15.46:0/1321274102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecec157b00 0x7fecec159770 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fece000c7f0 tx=0x7fece000ccb0 comp rx=0 tx=0).stop 2023-12-02T00:30:58.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.808+0000 7fecf4210640 1 -- 172.21.15.46:0/1321274102 shutdown_connections 2023-12-02T00:30:58.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.808+0000 7fecf4210640 1 --2- 172.21.15.46:0/1321274102 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fecd806ae40 0x7fecd806d300 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:58.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.808+0000 7fecf4210640 1 --2- 172.21.15.46:0/1321274102 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fecec15ad60 0x7fecec144250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:58.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.808+0000 7fecf4210640 1 --2- 172.21.15.46:0/1321274102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecec157b00 0x7fecec159770 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:58.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.808+0000 7fecf4210640 1 --2- 172.21.15.46:0/1321274102 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fecec071a00 0x7fecec143ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:58.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.808+0000 7fecf4210640 1 -- 172.21.15.46:0/1321274102 >> 172.21.15.46:0/1321274102 conn(0x7fecec0b1de0 msgr2=0x7fecec156b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:58.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.808+0000 7fecf4210640 1 -- 172.21.15.46:0/1321274102 shutdown_connections 2023-12-02T00:30:58.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.808+0000 7fecf4210640 1 -- 172.21.15.46:0/1321274102 wait complete. 2023-12-02T00:30:58.821 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-02T00:30:58.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.926+0000 7fa9621d0640 1 Processor -- start 2023-12-02T00:30:58.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.926+0000 7fa9621d0640 1 -- start start 2023-12-02T00:30:58.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.927+0000 7fa9621d0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa95c06ea50 0x7fa95c06ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:58.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.927+0000 7fa9621d0640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa95c161d20 con 0x7fa95c160fb0 2023-12-02T00:30:58.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.927+0000 7fa9621d0640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa95c06f370 con 0x7fa95c0d0ed0 2023-12-02T00:30:58.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.927+0000 7fa9621d0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa95c161b20 con 0x7fa95c06ea50 2023-12-02T00:30:58.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.927+0000 7fa95affd640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa95c160fb0 0x7fa95c161410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53782/0 (socket says 172.21.15.46:53782) 2023-12-02T00:30:58.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.927+0000 7fa95affd640 1 -- 172.21.15.46:0/2681531290 learned_addr learned my addr 172.21.15.46:0/2681531290 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:30:58.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.927+0000 7fa95b7fe640 1 --2- 172.21.15.46:0/2681531290 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa95c06ea50 0x7fa95c06ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:58.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.927+0000 7fa95a7fc640 1 -- 172.21.15.46:0/2681531290 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3604360020 0 0) 0x7fa95c161d20 con 0x7fa95c160fb0 2023-12-02T00:30:58.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.927+0000 7fa95a7fc640 1 -- 172.21.15.46:0/2681531290 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa944003520 con 0x7fa95c160fb0 2023-12-02T00:30:58.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.928+0000 7fa95a7fc640 1 -- 172.21.15.46:0/2681531290 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 947735758 0 0) 0x7fa95c06f370 con 0x7fa95c0d0ed0 2023-12-02T00:30:58.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.928+0000 7fa95a7fc640 1 -- 172.21.15.46:0/2681531290 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa95c161d20 con 0x7fa95c0d0ed0 2023-12-02T00:30:58.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.928+0000 7fa95a7fc640 1 -- 172.21.15.46:0/2681531290 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1453188549 0 0) 0x7fa944003520 con 0x7fa95c160fb0 2023-12-02T00:30:58.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.928+0000 7fa95a7fc640 1 -- 172.21.15.46:0/2681531290 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa95c06ea50 msgr2=0x7fa95c06ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:58.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.928+0000 7fa95a7fc640 1 --2- 172.21.15.46:0/2681531290 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa95c06ea50 0x7fa95c06ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:58.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.928+0000 7fa95a7fc640 1 -- 172.21.15.46:0/2681531290 >> v1:172.21.15.162:6789/0 conn(0x7fa95c0d0ed0 legacy=0x7fa95c0d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:58.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.928+0000 7fa95a7fc640 1 -- 172.21.15.46:0/2681531290 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa95c15e760 con 0x7fa95c160fb0 2023-12-02T00:30:58.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.928+0000 7fa95a7fc640 1 -- 172.21.15.46:0/2681531290 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7fa950001670 con 0x7fa95c160fb0 2023-12-02T00:30:58.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.928+0000 7fa95a7fc640 1 -- 172.21.15.46:0/2681531290 >> v1:172.21.15.46:6789/0 conn(0x7fa95c160fb0 legacy=0x7fa95c161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:58.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.928+0000 7fa95a7fc640 1 --2- 172.21.15.46:0/2681531290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa944003b30 0x7fa944003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:58.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.928+0000 7fa95a7fc640 1 --2- 172.21.15.46:0/2681531290 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9440047c0 0x7fa944004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:58.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.928+0000 7fa95a7fc640 1 --2- 172.21.15.46:0/2681531290 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa944005180 0x7fa94400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:58.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.928+0000 7fa95a7fc640 1 -- 172.21.15.46:0/2681531290 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95c161b20 con 0x7fa944003b30 2023-12-02T00:30:58.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.928+0000 7fa95a7fc640 1 -- 172.21.15.46:0/2681531290 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94400bf70 con 0x7fa944005180 2023-12-02T00:30:58.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.928+0000 7fa95a7fc640 1 -- 172.21.15.46:0/2681531290 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa94400c130 con 0x7fa9440047c0 2023-12-02T00:30:58.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.928+0000 7fa95b7fe640 1 --2- 172.21.15.46:0/2681531290 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa95c06ea50 0x7fa95c06ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:30:58.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.928+0000 7fa95b7fe640 1 --2- 172.21.15.46:0/2681531290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa944003b30 0x7fa944003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:58.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.928+0000 7fa95affd640 1 --2- 172.21.15.46:0/2681531290 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9440047c0 0x7fa944004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:58.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.928+0000 7fa95bfff640 1 --2- 172.21.15.46:0/2681531290 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa944005180 0x7fa94400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:58.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.928+0000 7fa95b7fe640 1 -- 172.21.15.46:0/2681531290 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9440047c0 msgr2=0x7fa944004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:58.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.928+0000 7fa95b7fe640 1 --2- 172.21.15.46:0/2681531290 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9440047c0 0x7fa944004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:58.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.928+0000 7fa95b7fe640 1 -- 172.21.15.46:0/2681531290 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa944005180 msgr2=0x7fa94400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:58.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.928+0000 7fa95b7fe640 1 --2- 172.21.15.46:0/2681531290 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa944005180 0x7fa94400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:58.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.928+0000 7fa95b7fe640 1 -- 172.21.15.46:0/2681531290 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa944003520 con 0x7fa944003b30 2023-12-02T00:30:58.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.929+0000 7fa95b7fe640 1 --2- 172.21.15.46:0/2681531290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa944003b30 0x7fa944003f30 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fa95c0b3db0 tx=0x7fa9440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d681c3be68b8f38 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:58.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.929+0000 7fa95a7fc640 1 -- 172.21.15.46:0/2681531290 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94c004030 con 0x7fa944003b30 2023-12-02T00:30:58.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.929+0000 7fa95a7fc640 1 -- 172.21.15.46:0/2681531290 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa94c004190 con 0x7fa944003b30 2023-12-02T00:30:58.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.929+0000 7fa95a7fc640 1 -- 172.21.15.46:0/2681531290 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94c004520 con 0x7fa944003b30 2023-12-02T00:30:58.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.930+0000 7fa9621d0640 1 -- 172.21.15.46:0/2681531290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa944003b30 msgr2=0x7fa944003f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:58.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.930+0000 7fa9621d0640 1 --2- 172.21.15.46:0/2681531290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa944003b30 0x7fa944003f30 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fa95c0b3db0 tx=0x7fa9440057f0 comp rx=0 tx=0).stop 2023-12-02T00:30:58.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.930+0000 7fa95bfff640 1 -- 172.21.15.46:0/2681531290 reap_dead start 2023-12-02T00:30:58.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.930+0000 7fa9621d0640 1 -- 172.21.15.46:0/2681531290 shutdown_connections 2023-12-02T00:30:58.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.930+0000 7fa9621d0640 1 -- 172.21.15.46:0/2681531290 >> 172.21.15.46:0/2681531290 conn(0x7fa95c06a6b0 msgr2=0x7fa95c06aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:58.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.930+0000 7fa9621d0640 1 -- 172.21.15.46:0/2681531290 shutdown_connections 2023-12-02T00:30:58.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.930+0000 7fa9621d0640 1 -- 172.21.15.46:0/2681531290 wait complete. 2023-12-02T00:30:58.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.931+0000 7fa9621d0640 1 Processor -- start 2023-12-02T00:30:58.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.931+0000 7fa9621d0640 1 -- start start 2023-12-02T00:30:58.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.932+0000 7fa9621d0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa944003b30 0x7fa95c1559f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:58.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.932+0000 7fa9621d0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa95c13bb10 0x7fa95c131130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:58.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.932+0000 7fa9621d0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa95c131670 0x7fa95c0b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:58.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.932+0000 7fa9621d0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95c132090 con 0x7fa95c13bb10 2023-12-02T00:30:58.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.932+0000 7fa9621d0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95c131af0 con 0x7fa944003b30 2023-12-02T00:30:58.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.932+0000 7fa9621d0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa95c13bf90 con 0x7fa95c131670 2023-12-02T00:30:58.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.932+0000 7fa95affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa95c13bb10 0x7fa95c131130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:58.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.932+0000 7fa95affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa95c13bb10 0x7fa95c131130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37412/0 (socket says 172.21.15.46:37412) 2023-12-02T00:30:58.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.932+0000 7fa95affd640 1 -- 172.21.15.46:0/1767123811 learned_addr learned my addr 172.21.15.46:0/1767123811 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:58.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.932+0000 7fa95b7fe640 1 --2- 172.21.15.46:0/1767123811 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa944003b30 0x7fa95c1559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:58.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.932+0000 7fa95bfff640 1 --2- 172.21.15.46:0/1767123811 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa95c131670 0x7fa95c0b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:58.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.932+0000 7fa95affd640 1 -- 172.21.15.46:0/1767123811 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa95c131670 msgr2=0x7fa95c0b0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:58.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.932+0000 7fa95affd640 1 --2- 172.21.15.46:0/1767123811 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa95c131670 0x7fa95c0b0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:58.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.932+0000 7fa95affd640 1 -- 172.21.15.46:0/1767123811 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa944003b30 msgr2=0x7fa95c1559f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:58.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.932+0000 7fa95affd640 1 --2- 172.21.15.46:0/1767123811 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa944003b30 0x7fa95c1559f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:58.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.932+0000 7fa95affd640 1 -- 172.21.15.46:0/1767123811 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa95c0b3760 con 0x7fa95c13bb10 2023-12-02T00:30:58.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.932+0000 7fa95bfff640 1 --2- 172.21.15.46:0/1767123811 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa95c131670 0x7fa95c0b0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:30:58.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.933+0000 7fa95affd640 1 --2- 172.21.15.46:0/1767123811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa95c13bb10 0x7fa95c131130 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fa950001400 tx=0x7fa9500077a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:58.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.933+0000 7fa943fff640 1 -- 172.21.15.46:0/1767123811 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9500015f0 con 0x7fa95c13bb10 2023-12-02T00:30:58.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.933+0000 7fa943fff640 1 -- 172.21.15.46:0/1767123811 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa95000f960 con 0x7fa95c13bb10 2023-12-02T00:30:58.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.933+0000 7fa9621d0640 1 -- 172.21.15.46:0/1767123811 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa95c0b3a40 con 0x7fa95c13bb10 2023-12-02T00:30:58.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.933+0000 7fa943fff640 1 -- 172.21.15.46:0/1767123811 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95000fcd0 con 0x7fa95c13bb10 2023-12-02T00:30:58.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.933+0000 7fa9621d0640 1 -- 172.21.15.46:0/1767123811 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa95c1551f0 con 0x7fa95c13bb10 2023-12-02T00:30:58.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.934+0000 7fa9621d0640 1 -- 172.21.15.46:0/1767123811 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa95c069400 con 0x7fa95c13bb10 2023-12-02T00:30:58.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.934+0000 7fa943fff640 1 -- 172.21.15.46:0/1767123811 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa95001d460 con 0x7fa95c13bb10 2023-12-02T00:30:58.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.935+0000 7fa943fff640 1 --2- 172.21.15.46:0/1767123811 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa94806ae70 0x7fa94806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:58.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.935+0000 7fa943fff640 1 -- 172.21.15.46:0/1767123811 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa950089280 con 0x7fa95c13bb10 2023-12-02T00:30:58.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.935+0000 7fa95b7fe640 1 --2- 172.21.15.46:0/1767123811 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa94806ae70 0x7fa94806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:58.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.935+0000 7fa95b7fe640 1 --2- 172.21.15.46:0/1767123811 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa94806ae70 0x7fa94806d330 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa94c035000 tx=0x7fa94c030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:58.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:58.939+0000 7fa943fff640 1 -- 172.21.15.46:0/1767123811 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa950054680 con 0x7fa95c13bb10 2023-12-02T00:30:59.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.119+0000 7fa9621d0640 1 -- 172.21.15.46:0/1767123811 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa95c15f170 con 0x7fa95c13bb10 2023-12-02T00:30:59.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.119+0000 7fa943fff640 1 -- 172.21.15.46:0/1767123811 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v47) v1 ==== 84+0+65 (secure 0 0 0) 0x7fa950057ca0 con 0x7fa95c13bb10 2023-12-02T00:30:59.122 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:30:59.122 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2023-12-02T00:30:59.122 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-02T00:30:59.122 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2023-12-02T00:30:59.122 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2023-12-02T00:30:59.122 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:30:59.122 INFO:teuthology.orchestra.run.smithi046.stdout:] 2023-12-02T00:30:59.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.121+0000 7fa9621d0640 1 -- 172.21.15.46:0/1767123811 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa94806ae70 msgr2=0x7fa94806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:59.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.121+0000 7fa9621d0640 1 --2- 172.21.15.46:0/1767123811 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa94806ae70 0x7fa94806d330 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa94c035000 tx=0x7fa94c030000 comp rx=0 tx=0).stop 2023-12-02T00:30:59.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.121+0000 7fa9621d0640 1 -- 172.21.15.46:0/1767123811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa95c13bb10 msgr2=0x7fa95c131130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:59.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.121+0000 7fa9621d0640 1 --2- 172.21.15.46:0/1767123811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa95c13bb10 0x7fa95c131130 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fa950001400 tx=0x7fa9500077a0 comp rx=0 tx=0).stop 2023-12-02T00:30:59.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.121+0000 7fa9621d0640 1 -- 172.21.15.46:0/1767123811 shutdown_connections 2023-12-02T00:30:59.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.121+0000 7fa9621d0640 1 --2- 172.21.15.46:0/1767123811 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fa94806ae70 0x7fa94806d330 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:59.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.121+0000 7fa9621d0640 1 --2- 172.21.15.46:0/1767123811 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa95c131670 0x7fa95c0b0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:59.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.121+0000 7fa9621d0640 1 --2- 172.21.15.46:0/1767123811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa95c13bb10 0x7fa95c131130 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:59.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.121+0000 7fa9621d0640 1 --2- 172.21.15.46:0/1767123811 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa944003b30 0x7fa95c1559f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:59.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.121+0000 7fa9621d0640 1 -- 172.21.15.46:0/1767123811 >> 172.21.15.46:0/1767123811 conn(0x7fa95c06a6b0 msgr2=0x7fa95c15e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:59.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.121+0000 7fa9621d0640 1 -- 172.21.15.46:0/1767123811 shutdown_connections 2023-12-02T00:30:59.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.121+0000 7fa9621d0640 1 -- 172.21.15.46:0/1767123811 wait complete. 2023-12-02T00:30:59.134 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-02T00:30:59.135 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-02T00:30:59.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.244+0000 7f4f4b875640 1 Processor -- start 2023-12-02T00:30:59.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.244+0000 7f4f4b875640 1 -- start start 2023-12-02T00:30:59.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.245+0000 7f4f4b875640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4f44160fb0 0x7f4f44161440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:59.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.245+0000 7f4f4b875640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f440d0ed0 0x7f4f440d1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:59.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.245+0000 7f4f4b875640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f44161d20 con 0x7f4f4406ea50 2023-12-02T00:30:59.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.245+0000 7f4f4b875640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f4415e760 con 0x7f4f44160fb0 2023-12-02T00:30:59.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.245+0000 7f4f4b875640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4415d9a0 con 0x7f4f440d0ed0 2023-12-02T00:30:59.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.245+0000 7f4f49611640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4f4406ea50 0x7f4f4406ee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53790/0 (socket says 172.21.15.46:53790) 2023-12-02T00:30:59.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.245+0000 7f4f49611640 1 -- 172.21.15.46:0/2056487522 learned_addr learned my addr 172.21.15.46:0/2056487522 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:30:59.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.245+0000 7f4f43fff640 1 -- 172.21.15.46:0/2056487522 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4007520058 0 0) 0x7f4f44161d20 con 0x7f4f4406ea50 2023-12-02T00:30:59.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.245+0000 7f4f43fff640 1 -- 172.21.15.46:0/2056487522 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f2c003520 con 0x7f4f4406ea50 2023-12-02T00:30:59.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.245+0000 7f4f48e10640 1 --2- 172.21.15.46:0/2056487522 >> v2:172.21.15.162:3300/0 conn(0x7f4f44160fb0 0x7f4f44161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:59.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.245+0000 7f4f49e12640 1 --2- 172.21.15.46:0/2056487522 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f440d0ed0 0x7f4f440d1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:59.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.246+0000 7f4f43fff640 1 -- 172.21.15.46:0/2056487522 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2745318185 0 0) 0x7f4f2c003520 con 0x7f4f4406ea50 2023-12-02T00:30:59.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.246+0000 7f4f43fff640 1 -- 172.21.15.46:0/2056487522 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f440d0ed0 msgr2=0x7f4f440d1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:59.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.246+0000 7f4f43fff640 1 --2- 172.21.15.46:0/2056487522 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f440d0ed0 0x7f4f440d1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:59.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.246+0000 7f4f43fff640 1 -- 172.21.15.46:0/2056487522 >> v2:172.21.15.162:3300/0 conn(0x7f4f44160fb0 msgr2=0x7f4f44161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:59.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.246+0000 7f4f43fff640 1 --2- 172.21.15.46:0/2056487522 >> v2:172.21.15.162:3300/0 conn(0x7f4f44160fb0 0x7f4f44161440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:59.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.246+0000 7f4f43fff640 1 -- 172.21.15.46:0/2056487522 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f4415dc80 con 0x7f4f4406ea50 2023-12-02T00:30:59.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.246+0000 7f4f43fff640 1 -- 172.21.15.46:0/2056487522 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7f4f3c001670 con 0x7f4f4406ea50 2023-12-02T00:30:59.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.246+0000 7f4f43fff640 1 -- 172.21.15.46:0/2056487522 >> v1:172.21.15.46:6789/0 conn(0x7f4f4406ea50 legacy=0x7f4f4406ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:59.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.246+0000 7f4f43fff640 1 --2- 172.21.15.46:0/2056487522 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f2c003970 0x7f4f2c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:59.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.246+0000 7f4f43fff640 1 --2- 172.21.15.46:0/2056487522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f2c0047c0 0x7f4f2c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:59.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.246+0000 7f4f43fff640 1 --2- 172.21.15.46:0/2056487522 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f440d0ed0 0x7f4f2c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:59.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.246+0000 7f4f43fff640 1 -- 172.21.15.46:0/2056487522 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4415e760 con 0x7f4f2c0047c0 2023-12-02T00:30:59.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.246+0000 7f4f43fff640 1 -- 172.21.15.46:0/2056487522 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4415d9a0 con 0x7f4f2c003970 2023-12-02T00:30:59.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.246+0000 7f4f43fff640 1 -- 172.21.15.46:0/2056487522 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2c00b980 con 0x7f4f440d0ed0 2023-12-02T00:30:59.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.246+0000 7f4f48e10640 1 --2- 172.21.15.46:0/2056487522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f2c0047c0 0x7f4f2c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:59.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.246+0000 7f4f49611640 1 --2- 172.21.15.46:0/2056487522 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f2c003970 0x7f4f2c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:59.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.246+0000 7f4f49e12640 1 --2- 172.21.15.46:0/2056487522 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f440d0ed0 0x7f4f2c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:59.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.246+0000 7f4f48e10640 1 -- 172.21.15.46:0/2056487522 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f440d0ed0 msgr2=0x7f4f2c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:59.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.246+0000 7f4f48e10640 1 --2- 172.21.15.46:0/2056487522 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f440d0ed0 0x7f4f2c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:59.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.246+0000 7f4f48e10640 1 -- 172.21.15.46:0/2056487522 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f2c003970 msgr2=0x7f4f2c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:59.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.246+0000 7f4f48e10640 1 --2- 172.21.15.46:0/2056487522 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f2c003970 0x7f4f2c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:59.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.246+0000 7f4f48e10640 1 -- 172.21.15.46:0/2056487522 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f2c003520 con 0x7f4f2c0047c0 2023-12-02T00:30:59.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.247+0000 7f4f48e10640 1 --2- 172.21.15.46:0/2056487522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f2c0047c0 0x7f4f2c004c40 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f4f38009ac0 tx=0x7f4f2c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8dc4caaaaed7bd6 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:59.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.247+0000 7f4f43fff640 1 -- 172.21.15.46:0/2056487522 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3802c710 con 0x7f4f2c0047c0 2023-12-02T00:30:59.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.247+0000 7f4f43fff640 1 -- 172.21.15.46:0/2056487522 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f3802c870 con 0x7f4f2c0047c0 2023-12-02T00:30:59.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.247+0000 7f4f43fff640 1 -- 172.21.15.46:0/2056487522 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3802cc00 con 0x7f4f2c0047c0 2023-12-02T00:30:59.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.248+0000 7f4f4b875640 1 -- 172.21.15.46:0/2056487522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f2c0047c0 msgr2=0x7f4f2c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:59.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.248+0000 7f4f4b875640 1 --2- 172.21.15.46:0/2056487522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f2c0047c0 0x7f4f2c004c40 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f4f38009ac0 tx=0x7f4f2c0057f0 comp rx=0 tx=0).stop 2023-12-02T00:30:59.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.248+0000 7f4f49e12640 1 -- 172.21.15.46:0/2056487522 reap_dead start 2023-12-02T00:30:59.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.248+0000 7f4f4b875640 1 -- 172.21.15.46:0/2056487522 shutdown_connections 2023-12-02T00:30:59.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.248+0000 7f4f4b875640 1 -- 172.21.15.46:0/2056487522 >> 172.21.15.46:0/2056487522 conn(0x7f4f4406a6b0 msgr2=0x7f4f4406aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:59.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.248+0000 7f4f4b875640 1 -- 172.21.15.46:0/2056487522 shutdown_connections 2023-12-02T00:30:59.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.248+0000 7f4f4b875640 1 -- 172.21.15.46:0/2056487522 wait complete. 2023-12-02T00:30:59.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.249+0000 7f4f4b875640 1 Processor -- start 2023-12-02T00:30:59.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.249+0000 7f4f4b875640 1 -- start start 2023-12-02T00:30:59.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.250+0000 7f4f4b875640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f2c0047c0 0x7f4f4413d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:59.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.250+0000 7f4f4b875640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f441e6c10 0x7f4f440d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:59.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.250+0000 7f4f4b875640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f441e7090 0x7f4f440d1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:59.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.250+0000 7f4f4b875640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f44155720 con 0x7f4f441e6c10 2023-12-02T00:30:59.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.250+0000 7f4f4b875640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f44155a00 con 0x7f4f441e7090 2023-12-02T00:30:59.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.250+0000 7f4f4b875640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f44155ce0 con 0x7f4f2c0047c0 2023-12-02T00:30:59.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.250+0000 7f4f48e10640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f441e6c10 0x7f4f440d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:59.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.250+0000 7f4f48e10640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f441e6c10 0x7f4f440d12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37426/0 (socket says 172.21.15.46:37426) 2023-12-02T00:30:59.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.250+0000 7f4f48e10640 1 -- 172.21.15.46:0/728462837 learned_addr learned my addr 172.21.15.46:0/728462837 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:59.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.250+0000 7f4f49e12640 1 --2- 172.21.15.46:0/728462837 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f441e7090 0x7f4f440d1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:59.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.250+0000 7f4f49611640 1 --2- 172.21.15.46:0/728462837 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f2c0047c0 0x7f4f4413d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:59.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.250+0000 7f4f48e10640 1 -- 172.21.15.46:0/728462837 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f2c0047c0 msgr2=0x7f4f4413d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:59.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.250+0000 7f4f48e10640 1 --2- 172.21.15.46:0/728462837 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f2c0047c0 0x7f4f4413d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:59.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.250+0000 7f4f48e10640 1 -- 172.21.15.46:0/728462837 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f441e7090 msgr2=0x7f4f440d1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:59.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.250+0000 7f4f48e10640 1 --2- 172.21.15.46:0/728462837 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f441e7090 0x7f4f440d1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:59.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.250+0000 7f4f48e10640 1 -- 172.21.15.46:0/728462837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f44155fc0 con 0x7f4f441e6c10 2023-12-02T00:30:59.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.250+0000 7f4f49611640 1 --2- 172.21.15.46:0/728462837 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f2c0047c0 0x7f4f4413d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:30:59.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.250+0000 7f4f48e10640 1 --2- 172.21.15.46:0/728462837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f441e6c10 0x7f4f440d12e0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f4f3802c2b0 tx=0x7f4f38004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:59.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.250+0000 7f4f41ffb640 1 -- 172.21.15.46:0/728462837 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f38003810 con 0x7f4f441e6c10 2023-12-02T00:30:59.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.250+0000 7f4f41ffb640 1 -- 172.21.15.46:0/728462837 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f3802de90 con 0x7f4f441e6c10 2023-12-02T00:30:59.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.250+0000 7f4f4b875640 1 -- 172.21.15.46:0/728462837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f441562a0 con 0x7f4f441e6c10 2023-12-02T00:30:59.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.251+0000 7f4f41ffb640 1 -- 172.21.15.46:0/728462837 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f38003b70 con 0x7f4f441e6c10 2023-12-02T00:30:59.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.251+0000 7f4f4b875640 1 -- 172.21.15.46:0/728462837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f441553b0 con 0x7f4f441e6c10 2023-12-02T00:30:59.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.251+0000 7f4f4b875640 1 -- 172.21.15.46:0/728462837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f44069400 con 0x7f4f441e6c10 2023-12-02T00:30:59.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.252+0000 7f4f41ffb640 1 -- 172.21.15.46:0/728462837 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4f38003cd0 con 0x7f4f441e6c10 2023-12-02T00:30:59.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.252+0000 7f4f41ffb640 1 --2- 172.21.15.46:0/728462837 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f4f3006ae40 0x7f4f3006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:59.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.253+0000 7f4f41ffb640 1 -- 172.21.15.46:0/728462837 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4f380ab150 con 0x7f4f441e6c10 2023-12-02T00:30:59.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.253+0000 7f4f49611640 1 --2- 172.21.15.46:0/728462837 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f4f3006ae40 0x7f4f3006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:59.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.253+0000 7f4f49611640 1 --2- 172.21.15.46:0/728462837 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f4f3006ae40 0x7f4f3006d300 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4f4413f200 tx=0x7f4f3c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:59.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.257+0000 7f4f41ffb640 1 -- 172.21.15.46:0/728462837 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4f38076720 con 0x7f4f441e6c10 2023-12-02T00:30:59.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.437+0000 7f4f4b875640 1 -- 172.21.15.46:0/728462837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f4f44157170 con 0x7f4f441e6c10 2023-12-02T00:30:59.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.811+0000 7f4f41ffb640 1 -- 172.21.15.46:0/728462837 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v48) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4f38079d40 con 0x7f4f441e6c10 2023-12-02T00:30:59.813 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2023-12-02T00:30:59.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.813+0000 7f4f4b875640 1 -- 172.21.15.46:0/728462837 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f4f3006ae40 msgr2=0x7f4f3006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:59.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.813+0000 7f4f4b875640 1 --2- 172.21.15.46:0/728462837 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f4f3006ae40 0x7f4f3006d300 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4f4413f200 tx=0x7f4f3c009040 comp rx=0 tx=0).stop 2023-12-02T00:30:59.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.813+0000 7f4f4b875640 1 -- 172.21.15.46:0/728462837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f441e6c10 msgr2=0x7f4f440d12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:59.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.813+0000 7f4f4b875640 1 --2- 172.21.15.46:0/728462837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f441e6c10 0x7f4f440d12e0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f4f3802c2b0 tx=0x7f4f38004730 comp rx=0 tx=0).stop 2023-12-02T00:30:59.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.813+0000 7f4f4b875640 1 -- 172.21.15.46:0/728462837 shutdown_connections 2023-12-02T00:30:59.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.813+0000 7f4f4b875640 1 --2- 172.21.15.46:0/728462837 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f4f3006ae40 0x7f4f3006d300 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:59.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.813+0000 7f4f4b875640 1 --2- 172.21.15.46:0/728462837 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f441e7090 0x7f4f440d1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:59.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.813+0000 7f4f4b875640 1 --2- 172.21.15.46:0/728462837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f441e6c10 0x7f4f440d12e0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:59.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.813+0000 7f4f4b875640 1 --2- 172.21.15.46:0/728462837 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f2c0047c0 0x7f4f4413d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:59.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.813+0000 7f4f4b875640 1 -- 172.21.15.46:0/728462837 >> 172.21.15.46:0/728462837 conn(0x7f4f4406a6b0 msgr2=0x7f4f4406aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:59.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.814+0000 7f4f4b875640 1 -- 172.21.15.46:0/728462837 shutdown_connections 2023-12-02T00:30:59.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.814+0000 7f4f4b875640 1 -- 172.21.15.46:0/728462837 wait complete. 2023-12-02T00:30:59.831 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-02T00:30:59.863 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2023-12-02T00:30:59.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.948+0000 7fbc209b0640 1 Processor -- start 2023-12-02T00:30:59.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.948+0000 7fbc209b0640 1 -- start start 2023-12-02T00:30:59.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.949+0000 7fbc209b0640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbc1c0ce6f0 0x7fbc1c15c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:59.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.949+0000 7fbc209b0640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbc1c0cfaf0 0x7fbc1c0b5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:59.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.949+0000 7fbc209b0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbc1c0b56a0 0x7fbc1c0b2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:59.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.949+0000 7fbc209b0640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc1c0d0b30 con 0x7fbc1c0ce6f0 2023-12-02T00:30:59.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.949+0000 7fbc209b0640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc1c0d0cc0 con 0x7fbc1c0cfaf0 2023-12-02T00:30:59.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.949+0000 7fbc209b0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1c0d0290 con 0x7fbc1c0b56a0 2023-12-02T00:30:59.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.949+0000 7fbc1a59c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbc1c0ce6f0 0x7fbc1c15c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:59.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.949+0000 7fbc1a59c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbc1c0ce6f0 0x7fbc1c15c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37436/0 (socket says 172.21.15.46:37436) 2023-12-02T00:30:59.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.949+0000 7fbc1a59c640 1 -- 172.21.15.46:0/4121366220 learned_addr learned my addr 172.21.15.46:0/4121366220 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:59.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.949+0000 7fbc19d9b640 1 --2- 172.21.15.46:0/4121366220 >> v2:172.21.15.162:3300/0 conn(0x7fbc1c0cfaf0 0x7fbc1c0b5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:59.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.949+0000 7fbc1ad9d640 1 --2- 172.21.15.46:0/4121366220 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbc1c0b56a0 0x7fbc1c0b2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:59.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.949+0000 7fbc1a59c640 1 -- 172.21.15.46:0/4121366220 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbc1c0b56a0 msgr2=0x7fbc1c0b2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:59.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.949+0000 7fbc1a59c640 1 --2- 172.21.15.46:0/4121366220 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbc1c0b56a0 0x7fbc1c0b2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:59.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.949+0000 7fbc1a59c640 1 -- 172.21.15.46:0/4121366220 >> v2:172.21.15.162:3300/0 conn(0x7fbc1c0cfaf0 msgr2=0x7fbc1c0b5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:30:59.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.949+0000 7fbc1a59c640 1 --2- 172.21.15.46:0/4121366220 >> v2:172.21.15.162:3300/0 conn(0x7fbc1c0cfaf0 0x7fbc1c0b5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:59.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.949+0000 7fbc1a59c640 1 -- 172.21.15.46:0/4121366220 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc1c0d0570 con 0x7fbc1c0ce6f0 2023-12-02T00:30:59.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.950+0000 7fbc1a59c640 1 --2- 172.21.15.46:0/4121366220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc1c0ce6f0 0x7fbc1c15c310 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c0b4390 tx=0x7fbc0c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99a59f002a3fa0cb server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:59.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.950+0000 7fbc1959a640 1 -- 172.21.15.46:0/4121366220 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c02ca70 con 0x7fbc1c0ce6f0 2023-12-02T00:30:59.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.950+0000 7fbc1959a640 1 -- 172.21.15.46:0/4121366220 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc0c02cbd0 con 0x7fbc1c0ce6f0 2023-12-02T00:30:59.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.950+0000 7fbc1959a640 1 -- 172.21.15.46:0/4121366220 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c0343f0 con 0x7fbc1c0ce6f0 2023-12-02T00:30:59.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.951+0000 7fbc209b0640 1 -- 172.21.15.46:0/4121366220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc1c0ce6f0 msgr2=0x7fbc1c15c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:59.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.951+0000 7fbc209b0640 1 --2- 172.21.15.46:0/4121366220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc1c0ce6f0 0x7fbc1c15c310 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c0b4390 tx=0x7fbc0c02bed0 comp rx=0 tx=0).stop 2023-12-02T00:30:59.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.951+0000 7fbc209b0640 1 -- 172.21.15.46:0/4121366220 shutdown_connections 2023-12-02T00:30:59.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.951+0000 7fbc209b0640 1 --2- 172.21.15.46:0/4121366220 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbc1c0b56a0 0x7fbc1c0b2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:59.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.951+0000 7fbc209b0640 1 --2- 172.21.15.46:0/4121366220 >> v2:172.21.15.162:3300/0 conn(0x7fbc1c0cfaf0 0x7fbc1c0b5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:59.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.951+0000 7fbc209b0640 1 --2- 172.21.15.46:0/4121366220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc1c0ce6f0 0x7fbc1c15c310 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:59.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.951+0000 7fbc209b0640 1 -- 172.21.15.46:0/4121366220 >> 172.21.15.46:0/4121366220 conn(0x7fbc1c06a4d0 msgr2=0x7fbc1c06a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:30:59.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.951+0000 7fbc209b0640 1 -- 172.21.15.46:0/4121366220 shutdown_connections 2023-12-02T00:30:59.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.951+0000 7fbc209b0640 1 -- 172.21.15.46:0/4121366220 wait complete. 2023-12-02T00:30:59.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.952+0000 7fbc209b0640 1 Processor -- start 2023-12-02T00:30:59.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.952+0000 7fbc209b0640 1 -- start start 2023-12-02T00:30:59.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.953+0000 7fbc209b0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc1c0b56a0 0x7fbc1c158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:59.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.953+0000 7fbc209b0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc1c0ce6f0 0x7fbc1c06cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:59.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.953+0000 7fbc209b0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbc1c0cfaf0 0x7fbc1c15dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:59.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.953+0000 7fbc209b0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1c06d680 con 0x7fbc1c0b56a0 2023-12-02T00:30:59.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.953+0000 7fbc209b0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1c0b07d0 con 0x7fbc1c0ce6f0 2023-12-02T00:30:59.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.953+0000 7fbc209b0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1c06d450 con 0x7fbc1c0cfaf0 2023-12-02T00:30:59.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.953+0000 7fbc1a59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc1c0b56a0 0x7fbc1c158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:59.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.953+0000 7fbc1a59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc1c0b56a0 0x7fbc1c158a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37440/0 (socket says 172.21.15.46:37440) 2023-12-02T00:30:59.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.953+0000 7fbc1a59c640 1 -- 172.21.15.46:0/3291651837 learned_addr learned my addr 172.21.15.46:0/3291651837 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:30:59.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.953+0000 7fbc19d9b640 1 --2- 172.21.15.46:0/3291651837 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc1c0ce6f0 0x7fbc1c06cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:59.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.953+0000 7fbc1ad9d640 1 --2- 172.21.15.46:0/3291651837 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbc1c0cfaf0 0x7fbc1c15dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:59.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.953+0000 7fbc1a59c640 1 -- 172.21.15.46:0/3291651837 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbc1c0cfaf0 msgr2=0x7fbc1c15dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:59.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.953+0000 7fbc1a59c640 1 --2- 172.21.15.46:0/3291651837 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbc1c0cfaf0 0x7fbc1c15dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:59.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.953+0000 7fbc1a59c640 1 -- 172.21.15.46:0/3291651837 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc1c0ce6f0 msgr2=0x7fbc1c06cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:30:59.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.953+0000 7fbc1a59c640 1 --2- 172.21.15.46:0/3291651837 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc1c0ce6f0 0x7fbc1c06cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:30:59.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.953+0000 7fbc1a59c640 1 -- 172.21.15.46:0/3291651837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc0c009b30 con 0x7fbc1c0b56a0 2023-12-02T00:30:59.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.954+0000 7fbc1ad9d640 1 --2- 172.21.15.46:0/3291651837 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbc1c0cfaf0 0x7fbc1c15dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:30:59.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.954+0000 7fbc1a59c640 1 --2- 172.21.15.46:0/3291651837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc1c0b56a0 0x7fbc1c158a30 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c000f80 tx=0x7fbc0c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:59.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.954+0000 7fbc0affd640 1 -- 172.21.15.46:0/3291651837 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c009510 con 0x7fbc1c0b56a0 2023-12-02T00:30:59.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.954+0000 7fbc0affd640 1 -- 172.21.15.46:0/3291651837 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc0c03e450 con 0x7fbc1c0b56a0 2023-12-02T00:30:59.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.954+0000 7fbc209b0640 1 -- 172.21.15.46:0/3291651837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc1c157b20 con 0x7fbc1c0b56a0 2023-12-02T00:30:59.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.954+0000 7fbc0affd640 1 -- 172.21.15.46:0/3291651837 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c03e800 con 0x7fbc1c0b56a0 2023-12-02T00:30:59.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.954+0000 7fbc209b0640 1 -- 172.21.15.46:0/3291651837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc1c1580e0 con 0x7fbc1c0b56a0 2023-12-02T00:30:59.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.955+0000 7fbc209b0640 1 -- 172.21.15.46:0/3291651837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc1c0b0910 con 0x7fbc1c0b56a0 2023-12-02T00:30:59.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.956+0000 7fbc0affd640 1 -- 172.21.15.46:0/3291651837 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbc0c03e960 con 0x7fbc1c0b56a0 2023-12-02T00:30:59.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.956+0000 7fbc0affd640 1 --2- 172.21.15.46:0/3291651837 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fbbfc06ae40 0x7fbbfc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:30:59.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.956+0000 7fbc0affd640 1 -- 172.21.15.46:0/3291651837 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fbc0c0ac240 con 0x7fbc1c0b56a0 2023-12-02T00:30:59.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.956+0000 7fbc19d9b640 1 --2- 172.21.15.46:0/3291651837 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fbbfc06ae40 0x7fbbfc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:30:59.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.957+0000 7fbc19d9b640 1 --2- 172.21.15.46:0/3291651837 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fbbfc06ae40 0x7fbbfc06d300 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c0d1800 tx=0x7fbc10008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:30:59.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:30:59.961+0000 7fbc0affd640 1 -- 172.21.15.46:0/3291651837 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbc1c0b0910 con 0x7fbc1c0b56a0 2023-12-02T00:31:00.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.142+0000 7fbc209b0640 1 -- 172.21.15.46:0/3291651837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fbc1c069400 con 0x7fbc1c0b56a0 2023-12-02T00:31:00.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.813+0000 7fbc0affd640 1 -- 172.21.15.46:0/3291651837 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v49) v1 ==== 170+0+0 (secure 0 0 0) 0x7fbc0c0774d0 con 0x7fbc1c0b56a0 2023-12-02T00:31:00.816 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2023-12-02T00:31:00.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.818+0000 7fbc209b0640 1 -- 172.21.15.46:0/3291651837 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fbbfc06ae40 msgr2=0x7fbbfc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:00.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.818+0000 7fbc209b0640 1 --2- 172.21.15.46:0/3291651837 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fbbfc06ae40 0x7fbbfc06d300 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c0d1800 tx=0x7fbc10008040 comp rx=0 tx=0).stop 2023-12-02T00:31:00.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.818+0000 7fbc209b0640 1 -- 172.21.15.46:0/3291651837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc1c0b56a0 msgr2=0x7fbc1c158a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:00.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.818+0000 7fbc209b0640 1 --2- 172.21.15.46:0/3291651837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc1c0b56a0 0x7fbc1c158a30 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c000f80 tx=0x7fbc0c004a40 comp rx=0 tx=0).stop 2023-12-02T00:31:00.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.818+0000 7fbc209b0640 1 -- 172.21.15.46:0/3291651837 shutdown_connections 2023-12-02T00:31:00.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.818+0000 7fbc209b0640 1 --2- 172.21.15.46:0/3291651837 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fbbfc06ae40 0x7fbbfc06d300 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:00.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.818+0000 7fbc209b0640 1 --2- 172.21.15.46:0/3291651837 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbc1c0cfaf0 0x7fbc1c15dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:00.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.818+0000 7fbc209b0640 1 --2- 172.21.15.46:0/3291651837 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc1c0ce6f0 0x7fbc1c06cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:00.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.818+0000 7fbc209b0640 1 --2- 172.21.15.46:0/3291651837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc1c0b56a0 0x7fbc1c158a30 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:00.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.818+0000 7fbc209b0640 1 -- 172.21.15.46:0/3291651837 >> 172.21.15.46:0/3291651837 conn(0x7fbc1c06a4d0 msgr2=0x7fbc1c15a870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:00.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.818+0000 7fbc209b0640 1 -- 172.21.15.46:0/3291651837 shutdown_connections 2023-12-02T00:31:00.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.818+0000 7fbc209b0640 1 -- 172.21.15.46:0/3291651837 wait complete. 2023-12-02T00:31:00.846 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-02T00:31:00.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.958+0000 7f2d1916f640 1 Processor -- start 2023-12-02T00:31:00.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.958+0000 7f2d1916f640 1 -- start start 2023-12-02T00:31:00.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.959+0000 7f2d1916f640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2d1415d340 0x7f2d1406b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:00.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.959+0000 7f2d1916f640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2d140ad640 0x7f2d1406ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:00.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.959+0000 7f2d1916f640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d1415d720 con 0x7f2d140aeb80 2023-12-02T00:31:00.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.959+0000 7f2d1916f640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d140adae0 con 0x7f2d1415d340 2023-12-02T00:31:00.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.959+0000 7f2d1916f640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d140addc0 con 0x7f2d140ad640 2023-12-02T00:31:00.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.959+0000 7f2d12d9d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2d1415d340 0x7f2d1406b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:00.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.959+0000 7f2d1359e640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2d140aeb80 0x7f2d14160ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53800/0 (socket says 172.21.15.46:53800) 2023-12-02T00:31:00.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.959+0000 7f2d1359e640 1 -- 172.21.15.46:0/1606260907 learned_addr learned my addr 172.21.15.46:0/1606260907 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:31:00.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.960+0000 7f2d0bfff640 1 --2- 172.21.15.46:0/1606260907 >> v2:172.21.15.162:3300/0 conn(0x7f2d140ad640 0x7f2d1406ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:00.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.960+0000 7f2d12d9d640 1 -- 172.21.15.46:0/1606260907 >> v2:172.21.15.162:3300/0 conn(0x7f2d140ad640 msgr2=0x7f2d1406ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:00.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.960+0000 7f2d12d9d640 1 --2- 172.21.15.46:0/1606260907 >> v2:172.21.15.162:3300/0 conn(0x7f2d140ad640 0x7f2d1406ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:00.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.960+0000 7f2d12d9d640 1 -- 172.21.15.46:0/1606260907 >> v1:172.21.15.46:6789/0 conn(0x7f2d140aeb80 legacy=0x7f2d14160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:00.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.960+0000 7f2d12d9d640 1 -- 172.21.15.46:0/1606260907 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d140ae0a0 con 0x7f2d1415d340 2023-12-02T00:31:00.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.960+0000 7f2d1259c640 1 -- 172.21.15.46:0/1606260907 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4235793809 0 0) 0x7f2d1415d720 con 0x7f2d140aeb80 2023-12-02T00:31:00.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.960+0000 7f2d12d9d640 1 --2- 172.21.15.46:0/1606260907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d1415d340 0x7f2d1406b4e0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f2d14072310 tx=0x7f2d0402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a646ae1e3b8d7a4 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:00.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.961+0000 7f2d1259c640 1 -- 172.21.15.46:0/1606260907 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d04031830 con 0x7f2d1415d340 2023-12-02T00:31:00.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.961+0000 7f2d1259c640 1 -- 172.21.15.46:0/1606260907 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d04031990 con 0x7f2d1415d340 2023-12-02T00:31:00.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.961+0000 7f2d1259c640 1 -- 172.21.15.46:0/1606260907 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d04031d20 con 0x7f2d1415d340 2023-12-02T00:31:00.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.961+0000 7f2d1916f640 1 -- 172.21.15.46:0/1606260907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d1415d340 msgr2=0x7f2d1406b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:00.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.961+0000 7f2d1916f640 1 --2- 172.21.15.46:0/1606260907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d1415d340 0x7f2d1406b4e0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f2d14072310 tx=0x7f2d0402b8d0 comp rx=0 tx=0).stop 2023-12-02T00:31:00.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.961+0000 7f2d1916f640 1 -- 172.21.15.46:0/1606260907 shutdown_connections 2023-12-02T00:31:00.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.961+0000 7f2d1916f640 1 --2- 172.21.15.46:0/1606260907 >> v2:172.21.15.162:3300/0 conn(0x7f2d140ad640 0x7f2d1406ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:00.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.961+0000 7f2d1916f640 1 --2- 172.21.15.46:0/1606260907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d1415d340 0x7f2d1406b4e0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:00.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.961+0000 7f2d1916f640 1 -- 172.21.15.46:0/1606260907 >> 172.21.15.46:0/1606260907 conn(0x7f2d14070e70 msgr2=0x7f2d14071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:00.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.961+0000 7f2d1916f640 1 -- 172.21.15.46:0/1606260907 shutdown_connections 2023-12-02T00:31:00.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.961+0000 7f2d1916f640 1 -- 172.21.15.46:0/1606260907 wait complete. 2023-12-02T00:31:00.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.963+0000 7f2d1916f640 1 Processor -- start 2023-12-02T00:31:00.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.963+0000 7f2d1916f640 1 -- start start 2023-12-02T00:31:00.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.963+0000 7f2d1916f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d140ad640 0x7f2d140c5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:00.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.963+0000 7f2d1916f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d140aeb80 0x7f2d141513d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:00.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.963+0000 7f2d1916f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d1415d340 0x7f2d140cf120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:00.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.963+0000 7f2d1916f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d140c57a0 con 0x7f2d140ad640 2023-12-02T00:31:00.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.963+0000 7f2d1916f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d140c4b90 con 0x7f2d140aeb80 2023-12-02T00:31:00.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.963+0000 7f2d1916f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d140c4e70 con 0x7f2d1415d340 2023-12-02T00:31:00.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.963+0000 7f2d12d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d140ad640 0x7f2d140c5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:00.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.963+0000 7f2d12d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d140ad640 0x7f2d140c5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37454/0 (socket says 172.21.15.46:37454) 2023-12-02T00:31:00.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.963+0000 7f2d12d9d640 1 -- 172.21.15.46:0/1399752735 learned_addr learned my addr 172.21.15.46:0/1399752735 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:00.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.964+0000 7f2d0bfff640 1 --2- 172.21.15.46:0/1399752735 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d140aeb80 0x7f2d141513d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:00.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.964+0000 7f2d1359e640 1 --2- 172.21.15.46:0/1399752735 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d1415d340 0x7f2d140cf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:00.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.964+0000 7f2d12d9d640 1 -- 172.21.15.46:0/1399752735 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d1415d340 msgr2=0x7f2d140cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:00.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.964+0000 7f2d12d9d640 1 --2- 172.21.15.46:0/1399752735 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d1415d340 0x7f2d140cf120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:00.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.964+0000 7f2d12d9d640 1 -- 172.21.15.46:0/1399752735 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d140aeb80 msgr2=0x7f2d141513d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:00.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.964+0000 7f2d12d9d640 1 --2- 172.21.15.46:0/1399752735 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d140aeb80 0x7f2d141513d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:00.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.964+0000 7f2d12d9d640 1 -- 172.21.15.46:0/1399752735 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d04009400 con 0x7f2d140ad640 2023-12-02T00:31:00.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.964+0000 7f2d12d9d640 1 --2- 172.21.15.46:0/1399752735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d140ad640 0x7f2d140c5c30 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f2d04000c00 tx=0x7f2d040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:00.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.964+0000 7f2d0b7fe640 1 -- 172.21.15.46:0/1399752735 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d04004230 con 0x7f2d140ad640 2023-12-02T00:31:00.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.964+0000 7f2d0b7fe640 1 -- 172.21.15.46:0/1399752735 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d04004390 con 0x7f2d140ad640 2023-12-02T00:31:00.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.964+0000 7f2d1916f640 1 -- 172.21.15.46:0/1399752735 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d140c5150 con 0x7f2d140ad640 2023-12-02T00:31:00.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.964+0000 7f2d0b7fe640 1 -- 172.21.15.46:0/1399752735 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0403a870 con 0x7f2d140ad640 2023-12-02T00:31:00.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.964+0000 7f2d1916f640 1 -- 172.21.15.46:0/1399752735 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d14151910 con 0x7f2d140ad640 2023-12-02T00:31:00.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.965+0000 7f2d1916f640 1 -- 172.21.15.46:0/1399752735 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d14069400 con 0x7f2d140ad640 2023-12-02T00:31:00.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.966+0000 7f2d0b7fe640 1 -- 172.21.15.46:0/1399752735 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2d0403a9d0 con 0x7f2d140ad640 2023-12-02T00:31:00.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.966+0000 7f2d0b7fe640 1 --2- 172.21.15.46:0/1399752735 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2cf406ae40 0x7f2cf406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:00.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.966+0000 7f2d0b7fe640 1 -- 172.21.15.46:0/1399752735 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f2d040ab730 con 0x7f2d140ad640 2023-12-02T00:31:00.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.966+0000 7f2d0bfff640 1 --2- 172.21.15.46:0/1399752735 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2cf406ae40 0x7f2cf406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:00.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.967+0000 7f2d0bfff640 1 --2- 172.21.15.46:0/1399752735 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2cf406ae40 0x7f2cf406d300 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2d1406e000 tx=0x7f2cfc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:00.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:00.971+0000 7f2d0b7fe640 1 -- 172.21.15.46:0/1399752735 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2d04076860 con 0x7f2d140ad640 2023-12-02T00:31:01.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.185+0000 7f2d1916f640 1 -- 172.21.15.46:0/1399752735 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f2d1415fb60 con 0x7f2d140ad640 2023-12-02T00:31:01.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.835+0000 7f2d0b7fe640 1 -- 172.21.15.46:0/1399752735 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v20) v1 ==== 172+0+0 (secure 0 0 0) 0x7f2d04035020 con 0x7f2d140ad640 2023-12-02T00:31:01.837 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 7 and data pool 8 2023-12-02T00:31:01.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.844+0000 7f2d1916f640 1 -- 172.21.15.46:0/1399752735 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2cf406ae40 msgr2=0x7f2cf406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:01.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.844+0000 7f2d1916f640 1 --2- 172.21.15.46:0/1399752735 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2cf406ae40 0x7f2cf406d300 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2d1406e000 tx=0x7f2cfc008040 comp rx=0 tx=0).stop 2023-12-02T00:31:01.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.844+0000 7f2d1916f640 1 -- 172.21.15.46:0/1399752735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d140ad640 msgr2=0x7f2d140c5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:01.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.844+0000 7f2d1916f640 1 --2- 172.21.15.46:0/1399752735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d140ad640 0x7f2d140c5c30 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f2d04000c00 tx=0x7f2d040046c0 comp rx=0 tx=0).stop 2023-12-02T00:31:01.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.845+0000 7f2d1916f640 1 -- 172.21.15.46:0/1399752735 shutdown_connections 2023-12-02T00:31:01.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.845+0000 7f2d1916f640 1 --2- 172.21.15.46:0/1399752735 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2cf406ae40 0x7f2cf406d300 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:01.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.845+0000 7f2d1916f640 1 --2- 172.21.15.46:0/1399752735 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d1415d340 0x7f2d140cf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:01.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.845+0000 7f2d1916f640 1 --2- 172.21.15.46:0/1399752735 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d140aeb80 0x7f2d141513d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:01.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.845+0000 7f2d1916f640 1 --2- 172.21.15.46:0/1399752735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d140ad640 0x7f2d140c5c30 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:01.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.845+0000 7f2d1916f640 1 -- 172.21.15.46:0/1399752735 >> 172.21.15.46:0/1399752735 conn(0x7f2d14070e70 msgr2=0x7f2d1406d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:01.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.845+0000 7f2d1916f640 1 -- 172.21.15.46:0/1399752735 shutdown_connections 2023-12-02T00:31:01.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.845+0000 7f2d1916f640 1 -- 172.21.15.46:0/1399752735 wait complete. 2023-12-02T00:31:01.871 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:31:01.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.982+0000 7f3fcdb2c640 1 Processor -- start 2023-12-02T00:31:01.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.982+0000 7f3fcdb2c640 1 -- start start 2023-12-02T00:31:01.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.982+0000 7f3fcdb2c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3fc80d2910 0x7f3fc80d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:01.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.982+0000 7f3fcdb2c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3fc80d3730 0x7f3fc80b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:01.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.982+0000 7f3fcdb2c640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fc80b2b10 con 0x7f3fc815d330 2023-12-02T00:31:01.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.982+0000 7f3fcdb2c640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fc80d3bc0 con 0x7f3fc80d2910 2023-12-02T00:31:01.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.982+0000 7f3fcdb2c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fc80d1590 con 0x7f3fc80d3730 2023-12-02T00:31:01.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.983+0000 7f3fc6ffd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3fc80d2910 0x7f3fc80d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:01.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.983+0000 7f3fc6ffd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3fc80d2910 0x7f3fc80d31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37460/0 (socket says 172.21.15.46:37460) 2023-12-02T00:31:01.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.983+0000 7f3fc6ffd640 1 -- 172.21.15.46:0/1548242158 learned_addr learned my addr 172.21.15.46:0/1548242158 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:01.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.983+0000 7f3fc7fff640 1 --2- 172.21.15.46:0/1548242158 >> v2:172.21.15.162:3300/0 conn(0x7f3fc80d3730 0x7f3fc80b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:01.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.983+0000 7f3fc6ffd640 1 -- 172.21.15.46:0/1548242158 >> v2:172.21.15.162:3300/0 conn(0x7f3fc80d3730 msgr2=0x7f3fc80b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:01.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.983+0000 7f3fc6ffd640 1 --2- 172.21.15.46:0/1548242158 >> v2:172.21.15.162:3300/0 conn(0x7f3fc80d3730 0x7f3fc80b6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:01.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.983+0000 7f3fc6ffd640 1 -- 172.21.15.46:0/1548242158 >> v1:172.21.15.162:6789/0 conn(0x7f3fc815d330 legacy=0x7f3fc80b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:01.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.983+0000 7f3fc6ffd640 1 -- 172.21.15.46:0/1548242158 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc80d1870 con 0x7f3fc80d2910 2023-12-02T00:31:01.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.983+0000 7f3fc7fff640 1 --2- 172.21.15.46:0/1548242158 >> v2:172.21.15.162:3300/0 conn(0x7f3fc80d3730 0x7f3fc80b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:31:01.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.984+0000 7f3fc6ffd640 1 --2- 172.21.15.46:0/1548242158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fc80d2910 0x7f3fc80d31f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f3fc806c500 tx=0x7f3fbc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb70344b1935112 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:01.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.984+0000 7f3fc67fc640 1 -- 172.21.15.46:0/1548242158 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc031870 con 0x7f3fc80d2910 2023-12-02T00:31:01.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.984+0000 7f3fc67fc640 1 -- 172.21.15.46:0/1548242158 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fbc0319d0 con 0x7f3fc80d2910 2023-12-02T00:31:01.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.984+0000 7f3fc67fc640 1 -- 172.21.15.46:0/1548242158 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc031d60 con 0x7f3fc80d2910 2023-12-02T00:31:01.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.984+0000 7f3fcdb2c640 1 -- 172.21.15.46:0/1548242158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fc80d2910 msgr2=0x7f3fc80d31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:01.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.984+0000 7f3fcdb2c640 1 --2- 172.21.15.46:0/1548242158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fc80d2910 0x7f3fc80d31f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f3fc806c500 tx=0x7f3fbc02b910 comp rx=0 tx=0).stop 2023-12-02T00:31:01.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.984+0000 7f3fcdb2c640 1 -- 172.21.15.46:0/1548242158 shutdown_connections 2023-12-02T00:31:01.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.984+0000 7f3fcdb2c640 1 --2- 172.21.15.46:0/1548242158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fc80d2910 0x7f3fc80d31f0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:01.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.984+0000 7f3fcdb2c640 1 --2- 172.21.15.46:0/1548242158 >> v2:172.21.15.162:3300/0 conn(0x7f3fc80d3730 0x7f3fc80b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:01.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.984+0000 7f3fcdb2c640 1 -- 172.21.15.46:0/1548242158 >> 172.21.15.46:0/1548242158 conn(0x7f3fc8072020 msgr2=0x7f3fc8072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:01.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.984+0000 7f3fcdb2c640 1 -- 172.21.15.46:0/1548242158 shutdown_connections 2023-12-02T00:31:01.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.984+0000 7f3fcdb2c640 1 -- 172.21.15.46:0/1548242158 wait complete. 2023-12-02T00:31:01.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.986+0000 7f3fcdb2c640 1 Processor -- start 2023-12-02T00:31:01.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.986+0000 7f3fcdb2c640 1 -- start start 2023-12-02T00:31:01.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.987+0000 7f3fcdb2c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3fc80d2910 0x7f3fc80cbd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:01.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.987+0000 7f3fcdb2c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3fc80d3730 0x7f3fc80c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:01.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.987+0000 7f3fcdb2c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fc815d330 0x7f3fc8153290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:01.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.987+0000 7f3fcdb2c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc80c6cd0 con 0x7f3fc815d330 2023-12-02T00:31:01.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.987+0000 7f3fcdb2c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc80cc380 con 0x7f3fc80d2910 2023-12-02T00:31:01.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.987+0000 7f3fcdb2c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc80cc660 con 0x7f3fc80d3730 2023-12-02T00:31:01.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.987+0000 7f3fc7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fc815d330 0x7f3fc8153290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:01.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.987+0000 7f3fc7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fc815d330 0x7f3fc8153290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37466/0 (socket says 172.21.15.46:37466) 2023-12-02T00:31:01.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.987+0000 7f3fc7fff640 1 -- 172.21.15.46:0/1784721362 learned_addr learned my addr 172.21.15.46:0/1784721362 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:01.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.987+0000 7f3fc6ffd640 1 --2- 172.21.15.46:0/1784721362 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3fc80d3730 0x7f3fc80c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:01.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.987+0000 7f3fc77fe640 1 --2- 172.21.15.46:0/1784721362 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3fc80d2910 0x7f3fc80cbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:01.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.987+0000 7f3fc7fff640 1 -- 172.21.15.46:0/1784721362 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3fc80d3730 msgr2=0x7f3fc80c8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:01.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.987+0000 7f3fc7fff640 1 --2- 172.21.15.46:0/1784721362 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3fc80d3730 0x7f3fc80c8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:01.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.987+0000 7f3fc7fff640 1 -- 172.21.15.46:0/1784721362 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3fc80d2910 msgr2=0x7f3fc80cbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:01.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.987+0000 7f3fc7fff640 1 --2- 172.21.15.46:0/1784721362 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3fc80d2910 0x7f3fc80cbd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:01.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.987+0000 7f3fc7fff640 1 -- 172.21.15.46:0/1784721362 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fbc0094d0 con 0x7f3fc815d330 2023-12-02T00:31:01.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.987+0000 7f3fc7fff640 1 --2- 172.21.15.46:0/1784721362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fc815d330 0x7f3fc8153290 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0007ef0 tx=0x7f3fc000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:01.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.987+0000 7f3faffff640 1 -- 172.21.15.46:0/1784721362 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc0003b90 con 0x7f3fc815d330 2023-12-02T00:31:01.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.987+0000 7f3faffff640 1 -- 172.21.15.46:0/1784721362 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fc000f040 con 0x7f3fc815d330 2023-12-02T00:31:01.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.987+0000 7f3fcdb2c640 1 -- 172.21.15.46:0/1784721362 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fc80cc940 con 0x7f3fc815d330 2023-12-02T00:31:01.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.987+0000 7f3faffff640 1 -- 172.21.15.46:0/1784721362 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc000de80 con 0x7f3fc815d330 2023-12-02T00:31:01.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.987+0000 7f3fcdb2c640 1 -- 172.21.15.46:0/1784721362 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fc80c86c0 con 0x7f3fc815d330 2023-12-02T00:31:01.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.989+0000 7f3fcdb2c640 1 -- 172.21.15.46:0/1784721362 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fc80c6e40 con 0x7f3fc815d330 2023-12-02T00:31:01.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.989+0000 7f3faffff640 1 -- 172.21.15.46:0/1784721362 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3fc0004030 con 0x7f3fc815d330 2023-12-02T00:31:01.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.990+0000 7f3faffff640 1 --2- 172.21.15.46:0/1784721362 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3fb406ae70 0x7f3fb406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:01.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.990+0000 7f3faffff640 1 -- 172.21.15.46:0/1784721362 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3fc0089940 con 0x7f3fc815d330 2023-12-02T00:31:01.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.990+0000 7f3fc77fe640 1 --2- 172.21.15.46:0/1784721362 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3fb406ae70 0x7f3fb406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:01.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.990+0000 7f3fc77fe640 1 --2- 172.21.15.46:0/1784721362 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3fb406ae70 0x7f3fb406d330 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8000f60 tx=0x7f3fb8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:01.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:01.994+0000 7f3faffff640 1 -- 172.21.15.46:0/1784721362 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3fc80c6e40 con 0x7f3fc815d330 2023-12-02T00:31:02.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.173+0000 7f3fcdb2c640 1 -- 172.21.15.46:0/1784721362 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3fc80c6e10 con 0x7f3fc815d330 2023-12-02T00:31:02.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.174+0000 7f3faffff640 1 -- 172.21.15.46:0/1784721362 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+16558 (secure 0 0 0) 0x7f3fc001a020 con 0x7f3fc815d330 2023-12-02T00:31:02.177 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:31:02.177 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":50,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:31:01.187236+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-12-02T00:30:59.469070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-12-02T00:31:00.174610+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:31:02.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.176+0000 7f3fcdb2c640 1 -- 172.21.15.46:0/1784721362 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3fb406ae70 msgr2=0x7f3fb406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:02.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.176+0000 7f3fcdb2c640 1 --2- 172.21.15.46:0/1784721362 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3fb406ae70 0x7f3fb406d330 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8000f60 tx=0x7f3fb8009040 comp rx=0 tx=0).stop 2023-12-02T00:31:02.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.176+0000 7f3fcdb2c640 1 -- 172.21.15.46:0/1784721362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fc815d330 msgr2=0x7f3fc8153290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:02.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.176+0000 7f3fcdb2c640 1 --2- 172.21.15.46:0/1784721362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fc815d330 0x7f3fc8153290 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0007ef0 tx=0x7f3fc000d4a0 comp rx=0 tx=0).stop 2023-12-02T00:31:02.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.176+0000 7f3fcdb2c640 1 -- 172.21.15.46:0/1784721362 shutdown_connections 2023-12-02T00:31:02.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.176+0000 7f3fcdb2c640 1 --2- 172.21.15.46:0/1784721362 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3fb406ae70 0x7f3fb406d330 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:02.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.176+0000 7f3fcdb2c640 1 --2- 172.21.15.46:0/1784721362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fc815d330 0x7f3fc8153290 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:02.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.176+0000 7f3fcdb2c640 1 --2- 172.21.15.46:0/1784721362 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3fc80d3730 0x7f3fc80c8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:02.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.176+0000 7f3fcdb2c640 1 --2- 172.21.15.46:0/1784721362 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3fc80d2910 0x7f3fc80cbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:02.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.176+0000 7f3fcdb2c640 1 -- 172.21.15.46:0/1784721362 >> 172.21.15.46:0/1784721362 conn(0x7f3fc8072020 msgr2=0x7f3fc80b5550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:02.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.176+0000 7f3fcdb2c640 1 -- 172.21.15.46:0/1784721362 shutdown_connections 2023-12-02T00:31:02.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.176+0000 7f3fcdb2c640 1 -- 172.21.15.46:0/1784721362 wait complete. 2023-12-02T00:31:02.190 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:31:02.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.294+0000 7fcdfd836640 1 Processor -- start 2023-12-02T00:31:02.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.294+0000 7fcdfd836640 1 -- start start 2023-12-02T00:31:02.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.295+0000 7fcdfd836640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcdf80ceb70 0x7fcdf80cc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:02.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.295+0000 7fcdfd836640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fcdf80b4370 0x7fcdf80b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:02.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.295+0000 7fcdfd836640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdf8071770 con 0x7fcdf8144980 2023-12-02T00:31:02.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.295+0000 7fcdfd836640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdf815ced0 con 0x7fcdf80b4370 2023-12-02T00:31:02.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.295+0000 7fcdfd836640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf80b4d20 con 0x7fcdf80ceb70 2023-12-02T00:31:02.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.295+0000 7fcdf6ffd640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fcdf8144980 0x7fcdf815c7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53816/0 (socket says 172.21.15.46:53816) 2023-12-02T00:31:02.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.295+0000 7fcdf6ffd640 1 -- 172.21.15.46:0/3505508752 learned_addr learned my addr 172.21.15.46:0/3505508752 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:31:02.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.295+0000 7fcdf5ffb640 1 -- 172.21.15.46:0/3505508752 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2413583046 0 0) 0x7fcdf8071770 con 0x7fcdf8144980 2023-12-02T00:31:02.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.295+0000 7fcdf67fc640 1 --2- 172.21.15.46:0/3505508752 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcdf80ceb70 0x7fcdf80cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:02.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.295+0000 7fcdf5ffb640 1 -- 172.21.15.46:0/3505508752 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcde4003520 con 0x7fcdf8144980 2023-12-02T00:31:02.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.295+0000 7fcdf77fe640 1 --2- 172.21.15.46:0/3505508752 >> v2:172.21.15.162:3300/0 conn(0x7fcdf80b4370 0x7fcdf80b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:02.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.295+0000 7fcdf5ffb640 1 -- 172.21.15.46:0/3505508752 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1471366987 0 0) 0x7fcde4003520 con 0x7fcdf8144980 2023-12-02T00:31:02.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.295+0000 7fcdf5ffb640 1 -- 172.21.15.46:0/3505508752 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcdf80ceb70 msgr2=0x7fcdf80cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:02.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.295+0000 7fcdf5ffb640 1 --2- 172.21.15.46:0/3505508752 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcdf80ceb70 0x7fcdf80cc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:02.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.295+0000 7fcdf5ffb640 1 -- 172.21.15.46:0/3505508752 >> v2:172.21.15.162:3300/0 conn(0x7fcdf80b4370 msgr2=0x7fcdf80b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:02.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.295+0000 7fcdf5ffb640 1 --2- 172.21.15.46:0/3505508752 >> v2:172.21.15.162:3300/0 conn(0x7fcdf80b4370 0x7fcdf80b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:02.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.295+0000 7fcdf5ffb640 1 -- 172.21.15.46:0/3505508752 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf80ccdc0 con 0x7fcdf8144980 2023-12-02T00:31:02.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.295+0000 7fcdf5ffb640 1 -- 172.21.15.46:0/3505508752 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7fcdf0001670 con 0x7fcdf8144980 2023-12-02T00:31:02.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.296+0000 7fcdf5ffb640 1 -- 172.21.15.46:0/3505508752 >> v1:172.21.15.46:6789/0 conn(0x7fcdf8144980 legacy=0x7fcdf815c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:02.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.296+0000 7fcdf5ffb640 1 --2- 172.21.15.46:0/3505508752 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcdf80ceb70 0x7fcde40039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:02.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.296+0000 7fcdf5ffb640 1 --2- 172.21.15.46:0/3505508752 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcde4003fd0 0x7fcde40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:02.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.296+0000 7fcdf5ffb640 1 --2- 172.21.15.46:0/3505508752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcde4004d00 0x7fcde4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:02.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.296+0000 7fcdf5ffb640 1 -- 172.21.15.46:0/3505508752 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf815ced0 con 0x7fcde4004d00 2023-12-02T00:31:02.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.296+0000 7fcdf5ffb640 1 -- 172.21.15.46:0/3505508752 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf80b4d20 con 0x7fcde4003fd0 2023-12-02T00:31:02.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.296+0000 7fcdf5ffb640 1 -- 172.21.15.46:0/3505508752 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcde400bab0 con 0x7fcdf80ceb70 2023-12-02T00:31:02.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.296+0000 7fcdf77fe640 1 --2- 172.21.15.46:0/3505508752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcde4004d00 0x7fcde4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:02.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.296+0000 7fcdf6ffd640 1 --2- 172.21.15.46:0/3505508752 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcdf80ceb70 0x7fcde40039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:02.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.296+0000 7fcdf67fc640 1 --2- 172.21.15.46:0/3505508752 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcde4003fd0 0x7fcde40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:02.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.296+0000 7fcdf77fe640 1 -- 172.21.15.46:0/3505508752 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcdf80ceb70 msgr2=0x7fcde40039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:02.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.296+0000 7fcdf77fe640 1 --2- 172.21.15.46:0/3505508752 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcdf80ceb70 0x7fcde40039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:02.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.296+0000 7fcdf77fe640 1 -- 172.21.15.46:0/3505508752 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcde4003fd0 msgr2=0x7fcde40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:02.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.296+0000 7fcdf77fe640 1 --2- 172.21.15.46:0/3505508752 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcde4003fd0 0x7fcde40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:02.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.296+0000 7fcdf77fe640 1 -- 172.21.15.46:0/3505508752 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde4003520 con 0x7fcde4004d00 2023-12-02T00:31:02.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.297+0000 7fcdf77fe640 1 --2- 172.21.15.46:0/3505508752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcde4004d00 0x7fcde4005180 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fcde8007360 tx=0x7fcde40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2028331dddc8485 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:02.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.297+0000 7fcdf5ffb640 1 -- 172.21.15.46:0/3505508752 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde802ccf0 con 0x7fcde4004d00 2023-12-02T00:31:02.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.297+0000 7fcdf5ffb640 1 -- 172.21.15.46:0/3505508752 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde802ce50 con 0x7fcde4004d00 2023-12-02T00:31:02.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.297+0000 7fcdf5ffb640 1 -- 172.21.15.46:0/3505508752 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde8034660 con 0x7fcde4004d00 2023-12-02T00:31:02.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.297+0000 7fcdfd836640 1 -- 172.21.15.46:0/3505508752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcde4004d00 msgr2=0x7fcde4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:02.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.297+0000 7fcdfd836640 1 --2- 172.21.15.46:0/3505508752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcde4004d00 0x7fcde4005180 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fcde8007360 tx=0x7fcde40057f0 comp rx=0 tx=0).stop 2023-12-02T00:31:02.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.297+0000 7fcdf77fe640 1 -- 172.21.15.46:0/3505508752 reap_dead start 2023-12-02T00:31:02.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.297+0000 7fcdfd836640 1 -- 172.21.15.46:0/3505508752 shutdown_connections 2023-12-02T00:31:02.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.297+0000 7fcdfd836640 1 -- 172.21.15.46:0/3505508752 >> 172.21.15.46:0/3505508752 conn(0x7fcdf806a4d0 msgr2=0x7fcdf806a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:02.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.298+0000 7fcdfd836640 1 -- 172.21.15.46:0/3505508752 shutdown_connections 2023-12-02T00:31:02.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.298+0000 7fcdfd836640 1 -- 172.21.15.46:0/3505508752 wait complete. 2023-12-02T00:31:02.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.299+0000 7fcdfd836640 1 Processor -- start 2023-12-02T00:31:02.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.299+0000 7fcdfd836640 1 -- start start 2023-12-02T00:31:02.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.299+0000 7fcdfd836640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcdf8156cf0 0x7fcdf8131a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:02.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.300+0000 7fcdfd836640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcdf8140c20 0x7fcdf81570d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:02.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.300+0000 7fcdfd836640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdf81f4720 0x7fcdf8136f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:02.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.300+0000 7fcdfd836640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf80c9a70 con 0x7fcdf81f4720 2023-12-02T00:31:02.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.300+0000 7fcdfd836640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf8157610 con 0x7fcdf8156cf0 2023-12-02T00:31:02.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.300+0000 7fcdfd836640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf81f4fe0 con 0x7fcdf8140c20 2023-12-02T00:31:02.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.300+0000 7fcdf77fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdf81f4720 0x7fcdf8136f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:02.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.300+0000 7fcdf77fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdf81f4720 0x7fcdf8136f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37480/0 (socket says 172.21.15.46:37480) 2023-12-02T00:31:02.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.300+0000 7fcdf77fe640 1 -- 172.21.15.46:0/4225445205 learned_addr learned my addr 172.21.15.46:0/4225445205 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:02.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.300+0000 7fcdf67fc640 1 --2- 172.21.15.46:0/4225445205 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcdf8140c20 0x7fcdf81570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:02.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.300+0000 7fcdf6ffd640 1 --2- 172.21.15.46:0/4225445205 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcdf8156cf0 0x7fcdf8131a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:02.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.300+0000 7fcdf77fe640 1 -- 172.21.15.46:0/4225445205 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcdf8140c20 msgr2=0x7fcdf81570d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:02.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.300+0000 7fcdf77fe640 1 --2- 172.21.15.46:0/4225445205 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcdf8140c20 0x7fcdf81570d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:02.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.300+0000 7fcdf77fe640 1 -- 172.21.15.46:0/4225445205 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcdf8156cf0 msgr2=0x7fcdf8131a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:02.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.300+0000 7fcdf77fe640 1 --2- 172.21.15.46:0/4225445205 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcdf8156cf0 0x7fcdf8131a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:02.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.300+0000 7fcdf77fe640 1 -- 172.21.15.46:0/4225445205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf81f52c0 con 0x7fcdf81f4720 2023-12-02T00:31:02.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.300+0000 7fcdf67fc640 1 --2- 172.21.15.46:0/4225445205 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcdf8140c20 0x7fcdf81570d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:31:02.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.300+0000 7fcdf77fe640 1 --2- 172.21.15.46:0/4225445205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdf81f4720 0x7fcdf8136f50 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fcde802dfd0 tx=0x7fcde8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:02.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.300+0000 7fcdeffff640 1 -- 172.21.15.46:0/4225445205 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde803e400 con 0x7fcdf81f4720 2023-12-02T00:31:02.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.300+0000 7fcdeffff640 1 -- 172.21.15.46:0/4225445205 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde803e580 con 0x7fcdf81f4720 2023-12-02T00:31:02.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.300+0000 7fcdfd836640 1 -- 172.21.15.46:0/4225445205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdf80c9ce0 con 0x7fcdf81f4720 2023-12-02T00:31:02.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.300+0000 7fcdeffff640 1 -- 172.21.15.46:0/4225445205 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde803e930 con 0x7fcdf81f4720 2023-12-02T00:31:02.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.300+0000 7fcdfd836640 1 -- 172.21.15.46:0/4225445205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf81f55a0 con 0x7fcdf81f4720 2023-12-02T00:31:02.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.302+0000 7fcdfd836640 1 -- 172.21.15.46:0/4225445205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdf80c98d0 con 0x7fcdf81f4720 2023-12-02T00:31:02.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.302+0000 7fcdeffff640 1 -- 172.21.15.46:0/4225445205 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcde803ea90 con 0x7fcdf81f4720 2023-12-02T00:31:02.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.302+0000 7fcdeffff640 1 --2- 172.21.15.46:0/4225445205 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fcdd806ae40 0x7fcdd806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:02.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.302+0000 7fcdeffff640 1 -- 172.21.15.46:0/4225445205 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcde80ab950 con 0x7fcdf81f4720 2023-12-02T00:31:02.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.303+0000 7fcdf6ffd640 1 --2- 172.21.15.46:0/4225445205 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fcdd806ae40 0x7fcdd806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:02.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.303+0000 7fcdf6ffd640 1 --2- 172.21.15.46:0/4225445205 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fcdd806ae40 0x7fcdd806d300 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0004840 tx=0x7fcdf0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:02.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.306+0000 7fcdeffff640 1 -- 172.21.15.46:0/4225445205 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcdf80c98d0 con 0x7fcdf81f4720 2023-12-02T00:31:02.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.485+0000 7fcdfd836640 1 -- 172.21.15.46:0/4225445205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcdf80c98d0 con 0x7fcdf81f4720 2023-12-02T00:31:02.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.486+0000 7fcdeffff640 1 -- 172.21.15.46:0/4225445205 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+16558 (secure 0 0 0) 0x7fcdf80c98d0 con 0x7fcdf81f4720 2023-12-02T00:31:02.488 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:31:02.489 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":50,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:31:01.187236+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-12-02T00:30:59.469070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-12-02T00:31:00.174610+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:31:02.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.488+0000 7fcdfd836640 1 -- 172.21.15.46:0/4225445205 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fcdd806ae40 msgr2=0x7fcdd806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:02.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.488+0000 7fcdfd836640 1 --2- 172.21.15.46:0/4225445205 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fcdd806ae40 0x7fcdd806d300 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0004840 tx=0x7fcdf0009040 comp rx=0 tx=0).stop 2023-12-02T00:31:02.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.488+0000 7fcdfd836640 1 -- 172.21.15.46:0/4225445205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdf81f4720 msgr2=0x7fcdf8136f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:02.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.488+0000 7fcdfd836640 1 --2- 172.21.15.46:0/4225445205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdf81f4720 0x7fcdf8136f50 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fcde802dfd0 tx=0x7fcde8004ab0 comp rx=0 tx=0).stop 2023-12-02T00:31:02.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.488+0000 7fcdfd836640 1 -- 172.21.15.46:0/4225445205 shutdown_connections 2023-12-02T00:31:02.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.488+0000 7fcdfd836640 1 --2- 172.21.15.46:0/4225445205 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fcdd806ae40 0x7fcdd806d300 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:02.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.488+0000 7fcdfd836640 1 --2- 172.21.15.46:0/4225445205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdf81f4720 0x7fcdf8136f50 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:02.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.488+0000 7fcdfd836640 1 --2- 172.21.15.46:0/4225445205 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcdf8140c20 0x7fcdf81570d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:02.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.488+0000 7fcdfd836640 1 --2- 172.21.15.46:0/4225445205 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcdf8156cf0 0x7fcdf8131a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:02.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.488+0000 7fcdfd836640 1 -- 172.21.15.46:0/4225445205 >> 172.21.15.46:0/4225445205 conn(0x7fcdf806a4d0 msgr2=0x7fcdf80cb250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:02.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.488+0000 7fcdfd836640 1 -- 172.21.15.46:0/4225445205 shutdown_connections 2023-12-02T00:31:02.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.488+0000 7fcdfd836640 1 -- 172.21.15.46:0/4225445205 wait complete. 2023-12-02T00:31:02.502 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-02T00:31:02.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.609+0000 7fafed39e640 1 Processor -- start 2023-12-02T00:31:02.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.609+0000 7fafed39e640 1 -- start start 2023-12-02T00:31:02.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.609+0000 7fafed39e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fafe81415f0 0x7fafe8141e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:02.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.609+0000 7fafed39e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fafe8144a00 0x7fafe806b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:02.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.609+0000 7fafed39e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fafe8144440 0x7fafe806ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:02.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.609+0000 7fafed39e640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafe80b2eb0 con 0x7fafe8144440 2023-12-02T00:31:02.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.609+0000 7fafed39e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafe80b3190 con 0x7fafe8144a00 2023-12-02T00:31:02.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.609+0000 7fafed39e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafe80b3470 con 0x7fafe81415f0 2023-12-02T00:31:02.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.610+0000 7fafe77fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fafe8144440 0x7fafe806ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:02.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.610+0000 7fafe77fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fafe8144440 0x7fafe806ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59620/0 (socket says 172.21.15.46:59620) 2023-12-02T00:31:02.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.610+0000 7fafe77fe640 1 -- 172.21.15.46:0/1947550767 learned_addr learned my addr 172.21.15.46:0/1947550767 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:02.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.610+0000 7fafe6ffd640 1 --2- 172.21.15.46:0/1947550767 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fafe81415f0 0x7fafe8141e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:02.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.610+0000 7fafe67fc640 1 --2- 172.21.15.46:0/1947550767 >> v2:172.21.15.162:3300/0 conn(0x7fafe8144a00 0x7fafe806b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:02.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.610+0000 7fafe77fe640 1 -- 172.21.15.46:0/1947550767 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fafe81415f0 msgr2=0x7fafe8141e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:02.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.610+0000 7fafe77fe640 1 --2- 172.21.15.46:0/1947550767 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fafe81415f0 0x7fafe8141e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:02.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.610+0000 7fafe77fe640 1 -- 172.21.15.46:0/1947550767 >> v2:172.21.15.162:3300/0 conn(0x7fafe8144a00 msgr2=0x7fafe806b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:02.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.610+0000 7fafe77fe640 1 --2- 172.21.15.46:0/1947550767 >> v2:172.21.15.162:3300/0 conn(0x7fafe8144a00 0x7fafe806b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:02.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.610+0000 7fafe77fe640 1 -- 172.21.15.46:0/1947550767 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafe80b3750 con 0x7fafe8144440 2023-12-02T00:31:02.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.610+0000 7fafe6ffd640 1 --2- 172.21.15.46:0/1947550767 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fafe81415f0 0x7fafe8141e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:31:02.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.611+0000 7fafe77fe640 1 --2- 172.21.15.46:0/1947550767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafe8144440 0x7fafe806ba20 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fafe80b2860 tx=0x7fafe002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c35d7d93504bde60 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:02.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.611+0000 7fafe5ffb640 1 -- 172.21.15.46:0/1947550767 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe002ca70 con 0x7fafe8144440 2023-12-02T00:31:02.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.611+0000 7fafe5ffb640 1 -- 172.21.15.46:0/1947550767 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafe002cbd0 con 0x7fafe8144440 2023-12-02T00:31:02.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.611+0000 7fafe5ffb640 1 -- 172.21.15.46:0/1947550767 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe00343f0 con 0x7fafe8144440 2023-12-02T00:31:02.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.611+0000 7fafed39e640 1 -- 172.21.15.46:0/1947550767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafe8144440 msgr2=0x7fafe806ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:02.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.611+0000 7fafed39e640 1 --2- 172.21.15.46:0/1947550767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafe8144440 0x7fafe806ba20 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fafe80b2860 tx=0x7fafe002bed0 comp rx=0 tx=0).stop 2023-12-02T00:31:02.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.611+0000 7fafed39e640 1 -- 172.21.15.46:0/1947550767 shutdown_connections 2023-12-02T00:31:02.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.611+0000 7fafed39e640 1 --2- 172.21.15.46:0/1947550767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafe8144440 0x7fafe806ba20 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:02.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.611+0000 7fafed39e640 1 --2- 172.21.15.46:0/1947550767 >> v2:172.21.15.162:3300/0 conn(0x7fafe8144a00 0x7fafe806b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:02.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.611+0000 7fafed39e640 1 --2- 172.21.15.46:0/1947550767 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fafe81415f0 0x7fafe8141e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:02.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.611+0000 7fafed39e640 1 -- 172.21.15.46:0/1947550767 >> 172.21.15.46:0/1947550767 conn(0x7fafe8070e70 msgr2=0x7fafe8071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:02.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.611+0000 7fafed39e640 1 -- 172.21.15.46:0/1947550767 shutdown_connections 2023-12-02T00:31:02.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.611+0000 7fafed39e640 1 -- 172.21.15.46:0/1947550767 wait complete. 2023-12-02T00:31:02.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.613+0000 7fafed39e640 1 Processor -- start 2023-12-02T00:31:02.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.613+0000 7fafed39e640 1 -- start start 2023-12-02T00:31:02.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.613+0000 7fafed39e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafe81415f0 0x7fafe806d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:02.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.613+0000 7fafed39e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafe8144440 0x7fafe8154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:02.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.613+0000 7fafed39e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fafe8144a00 0x7fafe80ae4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:02.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.614+0000 7fafed39e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe813ebe0 con 0x7fafe8144440 2023-12-02T00:31:02.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.614+0000 7fafed39e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe813ee80 con 0x7fafe81415f0 2023-12-02T00:31:02.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.614+0000 7fafed39e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafe813f160 con 0x7fafe8144a00 2023-12-02T00:31:02.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.614+0000 7fafe67fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafe8144440 0x7fafe8154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:02.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.614+0000 7fafe67fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafe8144440 0x7fafe8154430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59628/0 (socket says 172.21.15.46:59628) 2023-12-02T00:31:02.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.614+0000 7fafe67fc640 1 -- 172.21.15.46:0/4000832272 learned_addr learned my addr 172.21.15.46:0/4000832272 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:02.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.614+0000 7fafe77fe640 1 --2- 172.21.15.46:0/4000832272 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fafe8144a00 0x7fafe80ae4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:02.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.614+0000 7fafe6ffd640 1 --2- 172.21.15.46:0/4000832272 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafe81415f0 0x7fafe806d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:02.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.614+0000 7fafe67fc640 1 -- 172.21.15.46:0/4000832272 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fafe8144a00 msgr2=0x7fafe80ae4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:02.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.614+0000 7fafe67fc640 1 --2- 172.21.15.46:0/4000832272 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fafe8144a00 0x7fafe80ae4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:02.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.614+0000 7fafe67fc640 1 -- 172.21.15.46:0/4000832272 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafe81415f0 msgr2=0x7fafe806d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:02.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.614+0000 7fafe67fc640 1 --2- 172.21.15.46:0/4000832272 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafe81415f0 0x7fafe806d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:02.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.614+0000 7fafe67fc640 1 -- 172.21.15.46:0/4000832272 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafe0009b30 con 0x7fafe8144440 2023-12-02T00:31:02.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.614+0000 7fafe77fe640 1 --2- 172.21.15.46:0/4000832272 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fafe8144a00 0x7fafe80ae4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:31:02.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.614+0000 7fafe67fc640 1 --2- 172.21.15.46:0/4000832272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafe8144440 0x7fafe8154430 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fafdc00c7f0 tx=0x7fafdc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:02.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.614+0000 7fafcf7fe640 1 -- 172.21.15.46:0/4000832272 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafdc004220 con 0x7fafe8144440 2023-12-02T00:31:02.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.614+0000 7fafcf7fe640 1 -- 172.21.15.46:0/4000832272 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafdc0043a0 con 0x7fafe8144440 2023-12-02T00:31:02.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.614+0000 7fafcf7fe640 1 -- 172.21.15.46:0/4000832272 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafdc007860 con 0x7fafe8144440 2023-12-02T00:31:02.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.614+0000 7fafed39e640 1 -- 172.21.15.46:0/4000832272 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafe813f440 con 0x7fafe8144440 2023-12-02T00:31:02.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.614+0000 7fafed39e640 1 -- 172.21.15.46:0/4000832272 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafe813fa00 con 0x7fafe8144440 2023-12-02T00:31:02.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.615+0000 7fafed39e640 1 -- 172.21.15.46:0/4000832272 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafe80af9e0 con 0x7fafe8144440 2023-12-02T00:31:02.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.616+0000 7fafcf7fe640 1 -- 172.21.15.46:0/4000832272 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fafdc007bd0 con 0x7fafe8144440 2023-12-02T00:31:02.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.616+0000 7fafcf7fe640 1 --2- 172.21.15.46:0/4000832272 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fafd406ae40 0x7fafd406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:02.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.616+0000 7fafcf7fe640 1 -- 172.21.15.46:0/4000832272 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fafdc089500 con 0x7fafe8144440 2023-12-02T00:31:02.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.616+0000 7fafe6ffd640 1 --2- 172.21.15.46:0/4000832272 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fafd406ae40 0x7fafd406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:02.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.617+0000 7fafe6ffd640 1 --2- 172.21.15.46:0/4000832272 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fafd406ae40 0x7fafd406d300 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fafd800b7c0 tx=0x7fafd8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:02.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.620+0000 7fafcf7fe640 1 -- 172.21.15.46:0/4000832272 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fafdc054640 con 0x7fafe8144440 2023-12-02T00:31:02.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:02.837+0000 7fafed39e640 1 -- 172.21.15.46:0/4000832272 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fafe8069400 con 0x7fafe8144440 2023-12-02T00:31:03.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.846+0000 7fafcf7fe640 1 -- 172.21.15.46:0/4000832272 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v23) v1 ==== 118+0+0 (secure 0 0 0) 0x7fafdc057c60 con 0x7fafe8144440 2023-12-02T00:31:03.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.850+0000 7fafed39e640 1 -- 172.21.15.46:0/4000832272 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fafd406ae40 msgr2=0x7fafd406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:03.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.850+0000 7fafed39e640 1 --2- 172.21.15.46:0/4000832272 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fafd406ae40 0x7fafd406d300 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fafd800b7c0 tx=0x7fafd8008040 comp rx=0 tx=0).stop 2023-12-02T00:31:03.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.850+0000 7fafed39e640 1 -- 172.21.15.46:0/4000832272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafe8144440 msgr2=0x7fafe8154430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:03.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.850+0000 7fafed39e640 1 --2- 172.21.15.46:0/4000832272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafe8144440 0x7fafe8154430 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fafdc00c7f0 tx=0x7fafdc00ccb0 comp rx=0 tx=0).stop 2023-12-02T00:31:03.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.850+0000 7fafed39e640 1 -- 172.21.15.46:0/4000832272 shutdown_connections 2023-12-02T00:31:03.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.850+0000 7fafed39e640 1 --2- 172.21.15.46:0/4000832272 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fafd406ae40 0x7fafd406d300 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:03.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.850+0000 7fafed39e640 1 --2- 172.21.15.46:0/4000832272 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fafe8144a00 0x7fafe80ae4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:03.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.850+0000 7fafed39e640 1 --2- 172.21.15.46:0/4000832272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafe8144440 0x7fafe8154430 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:03.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.850+0000 7fafed39e640 1 --2- 172.21.15.46:0/4000832272 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafe81415f0 0x7fafe806d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:03.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.850+0000 7fafed39e640 1 -- 172.21.15.46:0/4000832272 >> 172.21.15.46:0/4000832272 conn(0x7fafe8070e70 msgr2=0x7fafe806b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:03.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.850+0000 7fafed39e640 1 -- 172.21.15.46:0/4000832272 shutdown_connections 2023-12-02T00:31:03.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.850+0000 7fafed39e640 1 -- 172.21.15.46:0/4000832272 wait complete. 2023-12-02T00:31:03.865 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-02T00:31:03.865 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-02T00:31:03.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.979+0000 7f3b950c2640 1 Processor -- start 2023-12-02T00:31:03.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.979+0000 7f3b950c2640 1 -- start start 2023-12-02T00:31:03.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.980+0000 7f3b950c2640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3b90146460 0x7f3b900b49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:03.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.980+0000 7f3b950c2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3b90146840 0x7f3b9006baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:03.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.980+0000 7f3b950c2640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b90146fd0 con 0x7f3b9006eab0 2023-12-02T00:31:03.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.980+0000 7f3b950c2640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b9006c180 con 0x7f3b90146460 2023-12-02T00:31:03.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.980+0000 7f3b950c2640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9006c460 con 0x7f3b90146840 2023-12-02T00:31:03.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.980+0000 7f3b8e59c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3b90146840 0x7f3b9006baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:03.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.980+0000 7f3b8ed9d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3b90146460 0x7f3b900b49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:03.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.980+0000 7f3b8e59c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3b90146840 0x7f3b9006baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.46:47300/0 (socket says 172.21.15.46:47300) 2023-12-02T00:31:03.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.980+0000 7f3b8e59c640 1 -- 172.21.15.46:0/899812549 learned_addr learned my addr 172.21.15.46:0/899812549 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:03.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.981+0000 7f3b8e59c640 1 -- 172.21.15.46:0/899812549 >> v2:172.21.15.162:3300/0 conn(0x7f3b90146460 msgr2=0x7f3b900b49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:03.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.981+0000 7f3b8e59c640 1 --2- 172.21.15.46:0/899812549 >> v2:172.21.15.162:3300/0 conn(0x7f3b90146460 0x7f3b900b49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:03.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.981+0000 7f3b8e59c640 1 -- 172.21.15.46:0/899812549 >> v1:172.21.15.162:6789/0 conn(0x7f3b9006eab0 legacy=0x7f3b90160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:03.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.981+0000 7f3b8dd9b640 1 -- 172.21.15.46:0/899812549 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1880691175 0 0) 0x7f3b90146fd0 con 0x7f3b9006eab0 2023-12-02T00:31:03.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.981+0000 7f3b8e59c640 1 -- 172.21.15.46:0/899812549 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b9006c740 con 0x7f3b90146840 2023-12-02T00:31:03.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.981+0000 7f3b8ed9d640 1 --2- 172.21.15.46:0/899812549 >> v2:172.21.15.162:3300/0 conn(0x7f3b90146460 0x7f3b900b49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:31:03.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.981+0000 7f3b8e59c640 1 --2- 172.21.15.46:0/899812549 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3b90146840 0x7f3b9006baf0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3b78009870 tx=0x7f3b7802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da5f8f660606a68 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:03.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.982+0000 7f3b8dd9b640 1 -- 172.21.15.46:0/899812549 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b78030840 con 0x7f3b90146840 2023-12-02T00:31:03.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.982+0000 7f3b8dd9b640 1 -- 172.21.15.46:0/899812549 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b780309a0 con 0x7f3b90146840 2023-12-02T00:31:03.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.982+0000 7f3b8dd9b640 1 -- 172.21.15.46:0/899812549 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b78030d30 con 0x7f3b90146840 2023-12-02T00:31:03.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.982+0000 7f3b950c2640 1 -- 172.21.15.46:0/899812549 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3b90146840 msgr2=0x7f3b9006baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:03.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.982+0000 7f3b950c2640 1 --2- 172.21.15.46:0/899812549 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3b90146840 0x7f3b9006baf0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3b78009870 tx=0x7f3b7802b980 comp rx=0 tx=0).stop 2023-12-02T00:31:03.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.982+0000 7f3b950c2640 1 -- 172.21.15.46:0/899812549 shutdown_connections 2023-12-02T00:31:03.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.982+0000 7f3b950c2640 1 --2- 172.21.15.46:0/899812549 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3b90146840 0x7f3b9006baf0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:03.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.982+0000 7f3b950c2640 1 --2- 172.21.15.46:0/899812549 >> v2:172.21.15.162:3300/0 conn(0x7f3b90146460 0x7f3b900b49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:03.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.982+0000 7f3b950c2640 1 -- 172.21.15.46:0/899812549 >> 172.21.15.46:0/899812549 conn(0x7f3b9006a6b0 msgr2=0x7f3b9006aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:03.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.982+0000 7f3b950c2640 1 -- 172.21.15.46:0/899812549 shutdown_connections 2023-12-02T00:31:03.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.982+0000 7f3b950c2640 1 -- 172.21.15.46:0/899812549 wait complete. 2023-12-02T00:31:03.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.984+0000 7f3b950c2640 1 Processor -- start 2023-12-02T00:31:03.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.984+0000 7f3b950c2640 1 -- start start 2023-12-02T00:31:03.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.984+0000 7f3b950c2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3b9006eab0 0x7f3b90144f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:03.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.984+0000 7f3b950c2640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b90146460 0x7f3b900aaa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:03.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.984+0000 7f3b950c2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b90146840 0x7f3b900aaf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:03.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.984+0000 7f3b950c2640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b900cf700 con 0x7f3b90146840 2023-12-02T00:31:03.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.984+0000 7f3b950c2640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b900ceb80 con 0x7f3b90146460 2023-12-02T00:31:03.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.984+0000 7f3b950c2640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b900cee60 con 0x7f3b9006eab0 2023-12-02T00:31:03.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.985+0000 7f3b8f59e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b90146840 0x7f3b900aaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:03.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.985+0000 7f3b8f59e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b90146840 0x7f3b900aaf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59630/0 (socket says 172.21.15.46:59630) 2023-12-02T00:31:03.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.985+0000 7f3b8f59e640 1 -- 172.21.15.46:0/3726878751 learned_addr learned my addr 172.21.15.46:0/3726878751 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:03.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.985+0000 7f3b8e59c640 1 --2- 172.21.15.46:0/3726878751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b90146460 0x7f3b900aaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:03.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.985+0000 7f3b8ed9d640 1 --2- 172.21.15.46:0/3726878751 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3b9006eab0 0x7f3b90144f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:03.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.985+0000 7f3b8f59e640 1 -- 172.21.15.46:0/3726878751 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3b9006eab0 msgr2=0x7f3b90144f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:03.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.985+0000 7f3b8f59e640 1 --2- 172.21.15.46:0/3726878751 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3b9006eab0 0x7f3b90144f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:03.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.985+0000 7f3b8f59e640 1 -- 172.21.15.46:0/3726878751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b90146460 msgr2=0x7f3b900aaa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:03.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.985+0000 7f3b8f59e640 1 --2- 172.21.15.46:0/3726878751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b90146460 0x7f3b900aaa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:03.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.985+0000 7f3b8f59e640 1 -- 172.21.15.46:0/3726878751 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b88009400 con 0x7f3b90146840 2023-12-02T00:31:03.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.985+0000 7f3b8f59e640 1 --2- 172.21.15.46:0/3726878751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b90146840 0x7f3b900aaf60 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f3b8000c780 tx=0x7f3b8000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:03.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.985+0000 7f3b877fe640 1 -- 172.21.15.46:0/3726878751 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b80003da0 con 0x7f3b90146840 2023-12-02T00:31:03.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.985+0000 7f3b877fe640 1 -- 172.21.15.46:0/3726878751 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b8000ce30 con 0x7f3b90146840 2023-12-02T00:31:03.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.985+0000 7f3b950c2640 1 -- 172.21.15.46:0/3726878751 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b780094d0 con 0x7f3b90146840 2023-12-02T00:31:03.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.985+0000 7f3b877fe640 1 -- 172.21.15.46:0/3726878751 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b800052b0 con 0x7f3b90146840 2023-12-02T00:31:03.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.985+0000 7f3b950c2640 1 -- 172.21.15.46:0/3726878751 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b900cf420 con 0x7f3b90146840 2023-12-02T00:31:03.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.986+0000 7f3b950c2640 1 -- 172.21.15.46:0/3726878751 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b90135a10 con 0x7f3b90146840 2023-12-02T00:31:03.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.987+0000 7f3b877fe640 1 -- 172.21.15.46:0/3726878751 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3b80005410 con 0x7f3b90146840 2023-12-02T00:31:03.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.987+0000 7f3b877fe640 1 --2- 172.21.15.46:0/3726878751 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3b7006ae40 0x7f3b7006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:03.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.987+0000 7f3b877fe640 1 -- 172.21.15.46:0/3726878751 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3b800891c0 con 0x7f3b90146840 2023-12-02T00:31:03.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.987+0000 7f3b8ed9d640 1 --2- 172.21.15.46:0/3726878751 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3b7006ae40 0x7f3b7006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:03.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.988+0000 7f3b8ed9d640 1 --2- 172.21.15.46:0/3726878751 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3b7006ae40 0x7f3b7006d300 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3b88009cd0 tx=0x7f3b88009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:03.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:03.991+0000 7f3b877fe640 1 -- 172.21.15.46:0/3726878751 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3b90135a10 con 0x7f3b90146840 2023-12-02T00:31:04.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.209+0000 7f3b950c2640 1 -- 172.21.15.46:0/3726878751 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f3b90135a10 con 0x7f3b90146840 2023-12-02T00:31:04.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.847+0000 7f3b877fe640 1 -- 172.21.15.46:0/3726878751 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3b90135a10 con 0x7f3b90146840 2023-12-02T00:31:04.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.850+0000 7f3b950c2640 1 -- 172.21.15.46:0/3726878751 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3b7006ae40 msgr2=0x7f3b7006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:04.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.850+0000 7f3b950c2640 1 --2- 172.21.15.46:0/3726878751 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3b7006ae40 0x7f3b7006d300 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3b88009cd0 tx=0x7f3b88009190 comp rx=0 tx=0).stop 2023-12-02T00:31:04.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.850+0000 7f3b950c2640 1 -- 172.21.15.46:0/3726878751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b90146840 msgr2=0x7f3b900aaf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:04.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.850+0000 7f3b950c2640 1 --2- 172.21.15.46:0/3726878751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b90146840 0x7f3b900aaf60 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f3b8000c780 tx=0x7f3b8000cc40 comp rx=0 tx=0).stop 2023-12-02T00:31:04.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.850+0000 7f3b950c2640 1 -- 172.21.15.46:0/3726878751 shutdown_connections 2023-12-02T00:31:04.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.850+0000 7f3b950c2640 1 --2- 172.21.15.46:0/3726878751 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3b7006ae40 0x7f3b7006d300 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:04.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.850+0000 7f3b950c2640 1 --2- 172.21.15.46:0/3726878751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b90146840 0x7f3b900aaf60 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:04.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.850+0000 7f3b950c2640 1 --2- 172.21.15.46:0/3726878751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b90146460 0x7f3b900aaa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:04.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.850+0000 7f3b950c2640 1 --2- 172.21.15.46:0/3726878751 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3b9006eab0 0x7f3b90144f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:04.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.850+0000 7f3b950c2640 1 -- 172.21.15.46:0/3726878751 >> 172.21.15.46:0/3726878751 conn(0x7f3b9006a6b0 msgr2=0x7f3b900d1070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:04.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.850+0000 7f3b950c2640 1 -- 172.21.15.46:0/3726878751 shutdown_connections 2023-12-02T00:31:04.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.850+0000 7f3b950c2640 1 -- 172.21.15.46:0/3726878751 wait complete. 2023-12-02T00:31:04.866 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-02T00:31:04.866 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-02T00:31:04.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.978+0000 7f58a1b2c640 1 Processor -- start 2023-12-02T00:31:04.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.978+0000 7f58a1b2c640 1 -- start start 2023-12-02T00:31:04.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.980+0000 7f58a1b2c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f589c155790 0x7f589c06f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:04.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.980+0000 7f58a1b2c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f589c06abe0 0x7f589c15ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:04.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.980+0000 7f58a1b2c640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f589c155b70 con 0x7f589c15b5e0 2023-12-02T00:31:04.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.980+0000 7f58a1b2c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f589c06ee30 con 0x7f589c06abe0 2023-12-02T00:31:04.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.980+0000 7f58a1b2c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f589c06f3b0 con 0x7f589c155790 2023-12-02T00:31:04.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.980+0000 7f589affd640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f589c15b5e0 0x7f589c06a4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41762/0 (socket says 172.21.15.46:41762) 2023-12-02T00:31:04.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.980+0000 7f589affd640 1 -- 172.21.15.46:0/2986642135 learned_addr learned my addr 172.21.15.46:0/2986642135 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:31:04.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.980+0000 7f589b7fe640 1 --2- 172.21.15.46:0/2986642135 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f589c155790 0x7f589c06f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:04.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.980+0000 7f589bfff640 1 --2- 172.21.15.46:0/2986642135 >> v2:172.21.15.162:3300/0 conn(0x7f589c06abe0 0x7f589c15ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:04.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.980+0000 7f589a7fc640 1 -- 172.21.15.46:0/2986642135 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1236563370 0 0) 0x7f589c155b70 con 0x7f589c15b5e0 2023-12-02T00:31:04.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.980+0000 7f589a7fc640 1 -- 172.21.15.46:0/2986642135 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5884003520 con 0x7f589c15b5e0 2023-12-02T00:31:04.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.980+0000 7f589a7fc640 1 -- 172.21.15.46:0/2986642135 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 886941564 0 0) 0x7f5884003520 con 0x7f589c15b5e0 2023-12-02T00:31:04.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.980+0000 7f589a7fc640 1 -- 172.21.15.46:0/2986642135 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f589c155790 msgr2=0x7f589c06f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:04.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.980+0000 7f589a7fc640 1 --2- 172.21.15.46:0/2986642135 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f589c155790 0x7f589c06f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:04.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.980+0000 7f589a7fc640 1 -- 172.21.15.46:0/2986642135 >> v2:172.21.15.162:3300/0 conn(0x7f589c06abe0 msgr2=0x7f589c15ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:04.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.980+0000 7f589a7fc640 1 --2- 172.21.15.46:0/2986642135 >> v2:172.21.15.162:3300/0 conn(0x7f589c06abe0 0x7f589c15ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:04.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.980+0000 7f589a7fc640 1 -- 172.21.15.46:0/2986642135 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f589c15c170 con 0x7f589c15b5e0 2023-12-02T00:31:04.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.980+0000 7f589a7fc640 1 -- 172.21.15.46:0/2986642135 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7f5890001670 con 0x7f589c15b5e0 2023-12-02T00:31:04.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.980+0000 7f589a7fc640 1 -- 172.21.15.46:0/2986642135 >> v1:172.21.15.46:6789/0 conn(0x7f589c15b5e0 legacy=0x7f589c06a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:04.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.980+0000 7f589a7fc640 1 --2- 172.21.15.46:0/2986642135 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f589c155790 0x7f5884003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:04.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.980+0000 7f589a7fc640 1 --2- 172.21.15.46:0/2986642135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5884003ef0 0x7f58840047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:04.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.980+0000 7f589a7fc640 1 --2- 172.21.15.46:0/2986642135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5884004d00 0x7f5884005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:04.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.980+0000 7f589a7fc640 1 -- 172.21.15.46:0/2986642135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f589c06ee30 con 0x7f5884004d00 2023-12-02T00:31:04.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.980+0000 7f589a7fc640 1 -- 172.21.15.46:0/2986642135 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f589c06f3b0 con 0x7f5884003ef0 2023-12-02T00:31:04.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.980+0000 7f589a7fc640 1 -- 172.21.15.46:0/2986642135 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f588400b980 con 0x7f589c155790 2023-12-02T00:31:04.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.980+0000 7f589bfff640 1 --2- 172.21.15.46:0/2986642135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5884004d00 0x7f5884005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:04.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.980+0000 7f589affd640 1 --2- 172.21.15.46:0/2986642135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5884003ef0 0x7f58840047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:04.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.980+0000 7f589b7fe640 1 --2- 172.21.15.46:0/2986642135 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f589c155790 0x7f5884003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:04.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.980+0000 7f589bfff640 1 -- 172.21.15.46:0/2986642135 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f589c155790 msgr2=0x7f5884003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:04.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.980+0000 7f589bfff640 1 --2- 172.21.15.46:0/2986642135 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f589c155790 0x7f5884003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:04.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.980+0000 7f589bfff640 1 -- 172.21.15.46:0/2986642135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5884003ef0 msgr2=0x7f58840047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:04.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.980+0000 7f589bfff640 1 --2- 172.21.15.46:0/2986642135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5884003ef0 0x7f58840047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:04.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.980+0000 7f589bfff640 1 -- 172.21.15.46:0/2986642135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5884003520 con 0x7f5884004d00 2023-12-02T00:31:04.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.981+0000 7f589bfff640 1 --2- 172.21.15.46:0/2986642135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5884004d00 0x7f5884005180 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f5894009f90 tx=0x7f58840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35a4a5584234cf68 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:04.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.981+0000 7f589a7fc640 1 -- 172.21.15.46:0/2986642135 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589402cc00 con 0x7f5884004d00 2023-12-02T00:31:04.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.981+0000 7f589a7fc640 1 -- 172.21.15.46:0/2986642135 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f589400f040 con 0x7f5884004d00 2023-12-02T00:31:04.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.981+0000 7f589a7fc640 1 -- 172.21.15.46:0/2986642135 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5894034440 con 0x7f5884004d00 2023-12-02T00:31:04.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.981+0000 7f58a1b2c640 1 -- 172.21.15.46:0/2986642135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5884004d00 msgr2=0x7f5884005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:04.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.981+0000 7f58a1b2c640 1 --2- 172.21.15.46:0/2986642135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5884004d00 0x7f5884005180 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f5894009f90 tx=0x7f58840057f0 comp rx=0 tx=0).stop 2023-12-02T00:31:04.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.981+0000 7f589bfff640 1 -- 172.21.15.46:0/2986642135 reap_dead start 2023-12-02T00:31:04.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.981+0000 7f58a1b2c640 1 -- 172.21.15.46:0/2986642135 shutdown_connections 2023-12-02T00:31:04.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.981+0000 7f58a1b2c640 1 -- 172.21.15.46:0/2986642135 >> 172.21.15.46:0/2986642135 conn(0x7f589c0b1e50 msgr2=0x7f589c0b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:04.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.981+0000 7f58a1b2c640 1 -- 172.21.15.46:0/2986642135 shutdown_connections 2023-12-02T00:31:04.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.981+0000 7f58a1b2c640 1 -- 172.21.15.46:0/2986642135 wait complete. 2023-12-02T00:31:04.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.983+0000 7f58a1b2c640 1 Processor -- start 2023-12-02T00:31:04.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.983+0000 7f58a1b2c640 1 -- start start 2023-12-02T00:31:04.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.983+0000 7f58a1b2c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5884004d00 0x7f589c1e5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:04.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.984+0000 7f58a1b2c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f589c1515b0 0x7f589c1578f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:04.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.984+0000 7f58a1b2c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f589c150d90 0x7f589c15a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:04.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.984+0000 7f58a1b2c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f589c1e58b0 con 0x7f589c1515b0 2023-12-02T00:31:04.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.984+0000 7f58a1b2c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f589c1e4310 con 0x7f589c150d90 2023-12-02T00:31:04.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.984+0000 7f58a1b2c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f589c157e30 con 0x7f5884004d00 2023-12-02T00:31:04.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.984+0000 7f589affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f589c1515b0 0x7f589c1578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:04.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.984+0000 7f589affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f589c1515b0 0x7f589c1578f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59634/0 (socket says 172.21.15.46:59634) 2023-12-02T00:31:04.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.984+0000 7f589affd640 1 -- 172.21.15.46:0/3598142441 learned_addr learned my addr 172.21.15.46:0/3598142441 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:04.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.984+0000 7f589bfff640 1 --2- 172.21.15.46:0/3598142441 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f589c150d90 0x7f589c15a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:04.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.984+0000 7f589b7fe640 1 --2- 172.21.15.46:0/3598142441 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5884004d00 0x7f589c1e5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:04.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.984+0000 7f589affd640 1 -- 172.21.15.46:0/3598142441 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5884004d00 msgr2=0x7f589c1e5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:04.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.984+0000 7f589affd640 1 --2- 172.21.15.46:0/3598142441 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5884004d00 0x7f589c1e5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:04.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.984+0000 7f589affd640 1 -- 172.21.15.46:0/3598142441 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f589c150d90 msgr2=0x7f589c15a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:04.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.984+0000 7f589affd640 1 --2- 172.21.15.46:0/3598142441 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f589c150d90 0x7f589c15a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:04.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.984+0000 7f589affd640 1 -- 172.21.15.46:0/3598142441 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f589c1e45b0 con 0x7f589c1515b0 2023-12-02T00:31:04.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.984+0000 7f589affd640 1 --2- 172.21.15.46:0/3598142441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f589c1515b0 0x7f589c1578f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f5890007310 tx=0x7f5890007710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:04.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.984+0000 7f5883fff640 1 -- 172.21.15.46:0/3598142441 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589000f960 con 0x7f589c1515b0 2023-12-02T00:31:04.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.984+0000 7f5883fff640 1 -- 172.21.15.46:0/3598142441 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f589000fb00 con 0x7f589c1515b0 2023-12-02T00:31:04.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.984+0000 7f58a1b2c640 1 -- 172.21.15.46:0/3598142441 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f589c1e4890 con 0x7f589c1515b0 2023-12-02T00:31:04.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.984+0000 7f5883fff640 1 -- 172.21.15.46:0/3598142441 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589000feb0 con 0x7f589c1515b0 2023-12-02T00:31:04.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.984+0000 7f58a1b2c640 1 -- 172.21.15.46:0/3598142441 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f589c1e4e50 con 0x7f589c1515b0 2023-12-02T00:31:04.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.986+0000 7f58a1b2c640 1 -- 172.21.15.46:0/3598142441 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f589c069400 con 0x7f589c1515b0 2023-12-02T00:31:04.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.986+0000 7f5883fff640 1 -- 172.21.15.46:0/3598142441 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f589001d460 con 0x7f589c1515b0 2023-12-02T00:31:04.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.986+0000 7f5883fff640 1 --2- 172.21.15.46:0/3598142441 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f588806ae40 0x7f588806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:04.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.986+0000 7f5883fff640 1 -- 172.21.15.46:0/3598142441 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5890089770 con 0x7f589c1515b0 2023-12-02T00:31:04.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.987+0000 7f589b7fe640 1 --2- 172.21.15.46:0/3598142441 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f588806ae40 0x7f588806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:04.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.987+0000 7f589b7fe640 1 --2- 172.21.15.46:0/3598142441 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f588806ae40 0x7f588806d300 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f588c009bc0 tx=0x7f588c009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:04.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:04.992+0000 7f5883fff640 1 -- 172.21.15.46:0/3598142441 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5890054820 con 0x7f589c1515b0 2023-12-02T00:31:05.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.201+0000 7f58a1b2c640 1 -- 172.21.15.46:0/3598142441 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f589c0b2c50 con 0x7f589c1515b0 2023-12-02T00:31:05.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.202+0000 7f5883fff640 1 -- 172.21.15.46:0/3598142441 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f5890057e40 con 0x7f589c1515b0 2023-12-02T00:31:05.204 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:31:05.204 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5189,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/4047806587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":4047806587},{"type":"v1","addr":"172.21.15.162:6837","nonce":4047806587}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5620,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/1418783715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1418783715},{"type":"v1","addr":"172.21.15.162:6836","nonce":1418783715}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5979,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/452915985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":452915985},{"type":"v1","addr":"172.21.15.46:6833","nonce":452915985}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-02T00:31:01.187535+0000","modified":"2023-12-02T00:31:04.211234+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5994":{"gid":5994,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/903694874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":903694874},{"type":"v1","addr":"172.21.15.46:6835","nonce":903694874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-12-02T00:31:05.205 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 24 2023-12-02T00:31:05.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.205+0000 7f58a1b2c640 1 -- 172.21.15.46:0/3598142441 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f588806ae40 msgr2=0x7f588806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:05.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.205+0000 7f58a1b2c640 1 --2- 172.21.15.46:0/3598142441 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f588806ae40 0x7f588806d300 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f588c009bc0 tx=0x7f588c009190 comp rx=0 tx=0).stop 2023-12-02T00:31:05.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.205+0000 7f58a1b2c640 1 -- 172.21.15.46:0/3598142441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f589c1515b0 msgr2=0x7f589c1578f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:05.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.205+0000 7f58a1b2c640 1 --2- 172.21.15.46:0/3598142441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f589c1515b0 0x7f589c1578f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f5890007310 tx=0x7f5890007710 comp rx=0 tx=0).stop 2023-12-02T00:31:05.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.205+0000 7f58a1b2c640 1 -- 172.21.15.46:0/3598142441 shutdown_connections 2023-12-02T00:31:05.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.205+0000 7f58a1b2c640 1 --2- 172.21.15.46:0/3598142441 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f588806ae40 0x7f588806d300 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:05.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.205+0000 7f58a1b2c640 1 --2- 172.21.15.46:0/3598142441 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f589c150d90 0x7f589c15a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:05.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.205+0000 7f58a1b2c640 1 --2- 172.21.15.46:0/3598142441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f589c1515b0 0x7f589c1578f0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:05.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.205+0000 7f58a1b2c640 1 --2- 172.21.15.46:0/3598142441 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5884004d00 0x7f589c1e5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:05.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.205+0000 7f58a1b2c640 1 -- 172.21.15.46:0/3598142441 >> 172.21.15.46:0/3598142441 conn(0x7f589c0b1e50 msgr2=0x7f589c15c690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:05.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.205+0000 7f58a1b2c640 1 -- 172.21.15.46:0/3598142441 shutdown_connections 2023-12-02T00:31:05.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.205+0000 7f58a1b2c640 1 -- 172.21.15.46:0/3598142441 wait complete. 2023-12-02T00:31:05.217 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:31:05.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.321+0000 7f6054887640 1 Processor -- start 2023-12-02T00:31:05.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.321+0000 7f6054887640 1 -- start start 2023-12-02T00:31:05.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.322+0000 7f6054887640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f605015d340 0x7f605006b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:05.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.322+0000 7f6054887640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60500ad640 0x7f605006ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:05.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.322+0000 7f6054887640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f605015d720 con 0x7f60500aeb80 2023-12-02T00:31:05.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.322+0000 7f6054887640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60500adae0 con 0x7f605015d340 2023-12-02T00:31:05.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.322+0000 7f6054887640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60500addc0 con 0x7f60500ad640 2023-12-02T00:31:05.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.322+0000 7f604e59c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f605015d340 0x7f605006b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:05.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.322+0000 7f604e59c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f605015d340 0x7f605006b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59642/0 (socket says 172.21.15.46:59642) 2023-12-02T00:31:05.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.322+0000 7f604e59c640 1 -- 172.21.15.46:0/1865239288 learned_addr learned my addr 172.21.15.46:0/1865239288 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:05.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.322+0000 7f604ed9d640 1 --1- 172.21.15.46:0/1865239288 >> v1:172.21.15.46:6789/0 conn(0x7f60500aeb80 0x7f6050160ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41772/0 (socket says 172.21.15.46:41772) 2023-12-02T00:31:05.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.323+0000 7f6045d9b640 1 --2- 172.21.15.46:0/1865239288 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60500ad640 0x7f605006ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:05.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.323+0000 7f604e59c640 1 -- 172.21.15.46:0/1865239288 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60500ad640 msgr2=0x7f605006ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:05.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.323+0000 7f604e59c640 1 --2- 172.21.15.46:0/1865239288 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60500ad640 0x7f605006ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:05.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.323+0000 7f604e59c640 1 -- 172.21.15.46:0/1865239288 >> v1:172.21.15.46:6789/0 conn(0x7f60500aeb80 legacy=0x7f6050160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:05.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.323+0000 7f604e59c640 1 -- 172.21.15.46:0/1865239288 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60500ae0a0 con 0x7f605015d340 2023-12-02T00:31:05.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.323+0000 7f604dd9b640 1 -- 172.21.15.46:0/1865239288 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2030627156 0 0) 0x7f605015d720 con 0x7f60500aeb80 2023-12-02T00:31:05.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.323+0000 7f604e59c640 1 --2- 172.21.15.46:0/1865239288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f605015d340 0x7f605006b4e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f6040009870 tx=0x7f604002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a1235603a94e053 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:05.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.324+0000 7f604dd9b640 1 -- 172.21.15.46:0/1865239288 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6040030840 con 0x7f605015d340 2023-12-02T00:31:05.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.324+0000 7f604dd9b640 1 -- 172.21.15.46:0/1865239288 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60400309a0 con 0x7f605015d340 2023-12-02T00:31:05.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.324+0000 7f604dd9b640 1 -- 172.21.15.46:0/1865239288 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6040030d30 con 0x7f605015d340 2023-12-02T00:31:05.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.324+0000 7f6054887640 1 -- 172.21.15.46:0/1865239288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f605015d340 msgr2=0x7f605006b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:05.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.324+0000 7f6054887640 1 --2- 172.21.15.46:0/1865239288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f605015d340 0x7f605006b4e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f6040009870 tx=0x7f604002b980 comp rx=0 tx=0).stop 2023-12-02T00:31:05.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.324+0000 7f6054887640 1 -- 172.21.15.46:0/1865239288 shutdown_connections 2023-12-02T00:31:05.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.324+0000 7f6054887640 1 --2- 172.21.15.46:0/1865239288 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60500ad640 0x7f605006ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:05.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.324+0000 7f6054887640 1 --2- 172.21.15.46:0/1865239288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f605015d340 0x7f605006b4e0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:05.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.324+0000 7f6054887640 1 -- 172.21.15.46:0/1865239288 >> 172.21.15.46:0/1865239288 conn(0x7f6050070e70 msgr2=0x7f6050071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:05.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.324+0000 7f6054887640 1 -- 172.21.15.46:0/1865239288 shutdown_connections 2023-12-02T00:31:05.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.324+0000 7f6054887640 1 -- 172.21.15.46:0/1865239288 wait complete. 2023-12-02T00:31:05.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.326+0000 7f6054887640 1 Processor -- start 2023-12-02T00:31:05.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.326+0000 7f6054887640 1 -- start start 2023-12-02T00:31:05.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.326+0000 7f6054887640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60500ad640 0x7f60500ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:05.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.326+0000 7f6054887640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60500aeb80 0x7f6050147190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:05.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.326+0000 7f6054887640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f605015d340 0x7f60500cd540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:05.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.326+0000 7f6054887640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60500cde00 con 0x7f605015d340 2023-12-02T00:31:05.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.326+0000 7f6054887640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60500ce0e0 con 0x7f60500aeb80 2023-12-02T00:31:05.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.326+0000 7f6054887640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60500ce3c0 con 0x7f60500ad640 2023-12-02T00:31:05.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.327+0000 7f604ed9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f605015d340 0x7f60500cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:05.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.327+0000 7f604ed9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f605015d340 0x7f60500cd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59644/0 (socket says 172.21.15.46:59644) 2023-12-02T00:31:05.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.327+0000 7f604ed9d640 1 -- 172.21.15.46:0/3487558873 learned_addr learned my addr 172.21.15.46:0/3487558873 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:05.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.327+0000 7f6045d9b640 1 --2- 172.21.15.46:0/3487558873 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60500aeb80 0x7f6050147190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:05.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.327+0000 7f604e59c640 1 --2- 172.21.15.46:0/3487558873 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60500ad640 0x7f60500ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:05.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.327+0000 7f604ed9d640 1 -- 172.21.15.46:0/3487558873 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60500ad640 msgr2=0x7f60500ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:05.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.327+0000 7f604ed9d640 1 --2- 172.21.15.46:0/3487558873 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60500ad640 0x7f60500ceea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:05.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.327+0000 7f604ed9d640 1 -- 172.21.15.46:0/3487558873 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60500aeb80 msgr2=0x7f6050147190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:05.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.327+0000 7f604ed9d640 1 --2- 172.21.15.46:0/3487558873 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60500aeb80 0x7f6050147190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:05.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.327+0000 7f604ed9d640 1 -- 172.21.15.46:0/3487558873 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60400094d0 con 0x7f605015d340 2023-12-02T00:31:05.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.327+0000 7f604e59c640 1 --2- 172.21.15.46:0/3487558873 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60500ad640 0x7f60500ceea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:31:05.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.327+0000 7f604ed9d640 1 --2- 172.21.15.46:0/3487558873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f605015d340 0x7f60500cd540 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f604800c7e0 tx=0x7f604800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:05.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.327+0000 7f60477fe640 1 -- 172.21.15.46:0/3487558873 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604800a640 con 0x7f605015d340 2023-12-02T00:31:05.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.327+0000 7f60477fe640 1 -- 172.21.15.46:0/3487558873 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f604800a7c0 con 0x7f605015d340 2023-12-02T00:31:05.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.327+0000 7f60477fe640 1 -- 172.21.15.46:0/3487558873 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604800ab70 con 0x7f605015d340 2023-12-02T00:31:05.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.327+0000 7f6054887640 1 -- 172.21.15.46:0/3487558873 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60500ce6a0 con 0x7f605015d340 2023-12-02T00:31:05.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.327+0000 7f6054887640 1 -- 172.21.15.46:0/3487558873 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60501476d0 con 0x7f605015d340 2023-12-02T00:31:05.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.328+0000 7f6054887640 1 -- 172.21.15.46:0/3487558873 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60500cebc0 con 0x7f605015d340 2023-12-02T00:31:05.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.329+0000 7f60477fe640 1 -- 172.21.15.46:0/3487558873 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f604800acd0 con 0x7f605015d340 2023-12-02T00:31:05.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.329+0000 7f60477fe640 1 --2- 172.21.15.46:0/3487558873 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f603006ae40 0x7f603006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:05.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.329+0000 7f60477fe640 1 -- 172.21.15.46:0/3487558873 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6048088be0 con 0x7f605015d340 2023-12-02T00:31:05.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.329+0000 7f604e59c640 1 --2- 172.21.15.46:0/3487558873 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f603006ae40 0x7f603006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:05.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.330+0000 7f604e59c640 1 --2- 172.21.15.46:0/3487558873 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f603006ae40 0x7f603006d300 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6040036000 tx=0x7f604003b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:05.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.333+0000 7f60477fe640 1 -- 172.21.15.46:0/3487558873 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f60500cebc0 con 0x7f605015d340 2023-12-02T00:31:05.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.511+0000 7f6054887640 1 -- 172.21.15.46:0/3487558873 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f60500cebc0 con 0x7f605015d340 2023-12-02T00:31:05.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.512+0000 7f60477fe640 1 -- 172.21.15.46:0/3487558873 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+16545 (secure 0 0 0) 0x7f60500cebc0 con 0x7f605015d340 2023-12-02T00:31:05.514 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:31:05.515 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":51,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:31:02.837102+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-12-02T00:30:59.469070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-12-02T00:31:00.174610+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:31:05.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.514+0000 7f6054887640 1 -- 172.21.15.46:0/3487558873 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f603006ae40 msgr2=0x7f603006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:05.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.514+0000 7f6054887640 1 --2- 172.21.15.46:0/3487558873 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f603006ae40 0x7f603006d300 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6040036000 tx=0x7f604003b040 comp rx=0 tx=0).stop 2023-12-02T00:31:05.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.514+0000 7f6054887640 1 -- 172.21.15.46:0/3487558873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f605015d340 msgr2=0x7f60500cd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:05.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.514+0000 7f6054887640 1 --2- 172.21.15.46:0/3487558873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f605015d340 0x7f60500cd540 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f604800c7e0 tx=0x7f604800cca0 comp rx=0 tx=0).stop 2023-12-02T00:31:05.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.514+0000 7f6054887640 1 -- 172.21.15.46:0/3487558873 shutdown_connections 2023-12-02T00:31:05.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.514+0000 7f6054887640 1 --2- 172.21.15.46:0/3487558873 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f603006ae40 0x7f603006d300 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:05.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.514+0000 7f6054887640 1 --2- 172.21.15.46:0/3487558873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f605015d340 0x7f60500cd540 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:05.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.514+0000 7f6054887640 1 --2- 172.21.15.46:0/3487558873 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60500aeb80 0x7f6050147190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:05.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.514+0000 7f6054887640 1 --2- 172.21.15.46:0/3487558873 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60500ad640 0x7f60500ceea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:05.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.514+0000 7f6054887640 1 -- 172.21.15.46:0/3487558873 >> 172.21.15.46:0/3487558873 conn(0x7f6050070e70 msgr2=0x7f605006b640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:05.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.514+0000 7f6054887640 1 -- 172.21.15.46:0/3487558873 shutdown_connections 2023-12-02T00:31:05.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.514+0000 7f6054887640 1 -- 172.21.15.46:0/3487558873 wait complete. 2023-12-02T00:31:05.527 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-02T00:31:05.528 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-02T00:31:05.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.633+0000 7f2ab599e640 1 Processor -- start 2023-12-02T00:31:05.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.633+0000 7f2ab599e640 1 -- start start 2023-12-02T00:31:05.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.635+0000 7f2ab599e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2ab00d15c0 0x7f2ab015e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:05.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.635+0000 7f2ab599e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2ab00cdb60 0x7f2ab00cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:05.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.635+0000 7f2ab599e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ab015d6b0 con 0x7f2ab00d0bb0 2023-12-02T00:31:05.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.635+0000 7f2ab599e640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ab00d1390 con 0x7f2ab00d15c0 2023-12-02T00:31:05.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.635+0000 7f2ab599e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ab00d08e0 con 0x7f2ab00cdb60 2023-12-02T00:31:05.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.635+0000 7f2ab499c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2ab00d15c0 0x7f2ab015e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:05.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.635+0000 7f2ab499c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2ab00d15c0 0x7f2ab015e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59652/0 (socket says 172.21.15.46:59652) 2023-12-02T00:31:05.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.635+0000 7f2ab499c640 1 -- 172.21.15.46:0/3703880548 learned_addr learned my addr 172.21.15.46:0/3703880548 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:05.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.635+0000 7f2ab519d640 1 --2- 172.21.15.46:0/3703880548 >> v2:172.21.15.162:3300/0 conn(0x7f2ab00cdb60 0x7f2ab00cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:05.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.635+0000 7f2ab499c640 1 -- 172.21.15.46:0/3703880548 >> v2:172.21.15.162:3300/0 conn(0x7f2ab00cdb60 msgr2=0x7f2ab00cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:05.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.635+0000 7f2ab499c640 1 --2- 172.21.15.46:0/3703880548 >> v2:172.21.15.162:3300/0 conn(0x7f2ab00cdb60 0x7f2ab00cdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:05.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.635+0000 7f2ab499c640 1 -- 172.21.15.46:0/3703880548 >> v1:172.21.15.162:6789/0 conn(0x7f2ab00d0bb0 legacy=0x7f2ab00d19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:05.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.635+0000 7f2ab499c640 1 -- 172.21.15.46:0/3703880548 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab00cfb20 con 0x7f2ab00d15c0 2023-12-02T00:31:05.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.635+0000 7f2ab519d640 1 --2- 172.21.15.46:0/3703880548 >> v2:172.21.15.162:3300/0 conn(0x7f2ab00cdb60 0x7f2ab00cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:31:05.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.635+0000 7f2ab499c640 1 --2- 172.21.15.46:0/3703880548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ab00d15c0 0x7f2ab015e3d0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f2ab00c6a40 tx=0x7f2aa802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4588ba4010d5e46c server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:05.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.635+0000 7f2aaf7fe640 1 -- 172.21.15.46:0/3703880548 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa8030860 con 0x7f2ab00d15c0 2023-12-02T00:31:05.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.635+0000 7f2aaf7fe640 1 -- 172.21.15.46:0/3703880548 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa80309c0 con 0x7f2ab00d15c0 2023-12-02T00:31:05.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.635+0000 7f2aaf7fe640 1 -- 172.21.15.46:0/3703880548 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa8030d50 con 0x7f2ab00d15c0 2023-12-02T00:31:05.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.636+0000 7f2ab599e640 1 -- 172.21.15.46:0/3703880548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ab00d15c0 msgr2=0x7f2ab015e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:05.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.636+0000 7f2ab599e640 1 --2- 172.21.15.46:0/3703880548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ab00d15c0 0x7f2ab015e3d0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f2ab00c6a40 tx=0x7f2aa802b9a0 comp rx=0 tx=0).stop 2023-12-02T00:31:05.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.636+0000 7f2ab599e640 1 -- 172.21.15.46:0/3703880548 shutdown_connections 2023-12-02T00:31:05.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.636+0000 7f2ab599e640 1 --2- 172.21.15.46:0/3703880548 >> v2:172.21.15.162:3300/0 conn(0x7f2ab00cdb60 0x7f2ab00cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:05.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.636+0000 7f2ab599e640 1 --2- 172.21.15.46:0/3703880548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ab00d15c0 0x7f2ab015e3d0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:05.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.636+0000 7f2ab599e640 1 -- 172.21.15.46:0/3703880548 >> 172.21.15.46:0/3703880548 conn(0x7f2ab00d2fe0 msgr2=0x7f2ab00d33f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:05.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.636+0000 7f2ab599e640 1 -- 172.21.15.46:0/3703880548 shutdown_connections 2023-12-02T00:31:05.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.636+0000 7f2ab599e640 1 -- 172.21.15.46:0/3703880548 wait complete. 2023-12-02T00:31:05.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.638+0000 7f2ab599e640 1 Processor -- start 2023-12-02T00:31:05.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.638+0000 7f2ab599e640 1 -- start start 2023-12-02T00:31:05.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.638+0000 7f2ab599e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ab00cdb60 0x7f2ab00ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:05.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.638+0000 7f2ab599e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ab00d0bb0 0x7f2ab00b2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:05.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.638+0000 7f2ab599e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ab00d15c0 0x7f2ab00cb100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:05.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.638+0000 7f2ab599e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab00cc6f0 con 0x7f2ab00d15c0 2023-12-02T00:31:05.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.638+0000 7f2ab599e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab00cba70 con 0x7f2ab00d0bb0 2023-12-02T00:31:05.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.638+0000 7f2ab599e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab00cbd50 con 0x7f2ab00cdb60 2023-12-02T00:31:05.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.638+0000 7f2ab519d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ab00d15c0 0x7f2ab00cb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:05.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.638+0000 7f2ab519d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ab00d15c0 0x7f2ab00cb100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59660/0 (socket says 172.21.15.46:59660) 2023-12-02T00:31:05.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.638+0000 7f2ab519d640 1 -- 172.21.15.46:0/1145168794 learned_addr learned my addr 172.21.15.46:0/1145168794 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:05.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.639+0000 7f2ab499c640 1 --2- 172.21.15.46:0/1145168794 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ab00cdb60 0x7f2ab00ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:05.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.639+0000 7f2aaffff640 1 --2- 172.21.15.46:0/1145168794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ab00d0bb0 0x7f2ab00b2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:05.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.639+0000 7f2ab519d640 1 -- 172.21.15.46:0/1145168794 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ab00cdb60 msgr2=0x7f2ab00ccb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:05.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.639+0000 7f2ab519d640 1 --2- 172.21.15.46:0/1145168794 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ab00cdb60 0x7f2ab00ccb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:05.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.639+0000 7f2ab519d640 1 -- 172.21.15.46:0/1145168794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ab00d0bb0 msgr2=0x7f2ab00b2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:05.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.639+0000 7f2ab519d640 1 --2- 172.21.15.46:0/1145168794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ab00d0bb0 0x7f2ab00b2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:05.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.639+0000 7f2ab519d640 1 -- 172.21.15.46:0/1145168794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa80094d0 con 0x7f2ab00d15c0 2023-12-02T00:31:05.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.639+0000 7f2ab519d640 1 --2- 172.21.15.46:0/1145168794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ab00d15c0 0x7f2ab00cb100 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f2a9800d7d0 tx=0x7f2a9800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:05.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.639+0000 7f2aad7fa640 1 -- 172.21.15.46:0/1145168794 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a98004030 con 0x7f2ab00d15c0 2023-12-02T00:31:05.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.639+0000 7f2aad7fa640 1 -- 172.21.15.46:0/1145168794 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a98004190 con 0x7f2ab00d15c0 2023-12-02T00:31:05.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.639+0000 7f2ab599e640 1 -- 172.21.15.46:0/1145168794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ab00cc030 con 0x7f2ab00d15c0 2023-12-02T00:31:05.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.639+0000 7f2aad7fa640 1 -- 172.21.15.46:0/1145168794 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a98004500 con 0x7f2ab00d15c0 2023-12-02T00:31:05.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.639+0000 7f2ab599e640 1 -- 172.21.15.46:0/1145168794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ab00b30f0 con 0x7f2ab00d15c0 2023-12-02T00:31:05.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.640+0000 7f2ab599e640 1 -- 172.21.15.46:0/1145168794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ab00cc830 con 0x7f2ab00d15c0 2023-12-02T00:31:05.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.641+0000 7f2aad7fa640 1 -- 172.21.15.46:0/1145168794 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2a980096a0 con 0x7f2ab00d15c0 2023-12-02T00:31:05.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.641+0000 7f2aad7fa640 1 --2- 172.21.15.46:0/1145168794 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2a9006ae40 0x7f2a9006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:05.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.641+0000 7f2aad7fa640 1 -- 172.21.15.46:0/1145168794 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2a98089790 con 0x7f2ab00d15c0 2023-12-02T00:31:05.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.641+0000 7f2ab499c640 1 --2- 172.21.15.46:0/1145168794 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2a9006ae40 0x7f2a9006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:05.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.642+0000 7f2ab499c640 1 --2- 172.21.15.46:0/1145168794 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2a9006ae40 0x7f2a9006d300 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8036000 tx=0x7f2aa8008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:05.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.645+0000 7f2aad7fa640 1 -- 172.21.15.46:0/1145168794 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2ab00cc830 con 0x7f2ab00d15c0 2023-12-02T00:31:05.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.817+0000 7f2ab599e640 1 -- 172.21.15.46:0/1145168794 --> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2ab00d03c0 con 0x7f2a9006ae40 2023-12-02T00:31:05.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.826+0000 7f2aad7fa640 1 -- 172.21.15.46:0/1145168794 <== mgr.4108 v2:172.21.15.162:6832/3248644722 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284257 (secure 0 0 0) 0x7f2aa8039450 con 0x7f2a9006ae40 2023-12-02T00:31:05.831 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:31:05.832 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-12-02T00:31:05.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.828+0000 7f2ab599e640 1 -- 172.21.15.46:0/1145168794 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2a9006ae40 msgr2=0x7f2a9006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:05.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.828+0000 7f2ab599e640 1 --2- 172.21.15.46:0/1145168794 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2a9006ae40 0x7f2a9006d300 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8036000 tx=0x7f2aa8008030 comp rx=0 tx=0).stop 2023-12-02T00:31:05.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.828+0000 7f2ab599e640 1 -- 172.21.15.46:0/1145168794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ab00d15c0 msgr2=0x7f2ab00cb100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:05.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.828+0000 7f2ab599e640 1 --2- 172.21.15.46:0/1145168794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ab00d15c0 0x7f2ab00cb100 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f2a9800d7d0 tx=0x7f2a9800dc90 comp rx=0 tx=0).stop 2023-12-02T00:31:05.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.829+0000 7f2ab599e640 1 -- 172.21.15.46:0/1145168794 shutdown_connections 2023-12-02T00:31:05.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.829+0000 7f2ab599e640 1 --2- 172.21.15.46:0/1145168794 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2a9006ae40 0x7f2a9006d300 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:05.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.829+0000 7f2ab599e640 1 --2- 172.21.15.46:0/1145168794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ab00d15c0 0x7f2ab00cb100 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:05.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.829+0000 7f2ab599e640 1 --2- 172.21.15.46:0/1145168794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ab00d0bb0 0x7f2ab00b2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:05.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.829+0000 7f2ab599e640 1 --2- 172.21.15.46:0/1145168794 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ab00cdb60 0x7f2ab00ccb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:05.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.829+0000 7f2ab599e640 1 -- 172.21.15.46:0/1145168794 >> 172.21.15.46:0/1145168794 conn(0x7f2ab00d2fe0 msgr2=0x7f2ab00b12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:05.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.829+0000 7f2ab599e640 1 -- 172.21.15.46:0/1145168794 shutdown_connections 2023-12-02T00:31:05.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.829+0000 7f2ab599e640 1 -- 172.21.15.46:0/1145168794 wait complete. 2023-12-02T00:31:05.842 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":119,"stamp":"2023-12-02T00:31:05.191363+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":32,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238420,"kb_used_data":5612,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736300,"statfs":{"total":773094113280,"available":772849971200,"internally_reserved":0,"allocated":5746688,"data_stored":3047349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.377699"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.827720+0000","last_change":"2023-12-02T00:31:00.827720+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.827720+0000","last_undegraded":"2023-12-02T00:31:00.827720+0000","last_fullsized":"2023-12-02T00:31:00.827720+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.824870+0000","last_change":"2023-12-02T00:31:00.824870+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.824870+0000","last_undegraded":"2023-12-02T00:31:00.824870+0000","last_fullsized":"2023-12-02T00:31:00.824870+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.826847+0000","last_change":"2023-12-02T00:31:00.826847+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.826847+0000","last_undegraded":"2023-12-02T00:31:00.826847+0000","last_fullsized":"2023-12-02T00:31:00.826847+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.823450+0000","last_change":"2023-12-02T00:31:00.823450+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.823450+0000","last_undegraded":"2023-12-02T00:31:00.823450+0000","last_fullsized":"2023-12-02T00:31:00.823450+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.818839+0000","last_change":"2023-12-02T00:31:00.818839+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.818839+0000","last_undegraded":"2023-12-02T00:31:00.818839+0000","last_fullsized":"2023-12-02T00:31:00.818839+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.828726+0000","last_change":"2023-12-02T00:31:00.828726+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.828726+0000","last_undegraded":"2023-12-02T00:31:00.828726+0000","last_fullsized":"2023-12-02T00:31:00.828726+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.827162+0000","last_change":"2023-12-02T00:31:00.827162+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.827162+0000","last_undegraded":"2023-12-02T00:31:00.827162+0000","last_fullsized":"2023-12-02T00:31:00.827162+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.834673+0000","last_change":"2023-12-02T00:31:00.834673+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.834673+0000","last_undegraded":"2023-12-02T00:31:00.834673+0000","last_fullsized":"2023-12-02T00:31:00.834673+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.833708+0000","last_change":"2023-12-02T00:31:00.833708+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.833708+0000","last_undegraded":"2023-12-02T00:31:00.833708+0000","last_fullsized":"2023-12-02T00:31:00.833708+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.829160+0000","last_change":"2023-12-02T00:31:00.829160+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.829160+0000","last_undegraded":"2023-12-02T00:31:00.829160+0000","last_fullsized":"2023-12-02T00:31:00.829160+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.818160+0000","last_change":"2023-12-02T00:31:00.818160+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.818160+0000","last_undegraded":"2023-12-02T00:31:00.818160+0000","last_fullsized":"2023-12-02T00:31:00.818160+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.831142+0000","last_change":"2023-12-02T00:31:00.831142+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.831142+0000","last_undegraded":"2023-12-02T00:31:00.831142+0000","last_fullsized":"2023-12-02T00:31:00.831142+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.827013+0000","last_change":"2023-12-02T00:31:00.827013+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.827013+0000","last_undegraded":"2023-12-02T00:31:00.827013+0000","last_fullsized":"2023-12-02T00:31:00.827013+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.826562+0000","last_change":"2023-12-02T00:31:00.826562+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.826562+0000","last_undegraded":"2023-12-02T00:31:00.826562+0000","last_fullsized":"2023-12-02T00:31:00.826562+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.823060+0000","last_change":"2023-12-02T00:31:00.823060+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.823060+0000","last_undegraded":"2023-12-02T00:31:00.823060+0000","last_fullsized":"2023-12-02T00:31:00.823060+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.825606+0000","last_change":"2023-12-02T00:31:00.825606+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.825606+0000","last_undegraded":"2023-12-02T00:31:00.825606+0000","last_fullsized":"2023-12-02T00:31:00.825606+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.825954+0000","last_change":"2023-12-02T00:31:00.825954+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.825954+0000","last_undegraded":"2023-12-02T00:31:00.825954+0000","last_fullsized":"2023-12-02T00:31:00.825954+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.824312+0000","last_change":"2023-12-02T00:31:00.824312+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.824312+0000","last_undegraded":"2023-12-02T00:31:00.824312+0000","last_fullsized":"2023-12-02T00:31:00.824312+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.825253+0000","last_change":"2023-12-02T00:31:00.825253+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.825253+0000","last_undegraded":"2023-12-02T00:31:00.825253+0000","last_fullsized":"2023-12-02T00:31:00.825253+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.825702+0000","last_change":"2023-12-02T00:31:00.825702+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.825702+0000","last_undegraded":"2023-12-02T00:31:00.825702+0000","last_fullsized":"2023-12-02T00:31:00.825702+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.826391+0000","last_change":"2023-12-02T00:31:00.826391+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.826391+0000","last_undegraded":"2023-12-02T00:31:00.826391+0000","last_fullsized":"2023-12-02T00:31:00.826391+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.831192+0000","last_change":"2023-12-02T00:31:00.831192+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.831192+0000","last_undegraded":"2023-12-02T00:31:00.831192+0000","last_fullsized":"2023-12-02T00:31:00.831192+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.818154+0000","last_change":"2023-12-02T00:31:00.818154+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.818154+0000","last_undegraded":"2023-12-02T00:31:00.818154+0000","last_fullsized":"2023-12-02T00:31:00.818154+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.824629+0000","last_change":"2023-12-02T00:31:00.824629+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.824629+0000","last_undegraded":"2023-12-02T00:31:00.824629+0000","last_fullsized":"2023-12-02T00:31:00.824629+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.824088+0000","last_change":"2023-12-02T00:31:00.824088+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.824088+0000","last_undegraded":"2023-12-02T00:31:00.824088+0000","last_fullsized":"2023-12-02T00:31:00.824088+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.829648+0000","last_change":"2023-12-02T00:31:00.829648+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.829648+0000","last_undegraded":"2023-12-02T00:31:00.829648+0000","last_fullsized":"2023-12-02T00:31:00.829648+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.827734+0000","last_change":"2023-12-02T00:31:00.827734+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.827734+0000","last_undegraded":"2023-12-02T00:31:00.827734+0000","last_fullsized":"2023-12-02T00:31:00.827734+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.824743+0000","last_change":"2023-12-02T00:31:00.824743+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.824743+0000","last_undegraded":"2023-12-02T00:31:00.824743+0000","last_fullsized":"2023-12-02T00:31:00.824743+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.823555+0000","last_change":"2023-12-02T00:31:00.823555+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.823555+0000","last_undegraded":"2023-12-02T00:31:00.823555+0000","last_fullsized":"2023-12-02T00:31:00.823555+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.823051+0000","last_change":"2023-12-02T00:31:00.823051+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.823051+0000","last_undegraded":"2023-12-02T00:31:00.823051+0000","last_fullsized":"2023-12-02T00:31:00.823051+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.823884+0000","last_change":"2023-12-02T00:31:00.823884+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.823884+0000","last_undegraded":"2023-12-02T00:31:00.823884+0000","last_fullsized":"2023-12-02T00:31:00.823884+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.827493+0000","last_change":"2023-12-02T00:31:00.827493+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.827493+0000","last_undegraded":"2023-12-02T00:31:00.827493+0000","last_fullsized":"2023-12-02T00:31:00.827493+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.824063+0000","last_change":"2023-12-02T00:31:00.824063+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.824063+0000","last_undegraded":"2023-12-02T00:31:00.824063+0000","last_fullsized":"2023-12-02T00:31:00.824063+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.836660+0000","last_change":"2023-12-02T00:31:00.832088+0000","last_active":"2023-12-02T00:31:00.836660+0000","last_peered":"2023-12-02T00:31:00.836660+0000","last_clean":"2023-12-02T00:31:00.836660+0000","last_became_active":"2023-12-02T00:31:00.830872+0000","last_became_peered":"2023-12-02T00:31:00.830872+0000","last_unstale":"2023-12-02T00:31:00.836660+0000","last_undegraded":"2023-12-02T00:31:00.836660+0000","last_fullsized":"2023-12-02T00:31:00.836660+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:18:11.861982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.825822+0000","last_change":"2023-12-02T00:31:00.825822+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.825822+0000","last_undegraded":"2023-12-02T00:31:00.825822+0000","last_fullsized":"2023-12-02T00:31:00.825822+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.836248+0000","last_change":"2023-12-02T00:31:00.834787+0000","last_active":"2023-12-02T00:31:00.836248+0000","last_peered":"2023-12-02T00:31:00.836248+0000","last_clean":"2023-12-02T00:31:00.836248+0000","last_became_active":"2023-12-02T00:31:00.834357+0000","last_became_peered":"2023-12-02T00:31:00.834357+0000","last_unstale":"2023-12-02T00:31:00.836248+0000","last_undegraded":"2023-12-02T00:31:00.836248+0000","last_fullsized":"2023-12-02T00:31:00.836248+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:57:02.728754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.826663+0000","last_change":"2023-12-02T00:31:00.826663+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.826663+0000","last_undegraded":"2023-12-02T00:31:00.826663+0000","last_fullsized":"2023-12-02T00:31:00.826663+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.839027+0000","last_change":"2023-12-02T00:31:00.837792+0000","last_active":"2023-12-02T00:31:00.839027+0000","last_peered":"2023-12-02T00:31:00.839027+0000","last_clean":"2023-12-02T00:31:00.839027+0000","last_became_active":"2023-12-02T00:31:00.837471+0000","last_became_peered":"2023-12-02T00:31:00.837471+0000","last_unstale":"2023-12-02T00:31:00.839027+0000","last_undegraded":"2023-12-02T00:31:00.839027+0000","last_fullsized":"2023-12-02T00:31:00.839027+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:33:23.942055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.823658+0000","last_change":"2023-12-02T00:31:00.823658+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.823658+0000","last_undegraded":"2023-12-02T00:31:00.823658+0000","last_fullsized":"2023-12-02T00:31:00.823658+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.837462+0000","last_change":"2023-12-02T00:31:00.834209+0000","last_active":"2023-12-02T00:31:00.837462+0000","last_peered":"2023-12-02T00:31:00.837462+0000","last_clean":"2023-12-02T00:31:00.837462+0000","last_became_active":"2023-12-02T00:31:00.833798+0000","last_became_peered":"2023-12-02T00:31:00.833798+0000","last_unstale":"2023-12-02T00:31:00.837462+0000","last_undegraded":"2023-12-02T00:31:00.837462+0000","last_fullsized":"2023-12-02T00:31:00.837462+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:20:10.886591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.818382+0000","last_change":"2023-12-02T00:31:00.818382+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.818382+0000","last_undegraded":"2023-12-02T00:31:00.818382+0000","last_fullsized":"2023-12-02T00:31:00.818382+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.824844+0000","last_change":"2023-12-02T00:31:00.821690+0000","last_active":"2023-12-02T00:31:00.824844+0000","last_peered":"2023-12-02T00:31:00.824844+0000","last_clean":"2023-12-02T00:31:00.824844+0000","last_became_active":"2023-12-02T00:31:00.821365+0000","last_became_peered":"2023-12-02T00:31:00.821365+0000","last_unstale":"2023-12-02T00:31:00.824844+0000","last_undegraded":"2023-12-02T00:31:00.824844+0000","last_fullsized":"2023-12-02T00:31:00.824844+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:57:02.728754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.823332+0000","last_change":"2023-12-02T00:31:00.823332+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.823332+0000","last_undegraded":"2023-12-02T00:31:00.823332+0000","last_fullsized":"2023-12-02T00:31:00.823332+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.834534+0000","last_change":"2023-12-02T00:31:00.831123+0000","last_active":"2023-12-02T00:31:00.834534+0000","last_peered":"2023-12-02T00:31:00.834534+0000","last_clean":"2023-12-02T00:31:00.834534+0000","last_became_active":"2023-12-02T00:31:00.830870+0000","last_became_peered":"2023-12-02T00:31:00.830870+0000","last_unstale":"2023-12-02T00:31:00.834534+0000","last_undegraded":"2023-12-02T00:31:00.834534+0000","last_fullsized":"2023-12-02T00:31:00.834534+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:56:41.004341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.822277+0000","last_change":"2023-12-02T00:31:00.822277+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.822277+0000","last_undegraded":"2023-12-02T00:31:00.822277+0000","last_fullsized":"2023-12-02T00:31:00.822277+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.839350+0000","last_change":"2023-12-02T00:31:00.834261+0000","last_active":"2023-12-02T00:31:00.839350+0000","last_peered":"2023-12-02T00:31:00.839350+0000","last_clean":"2023-12-02T00:31:00.839350+0000","last_became_active":"2023-12-02T00:31:00.833992+0000","last_became_peered":"2023-12-02T00:31:00.833992+0000","last_unstale":"2023-12-02T00:31:00.839350+0000","last_undegraded":"2023-12-02T00:31:00.839350+0000","last_fullsized":"2023-12-02T00:31:00.839350+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:09:23.731017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.823236+0000","last_change":"2023-12-02T00:31:00.823236+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.823236+0000","last_undegraded":"2023-12-02T00:31:00.823236+0000","last_fullsized":"2023-12-02T00:31:00.823236+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.832488+0000","last_change":"2023-12-02T00:31:00.828451+0000","last_active":"2023-12-02T00:31:00.832488+0000","last_peered":"2023-12-02T00:31:00.832488+0000","last_clean":"2023-12-02T00:31:00.832488+0000","last_became_active":"2023-12-02T00:31:00.828164+0000","last_became_peered":"2023-12-02T00:31:00.828164+0000","last_unstale":"2023-12-02T00:31:00.832488+0000","last_undegraded":"2023-12-02T00:31:00.832488+0000","last_fullsized":"2023-12-02T00:31:00.832488+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T12:24:56.280705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.818389+0000","last_change":"2023-12-02T00:31:00.818389+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.818389+0000","last_undegraded":"2023-12-02T00:31:00.818389+0000","last_fullsized":"2023-12-02T00:31:00.818389+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.839294+0000","last_change":"2023-12-02T00:31:00.835042+0000","last_active":"2023-12-02T00:31:00.839294+0000","last_peered":"2023-12-02T00:31:00.839294+0000","last_clean":"2023-12-02T00:31:00.839294+0000","last_became_active":"2023-12-02T00:31:00.834764+0000","last_became_peered":"2023-12-02T00:31:00.834764+0000","last_unstale":"2023-12-02T00:31:00.839294+0000","last_undegraded":"2023-12-02T00:31:00.839294+0000","last_fullsized":"2023-12-02T00:31:00.839294+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:54:07.912968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.827202+0000","last_change":"2023-12-02T00:31:00.827202+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.827202+0000","last_undegraded":"2023-12-02T00:31:00.827202+0000","last_fullsized":"2023-12-02T00:31:00.827202+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.826894+0000","last_change":"2023-12-02T00:31:00.821222+0000","last_active":"2023-12-02T00:31:00.826894+0000","last_peered":"2023-12-02T00:31:00.826894+0000","last_clean":"2023-12-02T00:31:00.826894+0000","last_became_active":"2023-12-02T00:31:00.820914+0000","last_became_peered":"2023-12-02T00:31:00.820914+0000","last_unstale":"2023-12-02T00:31:00.826894+0000","last_undegraded":"2023-12-02T00:31:00.826894+0000","last_fullsized":"2023-12-02T00:31:00.826894+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T10:06:37.837138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.823398+0000","last_change":"2023-12-02T00:31:00.823398+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.823398+0000","last_undegraded":"2023-12-02T00:31:00.823398+0000","last_fullsized":"2023-12-02T00:31:00.823398+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.840140+0000","last_change":"2023-12-02T00:31:00.837992+0000","last_active":"2023-12-02T00:31:00.840140+0000","last_peered":"2023-12-02T00:31:00.840140+0000","last_clean":"2023-12-02T00:31:00.840140+0000","last_became_active":"2023-12-02T00:31:00.837743+0000","last_became_peered":"2023-12-02T00:31:00.837743+0000","last_unstale":"2023-12-02T00:31:00.840140+0000","last_undegraded":"2023-12-02T00:31:00.840140+0000","last_fullsized":"2023-12-02T00:31:00.840140+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:00:47.235125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.818271+0000","last_change":"2023-12-02T00:31:00.818271+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.818271+0000","last_undegraded":"2023-12-02T00:31:00.818271+0000","last_fullsized":"2023-12-02T00:31:00.818271+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.842235+0000","last_change":"2023-12-02T00:31:00.840634+0000","last_active":"2023-12-02T00:31:00.842235+0000","last_peered":"2023-12-02T00:31:00.842235+0000","last_clean":"2023-12-02T00:31:00.842235+0000","last_became_active":"2023-12-02T00:31:00.840359+0000","last_became_peered":"2023-12-02T00:31:00.840359+0000","last_unstale":"2023-12-02T00:31:00.842235+0000","last_undegraded":"2023-12-02T00:31:00.842235+0000","last_fullsized":"2023-12-02T00:31:00.842235+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:34:01.547200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.826762+0000","last_change":"2023-12-02T00:31:00.826762+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.826762+0000","last_undegraded":"2023-12-02T00:31:00.826762+0000","last_fullsized":"2023-12-02T00:31:00.826762+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.832524+0000","last_change":"2023-12-02T00:31:00.829212+0000","last_active":"2023-12-02T00:31:00.832524+0000","last_peered":"2023-12-02T00:31:00.832524+0000","last_clean":"2023-12-02T00:31:00.832524+0000","last_became_active":"2023-12-02T00:31:00.828848+0000","last_became_peered":"2023-12-02T00:31:00.828848+0000","last_unstale":"2023-12-02T00:31:00.832524+0000","last_undegraded":"2023-12-02T00:31:00.832524+0000","last_fullsized":"2023-12-02T00:31:00.832524+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T01:12:56.020427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.822967+0000","last_change":"2023-12-02T00:31:00.822967+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.822967+0000","last_undegraded":"2023-12-02T00:31:00.822967+0000","last_fullsized":"2023-12-02T00:31:00.822967+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.839380+0000","last_change":"2023-12-02T00:31:00.834656+0000","last_active":"2023-12-02T00:31:00.839380+0000","last_peered":"2023-12-02T00:31:00.839380+0000","last_clean":"2023-12-02T00:31:00.839380+0000","last_became_active":"2023-12-02T00:31:00.834359+0000","last_became_peered":"2023-12-02T00:31:00.834359+0000","last_unstale":"2023-12-02T00:31:00.839380+0000","last_undegraded":"2023-12-02T00:31:00.839380+0000","last_fullsized":"2023-12-02T00:31:00.839380+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T01:33:59.617275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.824188+0000","last_change":"2023-12-02T00:31:00.824188+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.824188+0000","last_undegraded":"2023-12-02T00:31:00.824188+0000","last_fullsized":"2023-12-02T00:31:00.824188+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.840126+0000","last_change":"2023-12-02T00:31:00.837228+0000","last_active":"2023-12-02T00:31:00.840126+0000","last_peered":"2023-12-02T00:31:00.840126+0000","last_clean":"2023-12-02T00:31:00.840126+0000","last_became_active":"2023-12-02T00:31:00.836943+0000","last_became_peered":"2023-12-02T00:31:00.836943+0000","last_unstale":"2023-12-02T00:31:00.840126+0000","last_undegraded":"2023-12-02T00:31:00.840126+0000","last_fullsized":"2023-12-02T00:31:00.840126+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:51:08.104561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.823005+0000","last_change":"2023-12-02T00:31:00.823005+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.823005+0000","last_undegraded":"2023-12-02T00:31:00.823005+0000","last_fullsized":"2023-12-02T00:31:00.823005+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.842246+0000","last_change":"2023-12-02T00:31:00.840645+0000","last_active":"2023-12-02T00:31:00.842246+0000","last_peered":"2023-12-02T00:31:00.842246+0000","last_clean":"2023-12-02T00:31:00.842246+0000","last_became_active":"2023-12-02T00:31:00.840295+0000","last_became_peered":"2023-12-02T00:31:00.840295+0000","last_unstale":"2023-12-02T00:31:00.842246+0000","last_undegraded":"2023-12-02T00:31:00.842246+0000","last_fullsized":"2023-12-02T00:31:00.842246+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:41:19.966082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.822868+0000","last_change":"2023-12-02T00:31:00.822868+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.822868+0000","last_undegraded":"2023-12-02T00:31:00.822868+0000","last_fullsized":"2023-12-02T00:31:00.822868+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.840706+0000","last_change":"2023-12-02T00:31:00.839972+0000","last_active":"2023-12-02T00:31:00.840706+0000","last_peered":"2023-12-02T00:31:00.840706+0000","last_clean":"2023-12-02T00:31:00.840706+0000","last_became_active":"2023-12-02T00:31:00.839581+0000","last_became_peered":"2023-12-02T00:31:00.839581+0000","last_unstale":"2023-12-02T00:31:00.840706+0000","last_undegraded":"2023-12-02T00:31:00.840706+0000","last_fullsized":"2023-12-02T00:31:00.840706+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:33:23.942055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.825542+0000","last_change":"2023-12-02T00:31:00.825542+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.825542+0000","last_undegraded":"2023-12-02T00:31:00.825542+0000","last_fullsized":"2023-12-02T00:31:00.825542+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.833409+0000","last_change":"2023-12-02T00:31:00.832413+0000","last_active":"2023-12-02T00:31:00.833409+0000","last_peered":"2023-12-02T00:31:00.833409+0000","last_clean":"2023-12-02T00:31:00.833409+0000","last_became_active":"2023-12-02T00:31:00.832127+0000","last_became_peered":"2023-12-02T00:31:00.832127+0000","last_unstale":"2023-12-02T00:31:00.833409+0000","last_undegraded":"2023-12-02T00:31:00.833409+0000","last_fullsized":"2023-12-02T00:31:00.833409+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:55:28.223962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.818333+0000","last_change":"2023-12-02T00:31:00.818333+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.818333+0000","last_undegraded":"2023-12-02T00:31:00.818333+0000","last_fullsized":"2023-12-02T00:31:00.818333+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.840093+0000","last_change":"2023-12-02T00:31:00.837851+0000","last_active":"2023-12-02T00:31:00.840093+0000","last_peered":"2023-12-02T00:31:00.840093+0000","last_clean":"2023-12-02T00:31:00.840093+0000","last_became_active":"2023-12-02T00:31:00.837581+0000","last_became_peered":"2023-12-02T00:31:00.837581+0000","last_unstale":"2023-12-02T00:31:00.840093+0000","last_undegraded":"2023-12-02T00:31:00.840093+0000","last_fullsized":"2023-12-02T00:31:00.840093+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:17:18.074493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.826478+0000","last_change":"2023-12-02T00:31:00.826478+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.826478+0000","last_undegraded":"2023-12-02T00:31:00.826478+0000","last_fullsized":"2023-12-02T00:31:00.826478+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.836731+0000","last_change":"2023-12-02T00:31:00.832751+0000","last_active":"2023-12-02T00:31:00.836731+0000","last_peered":"2023-12-02T00:31:00.836731+0000","last_clean":"2023-12-02T00:31:00.836731+0000","last_became_active":"2023-12-02T00:31:00.832466+0000","last_became_peered":"2023-12-02T00:31:00.832466+0000","last_unstale":"2023-12-02T00:31:00.836731+0000","last_undegraded":"2023-12-02T00:31:00.836731+0000","last_fullsized":"2023-12-02T00:31:00.836731+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:01:15.991829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.826326+0000","last_change":"2023-12-02T00:31:00.826326+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.826326+0000","last_undegraded":"2023-12-02T00:31:00.826326+0000","last_fullsized":"2023-12-02T00:31:00.826326+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.836131+0000","last_change":"2023-12-02T00:31:00.833503+0000","last_active":"2023-12-02T00:31:00.836131+0000","last_peered":"2023-12-02T00:31:00.836131+0000","last_clean":"2023-12-02T00:31:00.836131+0000","last_became_active":"2023-12-02T00:31:00.833219+0000","last_became_peered":"2023-12-02T00:31:00.833219+0000","last_unstale":"2023-12-02T00:31:00.836131+0000","last_undegraded":"2023-12-02T00:31:00.836131+0000","last_fullsized":"2023-12-02T00:31:00.836131+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T09:28:07.117685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.824313+0000","last_change":"2023-12-02T00:31:00.824313+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.824313+0000","last_undegraded":"2023-12-02T00:31:00.824313+0000","last_fullsized":"2023-12-02T00:31:00.824313+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.842212+0000","last_change":"2023-12-02T00:31:00.840501+0000","last_active":"2023-12-02T00:31:00.842212+0000","last_peered":"2023-12-02T00:31:00.842212+0000","last_clean":"2023-12-02T00:31:00.842212+0000","last_became_active":"2023-12-02T00:31:00.840227+0000","last_became_peered":"2023-12-02T00:31:00.840227+0000","last_unstale":"2023-12-02T00:31:00.842212+0000","last_undegraded":"2023-12-02T00:31:00.842212+0000","last_fullsized":"2023-12-02T00:31:00.842212+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:40:05.075383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.822165+0000","last_change":"2023-12-02T00:31:00.822165+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.822165+0000","last_undegraded":"2023-12-02T00:31:00.822165+0000","last_fullsized":"2023-12-02T00:31:00.822165+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.836552+0000","last_change":"2023-12-02T00:31:00.833798+0000","last_active":"2023-12-02T00:31:00.836552+0000","last_peered":"2023-12-02T00:31:00.836552+0000","last_clean":"2023-12-02T00:31:00.836552+0000","last_became_active":"2023-12-02T00:31:00.833537+0000","last_became_peered":"2023-12-02T00:31:00.833537+0000","last_unstale":"2023-12-02T00:31:00.836552+0000","last_undegraded":"2023-12-02T00:31:00.836552+0000","last_fullsized":"2023-12-02T00:31:00.836552+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:54:17.615040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.826590+0000","last_change":"2023-12-02T00:31:00.826590+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.826590+0000","last_undegraded":"2023-12-02T00:31:00.826590+0000","last_fullsized":"2023-12-02T00:31:00.826590+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.830926+0000","last_change":"2023-12-02T00:31:00.829323+0000","last_active":"2023-12-02T00:31:00.830926+0000","last_peered":"2023-12-02T00:31:00.830926+0000","last_clean":"2023-12-02T00:31:00.830926+0000","last_became_active":"2023-12-02T00:31:00.829048+0000","last_became_peered":"2023-12-02T00:31:00.829048+0000","last_unstale":"2023-12-02T00:31:00.830926+0000","last_undegraded":"2023-12-02T00:31:00.830926+0000","last_fullsized":"2023-12-02T00:31:00.830926+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:33:51.372611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.825234+0000","last_change":"2023-12-02T00:31:00.825234+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.825234+0000","last_undegraded":"2023-12-02T00:31:00.825234+0000","last_fullsized":"2023-12-02T00:31:00.825234+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.839371+0000","last_change":"2023-12-02T00:31:00.835706+0000","last_active":"2023-12-02T00:31:00.839371+0000","last_peered":"2023-12-02T00:31:00.839371+0000","last_clean":"2023-12-02T00:31:00.839371+0000","last_became_active":"2023-12-02T00:31:00.835381+0000","last_became_peered":"2023-12-02T00:31:00.835381+0000","last_unstale":"2023-12-02T00:31:00.839371+0000","last_undegraded":"2023-12-02T00:31:00.839371+0000","last_fullsized":"2023-12-02T00:31:00.839371+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:14:37.638316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.828284+0000","last_change":"2023-12-02T00:31:00.828284+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.828284+0000","last_undegraded":"2023-12-02T00:31:00.828284+0000","last_fullsized":"2023-12-02T00:31:00.828284+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.836694+0000","last_change":"2023-12-02T00:31:00.832747+0000","last_active":"2023-12-02T00:31:00.836694+0000","last_peered":"2023-12-02T00:31:00.836694+0000","last_clean":"2023-12-02T00:31:00.836694+0000","last_became_active":"2023-12-02T00:31:00.832477+0000","last_became_peered":"2023-12-02T00:31:00.832477+0000","last_unstale":"2023-12-02T00:31:00.836694+0000","last_undegraded":"2023-12-02T00:31:00.836694+0000","last_fullsized":"2023-12-02T00:31:00.836694+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T10:47:44.799679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.823172+0000","last_change":"2023-12-02T00:31:00.823172+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.823172+0000","last_undegraded":"2023-12-02T00:31:00.823172+0000","last_fullsized":"2023-12-02T00:31:00.823172+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.832174+0000","last_change":"2023-12-02T00:31:00.828668+0000","last_active":"2023-12-02T00:31:00.832174+0000","last_peered":"2023-12-02T00:31:00.832174+0000","last_clean":"2023-12-02T00:31:00.832174+0000","last_became_active":"2023-12-02T00:31:00.828392+0000","last_became_peered":"2023-12-02T00:31:00.828392+0000","last_unstale":"2023-12-02T00:31:00.832174+0000","last_undegraded":"2023-12-02T00:31:00.832174+0000","last_fullsized":"2023-12-02T00:31:00.832174+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T09:28:07.117685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.822281+0000","last_change":"2023-12-02T00:31:00.822281+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.822281+0000","last_undegraded":"2023-12-02T00:31:00.822281+0000","last_fullsized":"2023-12-02T00:31:00.822281+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.842664+0000","last_change":"2023-12-02T00:31:00.841414+0000","last_active":"2023-12-02T00:31:00.842664+0000","last_peered":"2023-12-02T00:31:00.842664+0000","last_clean":"2023-12-02T00:31:00.842664+0000","last_became_active":"2023-12-02T00:31:00.841049+0000","last_became_peered":"2023-12-02T00:31:00.841049+0000","last_unstale":"2023-12-02T00:31:00.842664+0000","last_undegraded":"2023-12-02T00:31:00.842664+0000","last_fullsized":"2023-12-02T00:31:00.842664+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:34:41.347880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.826447+0000","last_change":"2023-12-02T00:31:00.826447+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.826447+0000","last_undegraded":"2023-12-02T00:31:00.826447+0000","last_fullsized":"2023-12-02T00:31:00.826447+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.839129+0000","last_change":"2023-12-02T00:31:00.838503+0000","last_active":"2023-12-02T00:31:00.839129+0000","last_peered":"2023-12-02T00:31:00.839129+0000","last_clean":"2023-12-02T00:31:00.839129+0000","last_became_active":"2023-12-02T00:31:00.838125+0000","last_became_peered":"2023-12-02T00:31:00.838125+0000","last_unstale":"2023-12-02T00:31:00.839129+0000","last_undegraded":"2023-12-02T00:31:00.839129+0000","last_fullsized":"2023-12-02T00:31:00.839129+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:00:03.107141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.822883+0000","last_change":"2023-12-02T00:31:00.822883+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.822883+0000","last_undegraded":"2023-12-02T00:31:00.822883+0000","last_fullsized":"2023-12-02T00:31:00.822883+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.833802+0000","last_change":"2023-12-02T00:31:00.828726+0000","last_active":"2023-12-02T00:31:00.833802+0000","last_peered":"2023-12-02T00:31:00.833802+0000","last_clean":"2023-12-02T00:31:00.833802+0000","last_became_active":"2023-12-02T00:31:00.828424+0000","last_became_peered":"2023-12-02T00:31:00.828424+0000","last_unstale":"2023-12-02T00:31:00.833802+0000","last_undegraded":"2023-12-02T00:31:00.833802+0000","last_fullsized":"2023-12-02T00:31:00.833802+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T02:42:58.558235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.823153+0000","last_change":"2023-12-02T00:31:00.823153+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.823153+0000","last_undegraded":"2023-12-02T00:31:00.823153+0000","last_fullsized":"2023-12-02T00:31:00.823153+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.840024+0000","last_change":"2023-12-02T00:31:00.837338+0000","last_active":"2023-12-02T00:31:00.840024+0000","last_peered":"2023-12-02T00:31:00.840024+0000","last_clean":"2023-12-02T00:31:00.840024+0000","last_became_active":"2023-12-02T00:31:00.837017+0000","last_became_peered":"2023-12-02T00:31:00.837017+0000","last_unstale":"2023-12-02T00:31:00.840024+0000","last_undegraded":"2023-12-02T00:31:00.840024+0000","last_fullsized":"2023-12-02T00:31:00.840024+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:06:02.637069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.824707+0000","last_change":"2023-12-02T00:31:00.824707+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.824707+0000","last_undegraded":"2023-12-02T00:31:00.824707+0000","last_fullsized":"2023-12-02T00:31:00.824707+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.832294+0000","last_change":"2023-12-02T00:31:00.829255+0000","last_active":"2023-12-02T00:31:00.832294+0000","last_peered":"2023-12-02T00:31:00.832294+0000","last_clean":"2023-12-02T00:31:00.832294+0000","last_became_active":"2023-12-02T00:31:00.828976+0000","last_became_peered":"2023-12-02T00:31:00.828976+0000","last_unstale":"2023-12-02T00:31:00.832294+0000","last_undegraded":"2023-12-02T00:31:00.832294+0000","last_fullsized":"2023-12-02T00:31:00.832294+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:30:57.889016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":192,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-12-02T00:30:59.819736+0000","last_change":"2023-12-02T00:28:39.379380+0000","last_active":"2023-12-02T00:30:59.819736+0000","last_peered":"2023-12-02T00:30:59.819736+0000","last_clean":"2023-12-02T00:30:59.819736+0000","last_became_active":"2023-12-02T00:28:39.378255+0000","last_became_peered":"2023-12-02T00:28:39.378255+0000","last_unstale":"2023-12-02T00:30:59.819736+0000","last_undegraded":"2023-12-02T00:30:59.819736+0000","last_fullsized":"2023-12-02T00:30:59.819736+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:38.177345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:38.177345+0000","last_clean_scrub_stamp":"2023-12-02T00:28:38.177345+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:21:08.582412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28156,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343684,"statfs":{"total":96636764160,"available":96607932416,"internally_reserved":0,"allocated":1163264,"data_stored":823065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.50900000000000001,"15min":0.499},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.48499999999999999}]},{"osd":1,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53400000000000003,"15min":0.53000000000000003},"min":{"1min":0.40400000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.54700000000000004,"15min":0.54400000000000004},"min":{"1min":0.46100000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.64500000000000002}]},{"osd":2,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54200000000000004,"15min":0.54400000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.67000000000000004,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.54300000000000004,"15min":0.53400000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.624}]},{"osd":3,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.53000000000000003,"15min":0.52200000000000002},"min":{"1min":0.48099999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.57299999999999995,"15min":0.57499999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.74399999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.68100000000000005}]},{"osd":4,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.50700000000000001,"15min":0.51100000000000001},"min":{"1min":0.29799999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.72199999999999998,"5min":0.745,"15min":0.745},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.51500000000000001,"15min":0.52100000000000002},"min":{"1min":0.33800000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.61799999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.60699999999999998}]},{"osd":5,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54700000000000004,"15min":0.55000000000000004},"min":{"1min":0.41899999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.78700000000000003,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.56100000000000005,"15min":0.56599999999999995},"min":{"1min":0.39500000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.63900000000000001,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.58199999999999996}]},{"osd":6,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.61799999999999999,"15min":0.627},"min":{"1min":0.40100000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.82899999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.60599999999999998,"15min":0.61299999999999999},"min":{"1min":0.44500000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76600000000000001,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.66500000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27592,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":581632,"data_stored":238972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.49199999999999999,"15min":0.48299999999999998},"min":{"1min":0.36499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.50700000000000001,"15min":0.501},"min":{"1min":0.40000000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.74099999999999999,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.64000000000000001}]},{"osd":1,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.51000000000000001,"15min":0.50600000000000001},"min":{"1min":0.33000000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.72599999999999998,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.53600000000000003,"15min":0.53300000000000003},"min":{"1min":0.39400000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.80900000000000005,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.59399999999999997}]},{"osd":2,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.52000000000000002,"15min":0.51200000000000001},"min":{"1min":0.38,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55500000000000005,"15min":0.55700000000000005},"min":{"1min":0.40300000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.77400000000000002,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.66100000000000003}]},{"osd":3,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57599999999999996,"15min":0.57299999999999995},"min":{"1min":0.45200000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.76300000000000001,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.622},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57199999999999995,"15min":0.56999999999999995},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.57699999999999996}]},{"osd":4,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.35399999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.71299999999999997,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52500000000000002,"15min":0.52300000000000002},"min":{"1min":0.34100000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73699999999999999,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.38600000000000001}]},{"osd":5,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.33400000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58799999999999997,"15min":0.59299999999999997},"min":{"1min":0.42399999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.40799999999999997}]},{"osd":7,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.35999999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.442},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.42399999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":569344,"data_stored":232697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.51700000000000002,"15min":0.50600000000000001},"min":{"1min":0.29099999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.73899999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.51600000000000001,"15min":0.51100000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.72899999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.54500000000000004}]},{"osd":1,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.51400000000000001,"15min":0.505},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.754,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55300000000000005,"15min":0.55000000000000004},"min":{"1min":0.36199999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.69499999999999995,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.42399999999999999}]},{"osd":2,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.56000000000000005,"15min":0.55100000000000005},"min":{"1min":0.41699999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.77500000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.53600000000000003,"15min":0.53300000000000003},"min":{"1min":0.34399999999999997,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.749,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.65100000000000002}]},{"osd":3,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58099999999999996,"15min":0.57699999999999996},"min":{"1min":0.39300000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.58299999999999996,"15min":0.58199999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.72499999999999998,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.58599999999999997}]},{"osd":4,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54300000000000004,"15min":0.54600000000000004},"min":{"1min":0.34799999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69899999999999995,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58699999999999997,"15min":0.58799999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.745}]},{"osd":6,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.56599999999999995,"15min":0.57099999999999995},"min":{"1min":0.36899999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.72899999999999998,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.624},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.59199999999999997,"15min":0.59599999999999997},"min":{"1min":0.435,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.73499999999999999}]},{"osd":7,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.42699999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.82099999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.63100000000000001,"15min":0.63500000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.76700000000000002,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.66800000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":569344,"data_stored":232697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.55100000000000005,"15min":0.54400000000000004},"min":{"1min":0.40699999999999997,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.74099999999999999,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.438,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.79500000000000004,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.83599999999999997}]},{"osd":1,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.57099999999999995,"15min":0.55800000000000005},"min":{"1min":0.50700000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.55800000000000005,"15min":0.55400000000000005},"min":{"1min":0.38600000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.77300000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.72099999999999997}]},{"osd":2,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.58099999999999996,"15min":0.57699999999999996},"min":{"1min":0.443,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.628,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.42799999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.75}]},{"osd":3,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.56999999999999995,"15min":0.56200000000000006},"min":{"1min":0.46600000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60599999999999998,"15min":0.60399999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.73499999999999999}]},{"osd":5,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52600000000000002,"15min":0.52300000000000002},"min":{"1min":0.36699999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.83799999999999997,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53900000000000003,"15min":0.54100000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.50600000000000001}]},{"osd":6,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.54200000000000004,"15min":0.53600000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.62},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.57699999999999996,"15min":0.56999999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.56899999999999995}]},{"osd":7,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.377,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.59499999999999997,"15min":0.59099999999999997},"min":{"1min":0.34699999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.70199999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574879,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31740,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340100,"statfs":{"total":96636764160,"available":96604262400,"internally_reserved":0,"allocated":569344,"data_stored":232697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.35699999999999998,"15min":0.35999999999999999},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.50800000000000001,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.38200000000000001,"15min":0.38100000000000001},"min":{"1min":0.27500000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.624,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.36899999999999999}]},{"osd":1,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.379,"15min":0.38300000000000001},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.59999999999999998,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.25800000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.58699999999999997,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.30399999999999999}]},{"osd":2,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.40600000000000003,"15min":0.40999999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.52900000000000003,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.42199999999999999,"15min":0.42099999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.66500000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.34999999999999998}]},{"osd":4,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.60899999999999999,"15min":0.61099999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76600000000000001,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.622,"15min":0.623},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.76300000000000001}]},{"osd":5,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.58299999999999996,"15min":0.58699999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85599999999999998,"5min":1.004,"15min":1.004},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.61299999999999999,"15min":0.61499999999999999},"min":{"1min":0.39300000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.90600000000000003,"5min":1.163,"15min":1.163},"last":0.50600000000000001}]},{"osd":6,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63800000000000001,"15min":0.63900000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.83199999999999996,"5min":1.095,"15min":1.095},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.63200000000000001,"15min":0.63500000000000001},"min":{"1min":0.45100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.96399999999999997,"5min":1.044,"15min":1.044},"last":0.746}]},{"osd":7,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.64700000000000002,"15min":0.65100000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.93899999999999995,"5min":1.177,"15min":1.177},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.64500000000000002,"15min":0.65100000000000002},"min":{"1min":0.45200000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.84799999999999998,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.72299999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31732,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340108,"statfs":{"total":96636764160,"available":96604270592,"internally_reserved":0,"allocated":565248,"data_stored":232078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.377,"15min":0.36299999999999999},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":2.649,"5min":2.649,"15min":2.649},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.42299999999999999,"15min":0.40799999999999997},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":3.173,"5min":3.173,"15min":3.173},"last":0.35099999999999998}]},{"osd":1,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.44800000000000001,"15min":0.41999999999999998},"min":{"1min":0.26300000000000001,"5min":0.246,"15min":0.246},"max":{"1min":4.6639999999999997,"5min":4.6639999999999997,"15min":4.6639999999999997},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.433,"15min":0.39100000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":6.093,"5min":6.093,"15min":6.093},"last":0.39500000000000002}]},{"osd":3,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.40300000000000002,"15min":0.376},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":4.1879999999999997,"5min":4.1879999999999997,"15min":4.1879999999999997},"last":0.317},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.44600000000000001,"15min":0.433},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":2.4159999999999999,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":0.33500000000000002}]},{"osd":4,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.60999999999999999,"15min":0.59599999999999997},"min":{"1min":0.40999999999999998,"5min":0.38,"15min":0.38},"max":{"1min":2.6840000000000002,"5min":2.6840000000000002,"15min":2.6840000000000002},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.65800000000000003,"15min":0.64600000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":3.0419999999999998,"5min":3.0419999999999998,"15min":3.0419999999999998},"last":0.45600000000000002}]},{"osd":5,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.66100000000000003,"15min":0.65300000000000002},"min":{"1min":0.41599999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":3.093,"5min":3.093,"15min":3.093},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.64700000000000002,"15min":0.65100000000000002},"min":{"1min":0.41799999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.98499999999999999,"5min":1.111,"15min":1.111},"last":0.51300000000000001}]},{"osd":6,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.64000000000000001,"15min":0.63800000000000001},"min":{"1min":0.42499999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.44},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.67800000000000005,"15min":0.67200000000000004},"min":{"1min":0.47799999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":2.6030000000000002,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":0.502}]},{"osd":7,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.65800000000000003,"15min":0.64800000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":3.1339999999999999,"5min":3.1339999999999999,"15min":3.1339999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.64200000000000002,"15min":0.64600000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.94199999999999995,"5min":1.2,"15min":1.2},"last":0.47699999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31736,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340104,"statfs":{"total":96636764160,"available":96604266496,"internally_reserved":0,"allocated":569344,"data_stored":232697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.33900000000000002,"15min":0.34100000000000003},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.436,"5min":0.48099999999999998,"15min":0.48099999999999998},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.39600000000000002,"15min":0.39800000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.58799999999999997,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.373}]},{"osd":2,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.35099999999999998,"15min":0.34899999999999998},"min":{"1min":0.249,"5min":0.246,"15min":0.246},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.36299999999999999,"15min":0.36699999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.53500000000000003,"5min":0.626,"15min":0.626},"last":0.29299999999999998}]},{"osd":3,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.40100000000000002,"15min":0.40200000000000002},"min":{"1min":0.30499999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.371,"15min":0.376},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.5,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.42099999999999999}]},{"osd":4,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.55400000000000005,"15min":0.54800000000000004},"min":{"1min":0.377,"5min":0.376,"15min":0.376},"max":{"1min":0.77100000000000002,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.55800000000000005,"15min":0.55100000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.78000000000000003,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.53200000000000003}]},{"osd":5,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.58599999999999997,"15min":0.57899999999999996},"min":{"1min":0.45900000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.96799999999999997,"5min":1.107,"15min":1.107},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57299999999999995,"15min":0.56599999999999995},"min":{"1min":0.437,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.63100000000000001}]},{"osd":6,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60399999999999998,"15min":0.60199999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.76600000000000001,"5min":1.044,"15min":1.044},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.61199999999999999,"15min":0.60599999999999998},"min":{"1min":0.51000000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.85399999999999998,"5min":0.998,"15min":0.998},"last":0.66600000000000004}]},{"osd":7,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58999999999999997,"15min":0.58799999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.78900000000000003,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.627,"5min":0.60599999999999998,"15min":0.60199999999999998},"min":{"1min":0.45300000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.80000000000000004,"5min":1.151,"15min":1.151},"last":0.55300000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32312,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339528,"statfs":{"total":96636764160,"available":96603676672,"internally_reserved":0,"allocated":1159168,"data_stored":822446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Dec 2 00:30:36 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.26000000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.501,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.37,"15min":0.36499999999999999},"min":{"1min":0.28299999999999997,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.58399999999999996,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.37}]},{"osd":2,"last update":"Sat Dec 2 00:30:36 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.54500000000000004,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40200000000000002,"15min":0.40100000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.32600000000000001}]},{"osd":3,"last update":"Sat Dec 2 00:30:36 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.39300000000000002,"15min":0.39400000000000002},"min":{"1min":0.29999999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.53100000000000003,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.436,"5min":0.42899999999999999,"15min":0.42799999999999999},"min":{"1min":0.32700000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.35099999999999998}]},{"osd":4,"last update":"Sat Dec 2 00:30:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57399999999999995,"15min":0.56899999999999995},"min":{"1min":0.42299999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.622,"15min":0.62},"min":{"1min":0.439,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.93100000000000005,"5min":1.002,"15min":1.002},"last":0.47899999999999998}]},{"osd":5,"last update":"Sat Dec 2 00:30:36 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.628,"15min":0.626},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.64700000000000002,"15min":0.64600000000000002},"min":{"1min":0.48099999999999998,"5min":0.433,"15min":0.433},"max":{"1min":0.91100000000000003,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.501}]},{"osd":6,"last update":"Sat Dec 2 00:30:36 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.60799999999999998,"15min":0.59999999999999998},"min":{"1min":0.435,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.89400000000000002,"5min":1.05,"15min":1.05},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.63800000000000001,"15min":0.64000000000000001},"min":{"1min":0.45300000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.94499999999999995,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.44500000000000001}]},{"osd":7,"last update":"Sat Dec 2 00:30:36 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.64300000000000002,"15min":0.63900000000000001},"min":{"1min":0.501,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.97699999999999998,"5min":1.028,"15min":1.028},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.626,"15min":0.626},"min":{"1min":0.46200000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.78000000000000003,"5min":0.874,"15min":0.874},"last":0.432}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-02T00:31:05.849 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-02T00:31:05.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.988+0000 7f16f0636640 1 Processor -- start 2023-12-02T00:31:05.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.988+0000 7f16f0636640 1 -- start start 2023-12-02T00:31:05.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.988+0000 7f16f0636640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f16e815d2d0 0x7f16e80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:05.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.988+0000 7f16f0636640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f16e8160f70 0x7f16e80cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:05.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.988+0000 7f16f0636640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16e81614e0 con 0x7f16e80ce480 2023-12-02T00:31:05.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.988+0000 7f16f0636640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16e81617c0 con 0x7f16e8160f70 2023-12-02T00:31:05.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.988+0000 7f16f0636640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16e8161aa0 con 0x7f16e815d2d0 2023-12-02T00:31:05.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.988+0000 7f16edbd1640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f16e8160f70 0x7f16e80cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:05.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.988+0000 7f16eebd3640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f16e80ce480 0x7f16e806b570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41784/0 (socket says 172.21.15.46:41784) 2023-12-02T00:31:05.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.988+0000 7f16edbd1640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f16e8160f70 0x7f16e80cdf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59668/0 (socket says 172.21.15.46:59668) 2023-12-02T00:31:05.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.988+0000 7f16eebd3640 1 -- 172.21.15.46:0/1182692532 learned_addr learned my addr 172.21.15.46:0/1182692532 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:31:05.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.988+0000 7f16ee3d2640 1 --2- 172.21.15.46:0/1182692532 >> v2:172.21.15.162:3300/0 conn(0x7f16e815d2d0 0x7f16e80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:05.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.988+0000 7f16edbd1640 1 -- 172.21.15.46:0/1182692532 >> v2:172.21.15.162:3300/0 conn(0x7f16e815d2d0 msgr2=0x7f16e80b8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:05.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.988+0000 7f16edbd1640 1 --2- 172.21.15.46:0/1182692532 >> v2:172.21.15.162:3300/0 conn(0x7f16e815d2d0 0x7f16e80b8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:05.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.988+0000 7f16edbd1640 1 -- 172.21.15.46:0/1182692532 >> v1:172.21.15.46:6789/0 conn(0x7f16e80ce480 legacy=0x7f16e806b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:05.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.988+0000 7f16edbd1640 1 -- 172.21.15.46:0/1182692532 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16e8161d80 con 0x7f16e8160f70 2023-12-02T00:31:05.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.988+0000 7f16ed3d0640 1 -- 172.21.15.46:0/1182692532 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1799247519 0 0) 0x7f16e81614e0 con 0x7f16e80ce480 2023-12-02T00:31:05.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.989+0000 7f16edbd1640 1 --2- 172.21.15.46:0/1182692532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16e8160f70 0x7f16e80cdf40 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f16e80b5db0 tx=0x7f16d802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f3737234736348 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:05.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.989+0000 7f16ed3d0640 1 -- 172.21.15.46:0/1182692532 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d8031830 con 0x7f16e8160f70 2023-12-02T00:31:05.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.989+0000 7f16ed3d0640 1 -- 172.21.15.46:0/1182692532 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16d8031990 con 0x7f16e8160f70 2023-12-02T00:31:05.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.989+0000 7f16ed3d0640 1 -- 172.21.15.46:0/1182692532 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d8031d20 con 0x7f16e8160f70 2023-12-02T00:31:05.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.990+0000 7f16f0636640 1 -- 172.21.15.46:0/1182692532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16e8160f70 msgr2=0x7f16e80cdf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:05.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.990+0000 7f16f0636640 1 --2- 172.21.15.46:0/1182692532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16e8160f70 0x7f16e80cdf40 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f16e80b5db0 tx=0x7f16d802b8d0 comp rx=0 tx=0).stop 2023-12-02T00:31:05.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.990+0000 7f16f0636640 1 -- 172.21.15.46:0/1182692532 shutdown_connections 2023-12-02T00:31:05.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.990+0000 7f16f0636640 1 --2- 172.21.15.46:0/1182692532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16e8160f70 0x7f16e80cdf40 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:05.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.990+0000 7f16f0636640 1 --2- 172.21.15.46:0/1182692532 >> v2:172.21.15.162:3300/0 conn(0x7f16e815d2d0 0x7f16e80b8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:05.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.990+0000 7f16f0636640 1 -- 172.21.15.46:0/1182692532 >> 172.21.15.46:0/1182692532 conn(0x7f16e8072230 msgr2=0x7f16e80cfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:05.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.990+0000 7f16f0636640 1 -- 172.21.15.46:0/1182692532 shutdown_connections 2023-12-02T00:31:05.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.990+0000 7f16f0636640 1 -- 172.21.15.46:0/1182692532 wait complete. 2023-12-02T00:31:05.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.991+0000 7f16f0636640 1 Processor -- start 2023-12-02T00:31:05.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.991+0000 7f16f0636640 1 -- start start 2023-12-02T00:31:05.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.992+0000 7f16f0636640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f16e80ce480 0x7f16e8148200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:05.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.992+0000 7f16f0636640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16e815d2d0 0x7f16e8147a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:05.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.992+0000 7f16f0636640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16e8160f70 0x7f16e8144180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:05.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.992+0000 7f16f0636640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16e8147f60 con 0x7f16e815d2d0 2023-12-02T00:31:05.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.992+0000 7f16f0636640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16e815f4a0 con 0x7f16e8160f70 2023-12-02T00:31:05.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.992+0000 7f16f0636640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16e815f780 con 0x7f16e80ce480 2023-12-02T00:31:05.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.992+0000 7f16edbd1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16e815d2d0 0x7f16e8147a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:05.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.992+0000 7f16edbd1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16e815d2d0 0x7f16e8147a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59676/0 (socket says 172.21.15.46:59676) 2023-12-02T00:31:05.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.992+0000 7f16edbd1640 1 -- 172.21.15.46:0/4078225530 learned_addr learned my addr 172.21.15.46:0/4078225530 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:05.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.992+0000 7f16ee3d2640 1 --2- 172.21.15.46:0/4078225530 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f16e80ce480 0x7f16e8148200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:05.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.992+0000 7f16eebd3640 1 --2- 172.21.15.46:0/4078225530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16e8160f70 0x7f16e8144180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:05.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.992+0000 7f16edbd1640 1 -- 172.21.15.46:0/4078225530 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f16e80ce480 msgr2=0x7f16e8148200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:05.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.992+0000 7f16edbd1640 1 --2- 172.21.15.46:0/4078225530 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f16e80ce480 0x7f16e8148200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:05.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.992+0000 7f16edbd1640 1 -- 172.21.15.46:0/4078225530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16e8160f70 msgr2=0x7f16e8144180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:05.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.992+0000 7f16edbd1640 1 --2- 172.21.15.46:0/4078225530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16e8160f70 0x7f16e8144180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:05.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.992+0000 7f16edbd1640 1 -- 172.21.15.46:0/4078225530 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d8009400 con 0x7f16e815d2d0 2023-12-02T00:31:05.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.992+0000 7f16eebd3640 1 --2- 172.21.15.46:0/4078225530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16e8160f70 0x7f16e8144180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:31:05.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.992+0000 7f16edbd1640 1 --2- 172.21.15.46:0/4078225530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16e815d2d0 0x7f16e8147a20 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f16d8000c00 tx=0x7f16d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:05.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.992+0000 7f16deffd640 1 -- 172.21.15.46:0/4078225530 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d803ba70 con 0x7f16e815d2d0 2023-12-02T00:31:05.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.993+0000 7f16deffd640 1 -- 172.21.15.46:0/4078225530 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16d803bbd0 con 0x7f16e815d2d0 2023-12-02T00:31:06.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.993+0000 7f16f0636640 1 -- 172.21.15.46:0/4078225530 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16e815fa60 con 0x7f16e815d2d0 2023-12-02T00:31:06.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.993+0000 7f16deffd640 1 -- 172.21.15.46:0/4078225530 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d80453f0 con 0x7f16e815d2d0 2023-12-02T00:31:06.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.993+0000 7f16f0636640 1 -- 172.21.15.46:0/4078225530 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16e8160020 con 0x7f16e815d2d0 2023-12-02T00:31:06.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.994+0000 7f16f0636640 1 -- 172.21.15.46:0/4078225530 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16e8160210 con 0x7f16e815d2d0 2023-12-02T00:31:06.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.994+0000 7f16deffd640 1 -- 172.21.15.46:0/4078225530 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f16d804e450 con 0x7f16e815d2d0 2023-12-02T00:31:06.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.994+0000 7f16deffd640 1 --2- 172.21.15.46:0/4078225530 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f16cc06ae40 0x7f16cc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:06.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.994+0000 7f16deffd640 1 -- 172.21.15.46:0/4078225530 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f16d80ab990 con 0x7f16e815d2d0 2023-12-02T00:31:06.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.995+0000 7f16ee3d2640 1 --2- 172.21.15.46:0/4078225530 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f16cc06ae40 0x7f16cc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:06.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.995+0000 7f16ee3d2640 1 --2- 172.21.15.46:0/4078225530 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f16cc06ae40 0x7f16cc06d300 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f16e4006fd0 tx=0x7f16e4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:06.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:05.999+0000 7f16deffd640 1 -- 172.21.15.46:0/4078225530 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f16d8076910 con 0x7f16e815d2d0 2023-12-02T00:31:06.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.171+0000 7f16f0636640 1 -- 172.21.15.46:0/4078225530 --> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f16e80b90d0 con 0x7f16cc06ae40 2023-12-02T00:31:06.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.179+0000 7f16deffd640 1 -- 172.21.15.46:0/4078225530 <== mgr.4108 v2:172.21.15.162:6832/3248644722 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284257 (secure 0 0 0) 0x7f16e400c040 con 0x7f16cc06ae40 2023-12-02T00:31:06.184 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:31:06.186 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-12-02T00:31:06.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.182+0000 7f16f0636640 1 -- 172.21.15.46:0/4078225530 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f16cc06ae40 msgr2=0x7f16cc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:06.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.182+0000 7f16f0636640 1 --2- 172.21.15.46:0/4078225530 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f16cc06ae40 0x7f16cc06d300 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f16e4006fd0 tx=0x7f16e4008040 comp rx=0 tx=0).stop 2023-12-02T00:31:06.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.182+0000 7f16f0636640 1 -- 172.21.15.46:0/4078225530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16e815d2d0 msgr2=0x7f16e8147a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:06.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.182+0000 7f16f0636640 1 --2- 172.21.15.46:0/4078225530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16e815d2d0 0x7f16e8147a20 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f16d8000c00 tx=0x7f16d80046c0 comp rx=0 tx=0).stop 2023-12-02T00:31:06.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.182+0000 7f16f0636640 1 -- 172.21.15.46:0/4078225530 shutdown_connections 2023-12-02T00:31:06.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.182+0000 7f16f0636640 1 --2- 172.21.15.46:0/4078225530 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f16cc06ae40 0x7f16cc06d300 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:06.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.182+0000 7f16f0636640 1 --2- 172.21.15.46:0/4078225530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16e8160f70 0x7f16e8144180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:06.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.182+0000 7f16f0636640 1 --2- 172.21.15.46:0/4078225530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16e815d2d0 0x7f16e8147a20 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:06.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.182+0000 7f16f0636640 1 --2- 172.21.15.46:0/4078225530 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f16e80ce480 0x7f16e8148200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:06.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.182+0000 7f16f0636640 1 -- 172.21.15.46:0/4078225530 >> 172.21.15.46:0/4078225530 conn(0x7f16e8072230 msgr2=0x7f16e806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:06.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.182+0000 7f16f0636640 1 -- 172.21.15.46:0/4078225530 shutdown_connections 2023-12-02T00:31:06.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.182+0000 7f16f0636640 1 -- 172.21.15.46:0/4078225530 wait complete. 2023-12-02T00:31:06.195 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":119,"stamp":"2023-12-02T00:31:05.191363+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":32,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238420,"kb_used_data":5612,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736300,"statfs":{"total":773094113280,"available":772849971200,"internally_reserved":0,"allocated":5746688,"data_stored":3047349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.377699"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.827720+0000","last_change":"2023-12-02T00:31:00.827720+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.827720+0000","last_undegraded":"2023-12-02T00:31:00.827720+0000","last_fullsized":"2023-12-02T00:31:00.827720+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.824870+0000","last_change":"2023-12-02T00:31:00.824870+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.824870+0000","last_undegraded":"2023-12-02T00:31:00.824870+0000","last_fullsized":"2023-12-02T00:31:00.824870+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.826847+0000","last_change":"2023-12-02T00:31:00.826847+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.826847+0000","last_undegraded":"2023-12-02T00:31:00.826847+0000","last_fullsized":"2023-12-02T00:31:00.826847+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.823450+0000","last_change":"2023-12-02T00:31:00.823450+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.823450+0000","last_undegraded":"2023-12-02T00:31:00.823450+0000","last_fullsized":"2023-12-02T00:31:00.823450+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.818839+0000","last_change":"2023-12-02T00:31:00.818839+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.818839+0000","last_undegraded":"2023-12-02T00:31:00.818839+0000","last_fullsized":"2023-12-02T00:31:00.818839+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.828726+0000","last_change":"2023-12-02T00:31:00.828726+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.828726+0000","last_undegraded":"2023-12-02T00:31:00.828726+0000","last_fullsized":"2023-12-02T00:31:00.828726+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.827162+0000","last_change":"2023-12-02T00:31:00.827162+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.827162+0000","last_undegraded":"2023-12-02T00:31:00.827162+0000","last_fullsized":"2023-12-02T00:31:00.827162+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.834673+0000","last_change":"2023-12-02T00:31:00.834673+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.834673+0000","last_undegraded":"2023-12-02T00:31:00.834673+0000","last_fullsized":"2023-12-02T00:31:00.834673+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.833708+0000","last_change":"2023-12-02T00:31:00.833708+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.833708+0000","last_undegraded":"2023-12-02T00:31:00.833708+0000","last_fullsized":"2023-12-02T00:31:00.833708+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.829160+0000","last_change":"2023-12-02T00:31:00.829160+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.829160+0000","last_undegraded":"2023-12-02T00:31:00.829160+0000","last_fullsized":"2023-12-02T00:31:00.829160+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.818160+0000","last_change":"2023-12-02T00:31:00.818160+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.818160+0000","last_undegraded":"2023-12-02T00:31:00.818160+0000","last_fullsized":"2023-12-02T00:31:00.818160+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.831142+0000","last_change":"2023-12-02T00:31:00.831142+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.831142+0000","last_undegraded":"2023-12-02T00:31:00.831142+0000","last_fullsized":"2023-12-02T00:31:00.831142+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.827013+0000","last_change":"2023-12-02T00:31:00.827013+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.827013+0000","last_undegraded":"2023-12-02T00:31:00.827013+0000","last_fullsized":"2023-12-02T00:31:00.827013+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.826562+0000","last_change":"2023-12-02T00:31:00.826562+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.826562+0000","last_undegraded":"2023-12-02T00:31:00.826562+0000","last_fullsized":"2023-12-02T00:31:00.826562+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.823060+0000","last_change":"2023-12-02T00:31:00.823060+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.823060+0000","last_undegraded":"2023-12-02T00:31:00.823060+0000","last_fullsized":"2023-12-02T00:31:00.823060+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.825606+0000","last_change":"2023-12-02T00:31:00.825606+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.825606+0000","last_undegraded":"2023-12-02T00:31:00.825606+0000","last_fullsized":"2023-12-02T00:31:00.825606+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.825954+0000","last_change":"2023-12-02T00:31:00.825954+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.825954+0000","last_undegraded":"2023-12-02T00:31:00.825954+0000","last_fullsized":"2023-12-02T00:31:00.825954+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.824312+0000","last_change":"2023-12-02T00:31:00.824312+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.824312+0000","last_undegraded":"2023-12-02T00:31:00.824312+0000","last_fullsized":"2023-12-02T00:31:00.824312+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.825253+0000","last_change":"2023-12-02T00:31:00.825253+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.825253+0000","last_undegraded":"2023-12-02T00:31:00.825253+0000","last_fullsized":"2023-12-02T00:31:00.825253+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.825702+0000","last_change":"2023-12-02T00:31:00.825702+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.825702+0000","last_undegraded":"2023-12-02T00:31:00.825702+0000","last_fullsized":"2023-12-02T00:31:00.825702+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.826391+0000","last_change":"2023-12-02T00:31:00.826391+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.826391+0000","last_undegraded":"2023-12-02T00:31:00.826391+0000","last_fullsized":"2023-12-02T00:31:00.826391+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.831192+0000","last_change":"2023-12-02T00:31:00.831192+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.831192+0000","last_undegraded":"2023-12-02T00:31:00.831192+0000","last_fullsized":"2023-12-02T00:31:00.831192+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.818154+0000","last_change":"2023-12-02T00:31:00.818154+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.818154+0000","last_undegraded":"2023-12-02T00:31:00.818154+0000","last_fullsized":"2023-12-02T00:31:00.818154+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.824629+0000","last_change":"2023-12-02T00:31:00.824629+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.824629+0000","last_undegraded":"2023-12-02T00:31:00.824629+0000","last_fullsized":"2023-12-02T00:31:00.824629+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.824088+0000","last_change":"2023-12-02T00:31:00.824088+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.824088+0000","last_undegraded":"2023-12-02T00:31:00.824088+0000","last_fullsized":"2023-12-02T00:31:00.824088+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.829648+0000","last_change":"2023-12-02T00:31:00.829648+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.829648+0000","last_undegraded":"2023-12-02T00:31:00.829648+0000","last_fullsized":"2023-12-02T00:31:00.829648+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.827734+0000","last_change":"2023-12-02T00:31:00.827734+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.827734+0000","last_undegraded":"2023-12-02T00:31:00.827734+0000","last_fullsized":"2023-12-02T00:31:00.827734+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.824743+0000","last_change":"2023-12-02T00:31:00.824743+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.824743+0000","last_undegraded":"2023-12-02T00:31:00.824743+0000","last_fullsized":"2023-12-02T00:31:00.824743+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.823555+0000","last_change":"2023-12-02T00:31:00.823555+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.823555+0000","last_undegraded":"2023-12-02T00:31:00.823555+0000","last_fullsized":"2023-12-02T00:31:00.823555+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.823051+0000","last_change":"2023-12-02T00:31:00.823051+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.823051+0000","last_undegraded":"2023-12-02T00:31:00.823051+0000","last_fullsized":"2023-12-02T00:31:00.823051+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.823884+0000","last_change":"2023-12-02T00:31:00.823884+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.823884+0000","last_undegraded":"2023-12-02T00:31:00.823884+0000","last_fullsized":"2023-12-02T00:31:00.823884+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.827493+0000","last_change":"2023-12-02T00:31:00.827493+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.827493+0000","last_undegraded":"2023-12-02T00:31:00.827493+0000","last_fullsized":"2023-12-02T00:31:00.827493+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.824063+0000","last_change":"2023-12-02T00:31:00.824063+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.824063+0000","last_undegraded":"2023-12-02T00:31:00.824063+0000","last_fullsized":"2023-12-02T00:31:00.824063+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.836660+0000","last_change":"2023-12-02T00:31:00.832088+0000","last_active":"2023-12-02T00:31:00.836660+0000","last_peered":"2023-12-02T00:31:00.836660+0000","last_clean":"2023-12-02T00:31:00.836660+0000","last_became_active":"2023-12-02T00:31:00.830872+0000","last_became_peered":"2023-12-02T00:31:00.830872+0000","last_unstale":"2023-12-02T00:31:00.836660+0000","last_undegraded":"2023-12-02T00:31:00.836660+0000","last_fullsized":"2023-12-02T00:31:00.836660+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:18:11.861982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.825822+0000","last_change":"2023-12-02T00:31:00.825822+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.825822+0000","last_undegraded":"2023-12-02T00:31:00.825822+0000","last_fullsized":"2023-12-02T00:31:00.825822+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.836248+0000","last_change":"2023-12-02T00:31:00.834787+0000","last_active":"2023-12-02T00:31:00.836248+0000","last_peered":"2023-12-02T00:31:00.836248+0000","last_clean":"2023-12-02T00:31:00.836248+0000","last_became_active":"2023-12-02T00:31:00.834357+0000","last_became_peered":"2023-12-02T00:31:00.834357+0000","last_unstale":"2023-12-02T00:31:00.836248+0000","last_undegraded":"2023-12-02T00:31:00.836248+0000","last_fullsized":"2023-12-02T00:31:00.836248+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:57:02.728754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.826663+0000","last_change":"2023-12-02T00:31:00.826663+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.826663+0000","last_undegraded":"2023-12-02T00:31:00.826663+0000","last_fullsized":"2023-12-02T00:31:00.826663+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.839027+0000","last_change":"2023-12-02T00:31:00.837792+0000","last_active":"2023-12-02T00:31:00.839027+0000","last_peered":"2023-12-02T00:31:00.839027+0000","last_clean":"2023-12-02T00:31:00.839027+0000","last_became_active":"2023-12-02T00:31:00.837471+0000","last_became_peered":"2023-12-02T00:31:00.837471+0000","last_unstale":"2023-12-02T00:31:00.839027+0000","last_undegraded":"2023-12-02T00:31:00.839027+0000","last_fullsized":"2023-12-02T00:31:00.839027+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:33:23.942055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.823658+0000","last_change":"2023-12-02T00:31:00.823658+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.823658+0000","last_undegraded":"2023-12-02T00:31:00.823658+0000","last_fullsized":"2023-12-02T00:31:00.823658+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.837462+0000","last_change":"2023-12-02T00:31:00.834209+0000","last_active":"2023-12-02T00:31:00.837462+0000","last_peered":"2023-12-02T00:31:00.837462+0000","last_clean":"2023-12-02T00:31:00.837462+0000","last_became_active":"2023-12-02T00:31:00.833798+0000","last_became_peered":"2023-12-02T00:31:00.833798+0000","last_unstale":"2023-12-02T00:31:00.837462+0000","last_undegraded":"2023-12-02T00:31:00.837462+0000","last_fullsized":"2023-12-02T00:31:00.837462+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:20:10.886591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.818382+0000","last_change":"2023-12-02T00:31:00.818382+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.818382+0000","last_undegraded":"2023-12-02T00:31:00.818382+0000","last_fullsized":"2023-12-02T00:31:00.818382+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.824844+0000","last_change":"2023-12-02T00:31:00.821690+0000","last_active":"2023-12-02T00:31:00.824844+0000","last_peered":"2023-12-02T00:31:00.824844+0000","last_clean":"2023-12-02T00:31:00.824844+0000","last_became_active":"2023-12-02T00:31:00.821365+0000","last_became_peered":"2023-12-02T00:31:00.821365+0000","last_unstale":"2023-12-02T00:31:00.824844+0000","last_undegraded":"2023-12-02T00:31:00.824844+0000","last_fullsized":"2023-12-02T00:31:00.824844+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:57:02.728754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.823332+0000","last_change":"2023-12-02T00:31:00.823332+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.823332+0000","last_undegraded":"2023-12-02T00:31:00.823332+0000","last_fullsized":"2023-12-02T00:31:00.823332+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.834534+0000","last_change":"2023-12-02T00:31:00.831123+0000","last_active":"2023-12-02T00:31:00.834534+0000","last_peered":"2023-12-02T00:31:00.834534+0000","last_clean":"2023-12-02T00:31:00.834534+0000","last_became_active":"2023-12-02T00:31:00.830870+0000","last_became_peered":"2023-12-02T00:31:00.830870+0000","last_unstale":"2023-12-02T00:31:00.834534+0000","last_undegraded":"2023-12-02T00:31:00.834534+0000","last_fullsized":"2023-12-02T00:31:00.834534+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:56:41.004341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.822277+0000","last_change":"2023-12-02T00:31:00.822277+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.822277+0000","last_undegraded":"2023-12-02T00:31:00.822277+0000","last_fullsized":"2023-12-02T00:31:00.822277+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.839350+0000","last_change":"2023-12-02T00:31:00.834261+0000","last_active":"2023-12-02T00:31:00.839350+0000","last_peered":"2023-12-02T00:31:00.839350+0000","last_clean":"2023-12-02T00:31:00.839350+0000","last_became_active":"2023-12-02T00:31:00.833992+0000","last_became_peered":"2023-12-02T00:31:00.833992+0000","last_unstale":"2023-12-02T00:31:00.839350+0000","last_undegraded":"2023-12-02T00:31:00.839350+0000","last_fullsized":"2023-12-02T00:31:00.839350+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:09:23.731017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.823236+0000","last_change":"2023-12-02T00:31:00.823236+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.823236+0000","last_undegraded":"2023-12-02T00:31:00.823236+0000","last_fullsized":"2023-12-02T00:31:00.823236+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.832488+0000","last_change":"2023-12-02T00:31:00.828451+0000","last_active":"2023-12-02T00:31:00.832488+0000","last_peered":"2023-12-02T00:31:00.832488+0000","last_clean":"2023-12-02T00:31:00.832488+0000","last_became_active":"2023-12-02T00:31:00.828164+0000","last_became_peered":"2023-12-02T00:31:00.828164+0000","last_unstale":"2023-12-02T00:31:00.832488+0000","last_undegraded":"2023-12-02T00:31:00.832488+0000","last_fullsized":"2023-12-02T00:31:00.832488+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T12:24:56.280705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.818389+0000","last_change":"2023-12-02T00:31:00.818389+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.818389+0000","last_undegraded":"2023-12-02T00:31:00.818389+0000","last_fullsized":"2023-12-02T00:31:00.818389+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.839294+0000","last_change":"2023-12-02T00:31:00.835042+0000","last_active":"2023-12-02T00:31:00.839294+0000","last_peered":"2023-12-02T00:31:00.839294+0000","last_clean":"2023-12-02T00:31:00.839294+0000","last_became_active":"2023-12-02T00:31:00.834764+0000","last_became_peered":"2023-12-02T00:31:00.834764+0000","last_unstale":"2023-12-02T00:31:00.839294+0000","last_undegraded":"2023-12-02T00:31:00.839294+0000","last_fullsized":"2023-12-02T00:31:00.839294+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:54:07.912968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.827202+0000","last_change":"2023-12-02T00:31:00.827202+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.827202+0000","last_undegraded":"2023-12-02T00:31:00.827202+0000","last_fullsized":"2023-12-02T00:31:00.827202+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.826894+0000","last_change":"2023-12-02T00:31:00.821222+0000","last_active":"2023-12-02T00:31:00.826894+0000","last_peered":"2023-12-02T00:31:00.826894+0000","last_clean":"2023-12-02T00:31:00.826894+0000","last_became_active":"2023-12-02T00:31:00.820914+0000","last_became_peered":"2023-12-02T00:31:00.820914+0000","last_unstale":"2023-12-02T00:31:00.826894+0000","last_undegraded":"2023-12-02T00:31:00.826894+0000","last_fullsized":"2023-12-02T00:31:00.826894+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T10:06:37.837138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.823398+0000","last_change":"2023-12-02T00:31:00.823398+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.823398+0000","last_undegraded":"2023-12-02T00:31:00.823398+0000","last_fullsized":"2023-12-02T00:31:00.823398+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.840140+0000","last_change":"2023-12-02T00:31:00.837992+0000","last_active":"2023-12-02T00:31:00.840140+0000","last_peered":"2023-12-02T00:31:00.840140+0000","last_clean":"2023-12-02T00:31:00.840140+0000","last_became_active":"2023-12-02T00:31:00.837743+0000","last_became_peered":"2023-12-02T00:31:00.837743+0000","last_unstale":"2023-12-02T00:31:00.840140+0000","last_undegraded":"2023-12-02T00:31:00.840140+0000","last_fullsized":"2023-12-02T00:31:00.840140+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:00:47.235125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.818271+0000","last_change":"2023-12-02T00:31:00.818271+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.818271+0000","last_undegraded":"2023-12-02T00:31:00.818271+0000","last_fullsized":"2023-12-02T00:31:00.818271+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.842235+0000","last_change":"2023-12-02T00:31:00.840634+0000","last_active":"2023-12-02T00:31:00.842235+0000","last_peered":"2023-12-02T00:31:00.842235+0000","last_clean":"2023-12-02T00:31:00.842235+0000","last_became_active":"2023-12-02T00:31:00.840359+0000","last_became_peered":"2023-12-02T00:31:00.840359+0000","last_unstale":"2023-12-02T00:31:00.842235+0000","last_undegraded":"2023-12-02T00:31:00.842235+0000","last_fullsized":"2023-12-02T00:31:00.842235+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:34:01.547200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.826762+0000","last_change":"2023-12-02T00:31:00.826762+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.826762+0000","last_undegraded":"2023-12-02T00:31:00.826762+0000","last_fullsized":"2023-12-02T00:31:00.826762+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.832524+0000","last_change":"2023-12-02T00:31:00.829212+0000","last_active":"2023-12-02T00:31:00.832524+0000","last_peered":"2023-12-02T00:31:00.832524+0000","last_clean":"2023-12-02T00:31:00.832524+0000","last_became_active":"2023-12-02T00:31:00.828848+0000","last_became_peered":"2023-12-02T00:31:00.828848+0000","last_unstale":"2023-12-02T00:31:00.832524+0000","last_undegraded":"2023-12-02T00:31:00.832524+0000","last_fullsized":"2023-12-02T00:31:00.832524+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T01:12:56.020427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.822967+0000","last_change":"2023-12-02T00:31:00.822967+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.822967+0000","last_undegraded":"2023-12-02T00:31:00.822967+0000","last_fullsized":"2023-12-02T00:31:00.822967+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.839380+0000","last_change":"2023-12-02T00:31:00.834656+0000","last_active":"2023-12-02T00:31:00.839380+0000","last_peered":"2023-12-02T00:31:00.839380+0000","last_clean":"2023-12-02T00:31:00.839380+0000","last_became_active":"2023-12-02T00:31:00.834359+0000","last_became_peered":"2023-12-02T00:31:00.834359+0000","last_unstale":"2023-12-02T00:31:00.839380+0000","last_undegraded":"2023-12-02T00:31:00.839380+0000","last_fullsized":"2023-12-02T00:31:00.839380+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T01:33:59.617275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.824188+0000","last_change":"2023-12-02T00:31:00.824188+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.824188+0000","last_undegraded":"2023-12-02T00:31:00.824188+0000","last_fullsized":"2023-12-02T00:31:00.824188+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.840126+0000","last_change":"2023-12-02T00:31:00.837228+0000","last_active":"2023-12-02T00:31:00.840126+0000","last_peered":"2023-12-02T00:31:00.840126+0000","last_clean":"2023-12-02T00:31:00.840126+0000","last_became_active":"2023-12-02T00:31:00.836943+0000","last_became_peered":"2023-12-02T00:31:00.836943+0000","last_unstale":"2023-12-02T00:31:00.840126+0000","last_undegraded":"2023-12-02T00:31:00.840126+0000","last_fullsized":"2023-12-02T00:31:00.840126+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:51:08.104561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.823005+0000","last_change":"2023-12-02T00:31:00.823005+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.823005+0000","last_undegraded":"2023-12-02T00:31:00.823005+0000","last_fullsized":"2023-12-02T00:31:00.823005+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.842246+0000","last_change":"2023-12-02T00:31:00.840645+0000","last_active":"2023-12-02T00:31:00.842246+0000","last_peered":"2023-12-02T00:31:00.842246+0000","last_clean":"2023-12-02T00:31:00.842246+0000","last_became_active":"2023-12-02T00:31:00.840295+0000","last_became_peered":"2023-12-02T00:31:00.840295+0000","last_unstale":"2023-12-02T00:31:00.842246+0000","last_undegraded":"2023-12-02T00:31:00.842246+0000","last_fullsized":"2023-12-02T00:31:00.842246+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:41:19.966082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.822868+0000","last_change":"2023-12-02T00:31:00.822868+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.822868+0000","last_undegraded":"2023-12-02T00:31:00.822868+0000","last_fullsized":"2023-12-02T00:31:00.822868+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.840706+0000","last_change":"2023-12-02T00:31:00.839972+0000","last_active":"2023-12-02T00:31:00.840706+0000","last_peered":"2023-12-02T00:31:00.840706+0000","last_clean":"2023-12-02T00:31:00.840706+0000","last_became_active":"2023-12-02T00:31:00.839581+0000","last_became_peered":"2023-12-02T00:31:00.839581+0000","last_unstale":"2023-12-02T00:31:00.840706+0000","last_undegraded":"2023-12-02T00:31:00.840706+0000","last_fullsized":"2023-12-02T00:31:00.840706+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:33:23.942055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.825542+0000","last_change":"2023-12-02T00:31:00.825542+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.825542+0000","last_undegraded":"2023-12-02T00:31:00.825542+0000","last_fullsized":"2023-12-02T00:31:00.825542+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.833409+0000","last_change":"2023-12-02T00:31:00.832413+0000","last_active":"2023-12-02T00:31:00.833409+0000","last_peered":"2023-12-02T00:31:00.833409+0000","last_clean":"2023-12-02T00:31:00.833409+0000","last_became_active":"2023-12-02T00:31:00.832127+0000","last_became_peered":"2023-12-02T00:31:00.832127+0000","last_unstale":"2023-12-02T00:31:00.833409+0000","last_undegraded":"2023-12-02T00:31:00.833409+0000","last_fullsized":"2023-12-02T00:31:00.833409+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:55:28.223962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.818333+0000","last_change":"2023-12-02T00:31:00.818333+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.818333+0000","last_undegraded":"2023-12-02T00:31:00.818333+0000","last_fullsized":"2023-12-02T00:31:00.818333+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.840093+0000","last_change":"2023-12-02T00:31:00.837851+0000","last_active":"2023-12-02T00:31:00.840093+0000","last_peered":"2023-12-02T00:31:00.840093+0000","last_clean":"2023-12-02T00:31:00.840093+0000","last_became_active":"2023-12-02T00:31:00.837581+0000","last_became_peered":"2023-12-02T00:31:00.837581+0000","last_unstale":"2023-12-02T00:31:00.840093+0000","last_undegraded":"2023-12-02T00:31:00.840093+0000","last_fullsized":"2023-12-02T00:31:00.840093+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:17:18.074493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.826478+0000","last_change":"2023-12-02T00:31:00.826478+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.826478+0000","last_undegraded":"2023-12-02T00:31:00.826478+0000","last_fullsized":"2023-12-02T00:31:00.826478+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.836731+0000","last_change":"2023-12-02T00:31:00.832751+0000","last_active":"2023-12-02T00:31:00.836731+0000","last_peered":"2023-12-02T00:31:00.836731+0000","last_clean":"2023-12-02T00:31:00.836731+0000","last_became_active":"2023-12-02T00:31:00.832466+0000","last_became_peered":"2023-12-02T00:31:00.832466+0000","last_unstale":"2023-12-02T00:31:00.836731+0000","last_undegraded":"2023-12-02T00:31:00.836731+0000","last_fullsized":"2023-12-02T00:31:00.836731+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:01:15.991829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.826326+0000","last_change":"2023-12-02T00:31:00.826326+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.826326+0000","last_undegraded":"2023-12-02T00:31:00.826326+0000","last_fullsized":"2023-12-02T00:31:00.826326+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.836131+0000","last_change":"2023-12-02T00:31:00.833503+0000","last_active":"2023-12-02T00:31:00.836131+0000","last_peered":"2023-12-02T00:31:00.836131+0000","last_clean":"2023-12-02T00:31:00.836131+0000","last_became_active":"2023-12-02T00:31:00.833219+0000","last_became_peered":"2023-12-02T00:31:00.833219+0000","last_unstale":"2023-12-02T00:31:00.836131+0000","last_undegraded":"2023-12-02T00:31:00.836131+0000","last_fullsized":"2023-12-02T00:31:00.836131+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T09:28:07.117685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.824313+0000","last_change":"2023-12-02T00:31:00.824313+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.824313+0000","last_undegraded":"2023-12-02T00:31:00.824313+0000","last_fullsized":"2023-12-02T00:31:00.824313+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.842212+0000","last_change":"2023-12-02T00:31:00.840501+0000","last_active":"2023-12-02T00:31:00.842212+0000","last_peered":"2023-12-02T00:31:00.842212+0000","last_clean":"2023-12-02T00:31:00.842212+0000","last_became_active":"2023-12-02T00:31:00.840227+0000","last_became_peered":"2023-12-02T00:31:00.840227+0000","last_unstale":"2023-12-02T00:31:00.842212+0000","last_undegraded":"2023-12-02T00:31:00.842212+0000","last_fullsized":"2023-12-02T00:31:00.842212+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:40:05.075383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.822165+0000","last_change":"2023-12-02T00:31:00.822165+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.822165+0000","last_undegraded":"2023-12-02T00:31:00.822165+0000","last_fullsized":"2023-12-02T00:31:00.822165+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.836552+0000","last_change":"2023-12-02T00:31:00.833798+0000","last_active":"2023-12-02T00:31:00.836552+0000","last_peered":"2023-12-02T00:31:00.836552+0000","last_clean":"2023-12-02T00:31:00.836552+0000","last_became_active":"2023-12-02T00:31:00.833537+0000","last_became_peered":"2023-12-02T00:31:00.833537+0000","last_unstale":"2023-12-02T00:31:00.836552+0000","last_undegraded":"2023-12-02T00:31:00.836552+0000","last_fullsized":"2023-12-02T00:31:00.836552+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:54:17.615040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.826590+0000","last_change":"2023-12-02T00:31:00.826590+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.826590+0000","last_undegraded":"2023-12-02T00:31:00.826590+0000","last_fullsized":"2023-12-02T00:31:00.826590+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.830926+0000","last_change":"2023-12-02T00:31:00.829323+0000","last_active":"2023-12-02T00:31:00.830926+0000","last_peered":"2023-12-02T00:31:00.830926+0000","last_clean":"2023-12-02T00:31:00.830926+0000","last_became_active":"2023-12-02T00:31:00.829048+0000","last_became_peered":"2023-12-02T00:31:00.829048+0000","last_unstale":"2023-12-02T00:31:00.830926+0000","last_undegraded":"2023-12-02T00:31:00.830926+0000","last_fullsized":"2023-12-02T00:31:00.830926+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:33:51.372611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.825234+0000","last_change":"2023-12-02T00:31:00.825234+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.825234+0000","last_undegraded":"2023-12-02T00:31:00.825234+0000","last_fullsized":"2023-12-02T00:31:00.825234+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.839371+0000","last_change":"2023-12-02T00:31:00.835706+0000","last_active":"2023-12-02T00:31:00.839371+0000","last_peered":"2023-12-02T00:31:00.839371+0000","last_clean":"2023-12-02T00:31:00.839371+0000","last_became_active":"2023-12-02T00:31:00.835381+0000","last_became_peered":"2023-12-02T00:31:00.835381+0000","last_unstale":"2023-12-02T00:31:00.839371+0000","last_undegraded":"2023-12-02T00:31:00.839371+0000","last_fullsized":"2023-12-02T00:31:00.839371+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:14:37.638316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.828284+0000","last_change":"2023-12-02T00:31:00.828284+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.828284+0000","last_undegraded":"2023-12-02T00:31:00.828284+0000","last_fullsized":"2023-12-02T00:31:00.828284+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.836694+0000","last_change":"2023-12-02T00:31:00.832747+0000","last_active":"2023-12-02T00:31:00.836694+0000","last_peered":"2023-12-02T00:31:00.836694+0000","last_clean":"2023-12-02T00:31:00.836694+0000","last_became_active":"2023-12-02T00:31:00.832477+0000","last_became_peered":"2023-12-02T00:31:00.832477+0000","last_unstale":"2023-12-02T00:31:00.836694+0000","last_undegraded":"2023-12-02T00:31:00.836694+0000","last_fullsized":"2023-12-02T00:31:00.836694+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T10:47:44.799679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.823172+0000","last_change":"2023-12-02T00:31:00.823172+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.823172+0000","last_undegraded":"2023-12-02T00:31:00.823172+0000","last_fullsized":"2023-12-02T00:31:00.823172+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.832174+0000","last_change":"2023-12-02T00:31:00.828668+0000","last_active":"2023-12-02T00:31:00.832174+0000","last_peered":"2023-12-02T00:31:00.832174+0000","last_clean":"2023-12-02T00:31:00.832174+0000","last_became_active":"2023-12-02T00:31:00.828392+0000","last_became_peered":"2023-12-02T00:31:00.828392+0000","last_unstale":"2023-12-02T00:31:00.832174+0000","last_undegraded":"2023-12-02T00:31:00.832174+0000","last_fullsized":"2023-12-02T00:31:00.832174+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T09:28:07.117685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.822281+0000","last_change":"2023-12-02T00:31:00.822281+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.822281+0000","last_undegraded":"2023-12-02T00:31:00.822281+0000","last_fullsized":"2023-12-02T00:31:00.822281+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.842664+0000","last_change":"2023-12-02T00:31:00.841414+0000","last_active":"2023-12-02T00:31:00.842664+0000","last_peered":"2023-12-02T00:31:00.842664+0000","last_clean":"2023-12-02T00:31:00.842664+0000","last_became_active":"2023-12-02T00:31:00.841049+0000","last_became_peered":"2023-12-02T00:31:00.841049+0000","last_unstale":"2023-12-02T00:31:00.842664+0000","last_undegraded":"2023-12-02T00:31:00.842664+0000","last_fullsized":"2023-12-02T00:31:00.842664+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:34:41.347880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.826447+0000","last_change":"2023-12-02T00:31:00.826447+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.826447+0000","last_undegraded":"2023-12-02T00:31:00.826447+0000","last_fullsized":"2023-12-02T00:31:00.826447+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.839129+0000","last_change":"2023-12-02T00:31:00.838503+0000","last_active":"2023-12-02T00:31:00.839129+0000","last_peered":"2023-12-02T00:31:00.839129+0000","last_clean":"2023-12-02T00:31:00.839129+0000","last_became_active":"2023-12-02T00:31:00.838125+0000","last_became_peered":"2023-12-02T00:31:00.838125+0000","last_unstale":"2023-12-02T00:31:00.839129+0000","last_undegraded":"2023-12-02T00:31:00.839129+0000","last_fullsized":"2023-12-02T00:31:00.839129+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:00:03.107141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.822883+0000","last_change":"2023-12-02T00:31:00.822883+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.822883+0000","last_undegraded":"2023-12-02T00:31:00.822883+0000","last_fullsized":"2023-12-02T00:31:00.822883+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.833802+0000","last_change":"2023-12-02T00:31:00.828726+0000","last_active":"2023-12-02T00:31:00.833802+0000","last_peered":"2023-12-02T00:31:00.833802+0000","last_clean":"2023-12-02T00:31:00.833802+0000","last_became_active":"2023-12-02T00:31:00.828424+0000","last_became_peered":"2023-12-02T00:31:00.828424+0000","last_unstale":"2023-12-02T00:31:00.833802+0000","last_undegraded":"2023-12-02T00:31:00.833802+0000","last_fullsized":"2023-12-02T00:31:00.833802+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T02:42:58.558235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.823153+0000","last_change":"2023-12-02T00:31:00.823153+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.823153+0000","last_undegraded":"2023-12-02T00:31:00.823153+0000","last_fullsized":"2023-12-02T00:31:00.823153+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.840024+0000","last_change":"2023-12-02T00:31:00.837338+0000","last_active":"2023-12-02T00:31:00.840024+0000","last_peered":"2023-12-02T00:31:00.840024+0000","last_clean":"2023-12-02T00:31:00.840024+0000","last_became_active":"2023-12-02T00:31:00.837017+0000","last_became_peered":"2023-12-02T00:31:00.837017+0000","last_unstale":"2023-12-02T00:31:00.840024+0000","last_undegraded":"2023-12-02T00:31:00.840024+0000","last_fullsized":"2023-12-02T00:31:00.840024+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:06:02.637069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.824707+0000","last_change":"2023-12-02T00:31:00.824707+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.824707+0000","last_undegraded":"2023-12-02T00:31:00.824707+0000","last_fullsized":"2023-12-02T00:31:00.824707+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.832294+0000","last_change":"2023-12-02T00:31:00.829255+0000","last_active":"2023-12-02T00:31:00.832294+0000","last_peered":"2023-12-02T00:31:00.832294+0000","last_clean":"2023-12-02T00:31:00.832294+0000","last_became_active":"2023-12-02T00:31:00.828976+0000","last_became_peered":"2023-12-02T00:31:00.828976+0000","last_unstale":"2023-12-02T00:31:00.832294+0000","last_undegraded":"2023-12-02T00:31:00.832294+0000","last_fullsized":"2023-12-02T00:31:00.832294+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:30:57.889016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":192,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-12-02T00:30:59.819736+0000","last_change":"2023-12-02T00:28:39.379380+0000","last_active":"2023-12-02T00:30:59.819736+0000","last_peered":"2023-12-02T00:30:59.819736+0000","last_clean":"2023-12-02T00:30:59.819736+0000","last_became_active":"2023-12-02T00:28:39.378255+0000","last_became_peered":"2023-12-02T00:28:39.378255+0000","last_unstale":"2023-12-02T00:30:59.819736+0000","last_undegraded":"2023-12-02T00:30:59.819736+0000","last_fullsized":"2023-12-02T00:30:59.819736+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:38.177345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:38.177345+0000","last_clean_scrub_stamp":"2023-12-02T00:28:38.177345+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:21:08.582412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28156,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343684,"statfs":{"total":96636764160,"available":96607932416,"internally_reserved":0,"allocated":1163264,"data_stored":823065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.50900000000000001,"15min":0.499},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.48499999999999999}]},{"osd":1,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53400000000000003,"15min":0.53000000000000003},"min":{"1min":0.40400000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.54700000000000004,"15min":0.54400000000000004},"min":{"1min":0.46100000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.64500000000000002}]},{"osd":2,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54200000000000004,"15min":0.54400000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.67000000000000004,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.54300000000000004,"15min":0.53400000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.624}]},{"osd":3,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.53000000000000003,"15min":0.52200000000000002},"min":{"1min":0.48099999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.57299999999999995,"15min":0.57499999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.74399999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.68100000000000005}]},{"osd":4,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.50700000000000001,"15min":0.51100000000000001},"min":{"1min":0.29799999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.72199999999999998,"5min":0.745,"15min":0.745},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.51500000000000001,"15min":0.52100000000000002},"min":{"1min":0.33800000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.61799999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.60699999999999998}]},{"osd":5,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54700000000000004,"15min":0.55000000000000004},"min":{"1min":0.41899999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.78700000000000003,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.56100000000000005,"15min":0.56599999999999995},"min":{"1min":0.39500000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.63900000000000001,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.58199999999999996}]},{"osd":6,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.61799999999999999,"15min":0.627},"min":{"1min":0.40100000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.82899999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.60599999999999998,"15min":0.61299999999999999},"min":{"1min":0.44500000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76600000000000001,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.66500000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27592,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":581632,"data_stored":238972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.49199999999999999,"15min":0.48299999999999998},"min":{"1min":0.36499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.50700000000000001,"15min":0.501},"min":{"1min":0.40000000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.74099999999999999,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.64000000000000001}]},{"osd":1,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.51000000000000001,"15min":0.50600000000000001},"min":{"1min":0.33000000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.72599999999999998,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.53600000000000003,"15min":0.53300000000000003},"min":{"1min":0.39400000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.80900000000000005,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.59399999999999997}]},{"osd":2,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.52000000000000002,"15min":0.51200000000000001},"min":{"1min":0.38,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55500000000000005,"15min":0.55700000000000005},"min":{"1min":0.40300000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.77400000000000002,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.66100000000000003}]},{"osd":3,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57599999999999996,"15min":0.57299999999999995},"min":{"1min":0.45200000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.76300000000000001,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.622},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57199999999999995,"15min":0.56999999999999995},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.57699999999999996}]},{"osd":4,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.35399999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.71299999999999997,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52500000000000002,"15min":0.52300000000000002},"min":{"1min":0.34100000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73699999999999999,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.38600000000000001}]},{"osd":5,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.33400000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58799999999999997,"15min":0.59299999999999997},"min":{"1min":0.42399999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.40799999999999997}]},{"osd":7,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.35999999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.442},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.42399999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":569344,"data_stored":232697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.51700000000000002,"15min":0.50600000000000001},"min":{"1min":0.29099999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.73899999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.51600000000000001,"15min":0.51100000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.72899999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.54500000000000004}]},{"osd":1,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.51400000000000001,"15min":0.505},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.754,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55300000000000005,"15min":0.55000000000000004},"min":{"1min":0.36199999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.69499999999999995,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.42399999999999999}]},{"osd":2,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.56000000000000005,"15min":0.55100000000000005},"min":{"1min":0.41699999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.77500000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.53600000000000003,"15min":0.53300000000000003},"min":{"1min":0.34399999999999997,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.749,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.65100000000000002}]},{"osd":3,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58099999999999996,"15min":0.57699999999999996},"min":{"1min":0.39300000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.58299999999999996,"15min":0.58199999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.72499999999999998,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.58599999999999997}]},{"osd":4,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54300000000000004,"15min":0.54600000000000004},"min":{"1min":0.34799999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69899999999999995,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58699999999999997,"15min":0.58799999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.745}]},{"osd":6,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.56599999999999995,"15min":0.57099999999999995},"min":{"1min":0.36899999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.72899999999999998,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.624},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.59199999999999997,"15min":0.59599999999999997},"min":{"1min":0.435,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.73499999999999999}]},{"osd":7,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.42699999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.82099999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.63100000000000001,"15min":0.63500000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.76700000000000002,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.66800000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":569344,"data_stored":232697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.55100000000000005,"15min":0.54400000000000004},"min":{"1min":0.40699999999999997,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.74099999999999999,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.438,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.79500000000000004,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.83599999999999997}]},{"osd":1,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.57099999999999995,"15min":0.55800000000000005},"min":{"1min":0.50700000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.55800000000000005,"15min":0.55400000000000005},"min":{"1min":0.38600000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.77300000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.72099999999999997}]},{"osd":2,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.58099999999999996,"15min":0.57699999999999996},"min":{"1min":0.443,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.628,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.42799999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.75}]},{"osd":3,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.56999999999999995,"15min":0.56200000000000006},"min":{"1min":0.46600000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60599999999999998,"15min":0.60399999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.73499999999999999}]},{"osd":5,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52600000000000002,"15min":0.52300000000000002},"min":{"1min":0.36699999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.83799999999999997,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53900000000000003,"15min":0.54100000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.50600000000000001}]},{"osd":6,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.54200000000000004,"15min":0.53600000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.62},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.57699999999999996,"15min":0.56999999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.56899999999999995}]},{"osd":7,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.377,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.59499999999999997,"15min":0.59099999999999997},"min":{"1min":0.34699999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.70199999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574879,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31740,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340100,"statfs":{"total":96636764160,"available":96604262400,"internally_reserved":0,"allocated":569344,"data_stored":232697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.35699999999999998,"15min":0.35999999999999999},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.50800000000000001,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.38200000000000001,"15min":0.38100000000000001},"min":{"1min":0.27500000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.624,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.36899999999999999}]},{"osd":1,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.379,"15min":0.38300000000000001},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.59999999999999998,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.25800000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.58699999999999997,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.30399999999999999}]},{"osd":2,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.40600000000000003,"15min":0.40999999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.52900000000000003,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.42199999999999999,"15min":0.42099999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.66500000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.34999999999999998}]},{"osd":4,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.60899999999999999,"15min":0.61099999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76600000000000001,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.622,"15min":0.623},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.76300000000000001}]},{"osd":5,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.58299999999999996,"15min":0.58699999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85599999999999998,"5min":1.004,"15min":1.004},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.61299999999999999,"15min":0.61499999999999999},"min":{"1min":0.39300000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.90600000000000003,"5min":1.163,"15min":1.163},"last":0.50600000000000001}]},{"osd":6,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63800000000000001,"15min":0.63900000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.83199999999999996,"5min":1.095,"15min":1.095},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.63200000000000001,"15min":0.63500000000000001},"min":{"1min":0.45100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.96399999999999997,"5min":1.044,"15min":1.044},"last":0.746}]},{"osd":7,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.64700000000000002,"15min":0.65100000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.93899999999999995,"5min":1.177,"15min":1.177},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.64500000000000002,"15min":0.65100000000000002},"min":{"1min":0.45200000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.84799999999999998,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.72299999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31732,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340108,"statfs":{"total":96636764160,"available":96604270592,"internally_reserved":0,"allocated":565248,"data_stored":232078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.377,"15min":0.36299999999999999},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":2.649,"5min":2.649,"15min":2.649},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.42299999999999999,"15min":0.40799999999999997},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":3.173,"5min":3.173,"15min":3.173},"last":0.35099999999999998}]},{"osd":1,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.44800000000000001,"15min":0.41999999999999998},"min":{"1min":0.26300000000000001,"5min":0.246,"15min":0.246},"max":{"1min":4.6639999999999997,"5min":4.6639999999999997,"15min":4.6639999999999997},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.433,"15min":0.39100000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":6.093,"5min":6.093,"15min":6.093},"last":0.39500000000000002}]},{"osd":3,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.40300000000000002,"15min":0.376},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":4.1879999999999997,"5min":4.1879999999999997,"15min":4.1879999999999997},"last":0.317},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.44600000000000001,"15min":0.433},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":2.4159999999999999,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":0.33500000000000002}]},{"osd":4,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.60999999999999999,"15min":0.59599999999999997},"min":{"1min":0.40999999999999998,"5min":0.38,"15min":0.38},"max":{"1min":2.6840000000000002,"5min":2.6840000000000002,"15min":2.6840000000000002},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.65800000000000003,"15min":0.64600000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":3.0419999999999998,"5min":3.0419999999999998,"15min":3.0419999999999998},"last":0.45600000000000002}]},{"osd":5,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.66100000000000003,"15min":0.65300000000000002},"min":{"1min":0.41599999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":3.093,"5min":3.093,"15min":3.093},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.64700000000000002,"15min":0.65100000000000002},"min":{"1min":0.41799999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.98499999999999999,"5min":1.111,"15min":1.111},"last":0.51300000000000001}]},{"osd":6,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.64000000000000001,"15min":0.63800000000000001},"min":{"1min":0.42499999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.44},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.67800000000000005,"15min":0.67200000000000004},"min":{"1min":0.47799999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":2.6030000000000002,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":0.502}]},{"osd":7,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.65800000000000003,"15min":0.64800000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":3.1339999999999999,"5min":3.1339999999999999,"15min":3.1339999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.64200000000000002,"15min":0.64600000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.94199999999999995,"5min":1.2,"15min":1.2},"last":0.47699999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31736,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340104,"statfs":{"total":96636764160,"available":96604266496,"internally_reserved":0,"allocated":569344,"data_stored":232697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.33900000000000002,"15min":0.34100000000000003},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.436,"5min":0.48099999999999998,"15min":0.48099999999999998},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.39600000000000002,"15min":0.39800000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.58799999999999997,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.373}]},{"osd":2,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.35099999999999998,"15min":0.34899999999999998},"min":{"1min":0.249,"5min":0.246,"15min":0.246},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.36299999999999999,"15min":0.36699999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.53500000000000003,"5min":0.626,"15min":0.626},"last":0.29299999999999998}]},{"osd":3,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.40100000000000002,"15min":0.40200000000000002},"min":{"1min":0.30499999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.371,"15min":0.376},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.5,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.42099999999999999}]},{"osd":4,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.55400000000000005,"15min":0.54800000000000004},"min":{"1min":0.377,"5min":0.376,"15min":0.376},"max":{"1min":0.77100000000000002,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.55800000000000005,"15min":0.55100000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.78000000000000003,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.53200000000000003}]},{"osd":5,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.58599999999999997,"15min":0.57899999999999996},"min":{"1min":0.45900000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.96799999999999997,"5min":1.107,"15min":1.107},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57299999999999995,"15min":0.56599999999999995},"min":{"1min":0.437,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.63100000000000001}]},{"osd":6,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60399999999999998,"15min":0.60199999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.76600000000000001,"5min":1.044,"15min":1.044},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.61199999999999999,"15min":0.60599999999999998},"min":{"1min":0.51000000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.85399999999999998,"5min":0.998,"15min":0.998},"last":0.66600000000000004}]},{"osd":7,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58999999999999997,"15min":0.58799999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.78900000000000003,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.627,"5min":0.60599999999999998,"15min":0.60199999999999998},"min":{"1min":0.45300000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.80000000000000004,"5min":1.151,"15min":1.151},"last":0.55300000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32312,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339528,"statfs":{"total":96636764160,"available":96603676672,"internally_reserved":0,"allocated":1159168,"data_stored":822446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Dec 2 00:30:36 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.26000000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.501,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.37,"15min":0.36499999999999999},"min":{"1min":0.28299999999999997,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.58399999999999996,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.37}]},{"osd":2,"last update":"Sat Dec 2 00:30:36 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.54500000000000004,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40200000000000002,"15min":0.40100000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.32600000000000001}]},{"osd":3,"last update":"Sat Dec 2 00:30:36 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.39300000000000002,"15min":0.39400000000000002},"min":{"1min":0.29999999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.53100000000000003,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.436,"5min":0.42899999999999999,"15min":0.42799999999999999},"min":{"1min":0.32700000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.35099999999999998}]},{"osd":4,"last update":"Sat Dec 2 00:30:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57399999999999995,"15min":0.56899999999999995},"min":{"1min":0.42299999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.622,"15min":0.62},"min":{"1min":0.439,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.93100000000000005,"5min":1.002,"15min":1.002},"last":0.47899999999999998}]},{"osd":5,"last update":"Sat Dec 2 00:30:36 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.628,"15min":0.626},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.64700000000000002,"15min":0.64600000000000002},"min":{"1min":0.48099999999999998,"5min":0.433,"15min":0.433},"max":{"1min":0.91100000000000003,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.501}]},{"osd":6,"last update":"Sat Dec 2 00:30:36 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.60799999999999998,"15min":0.59999999999999998},"min":{"1min":0.435,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.89400000000000002,"5min":1.05,"15min":1.05},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.63800000000000001,"15min":0.64000000000000001},"min":{"1min":0.45300000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.94499999999999995,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.44500000000000001}]},{"osd":7,"last update":"Sat Dec 2 00:30:36 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.64300000000000002,"15min":0.63900000000000001},"min":{"1min":0.501,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.97699999999999998,"5min":1.028,"15min":1.028},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.626,"15min":0.626},"min":{"1min":0.46200000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.78000000000000003,"5min":0.874,"15min":0.874},"last":0.432}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-02T00:31:06.200 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-02T00:31:06.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.340+0000 7f86bec7c640 1 Processor -- start 2023-12-02T00:31:06.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.340+0000 7f86bec7c640 1 -- start start 2023-12-02T00:31:06.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.340+0000 7f86bec7c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f86b8146460 0x7f86b80b49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:06.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.340+0000 7f86bec7c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f86b806f1c0 0x7f86b806baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:06.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.340+0000 7f86bec7c640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86b8146fd0 con 0x7f86b8146840 2023-12-02T00:31:06.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.340+0000 7f86bec7c640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86b806c180 con 0x7f86b8146460 2023-12-02T00:31:06.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.340+0000 7f86bec7c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86b806c460 con 0x7f86b806f1c0 2023-12-02T00:31:06.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.341+0000 7f86bca18640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f86b8146460 0x7f86b80b49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:06.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.341+0000 7f86bca18640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f86b8146460 0x7f86b80b49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59682/0 (socket says 172.21.15.46:59682) 2023-12-02T00:31:06.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.341+0000 7f86bca18640 1 -- 172.21.15.46:0/2598953506 learned_addr learned my addr 172.21.15.46:0/2598953506 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:06.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.341+0000 7f86bd219640 1 --2- 172.21.15.46:0/2598953506 >> v2:172.21.15.162:3300/0 conn(0x7f86b806f1c0 0x7f86b806baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:06.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.341+0000 7f86bca18640 1 -- 172.21.15.46:0/2598953506 >> v2:172.21.15.162:3300/0 conn(0x7f86b806f1c0 msgr2=0x7f86b806baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:06.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.341+0000 7f86bca18640 1 --2- 172.21.15.46:0/2598953506 >> v2:172.21.15.162:3300/0 conn(0x7f86b806f1c0 0x7f86b806baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:06.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.341+0000 7f86bca18640 1 -- 172.21.15.46:0/2598953506 >> v1:172.21.15.162:6789/0 conn(0x7f86b8146840 legacy=0x7f86b806eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:06.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.341+0000 7f86bca18640 1 -- 172.21.15.46:0/2598953506 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b806c740 con 0x7f86b8146460 2023-12-02T00:31:06.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.342+0000 7f86bca18640 1 --2- 172.21.15.46:0/2598953506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86b8146460 0x7f86b80b49d0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f86b80b4fe0 tx=0x7f86ac02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=232bce2b152a40ce server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:06.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.342+0000 7f86b77fe640 1 -- 172.21.15.46:0/2598953506 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ac031870 con 0x7f86b8146460 2023-12-02T00:31:06.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.342+0000 7f86b77fe640 1 -- 172.21.15.46:0/2598953506 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86ac0319d0 con 0x7f86b8146460 2023-12-02T00:31:06.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.342+0000 7f86b77fe640 1 -- 172.21.15.46:0/2598953506 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ac031d60 con 0x7f86b8146460 2023-12-02T00:31:06.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.342+0000 7f86bec7c640 1 -- 172.21.15.46:0/2598953506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86b8146460 msgr2=0x7f86b80b49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:06.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.342+0000 7f86bec7c640 1 --2- 172.21.15.46:0/2598953506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86b8146460 0x7f86b80b49d0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f86b80b4fe0 tx=0x7f86ac02b910 comp rx=0 tx=0).stop 2023-12-02T00:31:06.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.342+0000 7f86bec7c640 1 -- 172.21.15.46:0/2598953506 shutdown_connections 2023-12-02T00:31:06.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.342+0000 7f86bec7c640 1 --2- 172.21.15.46:0/2598953506 >> v2:172.21.15.162:3300/0 conn(0x7f86b806f1c0 0x7f86b806baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:06.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.342+0000 7f86bec7c640 1 --2- 172.21.15.46:0/2598953506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86b8146460 0x7f86b80b49d0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:06.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.342+0000 7f86bec7c640 1 -- 172.21.15.46:0/2598953506 >> 172.21.15.46:0/2598953506 conn(0x7f86b806a6b0 msgr2=0x7f86b806aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:06.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.342+0000 7f86bec7c640 1 -- 172.21.15.46:0/2598953506 shutdown_connections 2023-12-02T00:31:06.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.342+0000 7f86bec7c640 1 -- 172.21.15.46:0/2598953506 wait complete. 2023-12-02T00:31:06.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.343+0000 7f86bec7c640 1 Processor -- start 2023-12-02T00:31:06.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.344+0000 7f86bec7c640 1 -- start start 2023-12-02T00:31:06.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.344+0000 7f86bec7c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86b806f1c0 0x7f86b80cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:06.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.344+0000 7f86bec7c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86b8146460 0x7f86b80cad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:06.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.344+0000 7f86bec7c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f86b8146840 0x7f86b815cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:06.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.344+0000 7f86bec7c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b813feb0 con 0x7f86b8146460 2023-12-02T00:31:06.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.344+0000 7f86bec7c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b80cb2c0 con 0x7f86b806f1c0 2023-12-02T00:31:06.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.344+0000 7f86bec7c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86b813f580 con 0x7f86b8146840 2023-12-02T00:31:06.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.345+0000 7f86b7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86b8146460 0x7f86b80cad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:06.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.345+0000 7f86b7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86b8146460 0x7f86b80cad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59692/0 (socket says 172.21.15.46:59692) 2023-12-02T00:31:06.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.345+0000 7f86b7fff640 1 -- 172.21.15.46:0/2379260104 learned_addr learned my addr 172.21.15.46:0/2379260104 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:06.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.345+0000 7f86bd219640 1 --2- 172.21.15.46:0/2379260104 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f86b8146840 0x7f86b815cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:06.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.345+0000 7f86bca18640 1 --2- 172.21.15.46:0/2379260104 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86b806f1c0 0x7f86b80cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:06.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.345+0000 7f86b7fff640 1 -- 172.21.15.46:0/2379260104 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f86b8146840 msgr2=0x7f86b815cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:06.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.345+0000 7f86b7fff640 1 --2- 172.21.15.46:0/2379260104 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f86b8146840 0x7f86b815cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:06.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.345+0000 7f86b7fff640 1 -- 172.21.15.46:0/2379260104 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86b806f1c0 msgr2=0x7f86b80cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:06.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.345+0000 7f86b7fff640 1 --2- 172.21.15.46:0/2379260104 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86b806f1c0 0x7f86b80cbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:06.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.345+0000 7f86b7fff640 1 -- 172.21.15.46:0/2379260104 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86ac0094d0 con 0x7f86b8146460 2023-12-02T00:31:06.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.345+0000 7f86b7fff640 1 --2- 172.21.15.46:0/2379260104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86b8146460 0x7f86b80cad80 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f86a400b440 tx=0x7f86a400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:06.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.345+0000 7f86b57fa640 1 -- 172.21.15.46:0/2379260104 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a4011830 con 0x7f86b8146460 2023-12-02T00:31:06.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.345+0000 7f86b57fa640 1 -- 172.21.15.46:0/2379260104 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86a40119b0 con 0x7f86b8146460 2023-12-02T00:31:06.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.345+0000 7f86b57fa640 1 -- 172.21.15.46:0/2379260104 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a4011d60 con 0x7f86b8146460 2023-12-02T00:31:06.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.345+0000 7f86bec7c640 1 -- 172.21.15.46:0/2379260104 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86b813f860 con 0x7f86b8146460 2023-12-02T00:31:06.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.345+0000 7f86bec7c640 1 -- 172.21.15.46:0/2379260104 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86b815d0f0 con 0x7f86b8146460 2023-12-02T00:31:06.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.346+0000 7f86bec7c640 1 -- 172.21.15.46:0/2379260104 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86b8069400 con 0x7f86b8146460 2023-12-02T00:31:06.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.347+0000 7f86b57fa640 1 -- 172.21.15.46:0/2379260104 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f86a40193b0 con 0x7f86b8146460 2023-12-02T00:31:06.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.347+0000 7f86b57fa640 1 --2- 172.21.15.46:0/2379260104 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f869c06ae70 0x7f869c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:06.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.347+0000 7f86b57fa640 1 -- 172.21.15.46:0/2379260104 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f86a40891c0 con 0x7f86b8146460 2023-12-02T00:31:06.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.347+0000 7f86bca18640 1 --2- 172.21.15.46:0/2379260104 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f869c06ae70 0x7f869c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:06.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.348+0000 7f86bca18640 1 --2- 172.21.15.46:0/2379260104 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f869c06ae70 0x7f869c06d330 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f86ac036000 tx=0x7f86ac03c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:06.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.351+0000 7f86b57fa640 1 -- 172.21.15.46:0/2379260104 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f86a40543b0 con 0x7f86b8146460 2023-12-02T00:31:06.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.523+0000 7f86bec7c640 1 -- 172.21.15.46:0/2379260104 --> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f86b80b4da0 con 0x7f869c06ae70 2023-12-02T00:31:06.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.531+0000 7f86b57fa640 1 -- 172.21.15.46:0/2379260104 <== mgr.4108 v2:172.21.15.162:6832/3248644722 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284257 (secure 0 0 0) 0x7f86b80b4da0 con 0x7f869c06ae70 2023-12-02T00:31:06.536 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:31:06.538 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-12-02T00:31:06.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.534+0000 7f86bec7c640 1 -- 172.21.15.46:0/2379260104 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f869c06ae70 msgr2=0x7f869c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:06.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.534+0000 7f86bec7c640 1 --2- 172.21.15.46:0/2379260104 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f869c06ae70 0x7f869c06d330 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f86ac036000 tx=0x7f86ac03c040 comp rx=0 tx=0).stop 2023-12-02T00:31:06.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.534+0000 7f86bec7c640 1 -- 172.21.15.46:0/2379260104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86b8146460 msgr2=0x7f86b80cad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:06.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.534+0000 7f86bec7c640 1 --2- 172.21.15.46:0/2379260104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86b8146460 0x7f86b80cad80 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f86a400b440 tx=0x7f86a400b900 comp rx=0 tx=0).stop 2023-12-02T00:31:06.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.534+0000 7f86bec7c640 1 -- 172.21.15.46:0/2379260104 shutdown_connections 2023-12-02T00:31:06.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.534+0000 7f86bec7c640 1 --2- 172.21.15.46:0/2379260104 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f869c06ae70 0x7f869c06d330 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:06.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.534+0000 7f86bec7c640 1 --2- 172.21.15.46:0/2379260104 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f86b8146840 0x7f86b815cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:06.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.534+0000 7f86bec7c640 1 --2- 172.21.15.46:0/2379260104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86b8146460 0x7f86b80cad80 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:06.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.534+0000 7f86bec7c640 1 --2- 172.21.15.46:0/2379260104 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86b806f1c0 0x7f86b80cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:06.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.534+0000 7f86bec7c640 1 -- 172.21.15.46:0/2379260104 >> 172.21.15.46:0/2379260104 conn(0x7f86b806a6b0 msgr2=0x7f86b806d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:06.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.534+0000 7f86bec7c640 1 -- 172.21.15.46:0/2379260104 shutdown_connections 2023-12-02T00:31:06.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.534+0000 7f86bec7c640 1 -- 172.21.15.46:0/2379260104 wait complete. 2023-12-02T00:31:06.547 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":119,"stamp":"2023-12-02T00:31:05.191363+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":32,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238420,"kb_used_data":5612,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736300,"statfs":{"total":773094113280,"available":772849971200,"internally_reserved":0,"allocated":5746688,"data_stored":3047349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.377699"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.827720+0000","last_change":"2023-12-02T00:31:00.827720+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.827720+0000","last_undegraded":"2023-12-02T00:31:00.827720+0000","last_fullsized":"2023-12-02T00:31:00.827720+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.824870+0000","last_change":"2023-12-02T00:31:00.824870+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.824870+0000","last_undegraded":"2023-12-02T00:31:00.824870+0000","last_fullsized":"2023-12-02T00:31:00.824870+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.826847+0000","last_change":"2023-12-02T00:31:00.826847+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.826847+0000","last_undegraded":"2023-12-02T00:31:00.826847+0000","last_fullsized":"2023-12-02T00:31:00.826847+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.823450+0000","last_change":"2023-12-02T00:31:00.823450+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.823450+0000","last_undegraded":"2023-12-02T00:31:00.823450+0000","last_fullsized":"2023-12-02T00:31:00.823450+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.818839+0000","last_change":"2023-12-02T00:31:00.818839+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.818839+0000","last_undegraded":"2023-12-02T00:31:00.818839+0000","last_fullsized":"2023-12-02T00:31:00.818839+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.828726+0000","last_change":"2023-12-02T00:31:00.828726+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.828726+0000","last_undegraded":"2023-12-02T00:31:00.828726+0000","last_fullsized":"2023-12-02T00:31:00.828726+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.827162+0000","last_change":"2023-12-02T00:31:00.827162+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.827162+0000","last_undegraded":"2023-12-02T00:31:00.827162+0000","last_fullsized":"2023-12-02T00:31:00.827162+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.834673+0000","last_change":"2023-12-02T00:31:00.834673+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.834673+0000","last_undegraded":"2023-12-02T00:31:00.834673+0000","last_fullsized":"2023-12-02T00:31:00.834673+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.833708+0000","last_change":"2023-12-02T00:31:00.833708+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.833708+0000","last_undegraded":"2023-12-02T00:31:00.833708+0000","last_fullsized":"2023-12-02T00:31:00.833708+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.829160+0000","last_change":"2023-12-02T00:31:00.829160+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.829160+0000","last_undegraded":"2023-12-02T00:31:00.829160+0000","last_fullsized":"2023-12-02T00:31:00.829160+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.818160+0000","last_change":"2023-12-02T00:31:00.818160+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.818160+0000","last_undegraded":"2023-12-02T00:31:00.818160+0000","last_fullsized":"2023-12-02T00:31:00.818160+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.831142+0000","last_change":"2023-12-02T00:31:00.831142+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.831142+0000","last_undegraded":"2023-12-02T00:31:00.831142+0000","last_fullsized":"2023-12-02T00:31:00.831142+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.827013+0000","last_change":"2023-12-02T00:31:00.827013+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.827013+0000","last_undegraded":"2023-12-02T00:31:00.827013+0000","last_fullsized":"2023-12-02T00:31:00.827013+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.826562+0000","last_change":"2023-12-02T00:31:00.826562+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.826562+0000","last_undegraded":"2023-12-02T00:31:00.826562+0000","last_fullsized":"2023-12-02T00:31:00.826562+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.823060+0000","last_change":"2023-12-02T00:31:00.823060+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.823060+0000","last_undegraded":"2023-12-02T00:31:00.823060+0000","last_fullsized":"2023-12-02T00:31:00.823060+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.825606+0000","last_change":"2023-12-02T00:31:00.825606+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.825606+0000","last_undegraded":"2023-12-02T00:31:00.825606+0000","last_fullsized":"2023-12-02T00:31:00.825606+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.825954+0000","last_change":"2023-12-02T00:31:00.825954+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.825954+0000","last_undegraded":"2023-12-02T00:31:00.825954+0000","last_fullsized":"2023-12-02T00:31:00.825954+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.824312+0000","last_change":"2023-12-02T00:31:00.824312+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.824312+0000","last_undegraded":"2023-12-02T00:31:00.824312+0000","last_fullsized":"2023-12-02T00:31:00.824312+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.825253+0000","last_change":"2023-12-02T00:31:00.825253+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.825253+0000","last_undegraded":"2023-12-02T00:31:00.825253+0000","last_fullsized":"2023-12-02T00:31:00.825253+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.825702+0000","last_change":"2023-12-02T00:31:00.825702+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.825702+0000","last_undegraded":"2023-12-02T00:31:00.825702+0000","last_fullsized":"2023-12-02T00:31:00.825702+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.826391+0000","last_change":"2023-12-02T00:31:00.826391+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.826391+0000","last_undegraded":"2023-12-02T00:31:00.826391+0000","last_fullsized":"2023-12-02T00:31:00.826391+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.831192+0000","last_change":"2023-12-02T00:31:00.831192+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.831192+0000","last_undegraded":"2023-12-02T00:31:00.831192+0000","last_fullsized":"2023-12-02T00:31:00.831192+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.818154+0000","last_change":"2023-12-02T00:31:00.818154+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.818154+0000","last_undegraded":"2023-12-02T00:31:00.818154+0000","last_fullsized":"2023-12-02T00:31:00.818154+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.824629+0000","last_change":"2023-12-02T00:31:00.824629+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.824629+0000","last_undegraded":"2023-12-02T00:31:00.824629+0000","last_fullsized":"2023-12-02T00:31:00.824629+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.824088+0000","last_change":"2023-12-02T00:31:00.824088+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.824088+0000","last_undegraded":"2023-12-02T00:31:00.824088+0000","last_fullsized":"2023-12-02T00:31:00.824088+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.829648+0000","last_change":"2023-12-02T00:31:00.829648+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.829648+0000","last_undegraded":"2023-12-02T00:31:00.829648+0000","last_fullsized":"2023-12-02T00:31:00.829648+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.827734+0000","last_change":"2023-12-02T00:31:00.827734+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.827734+0000","last_undegraded":"2023-12-02T00:31:00.827734+0000","last_fullsized":"2023-12-02T00:31:00.827734+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.824743+0000","last_change":"2023-12-02T00:31:00.824743+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.824743+0000","last_undegraded":"2023-12-02T00:31:00.824743+0000","last_fullsized":"2023-12-02T00:31:00.824743+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.823555+0000","last_change":"2023-12-02T00:31:00.823555+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.823555+0000","last_undegraded":"2023-12-02T00:31:00.823555+0000","last_fullsized":"2023-12-02T00:31:00.823555+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.823051+0000","last_change":"2023-12-02T00:31:00.823051+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.823051+0000","last_undegraded":"2023-12-02T00:31:00.823051+0000","last_fullsized":"2023-12-02T00:31:00.823051+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.823884+0000","last_change":"2023-12-02T00:31:00.823884+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.823884+0000","last_undegraded":"2023-12-02T00:31:00.823884+0000","last_fullsized":"2023-12-02T00:31:00.823884+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.827493+0000","last_change":"2023-12-02T00:31:00.827493+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.827493+0000","last_undegraded":"2023-12-02T00:31:00.827493+0000","last_fullsized":"2023-12-02T00:31:00.827493+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.824063+0000","last_change":"2023-12-02T00:31:00.824063+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.824063+0000","last_undegraded":"2023-12-02T00:31:00.824063+0000","last_fullsized":"2023-12-02T00:31:00.824063+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.836660+0000","last_change":"2023-12-02T00:31:00.832088+0000","last_active":"2023-12-02T00:31:00.836660+0000","last_peered":"2023-12-02T00:31:00.836660+0000","last_clean":"2023-12-02T00:31:00.836660+0000","last_became_active":"2023-12-02T00:31:00.830872+0000","last_became_peered":"2023-12-02T00:31:00.830872+0000","last_unstale":"2023-12-02T00:31:00.836660+0000","last_undegraded":"2023-12-02T00:31:00.836660+0000","last_fullsized":"2023-12-02T00:31:00.836660+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:18:11.861982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.825822+0000","last_change":"2023-12-02T00:31:00.825822+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.825822+0000","last_undegraded":"2023-12-02T00:31:00.825822+0000","last_fullsized":"2023-12-02T00:31:00.825822+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.836248+0000","last_change":"2023-12-02T00:31:00.834787+0000","last_active":"2023-12-02T00:31:00.836248+0000","last_peered":"2023-12-02T00:31:00.836248+0000","last_clean":"2023-12-02T00:31:00.836248+0000","last_became_active":"2023-12-02T00:31:00.834357+0000","last_became_peered":"2023-12-02T00:31:00.834357+0000","last_unstale":"2023-12-02T00:31:00.836248+0000","last_undegraded":"2023-12-02T00:31:00.836248+0000","last_fullsized":"2023-12-02T00:31:00.836248+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:57:02.728754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.826663+0000","last_change":"2023-12-02T00:31:00.826663+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.826663+0000","last_undegraded":"2023-12-02T00:31:00.826663+0000","last_fullsized":"2023-12-02T00:31:00.826663+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.839027+0000","last_change":"2023-12-02T00:31:00.837792+0000","last_active":"2023-12-02T00:31:00.839027+0000","last_peered":"2023-12-02T00:31:00.839027+0000","last_clean":"2023-12-02T00:31:00.839027+0000","last_became_active":"2023-12-02T00:31:00.837471+0000","last_became_peered":"2023-12-02T00:31:00.837471+0000","last_unstale":"2023-12-02T00:31:00.839027+0000","last_undegraded":"2023-12-02T00:31:00.839027+0000","last_fullsized":"2023-12-02T00:31:00.839027+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:33:23.942055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.823658+0000","last_change":"2023-12-02T00:31:00.823658+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.823658+0000","last_undegraded":"2023-12-02T00:31:00.823658+0000","last_fullsized":"2023-12-02T00:31:00.823658+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.837462+0000","last_change":"2023-12-02T00:31:00.834209+0000","last_active":"2023-12-02T00:31:00.837462+0000","last_peered":"2023-12-02T00:31:00.837462+0000","last_clean":"2023-12-02T00:31:00.837462+0000","last_became_active":"2023-12-02T00:31:00.833798+0000","last_became_peered":"2023-12-02T00:31:00.833798+0000","last_unstale":"2023-12-02T00:31:00.837462+0000","last_undegraded":"2023-12-02T00:31:00.837462+0000","last_fullsized":"2023-12-02T00:31:00.837462+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:20:10.886591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.818382+0000","last_change":"2023-12-02T00:31:00.818382+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.818382+0000","last_undegraded":"2023-12-02T00:31:00.818382+0000","last_fullsized":"2023-12-02T00:31:00.818382+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.824844+0000","last_change":"2023-12-02T00:31:00.821690+0000","last_active":"2023-12-02T00:31:00.824844+0000","last_peered":"2023-12-02T00:31:00.824844+0000","last_clean":"2023-12-02T00:31:00.824844+0000","last_became_active":"2023-12-02T00:31:00.821365+0000","last_became_peered":"2023-12-02T00:31:00.821365+0000","last_unstale":"2023-12-02T00:31:00.824844+0000","last_undegraded":"2023-12-02T00:31:00.824844+0000","last_fullsized":"2023-12-02T00:31:00.824844+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:57:02.728754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.823332+0000","last_change":"2023-12-02T00:31:00.823332+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.823332+0000","last_undegraded":"2023-12-02T00:31:00.823332+0000","last_fullsized":"2023-12-02T00:31:00.823332+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.834534+0000","last_change":"2023-12-02T00:31:00.831123+0000","last_active":"2023-12-02T00:31:00.834534+0000","last_peered":"2023-12-02T00:31:00.834534+0000","last_clean":"2023-12-02T00:31:00.834534+0000","last_became_active":"2023-12-02T00:31:00.830870+0000","last_became_peered":"2023-12-02T00:31:00.830870+0000","last_unstale":"2023-12-02T00:31:00.834534+0000","last_undegraded":"2023-12-02T00:31:00.834534+0000","last_fullsized":"2023-12-02T00:31:00.834534+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:56:41.004341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.822277+0000","last_change":"2023-12-02T00:31:00.822277+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.822277+0000","last_undegraded":"2023-12-02T00:31:00.822277+0000","last_fullsized":"2023-12-02T00:31:00.822277+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.839350+0000","last_change":"2023-12-02T00:31:00.834261+0000","last_active":"2023-12-02T00:31:00.839350+0000","last_peered":"2023-12-02T00:31:00.839350+0000","last_clean":"2023-12-02T00:31:00.839350+0000","last_became_active":"2023-12-02T00:31:00.833992+0000","last_became_peered":"2023-12-02T00:31:00.833992+0000","last_unstale":"2023-12-02T00:31:00.839350+0000","last_undegraded":"2023-12-02T00:31:00.839350+0000","last_fullsized":"2023-12-02T00:31:00.839350+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:09:23.731017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.823236+0000","last_change":"2023-12-02T00:31:00.823236+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.823236+0000","last_undegraded":"2023-12-02T00:31:00.823236+0000","last_fullsized":"2023-12-02T00:31:00.823236+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.832488+0000","last_change":"2023-12-02T00:31:00.828451+0000","last_active":"2023-12-02T00:31:00.832488+0000","last_peered":"2023-12-02T00:31:00.832488+0000","last_clean":"2023-12-02T00:31:00.832488+0000","last_became_active":"2023-12-02T00:31:00.828164+0000","last_became_peered":"2023-12-02T00:31:00.828164+0000","last_unstale":"2023-12-02T00:31:00.832488+0000","last_undegraded":"2023-12-02T00:31:00.832488+0000","last_fullsized":"2023-12-02T00:31:00.832488+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T12:24:56.280705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.818389+0000","last_change":"2023-12-02T00:31:00.818389+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.818389+0000","last_undegraded":"2023-12-02T00:31:00.818389+0000","last_fullsized":"2023-12-02T00:31:00.818389+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.839294+0000","last_change":"2023-12-02T00:31:00.835042+0000","last_active":"2023-12-02T00:31:00.839294+0000","last_peered":"2023-12-02T00:31:00.839294+0000","last_clean":"2023-12-02T00:31:00.839294+0000","last_became_active":"2023-12-02T00:31:00.834764+0000","last_became_peered":"2023-12-02T00:31:00.834764+0000","last_unstale":"2023-12-02T00:31:00.839294+0000","last_undegraded":"2023-12-02T00:31:00.839294+0000","last_fullsized":"2023-12-02T00:31:00.839294+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:54:07.912968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.827202+0000","last_change":"2023-12-02T00:31:00.827202+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.827202+0000","last_undegraded":"2023-12-02T00:31:00.827202+0000","last_fullsized":"2023-12-02T00:31:00.827202+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.826894+0000","last_change":"2023-12-02T00:31:00.821222+0000","last_active":"2023-12-02T00:31:00.826894+0000","last_peered":"2023-12-02T00:31:00.826894+0000","last_clean":"2023-12-02T00:31:00.826894+0000","last_became_active":"2023-12-02T00:31:00.820914+0000","last_became_peered":"2023-12-02T00:31:00.820914+0000","last_unstale":"2023-12-02T00:31:00.826894+0000","last_undegraded":"2023-12-02T00:31:00.826894+0000","last_fullsized":"2023-12-02T00:31:00.826894+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T10:06:37.837138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.823398+0000","last_change":"2023-12-02T00:31:00.823398+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.823398+0000","last_undegraded":"2023-12-02T00:31:00.823398+0000","last_fullsized":"2023-12-02T00:31:00.823398+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.840140+0000","last_change":"2023-12-02T00:31:00.837992+0000","last_active":"2023-12-02T00:31:00.840140+0000","last_peered":"2023-12-02T00:31:00.840140+0000","last_clean":"2023-12-02T00:31:00.840140+0000","last_became_active":"2023-12-02T00:31:00.837743+0000","last_became_peered":"2023-12-02T00:31:00.837743+0000","last_unstale":"2023-12-02T00:31:00.840140+0000","last_undegraded":"2023-12-02T00:31:00.840140+0000","last_fullsized":"2023-12-02T00:31:00.840140+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:00:47.235125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.818271+0000","last_change":"2023-12-02T00:31:00.818271+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.818271+0000","last_undegraded":"2023-12-02T00:31:00.818271+0000","last_fullsized":"2023-12-02T00:31:00.818271+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.842235+0000","last_change":"2023-12-02T00:31:00.840634+0000","last_active":"2023-12-02T00:31:00.842235+0000","last_peered":"2023-12-02T00:31:00.842235+0000","last_clean":"2023-12-02T00:31:00.842235+0000","last_became_active":"2023-12-02T00:31:00.840359+0000","last_became_peered":"2023-12-02T00:31:00.840359+0000","last_unstale":"2023-12-02T00:31:00.842235+0000","last_undegraded":"2023-12-02T00:31:00.842235+0000","last_fullsized":"2023-12-02T00:31:00.842235+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:34:01.547200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.826762+0000","last_change":"2023-12-02T00:31:00.826762+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.826762+0000","last_undegraded":"2023-12-02T00:31:00.826762+0000","last_fullsized":"2023-12-02T00:31:00.826762+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.832524+0000","last_change":"2023-12-02T00:31:00.829212+0000","last_active":"2023-12-02T00:31:00.832524+0000","last_peered":"2023-12-02T00:31:00.832524+0000","last_clean":"2023-12-02T00:31:00.832524+0000","last_became_active":"2023-12-02T00:31:00.828848+0000","last_became_peered":"2023-12-02T00:31:00.828848+0000","last_unstale":"2023-12-02T00:31:00.832524+0000","last_undegraded":"2023-12-02T00:31:00.832524+0000","last_fullsized":"2023-12-02T00:31:00.832524+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T01:12:56.020427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.822967+0000","last_change":"2023-12-02T00:31:00.822967+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.822967+0000","last_undegraded":"2023-12-02T00:31:00.822967+0000","last_fullsized":"2023-12-02T00:31:00.822967+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.839380+0000","last_change":"2023-12-02T00:31:00.834656+0000","last_active":"2023-12-02T00:31:00.839380+0000","last_peered":"2023-12-02T00:31:00.839380+0000","last_clean":"2023-12-02T00:31:00.839380+0000","last_became_active":"2023-12-02T00:31:00.834359+0000","last_became_peered":"2023-12-02T00:31:00.834359+0000","last_unstale":"2023-12-02T00:31:00.839380+0000","last_undegraded":"2023-12-02T00:31:00.839380+0000","last_fullsized":"2023-12-02T00:31:00.839380+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T01:33:59.617275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.824188+0000","last_change":"2023-12-02T00:31:00.824188+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.824188+0000","last_undegraded":"2023-12-02T00:31:00.824188+0000","last_fullsized":"2023-12-02T00:31:00.824188+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.840126+0000","last_change":"2023-12-02T00:31:00.837228+0000","last_active":"2023-12-02T00:31:00.840126+0000","last_peered":"2023-12-02T00:31:00.840126+0000","last_clean":"2023-12-02T00:31:00.840126+0000","last_became_active":"2023-12-02T00:31:00.836943+0000","last_became_peered":"2023-12-02T00:31:00.836943+0000","last_unstale":"2023-12-02T00:31:00.840126+0000","last_undegraded":"2023-12-02T00:31:00.840126+0000","last_fullsized":"2023-12-02T00:31:00.840126+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:51:08.104561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.823005+0000","last_change":"2023-12-02T00:31:00.823005+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.823005+0000","last_undegraded":"2023-12-02T00:31:00.823005+0000","last_fullsized":"2023-12-02T00:31:00.823005+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.842246+0000","last_change":"2023-12-02T00:31:00.840645+0000","last_active":"2023-12-02T00:31:00.842246+0000","last_peered":"2023-12-02T00:31:00.842246+0000","last_clean":"2023-12-02T00:31:00.842246+0000","last_became_active":"2023-12-02T00:31:00.840295+0000","last_became_peered":"2023-12-02T00:31:00.840295+0000","last_unstale":"2023-12-02T00:31:00.842246+0000","last_undegraded":"2023-12-02T00:31:00.842246+0000","last_fullsized":"2023-12-02T00:31:00.842246+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:41:19.966082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.822868+0000","last_change":"2023-12-02T00:31:00.822868+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.822868+0000","last_undegraded":"2023-12-02T00:31:00.822868+0000","last_fullsized":"2023-12-02T00:31:00.822868+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.840706+0000","last_change":"2023-12-02T00:31:00.839972+0000","last_active":"2023-12-02T00:31:00.840706+0000","last_peered":"2023-12-02T00:31:00.840706+0000","last_clean":"2023-12-02T00:31:00.840706+0000","last_became_active":"2023-12-02T00:31:00.839581+0000","last_became_peered":"2023-12-02T00:31:00.839581+0000","last_unstale":"2023-12-02T00:31:00.840706+0000","last_undegraded":"2023-12-02T00:31:00.840706+0000","last_fullsized":"2023-12-02T00:31:00.840706+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:33:23.942055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.825542+0000","last_change":"2023-12-02T00:31:00.825542+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.825542+0000","last_undegraded":"2023-12-02T00:31:00.825542+0000","last_fullsized":"2023-12-02T00:31:00.825542+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.833409+0000","last_change":"2023-12-02T00:31:00.832413+0000","last_active":"2023-12-02T00:31:00.833409+0000","last_peered":"2023-12-02T00:31:00.833409+0000","last_clean":"2023-12-02T00:31:00.833409+0000","last_became_active":"2023-12-02T00:31:00.832127+0000","last_became_peered":"2023-12-02T00:31:00.832127+0000","last_unstale":"2023-12-02T00:31:00.833409+0000","last_undegraded":"2023-12-02T00:31:00.833409+0000","last_fullsized":"2023-12-02T00:31:00.833409+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:55:28.223962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.818333+0000","last_change":"2023-12-02T00:31:00.818333+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.818333+0000","last_undegraded":"2023-12-02T00:31:00.818333+0000","last_fullsized":"2023-12-02T00:31:00.818333+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.840093+0000","last_change":"2023-12-02T00:31:00.837851+0000","last_active":"2023-12-02T00:31:00.840093+0000","last_peered":"2023-12-02T00:31:00.840093+0000","last_clean":"2023-12-02T00:31:00.840093+0000","last_became_active":"2023-12-02T00:31:00.837581+0000","last_became_peered":"2023-12-02T00:31:00.837581+0000","last_unstale":"2023-12-02T00:31:00.840093+0000","last_undegraded":"2023-12-02T00:31:00.840093+0000","last_fullsized":"2023-12-02T00:31:00.840093+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:17:18.074493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.826478+0000","last_change":"2023-12-02T00:31:00.826478+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.826478+0000","last_undegraded":"2023-12-02T00:31:00.826478+0000","last_fullsized":"2023-12-02T00:31:00.826478+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.836731+0000","last_change":"2023-12-02T00:31:00.832751+0000","last_active":"2023-12-02T00:31:00.836731+0000","last_peered":"2023-12-02T00:31:00.836731+0000","last_clean":"2023-12-02T00:31:00.836731+0000","last_became_active":"2023-12-02T00:31:00.832466+0000","last_became_peered":"2023-12-02T00:31:00.832466+0000","last_unstale":"2023-12-02T00:31:00.836731+0000","last_undegraded":"2023-12-02T00:31:00.836731+0000","last_fullsized":"2023-12-02T00:31:00.836731+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:01:15.991829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.826326+0000","last_change":"2023-12-02T00:31:00.826326+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.826326+0000","last_undegraded":"2023-12-02T00:31:00.826326+0000","last_fullsized":"2023-12-02T00:31:00.826326+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.836131+0000","last_change":"2023-12-02T00:31:00.833503+0000","last_active":"2023-12-02T00:31:00.836131+0000","last_peered":"2023-12-02T00:31:00.836131+0000","last_clean":"2023-12-02T00:31:00.836131+0000","last_became_active":"2023-12-02T00:31:00.833219+0000","last_became_peered":"2023-12-02T00:31:00.833219+0000","last_unstale":"2023-12-02T00:31:00.836131+0000","last_undegraded":"2023-12-02T00:31:00.836131+0000","last_fullsized":"2023-12-02T00:31:00.836131+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T09:28:07.117685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.824313+0000","last_change":"2023-12-02T00:31:00.824313+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.824313+0000","last_undegraded":"2023-12-02T00:31:00.824313+0000","last_fullsized":"2023-12-02T00:31:00.824313+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.842212+0000","last_change":"2023-12-02T00:31:00.840501+0000","last_active":"2023-12-02T00:31:00.842212+0000","last_peered":"2023-12-02T00:31:00.842212+0000","last_clean":"2023-12-02T00:31:00.842212+0000","last_became_active":"2023-12-02T00:31:00.840227+0000","last_became_peered":"2023-12-02T00:31:00.840227+0000","last_unstale":"2023-12-02T00:31:00.842212+0000","last_undegraded":"2023-12-02T00:31:00.842212+0000","last_fullsized":"2023-12-02T00:31:00.842212+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:40:05.075383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.822165+0000","last_change":"2023-12-02T00:31:00.822165+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.822165+0000","last_undegraded":"2023-12-02T00:31:00.822165+0000","last_fullsized":"2023-12-02T00:31:00.822165+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.836552+0000","last_change":"2023-12-02T00:31:00.833798+0000","last_active":"2023-12-02T00:31:00.836552+0000","last_peered":"2023-12-02T00:31:00.836552+0000","last_clean":"2023-12-02T00:31:00.836552+0000","last_became_active":"2023-12-02T00:31:00.833537+0000","last_became_peered":"2023-12-02T00:31:00.833537+0000","last_unstale":"2023-12-02T00:31:00.836552+0000","last_undegraded":"2023-12-02T00:31:00.836552+0000","last_fullsized":"2023-12-02T00:31:00.836552+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:54:17.615040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.826590+0000","last_change":"2023-12-02T00:31:00.826590+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.826590+0000","last_undegraded":"2023-12-02T00:31:00.826590+0000","last_fullsized":"2023-12-02T00:31:00.826590+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.830926+0000","last_change":"2023-12-02T00:31:00.829323+0000","last_active":"2023-12-02T00:31:00.830926+0000","last_peered":"2023-12-02T00:31:00.830926+0000","last_clean":"2023-12-02T00:31:00.830926+0000","last_became_active":"2023-12-02T00:31:00.829048+0000","last_became_peered":"2023-12-02T00:31:00.829048+0000","last_unstale":"2023-12-02T00:31:00.830926+0000","last_undegraded":"2023-12-02T00:31:00.830926+0000","last_fullsized":"2023-12-02T00:31:00.830926+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:33:51.372611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.825234+0000","last_change":"2023-12-02T00:31:00.825234+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.825234+0000","last_undegraded":"2023-12-02T00:31:00.825234+0000","last_fullsized":"2023-12-02T00:31:00.825234+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.839371+0000","last_change":"2023-12-02T00:31:00.835706+0000","last_active":"2023-12-02T00:31:00.839371+0000","last_peered":"2023-12-02T00:31:00.839371+0000","last_clean":"2023-12-02T00:31:00.839371+0000","last_became_active":"2023-12-02T00:31:00.835381+0000","last_became_peered":"2023-12-02T00:31:00.835381+0000","last_unstale":"2023-12-02T00:31:00.839371+0000","last_undegraded":"2023-12-02T00:31:00.839371+0000","last_fullsized":"2023-12-02T00:31:00.839371+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:14:37.638316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.828284+0000","last_change":"2023-12-02T00:31:00.828284+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.828284+0000","last_undegraded":"2023-12-02T00:31:00.828284+0000","last_fullsized":"2023-12-02T00:31:00.828284+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.836694+0000","last_change":"2023-12-02T00:31:00.832747+0000","last_active":"2023-12-02T00:31:00.836694+0000","last_peered":"2023-12-02T00:31:00.836694+0000","last_clean":"2023-12-02T00:31:00.836694+0000","last_became_active":"2023-12-02T00:31:00.832477+0000","last_became_peered":"2023-12-02T00:31:00.832477+0000","last_unstale":"2023-12-02T00:31:00.836694+0000","last_undegraded":"2023-12-02T00:31:00.836694+0000","last_fullsized":"2023-12-02T00:31:00.836694+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T10:47:44.799679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.823172+0000","last_change":"2023-12-02T00:31:00.823172+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.823172+0000","last_undegraded":"2023-12-02T00:31:00.823172+0000","last_fullsized":"2023-12-02T00:31:00.823172+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.832174+0000","last_change":"2023-12-02T00:31:00.828668+0000","last_active":"2023-12-02T00:31:00.832174+0000","last_peered":"2023-12-02T00:31:00.832174+0000","last_clean":"2023-12-02T00:31:00.832174+0000","last_became_active":"2023-12-02T00:31:00.828392+0000","last_became_peered":"2023-12-02T00:31:00.828392+0000","last_unstale":"2023-12-02T00:31:00.832174+0000","last_undegraded":"2023-12-02T00:31:00.832174+0000","last_fullsized":"2023-12-02T00:31:00.832174+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T09:28:07.117685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.822281+0000","last_change":"2023-12-02T00:31:00.822281+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.822281+0000","last_undegraded":"2023-12-02T00:31:00.822281+0000","last_fullsized":"2023-12-02T00:31:00.822281+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.842664+0000","last_change":"2023-12-02T00:31:00.841414+0000","last_active":"2023-12-02T00:31:00.842664+0000","last_peered":"2023-12-02T00:31:00.842664+0000","last_clean":"2023-12-02T00:31:00.842664+0000","last_became_active":"2023-12-02T00:31:00.841049+0000","last_became_peered":"2023-12-02T00:31:00.841049+0000","last_unstale":"2023-12-02T00:31:00.842664+0000","last_undegraded":"2023-12-02T00:31:00.842664+0000","last_fullsized":"2023-12-02T00:31:00.842664+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:34:41.347880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.826447+0000","last_change":"2023-12-02T00:31:00.826447+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.826447+0000","last_undegraded":"2023-12-02T00:31:00.826447+0000","last_fullsized":"2023-12-02T00:31:00.826447+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.839129+0000","last_change":"2023-12-02T00:31:00.838503+0000","last_active":"2023-12-02T00:31:00.839129+0000","last_peered":"2023-12-02T00:31:00.839129+0000","last_clean":"2023-12-02T00:31:00.839129+0000","last_became_active":"2023-12-02T00:31:00.838125+0000","last_became_peered":"2023-12-02T00:31:00.838125+0000","last_unstale":"2023-12-02T00:31:00.839129+0000","last_undegraded":"2023-12-02T00:31:00.839129+0000","last_fullsized":"2023-12-02T00:31:00.839129+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:00:03.107141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.822883+0000","last_change":"2023-12-02T00:31:00.822883+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.822883+0000","last_undegraded":"2023-12-02T00:31:00.822883+0000","last_fullsized":"2023-12-02T00:31:00.822883+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.833802+0000","last_change":"2023-12-02T00:31:00.828726+0000","last_active":"2023-12-02T00:31:00.833802+0000","last_peered":"2023-12-02T00:31:00.833802+0000","last_clean":"2023-12-02T00:31:00.833802+0000","last_became_active":"2023-12-02T00:31:00.828424+0000","last_became_peered":"2023-12-02T00:31:00.828424+0000","last_unstale":"2023-12-02T00:31:00.833802+0000","last_undegraded":"2023-12-02T00:31:00.833802+0000","last_fullsized":"2023-12-02T00:31:00.833802+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T02:42:58.558235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.823153+0000","last_change":"2023-12-02T00:31:00.823153+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.823153+0000","last_undegraded":"2023-12-02T00:31:00.823153+0000","last_fullsized":"2023-12-02T00:31:00.823153+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.840024+0000","last_change":"2023-12-02T00:31:00.837338+0000","last_active":"2023-12-02T00:31:00.840024+0000","last_peered":"2023-12-02T00:31:00.840024+0000","last_clean":"2023-12-02T00:31:00.840024+0000","last_became_active":"2023-12-02T00:31:00.837017+0000","last_became_peered":"2023-12-02T00:31:00.837017+0000","last_unstale":"2023-12-02T00:31:00.840024+0000","last_undegraded":"2023-12-02T00:31:00.840024+0000","last_fullsized":"2023-12-02T00:31:00.840024+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:06:02.637069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2023-12-02T00:31:00.824707+0000","last_change":"2023-12-02T00:31:00.824707+0000","last_active":"2023-12-02T00:31:00.812881+0000","last_peered":"2023-12-02T00:31:00.812881+0000","last_clean":"2023-12-02T00:31:00.812881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-02T00:31:00.824707+0000","last_undegraded":"2023-12-02T00:31:00.824707+0000","last_fullsized":"2023-12-02T00:31:00.824707+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-02T00:31:00.832294+0000","last_change":"2023-12-02T00:31:00.829255+0000","last_active":"2023-12-02T00:31:00.832294+0000","last_peered":"2023-12-02T00:31:00.832294+0000","last_clean":"2023-12-02T00:31:00.832294+0000","last_became_active":"2023-12-02T00:31:00.828976+0000","last_became_peered":"2023-12-02T00:31:00.828976+0000","last_unstale":"2023-12-02T00:31:00.832294+0000","last_undegraded":"2023-12-02T00:31:00.832294+0000","last_fullsized":"2023-12-02T00:31:00.832294+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:30:57.889016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":192,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-12-02T00:30:59.819736+0000","last_change":"2023-12-02T00:28:39.379380+0000","last_active":"2023-12-02T00:30:59.819736+0000","last_peered":"2023-12-02T00:30:59.819736+0000","last_clean":"2023-12-02T00:30:59.819736+0000","last_became_active":"2023-12-02T00:28:39.378255+0000","last_became_peered":"2023-12-02T00:28:39.378255+0000","last_unstale":"2023-12-02T00:30:59.819736+0000","last_undegraded":"2023-12-02T00:30:59.819736+0000","last_fullsized":"2023-12-02T00:30:59.819736+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:38.177345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:38.177345+0000","last_clean_scrub_stamp":"2023-12-02T00:28:38.177345+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:21:08.582412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28156,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343684,"statfs":{"total":96636764160,"available":96607932416,"internally_reserved":0,"allocated":1163264,"data_stored":823065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.50900000000000001,"15min":0.499},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.48499999999999999}]},{"osd":1,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53400000000000003,"15min":0.53000000000000003},"min":{"1min":0.40400000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.54700000000000004,"15min":0.54400000000000004},"min":{"1min":0.46100000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.64500000000000002}]},{"osd":2,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54200000000000004,"15min":0.54400000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.67000000000000004,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.54300000000000004,"15min":0.53400000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.624}]},{"osd":3,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.53000000000000003,"15min":0.52200000000000002},"min":{"1min":0.48099999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.57299999999999995,"15min":0.57499999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.74399999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.68100000000000005}]},{"osd":4,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.50700000000000001,"15min":0.51100000000000001},"min":{"1min":0.29799999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.72199999999999998,"5min":0.745,"15min":0.745},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.51500000000000001,"15min":0.52100000000000002},"min":{"1min":0.33800000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.61799999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.60699999999999998}]},{"osd":5,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54700000000000004,"15min":0.55000000000000004},"min":{"1min":0.41899999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.78700000000000003,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.56100000000000005,"15min":0.56599999999999995},"min":{"1min":0.39500000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.63900000000000001,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.58199999999999996}]},{"osd":6,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.61799999999999999,"15min":0.627},"min":{"1min":0.40100000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.82899999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.60599999999999998,"15min":0.61299999999999999},"min":{"1min":0.44500000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76600000000000001,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.66500000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27592,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":581632,"data_stored":238972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.49199999999999999,"15min":0.48299999999999998},"min":{"1min":0.36499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.50700000000000001,"15min":0.501},"min":{"1min":0.40000000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.74099999999999999,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.64000000000000001}]},{"osd":1,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.51000000000000001,"15min":0.50600000000000001},"min":{"1min":0.33000000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.72599999999999998,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.53600000000000003,"15min":0.53300000000000003},"min":{"1min":0.39400000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.80900000000000005,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.59399999999999997}]},{"osd":2,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.52000000000000002,"15min":0.51200000000000001},"min":{"1min":0.38,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55500000000000005,"15min":0.55700000000000005},"min":{"1min":0.40300000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.77400000000000002,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.66100000000000003}]},{"osd":3,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57599999999999996,"15min":0.57299999999999995},"min":{"1min":0.45200000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.76300000000000001,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.622},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57199999999999995,"15min":0.56999999999999995},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.57699999999999996}]},{"osd":4,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.35399999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.71299999999999997,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52500000000000002,"15min":0.52300000000000002},"min":{"1min":0.34100000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73699999999999999,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.38600000000000001}]},{"osd":5,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.33400000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58799999999999997,"15min":0.59299999999999997},"min":{"1min":0.42399999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.40799999999999997}]},{"osd":7,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.35999999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.442},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.42399999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":569344,"data_stored":232697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.51700000000000002,"15min":0.50600000000000001},"min":{"1min":0.29099999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.73899999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.51600000000000001,"15min":0.51100000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.72899999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.54500000000000004}]},{"osd":1,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.51400000000000001,"15min":0.505},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.754,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55300000000000005,"15min":0.55000000000000004},"min":{"1min":0.36199999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.69499999999999995,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.42399999999999999}]},{"osd":2,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.56000000000000005,"15min":0.55100000000000005},"min":{"1min":0.41699999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.77500000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.53600000000000003,"15min":0.53300000000000003},"min":{"1min":0.34399999999999997,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.749,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.65100000000000002}]},{"osd":3,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58099999999999996,"15min":0.57699999999999996},"min":{"1min":0.39300000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.58299999999999996,"15min":0.58199999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.72499999999999998,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.58599999999999997}]},{"osd":4,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54300000000000004,"15min":0.54600000000000004},"min":{"1min":0.34799999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69899999999999995,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58699999999999997,"15min":0.58799999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.745}]},{"osd":6,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.56599999999999995,"15min":0.57099999999999995},"min":{"1min":0.36899999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.72899999999999998,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.624},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.59199999999999997,"15min":0.59599999999999997},"min":{"1min":0.435,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.73499999999999999}]},{"osd":7,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.42699999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.82099999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.63100000000000001,"15min":0.63500000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.76700000000000002,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.66800000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":569344,"data_stored":232697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.55100000000000005,"15min":0.54400000000000004},"min":{"1min":0.40699999999999997,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.74099999999999999,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.438,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.79500000000000004,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.83599999999999997}]},{"osd":1,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.57099999999999995,"15min":0.55800000000000005},"min":{"1min":0.50700000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.55800000000000005,"15min":0.55400000000000005},"min":{"1min":0.38600000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.77300000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.72099999999999997}]},{"osd":2,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.58099999999999996,"15min":0.57699999999999996},"min":{"1min":0.443,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.628,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.42799999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.75}]},{"osd":3,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.56999999999999995,"15min":0.56200000000000006},"min":{"1min":0.46600000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60599999999999998,"15min":0.60399999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.73499999999999999}]},{"osd":5,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52600000000000002,"15min":0.52300000000000002},"min":{"1min":0.36699999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.83799999999999997,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53900000000000003,"15min":0.54100000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.50600000000000001}]},{"osd":6,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.54200000000000004,"15min":0.53600000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.62},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.57699999999999996,"15min":0.56999999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.56899999999999995}]},{"osd":7,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.377,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.59499999999999997,"15min":0.59099999999999997},"min":{"1min":0.34699999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.70199999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574879,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31740,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340100,"statfs":{"total":96636764160,"available":96604262400,"internally_reserved":0,"allocated":569344,"data_stored":232697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.35699999999999998,"15min":0.35999999999999999},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.50800000000000001,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.38200000000000001,"15min":0.38100000000000001},"min":{"1min":0.27500000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.624,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.36899999999999999}]},{"osd":1,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.379,"15min":0.38300000000000001},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.59999999999999998,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.25800000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.58699999999999997,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.30399999999999999}]},{"osd":2,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.40600000000000003,"15min":0.40999999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.52900000000000003,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.42199999999999999,"15min":0.42099999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.66500000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.34999999999999998}]},{"osd":4,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.60899999999999999,"15min":0.61099999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76600000000000001,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.622,"15min":0.623},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.76300000000000001}]},{"osd":5,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.58299999999999996,"15min":0.58699999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85599999999999998,"5min":1.004,"15min":1.004},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.61299999999999999,"15min":0.61499999999999999},"min":{"1min":0.39300000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.90600000000000003,"5min":1.163,"15min":1.163},"last":0.50600000000000001}]},{"osd":6,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63800000000000001,"15min":0.63900000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.83199999999999996,"5min":1.095,"15min":1.095},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.63200000000000001,"15min":0.63500000000000001},"min":{"1min":0.45100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.96399999999999997,"5min":1.044,"15min":1.044},"last":0.746}]},{"osd":7,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.64700000000000002,"15min":0.65100000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.93899999999999995,"5min":1.177,"15min":1.177},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.64500000000000002,"15min":0.65100000000000002},"min":{"1min":0.45200000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.84799999999999998,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.72299999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31732,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340108,"statfs":{"total":96636764160,"available":96604270592,"internally_reserved":0,"allocated":565248,"data_stored":232078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.377,"15min":0.36299999999999999},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":2.649,"5min":2.649,"15min":2.649},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.42299999999999999,"15min":0.40799999999999997},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":3.173,"5min":3.173,"15min":3.173},"last":0.35099999999999998}]},{"osd":1,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.44800000000000001,"15min":0.41999999999999998},"min":{"1min":0.26300000000000001,"5min":0.246,"15min":0.246},"max":{"1min":4.6639999999999997,"5min":4.6639999999999997,"15min":4.6639999999999997},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.433,"15min":0.39100000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":6.093,"5min":6.093,"15min":6.093},"last":0.39500000000000002}]},{"osd":3,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.40300000000000002,"15min":0.376},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":4.1879999999999997,"5min":4.1879999999999997,"15min":4.1879999999999997},"last":0.317},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.44600000000000001,"15min":0.433},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":2.4159999999999999,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":0.33500000000000002}]},{"osd":4,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.60999999999999999,"15min":0.59599999999999997},"min":{"1min":0.40999999999999998,"5min":0.38,"15min":0.38},"max":{"1min":2.6840000000000002,"5min":2.6840000000000002,"15min":2.6840000000000002},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.65800000000000003,"15min":0.64600000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":3.0419999999999998,"5min":3.0419999999999998,"15min":3.0419999999999998},"last":0.45600000000000002}]},{"osd":5,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.66100000000000003,"15min":0.65300000000000002},"min":{"1min":0.41599999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":3.093,"5min":3.093,"15min":3.093},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.64700000000000002,"15min":0.65100000000000002},"min":{"1min":0.41799999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.98499999999999999,"5min":1.111,"15min":1.111},"last":0.51300000000000001}]},{"osd":6,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.64000000000000001,"15min":0.63800000000000001},"min":{"1min":0.42499999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.44},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.67800000000000005,"15min":0.67200000000000004},"min":{"1min":0.47799999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":2.6030000000000002,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":0.502}]},{"osd":7,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.65800000000000003,"15min":0.64800000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":3.1339999999999999,"5min":3.1339999999999999,"15min":3.1339999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.64200000000000002,"15min":0.64600000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.94199999999999995,"5min":1.2,"15min":1.2},"last":0.47699999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31736,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340104,"statfs":{"total":96636764160,"available":96604266496,"internally_reserved":0,"allocated":569344,"data_stored":232697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.33900000000000002,"15min":0.34100000000000003},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.436,"5min":0.48099999999999998,"15min":0.48099999999999998},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.39600000000000002,"15min":0.39800000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.58799999999999997,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.373}]},{"osd":2,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.35099999999999998,"15min":0.34899999999999998},"min":{"1min":0.249,"5min":0.246,"15min":0.246},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.36299999999999999,"15min":0.36699999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.53500000000000003,"5min":0.626,"15min":0.626},"last":0.29299999999999998}]},{"osd":3,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.40100000000000002,"15min":0.40200000000000002},"min":{"1min":0.30499999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.371,"15min":0.376},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.5,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.42099999999999999}]},{"osd":4,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.55400000000000005,"15min":0.54800000000000004},"min":{"1min":0.377,"5min":0.376,"15min":0.376},"max":{"1min":0.77100000000000002,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.55800000000000005,"15min":0.55100000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.78000000000000003,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.53200000000000003}]},{"osd":5,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.58599999999999997,"15min":0.57899999999999996},"min":{"1min":0.45900000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.96799999999999997,"5min":1.107,"15min":1.107},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57299999999999995,"15min":0.56599999999999995},"min":{"1min":0.437,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.63100000000000001}]},{"osd":6,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60399999999999998,"15min":0.60199999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.76600000000000001,"5min":1.044,"15min":1.044},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.61199999999999999,"15min":0.60599999999999998},"min":{"1min":0.51000000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.85399999999999998,"5min":0.998,"15min":0.998},"last":0.66600000000000004}]},{"osd":7,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58999999999999997,"15min":0.58799999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.78900000000000003,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.627,"5min":0.60599999999999998,"15min":0.60199999999999998},"min":{"1min":0.45300000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.80000000000000004,"5min":1.151,"15min":1.151},"last":0.55300000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32312,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339528,"statfs":{"total":96636764160,"available":96603676672,"internally_reserved":0,"allocated":1159168,"data_stored":822446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Dec 2 00:30:36 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.26000000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.501,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.37,"15min":0.36499999999999999},"min":{"1min":0.28299999999999997,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.58399999999999996,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.37}]},{"osd":2,"last update":"Sat Dec 2 00:30:36 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.54500000000000004,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40200000000000002,"15min":0.40100000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.32600000000000001}]},{"osd":3,"last update":"Sat Dec 2 00:30:36 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.39300000000000002,"15min":0.39400000000000002},"min":{"1min":0.29999999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.53100000000000003,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.436,"5min":0.42899999999999999,"15min":0.42799999999999999},"min":{"1min":0.32700000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.35099999999999998}]},{"osd":4,"last update":"Sat Dec 2 00:30:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57399999999999995,"15min":0.56899999999999995},"min":{"1min":0.42299999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.622,"15min":0.62},"min":{"1min":0.439,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.93100000000000005,"5min":1.002,"15min":1.002},"last":0.47899999999999998}]},{"osd":5,"last update":"Sat Dec 2 00:30:36 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.628,"15min":0.626},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.64700000000000002,"15min":0.64600000000000002},"min":{"1min":0.48099999999999998,"5min":0.433,"15min":0.433},"max":{"1min":0.91100000000000003,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.501}]},{"osd":6,"last update":"Sat Dec 2 00:30:36 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.60799999999999998,"15min":0.59999999999999998},"min":{"1min":0.435,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.89400000000000002,"5min":1.05,"15min":1.05},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.63800000000000001,"15min":0.64000000000000001},"min":{"1min":0.45300000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.94499999999999995,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.44500000000000001}]},{"osd":7,"last update":"Sat Dec 2 00:30:36 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.64300000000000002,"15min":0.63900000000000001},"min":{"1min":0.501,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.97699999999999998,"5min":1.028,"15min":1.028},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.626,"15min":0.626},"min":{"1min":0.46200000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.78000000000000003,"5min":0.874,"15min":0.874},"last":0.432}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-02T00:31:06.552 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3f is not active+clean 2023-12-02T00:31:06.552 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.827720+0000', 'last_change': '2023-12-02T00:31:00.827720+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.827720+0000', 'last_undegraded': '2023-12-02T00:31:00.827720+0000', 'last_fullsized': '2023-12-02T00:31:00.827720+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-02T00:31:06.552 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3e is not active+clean 2023-12-02T00:31:06.552 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.824870+0000', 'last_change': '2023-12-02T00:31:00.824870+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.824870+0000', 'last_undegraded': '2023-12-02T00:31:00.824870+0000', 'last_fullsized': '2023-12-02T00:31:00.824870+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-02T00:31:06.552 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3d is not active+clean 2023-12-02T00:31:06.553 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.826847+0000', 'last_change': '2023-12-02T00:31:00.826847+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.826847+0000', 'last_undegraded': '2023-12-02T00:31:00.826847+0000', 'last_fullsized': '2023-12-02T00:31:00.826847+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-02T00:31:06.553 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3c is not active+clean 2023-12-02T00:31:06.553 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.823450+0000', 'last_change': '2023-12-02T00:31:00.823450+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.823450+0000', 'last_undegraded': '2023-12-02T00:31:00.823450+0000', 'last_fullsized': '2023-12-02T00:31:00.823450+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-02T00:31:06.553 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3b is not active+clean 2023-12-02T00:31:06.553 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.818839+0000', 'last_change': '2023-12-02T00:31:00.818839+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.818839+0000', 'last_undegraded': '2023-12-02T00:31:00.818839+0000', 'last_fullsized': '2023-12-02T00:31:00.818839+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-02T00:31:06.553 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3a is not active+clean 2023-12-02T00:31:06.553 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.828726+0000', 'last_change': '2023-12-02T00:31:00.828726+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.828726+0000', 'last_undegraded': '2023-12-02T00:31:00.828726+0000', 'last_fullsized': '2023-12-02T00:31:00.828726+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-02T00:31:06.553 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.39 is not active+clean 2023-12-02T00:31:06.553 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.827162+0000', 'last_change': '2023-12-02T00:31:00.827162+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.827162+0000', 'last_undegraded': '2023-12-02T00:31:00.827162+0000', 'last_fullsized': '2023-12-02T00:31:00.827162+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-02T00:31:06.553 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.38 is not active+clean 2023-12-02T00:31:06.553 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.834673+0000', 'last_change': '2023-12-02T00:31:00.834673+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.834673+0000', 'last_undegraded': '2023-12-02T00:31:00.834673+0000', 'last_fullsized': '2023-12-02T00:31:00.834673+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-02T00:31:06.553 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.37 is not active+clean 2023-12-02T00:31:06.554 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.833708+0000', 'last_change': '2023-12-02T00:31:00.833708+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.833708+0000', 'last_undegraded': '2023-12-02T00:31:00.833708+0000', 'last_fullsized': '2023-12-02T00:31:00.833708+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-02T00:31:06.554 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.36 is not active+clean 2023-12-02T00:31:06.554 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.829160+0000', 'last_change': '2023-12-02T00:31:00.829160+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.829160+0000', 'last_undegraded': '2023-12-02T00:31:00.829160+0000', 'last_fullsized': '2023-12-02T00:31:00.829160+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-02T00:31:06.554 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.35 is not active+clean 2023-12-02T00:31:06.554 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.818160+0000', 'last_change': '2023-12-02T00:31:00.818160+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.818160+0000', 'last_undegraded': '2023-12-02T00:31:00.818160+0000', 'last_fullsized': '2023-12-02T00:31:00.818160+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-02T00:31:06.554 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.34 is not active+clean 2023-12-02T00:31:06.554 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.831142+0000', 'last_change': '2023-12-02T00:31:00.831142+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.831142+0000', 'last_undegraded': '2023-12-02T00:31:00.831142+0000', 'last_fullsized': '2023-12-02T00:31:00.831142+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-02T00:31:06.554 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.33 is not active+clean 2023-12-02T00:31:06.554 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.827013+0000', 'last_change': '2023-12-02T00:31:00.827013+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.827013+0000', 'last_undegraded': '2023-12-02T00:31:00.827013+0000', 'last_fullsized': '2023-12-02T00:31:00.827013+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-02T00:31:06.555 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.32 is not active+clean 2023-12-02T00:31:06.555 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.826562+0000', 'last_change': '2023-12-02T00:31:00.826562+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.826562+0000', 'last_undegraded': '2023-12-02T00:31:00.826562+0000', 'last_fullsized': '2023-12-02T00:31:00.826562+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-02T00:31:06.555 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.31 is not active+clean 2023-12-02T00:31:06.555 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.823060+0000', 'last_change': '2023-12-02T00:31:00.823060+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.823060+0000', 'last_undegraded': '2023-12-02T00:31:00.823060+0000', 'last_fullsized': '2023-12-02T00:31:00.823060+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-02T00:31:06.555 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.30 is not active+clean 2023-12-02T00:31:06.555 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.825606+0000', 'last_change': '2023-12-02T00:31:00.825606+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.825606+0000', 'last_undegraded': '2023-12-02T00:31:00.825606+0000', 'last_fullsized': '2023-12-02T00:31:00.825606+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-02T00:31:06.555 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2f is not active+clean 2023-12-02T00:31:06.555 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.825954+0000', 'last_change': '2023-12-02T00:31:00.825954+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.825954+0000', 'last_undegraded': '2023-12-02T00:31:00.825954+0000', 'last_fullsized': '2023-12-02T00:31:00.825954+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-02T00:31:06.555 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2e is not active+clean 2023-12-02T00:31:06.556 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.824312+0000', 'last_change': '2023-12-02T00:31:00.824312+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.824312+0000', 'last_undegraded': '2023-12-02T00:31:00.824312+0000', 'last_fullsized': '2023-12-02T00:31:00.824312+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-02T00:31:06.556 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2d is not active+clean 2023-12-02T00:31:06.556 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.825253+0000', 'last_change': '2023-12-02T00:31:00.825253+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.825253+0000', 'last_undegraded': '2023-12-02T00:31:00.825253+0000', 'last_fullsized': '2023-12-02T00:31:00.825253+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-02T00:31:06.556 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2c is not active+clean 2023-12-02T00:31:06.556 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.825702+0000', 'last_change': '2023-12-02T00:31:00.825702+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.825702+0000', 'last_undegraded': '2023-12-02T00:31:00.825702+0000', 'last_fullsized': '2023-12-02T00:31:00.825702+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-02T00:31:06.556 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2b is not active+clean 2023-12-02T00:31:06.556 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.826391+0000', 'last_change': '2023-12-02T00:31:00.826391+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.826391+0000', 'last_undegraded': '2023-12-02T00:31:00.826391+0000', 'last_fullsized': '2023-12-02T00:31:00.826391+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-02T00:31:06.556 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2a is not active+clean 2023-12-02T00:31:06.556 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.831192+0000', 'last_change': '2023-12-02T00:31:00.831192+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.831192+0000', 'last_undegraded': '2023-12-02T00:31:00.831192+0000', 'last_fullsized': '2023-12-02T00:31:00.831192+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-02T00:31:06.556 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.29 is not active+clean 2023-12-02T00:31:06.556 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.818154+0000', 'last_change': '2023-12-02T00:31:00.818154+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.818154+0000', 'last_undegraded': '2023-12-02T00:31:00.818154+0000', 'last_fullsized': '2023-12-02T00:31:00.818154+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-02T00:31:06.557 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.28 is not active+clean 2023-12-02T00:31:06.557 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.824629+0000', 'last_change': '2023-12-02T00:31:00.824629+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.824629+0000', 'last_undegraded': '2023-12-02T00:31:00.824629+0000', 'last_fullsized': '2023-12-02T00:31:00.824629+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-02T00:31:06.557 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.27 is not active+clean 2023-12-02T00:31:06.557 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.824088+0000', 'last_change': '2023-12-02T00:31:00.824088+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.824088+0000', 'last_undegraded': '2023-12-02T00:31:00.824088+0000', 'last_fullsized': '2023-12-02T00:31:00.824088+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-02T00:31:06.557 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.26 is not active+clean 2023-12-02T00:31:06.557 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.829648+0000', 'last_change': '2023-12-02T00:31:00.829648+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.829648+0000', 'last_undegraded': '2023-12-02T00:31:00.829648+0000', 'last_fullsized': '2023-12-02T00:31:00.829648+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-02T00:31:06.557 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.25 is not active+clean 2023-12-02T00:31:06.557 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.827734+0000', 'last_change': '2023-12-02T00:31:00.827734+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.827734+0000', 'last_undegraded': '2023-12-02T00:31:00.827734+0000', 'last_fullsized': '2023-12-02T00:31:00.827734+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-02T00:31:06.557 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.24 is not active+clean 2023-12-02T00:31:06.557 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.824743+0000', 'last_change': '2023-12-02T00:31:00.824743+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.824743+0000', 'last_undegraded': '2023-12-02T00:31:00.824743+0000', 'last_fullsized': '2023-12-02T00:31:00.824743+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-02T00:31:06.557 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.23 is not active+clean 2023-12-02T00:31:06.558 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.823555+0000', 'last_change': '2023-12-02T00:31:00.823555+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.823555+0000', 'last_undegraded': '2023-12-02T00:31:00.823555+0000', 'last_fullsized': '2023-12-02T00:31:00.823555+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-02T00:31:06.558 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.22 is not active+clean 2023-12-02T00:31:06.558 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.823051+0000', 'last_change': '2023-12-02T00:31:00.823051+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.823051+0000', 'last_undegraded': '2023-12-02T00:31:00.823051+0000', 'last_fullsized': '2023-12-02T00:31:00.823051+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-02T00:31:06.558 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.21 is not active+clean 2023-12-02T00:31:06.558 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.823884+0000', 'last_change': '2023-12-02T00:31:00.823884+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.823884+0000', 'last_undegraded': '2023-12-02T00:31:00.823884+0000', 'last_fullsized': '2023-12-02T00:31:00.823884+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-02T00:31:06.558 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.20 is not active+clean 2023-12-02T00:31:06.558 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.827493+0000', 'last_change': '2023-12-02T00:31:00.827493+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.827493+0000', 'last_undegraded': '2023-12-02T00:31:00.827493+0000', 'last_fullsized': '2023-12-02T00:31:00.827493+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-02T00:31:06.558 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.10 is not active+clean 2023-12-02T00:31:06.559 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.824063+0000', 'last_change': '2023-12-02T00:31:00.824063+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.824063+0000', 'last_undegraded': '2023-12-02T00:31:00.824063+0000', 'last_fullsized': '2023-12-02T00:31:00.824063+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-02T00:31:06.559 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.11 is not active+clean 2023-12-02T00:31:06.559 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.825822+0000', 'last_change': '2023-12-02T00:31:00.825822+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.825822+0000', 'last_undegraded': '2023-12-02T00:31:00.825822+0000', 'last_fullsized': '2023-12-02T00:31:00.825822+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-02T00:31:06.559 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.12 is not active+clean 2023-12-02T00:31:06.559 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.826663+0000', 'last_change': '2023-12-02T00:31:00.826663+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.826663+0000', 'last_undegraded': '2023-12-02T00:31:00.826663+0000', 'last_fullsized': '2023-12-02T00:31:00.826663+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-02T00:31:06.559 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.13 is not active+clean 2023-12-02T00:31:06.559 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.823658+0000', 'last_change': '2023-12-02T00:31:00.823658+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.823658+0000', 'last_undegraded': '2023-12-02T00:31:00.823658+0000', 'last_fullsized': '2023-12-02T00:31:00.823658+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-02T00:31:06.559 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.14 is not active+clean 2023-12-02T00:31:06.559 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.818382+0000', 'last_change': '2023-12-02T00:31:00.818382+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.818382+0000', 'last_undegraded': '2023-12-02T00:31:00.818382+0000', 'last_fullsized': '2023-12-02T00:31:00.818382+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-02T00:31:06.560 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.15 is not active+clean 2023-12-02T00:31:06.560 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.823332+0000', 'last_change': '2023-12-02T00:31:00.823332+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.823332+0000', 'last_undegraded': '2023-12-02T00:31:00.823332+0000', 'last_fullsized': '2023-12-02T00:31:00.823332+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-02T00:31:06.560 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.16 is not active+clean 2023-12-02T00:31:06.560 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.822277+0000', 'last_change': '2023-12-02T00:31:00.822277+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.822277+0000', 'last_undegraded': '2023-12-02T00:31:00.822277+0000', 'last_fullsized': '2023-12-02T00:31:00.822277+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-02T00:31:06.560 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.17 is not active+clean 2023-12-02T00:31:06.560 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.823236+0000', 'last_change': '2023-12-02T00:31:00.823236+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.823236+0000', 'last_undegraded': '2023-12-02T00:31:00.823236+0000', 'last_fullsized': '2023-12-02T00:31:00.823236+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-02T00:31:06.560 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.18 is not active+clean 2023-12-02T00:31:06.560 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.818389+0000', 'last_change': '2023-12-02T00:31:00.818389+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.818389+0000', 'last_undegraded': '2023-12-02T00:31:00.818389+0000', 'last_fullsized': '2023-12-02T00:31:00.818389+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-02T00:31:06.561 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.19 is not active+clean 2023-12-02T00:31:06.561 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.827202+0000', 'last_change': '2023-12-02T00:31:00.827202+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.827202+0000', 'last_undegraded': '2023-12-02T00:31:00.827202+0000', 'last_fullsized': '2023-12-02T00:31:00.827202+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-02T00:31:06.561 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1a is not active+clean 2023-12-02T00:31:06.561 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.823398+0000', 'last_change': '2023-12-02T00:31:00.823398+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.823398+0000', 'last_undegraded': '2023-12-02T00:31:00.823398+0000', 'last_fullsized': '2023-12-02T00:31:00.823398+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-02T00:31:06.561 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1b is not active+clean 2023-12-02T00:31:06.561 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.818271+0000', 'last_change': '2023-12-02T00:31:00.818271+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.818271+0000', 'last_undegraded': '2023-12-02T00:31:00.818271+0000', 'last_fullsized': '2023-12-02T00:31:00.818271+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-02T00:31:06.561 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1c is not active+clean 2023-12-02T00:31:06.561 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.826762+0000', 'last_change': '2023-12-02T00:31:00.826762+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.826762+0000', 'last_undegraded': '2023-12-02T00:31:00.826762+0000', 'last_fullsized': '2023-12-02T00:31:00.826762+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-02T00:31:06.561 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1d is not active+clean 2023-12-02T00:31:06.561 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.822967+0000', 'last_change': '2023-12-02T00:31:00.822967+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.822967+0000', 'last_undegraded': '2023-12-02T00:31:00.822967+0000', 'last_fullsized': '2023-12-02T00:31:00.822967+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-02T00:31:06.561 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1e is not active+clean 2023-12-02T00:31:06.562 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.824188+0000', 'last_change': '2023-12-02T00:31:00.824188+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.824188+0000', 'last_undegraded': '2023-12-02T00:31:00.824188+0000', 'last_fullsized': '2023-12-02T00:31:00.824188+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-02T00:31:06.562 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1f is not active+clean 2023-12-02T00:31:06.562 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.823005+0000', 'last_change': '2023-12-02T00:31:00.823005+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.823005+0000', 'last_undegraded': '2023-12-02T00:31:00.823005+0000', 'last_fullsized': '2023-12-02T00:31:00.823005+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-02T00:31:06.562 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.0 is not active+clean 2023-12-02T00:31:06.562 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.822868+0000', 'last_change': '2023-12-02T00:31:00.822868+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.822868+0000', 'last_undegraded': '2023-12-02T00:31:00.822868+0000', 'last_fullsized': '2023-12-02T00:31:00.822868+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-02T00:31:06.562 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1 is not active+clean 2023-12-02T00:31:06.562 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.825542+0000', 'last_change': '2023-12-02T00:31:00.825542+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.825542+0000', 'last_undegraded': '2023-12-02T00:31:00.825542+0000', 'last_fullsized': '2023-12-02T00:31:00.825542+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-02T00:31:06.562 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2 is not active+clean 2023-12-02T00:31:06.562 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.818333+0000', 'last_change': '2023-12-02T00:31:00.818333+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.818333+0000', 'last_undegraded': '2023-12-02T00:31:00.818333+0000', 'last_fullsized': '2023-12-02T00:31:00.818333+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-02T00:31:06.562 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3 is not active+clean 2023-12-02T00:31:06.562 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.826478+0000', 'last_change': '2023-12-02T00:31:00.826478+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.826478+0000', 'last_undegraded': '2023-12-02T00:31:00.826478+0000', 'last_fullsized': '2023-12-02T00:31:00.826478+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-02T00:31:06.563 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.4 is not active+clean 2023-12-02T00:31:06.563 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.826326+0000', 'last_change': '2023-12-02T00:31:00.826326+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.826326+0000', 'last_undegraded': '2023-12-02T00:31:00.826326+0000', 'last_fullsized': '2023-12-02T00:31:00.826326+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-02T00:31:06.563 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.5 is not active+clean 2023-12-02T00:31:06.563 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.824313+0000', 'last_change': '2023-12-02T00:31:00.824313+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.824313+0000', 'last_undegraded': '2023-12-02T00:31:00.824313+0000', 'last_fullsized': '2023-12-02T00:31:00.824313+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-02T00:31:06.563 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.6 is not active+clean 2023-12-02T00:31:06.563 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.822165+0000', 'last_change': '2023-12-02T00:31:00.822165+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.822165+0000', 'last_undegraded': '2023-12-02T00:31:00.822165+0000', 'last_fullsized': '2023-12-02T00:31:00.822165+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-02T00:31:06.563 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.7 is not active+clean 2023-12-02T00:31:06.563 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.826590+0000', 'last_change': '2023-12-02T00:31:00.826590+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.826590+0000', 'last_undegraded': '2023-12-02T00:31:00.826590+0000', 'last_fullsized': '2023-12-02T00:31:00.826590+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-02T00:31:06.563 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.8 is not active+clean 2023-12-02T00:31:06.563 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.825234+0000', 'last_change': '2023-12-02T00:31:00.825234+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.825234+0000', 'last_undegraded': '2023-12-02T00:31:00.825234+0000', 'last_fullsized': '2023-12-02T00:31:00.825234+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-02T00:31:06.563 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.a is not active+clean 2023-12-02T00:31:06.563 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.828284+0000', 'last_change': '2023-12-02T00:31:00.828284+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.828284+0000', 'last_undegraded': '2023-12-02T00:31:00.828284+0000', 'last_fullsized': '2023-12-02T00:31:00.828284+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-02T00:31:06.564 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.b is not active+clean 2023-12-02T00:31:06.564 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.823172+0000', 'last_change': '2023-12-02T00:31:00.823172+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.823172+0000', 'last_undegraded': '2023-12-02T00:31:00.823172+0000', 'last_fullsized': '2023-12-02T00:31:00.823172+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-02T00:31:06.564 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.c is not active+clean 2023-12-02T00:31:06.564 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.822281+0000', 'last_change': '2023-12-02T00:31:00.822281+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.822281+0000', 'last_undegraded': '2023-12-02T00:31:00.822281+0000', 'last_fullsized': '2023-12-02T00:31:00.822281+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-02T00:31:06.564 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.d is not active+clean 2023-12-02T00:31:06.564 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.826447+0000', 'last_change': '2023-12-02T00:31:00.826447+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.826447+0000', 'last_undegraded': '2023-12-02T00:31:00.826447+0000', 'last_fullsized': '2023-12-02T00:31:00.826447+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-02T00:31:06.564 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.e is not active+clean 2023-12-02T00:31:06.564 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.822883+0000', 'last_change': '2023-12-02T00:31:00.822883+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.822883+0000', 'last_undegraded': '2023-12-02T00:31:00.822883+0000', 'last_fullsized': '2023-12-02T00:31:00.822883+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-02T00:31:06.564 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.f is not active+clean 2023-12-02T00:31:06.564 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.823153+0000', 'last_change': '2023-12-02T00:31:00.823153+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.823153+0000', 'last_undegraded': '2023-12-02T00:31:00.823153+0000', 'last_fullsized': '2023-12-02T00:31:00.823153+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-02T00:31:06.564 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.9 is not active+clean 2023-12-02T00:31:06.564 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2023-12-02T00:31:00.824707+0000', 'last_change': '2023-12-02T00:31:00.824707+0000', 'last_active': '2023-12-02T00:31:00.812881+0000', 'last_peered': '2023-12-02T00:31:00.812881+0000', 'last_clean': '2023-12-02T00:31:00.812881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-02T00:31:00.824707+0000', 'last_undegraded': '2023-12-02T00:31:00.824707+0000', 'last_fullsized': '2023-12-02T00:31:00.824707+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'last_clean_scrub_stamp': '2023-12-02T00:31:00.812881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-02T00:31:06.565 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-12-02T00:31:06.693 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.692+0000 7f5c866ad640 1 Processor -- start 2023-12-02T00:31:06.693 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.692+0000 7f5c866ad640 1 -- start start 2023-12-02T00:31:06.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.693+0000 7f5c866ad640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5c8015e730 0x7f5c8015abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:06.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.693+0000 7f5c866ad640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c8015e2a0 con 0x7f5c80143490 2023-12-02T00:31:06.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.693+0000 7f5c866ad640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c8015b240 con 0x7f5c800b1400 2023-12-02T00:31:06.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.693+0000 7f5c866ad640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c8015b520 con 0x7f5c8015e730 2023-12-02T00:31:06.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.693+0000 7f5c84c4a640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5c8015e730 0x7f5c8015abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:06.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.693+0000 7f5c84c4a640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5c8015e730 0x7f5c8015abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59694/0 (socket says 172.21.15.46:59694) 2023-12-02T00:31:06.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.693+0000 7f5c84c4a640 1 -- 172.21.15.46:0/993605685 learned_addr learned my addr 172.21.15.46:0/993605685 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:06.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.693+0000 7f5c7ffff640 1 --1- 172.21.15.46:0/993605685 >> v1:172.21.15.46:6789/0 conn(0x7f5c80143490 0x7f5c800b0cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41788/0 (socket says 172.21.15.46:41788) 2023-12-02T00:31:06.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.694+0000 7f5c84c4a640 1 -- 172.21.15.46:0/993605685 >> v1:172.21.15.162:6789/0 conn(0x7f5c800b1400 legacy=0x7f5c8015a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:06.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.694+0000 7f5c84c4a640 1 -- 172.21.15.46:0/993605685 >> v1:172.21.15.46:6789/0 conn(0x7f5c80143490 legacy=0x7f5c800b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:06.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.694+0000 7f5c7effd640 1 -- 172.21.15.46:0/993605685 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1986264446 0 0) 0x7f5c8015e2a0 con 0x7f5c80143490 2023-12-02T00:31:06.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.694+0000 7f5c84c4a640 1 -- 172.21.15.46:0/993605685 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c8015dc50 con 0x7f5c8015e730 2023-12-02T00:31:06.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.694+0000 7f5c84c4a640 1 --2- 172.21.15.46:0/993605685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c8015e730 0x7f5c8015abf0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f5c78009e50 tx=0x7f5c7802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63bb6f113e057c75 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:06.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.694+0000 7f5c7effd640 1 -- 172.21.15.46:0/993605685 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7802cb20 con 0x7f5c8015e730 2023-12-02T00:31:06.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.695+0000 7f5c7effd640 1 -- 172.21.15.46:0/993605685 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c7802cc80 con 0x7f5c8015e730 2023-12-02T00:31:06.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.695+0000 7f5c7effd640 1 -- 172.21.15.46:0/993605685 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c780343f0 con 0x7f5c8015e730 2023-12-02T00:31:06.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.695+0000 7f5c866ad640 1 -- 172.21.15.46:0/993605685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c8015e730 msgr2=0x7f5c8015abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:06.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.695+0000 7f5c866ad640 1 --2- 172.21.15.46:0/993605685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c8015e730 0x7f5c8015abf0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f5c78009e50 tx=0x7f5c7802bf80 comp rx=0 tx=0).stop 2023-12-02T00:31:06.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.695+0000 7f5c866ad640 1 -- 172.21.15.46:0/993605685 shutdown_connections 2023-12-02T00:31:06.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.695+0000 7f5c866ad640 1 --2- 172.21.15.46:0/993605685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c8015e730 0x7f5c8015abf0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:06.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.695+0000 7f5c866ad640 1 -- 172.21.15.46:0/993605685 >> 172.21.15.46:0/993605685 conn(0x7f5c8006a4d0 msgr2=0x7f5c8006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:06.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.695+0000 7f5c866ad640 1 -- 172.21.15.46:0/993605685 shutdown_connections 2023-12-02T00:31:06.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.695+0000 7f5c866ad640 1 -- 172.21.15.46:0/993605685 wait complete. 2023-12-02T00:31:06.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.696+0000 7f5c866ad640 1 Processor -- start 2023-12-02T00:31:06.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.696+0000 7f5c866ad640 1 -- start start 2023-12-02T00:31:06.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.697+0000 7f5c866ad640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c800b1400 0x7f5c801558c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:06.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.697+0000 7f5c866ad640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c80143490 0x7f5c80154970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:06.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.697+0000 7f5c866ad640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c8015e730 0x7f5c80140810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:06.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.697+0000 7f5c866ad640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c801561c0 con 0x7f5c80143490 2023-12-02T00:31:06.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.697+0000 7f5c866ad640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c801564a0 con 0x7f5c800b1400 2023-12-02T00:31:06.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.697+0000 7f5c866ad640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c800aecd0 con 0x7f5c8015e730 2023-12-02T00:31:06.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.697+0000 7f5c7f7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c80143490 0x7f5c80154970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:06.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.697+0000 7f5c7f7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c80143490 0x7f5c80154970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59702/0 (socket says 172.21.15.46:59702) 2023-12-02T00:31:06.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.697+0000 7f5c7f7fe640 1 -- 172.21.15.46:0/366719733 learned_addr learned my addr 172.21.15.46:0/366719733 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:06.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.697+0000 7f5c84c4a640 1 --2- 172.21.15.46:0/366719733 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c8015e730 0x7f5c80140810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:06.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.697+0000 7f5c7ffff640 1 --2- 172.21.15.46:0/366719733 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c800b1400 0x7f5c801558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:06.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.698+0000 7f5c7f7fe640 1 -- 172.21.15.46:0/366719733 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c8015e730 msgr2=0x7f5c80140810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:06.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.698+0000 7f5c7f7fe640 1 --2- 172.21.15.46:0/366719733 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c8015e730 0x7f5c80140810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:06.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.698+0000 7f5c7f7fe640 1 -- 172.21.15.46:0/366719733 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c800b1400 msgr2=0x7f5c801558c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:06.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.698+0000 7f5c7f7fe640 1 --2- 172.21.15.46:0/366719733 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c800b1400 0x7f5c801558c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:06.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.698+0000 7f5c7f7fe640 1 -- 172.21.15.46:0/366719733 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c78009b30 con 0x7f5c80143490 2023-12-02T00:31:06.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.698+0000 7f5c7ffff640 1 --2- 172.21.15.46:0/366719733 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c800b1400 0x7f5c801558c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:31:06.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.698+0000 7f5c7f7fe640 1 --2- 172.21.15.46:0/366719733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c80143490 0x7f5c80154970 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f5c7000c430 tx=0x7f5c7000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:06.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.698+0000 7f5c7cff9640 1 -- 172.21.15.46:0/366719733 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c70011790 con 0x7f5c80143490 2023-12-02T00:31:06.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.698+0000 7f5c7cff9640 1 -- 172.21.15.46:0/366719733 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c700118f0 con 0x7f5c80143490 2023-12-02T00:31:06.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.698+0000 7f5c866ad640 1 -- 172.21.15.46:0/366719733 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c800aefb0 con 0x7f5c80143490 2023-12-02T00:31:06.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.698+0000 7f5c7cff9640 1 -- 172.21.15.46:0/366719733 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c70011ce0 con 0x7f5c80143490 2023-12-02T00:31:06.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.698+0000 7f5c866ad640 1 -- 172.21.15.46:0/366719733 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c80155e00 con 0x7f5c80143490 2023-12-02T00:31:06.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.699+0000 7f5c866ad640 1 -- 172.21.15.46:0/366719733 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c800af1a0 con 0x7f5c80143490 2023-12-02T00:31:06.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.700+0000 7f5c7cff9640 1 -- 172.21.15.46:0/366719733 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5c7001c460 con 0x7f5c80143490 2023-12-02T00:31:06.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.700+0000 7f5c7cff9640 1 --2- 172.21.15.46:0/366719733 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5c6c06ae40 0x7f5c6c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:06.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.700+0000 7f5c7cff9640 1 -- 172.21.15.46:0/366719733 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5c700894d0 con 0x7f5c80143490 2023-12-02T00:31:06.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.700+0000 7f5c7ffff640 1 --2- 172.21.15.46:0/366719733 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5c6c06ae40 0x7f5c6c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:06.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.701+0000 7f5c7ffff640 1 --2- 172.21.15.46:0/366719733 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5c6c06ae40 0x7f5c6c06d300 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5c74000f60 tx=0x7f5c74007c10 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:06.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.704+0000 7f5c7cff9640 1 -- 172.21.15.46:0/366719733 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5c70054590 con 0x7f5c80143490 2023-12-02T00:31:06.804 INFO:tasks.ceph.mds.a.smithi046.stderr:2023-12-02T00:31:06.803+0000 7fb7a25a8640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-02T00:31:06.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.952+0000 7f5c866ad640 1 -- 172.21.15.46:0/366719733 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f5c80069400 con 0x7f5c80143490 2023-12-02T00:31:06.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.952+0000 7f5c7cff9640 1 -- 172.21.15.46:0/366719733 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1194 (secure 0 0 0) 0x7f5c70057bb0 con 0x7f5c80143490 2023-12-02T00:31:06.953 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:31:06.953 INFO:teuthology.orchestra.run.smithi046.stdout:{"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":162,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":51,"num_osds":8,"num_up_osds":8,"osd_up_since":1701476915,"num_in_osds":8,"osd_in_since":1701476907,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":244142080,"bytes_avail":772849971200,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":24,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"a","status":"up:active","gid":5994}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2023-12-02T00:30:51.188564+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-12-02T00:31:06.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.954+0000 7f5c866ad640 1 -- 172.21.15.46:0/366719733 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5c6c06ae40 msgr2=0x7f5c6c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:06.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.954+0000 7f5c866ad640 1 --2- 172.21.15.46:0/366719733 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5c6c06ae40 0x7f5c6c06d300 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5c74000f60 tx=0x7f5c74007c10 comp rx=0 tx=0).stop 2023-12-02T00:31:06.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.954+0000 7f5c866ad640 1 -- 172.21.15.46:0/366719733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c80143490 msgr2=0x7f5c80154970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:06.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.954+0000 7f5c866ad640 1 --2- 172.21.15.46:0/366719733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c80143490 0x7f5c80154970 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f5c7000c430 tx=0x7f5c7000c8f0 comp rx=0 tx=0).stop 2023-12-02T00:31:06.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.954+0000 7f5c866ad640 1 -- 172.21.15.46:0/366719733 shutdown_connections 2023-12-02T00:31:06.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.954+0000 7f5c866ad640 1 --2- 172.21.15.46:0/366719733 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f5c6c06ae40 0x7f5c6c06d300 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:06.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.954+0000 7f5c866ad640 1 --2- 172.21.15.46:0/366719733 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c8015e730 0x7f5c80140810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:06.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.954+0000 7f5c866ad640 1 --2- 172.21.15.46:0/366719733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c80143490 0x7f5c80154970 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:06.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.954+0000 7f5c866ad640 1 --2- 172.21.15.46:0/366719733 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c800b1400 0x7f5c801558c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:06.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.954+0000 7f5c866ad640 1 -- 172.21.15.46:0/366719733 >> 172.21.15.46:0/366719733 conn(0x7f5c8006a4d0 msgr2=0x7f5c8006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:06.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.954+0000 7f5c866ad640 1 -- 172.21.15.46:0/366719733 shutdown_connections 2023-12-02T00:31:06.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:06.954+0000 7f5c866ad640 1 -- 172.21.15.46:0/366719733 wait complete. 2023-12-02T00:31:06.966 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-12-02T00:31:06.966 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-02T00:31:07.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.072+0000 7f92e1fa0640 1 Processor -- start 2023-12-02T00:31:07.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.072+0000 7f92e1fa0640 1 -- start start 2023-12-02T00:31:07.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.073+0000 7f92e1fa0640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f92dc160960 0x7f92dc0b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:07.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.073+0000 7f92e1fa0640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f92dc0d0020 0x7f92dc0d0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:07.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.073+0000 7f92e1fa0640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92dc0afb60 con 0x7f92dc0b39b0 2023-12-02T00:31:07.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.073+0000 7f92e1fa0640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92dc160d40 con 0x7f92dc0d0020 2023-12-02T00:31:07.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.073+0000 7f92e1fa0640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92dc0ce790 con 0x7f92dc160960 2023-12-02T00:31:07.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.073+0000 7f92e179f640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f92dc0d0020 0x7f92dc0d0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:07.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.073+0000 7f92e179f640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f92dc0d0020 0x7f92dc0d0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59718/0 (socket says 172.21.15.46:59718) 2023-12-02T00:31:07.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.073+0000 7f92e179f640 1 -- 172.21.15.46:0/3378524649 learned_addr learned my addr 172.21.15.46:0/3378524649 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:07.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.073+0000 7f92dbfff640 1 --1- 172.21.15.46:0/3378524649 >> v1:172.21.15.46:6789/0 conn(0x7f92dc0b39b0 0x7f92dc0cf910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41798/0 (socket says 172.21.15.46:41798) 2023-12-02T00:31:07.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.073+0000 7f92e0f9e640 1 --2- 172.21.15.46:0/3378524649 >> v2:172.21.15.162:3300/0 conn(0x7f92dc160960 0x7f92dc0b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:07.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.073+0000 7f92e179f640 1 -- 172.21.15.46:0/3378524649 >> v2:172.21.15.162:3300/0 conn(0x7f92dc160960 msgr2=0x7f92dc0b3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:07.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.073+0000 7f92e179f640 1 --2- 172.21.15.46:0/3378524649 >> v2:172.21.15.162:3300/0 conn(0x7f92dc160960 0x7f92dc0b3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:07.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.073+0000 7f92e179f640 1 -- 172.21.15.46:0/3378524649 >> v1:172.21.15.46:6789/0 conn(0x7f92dc0b39b0 legacy=0x7f92dc0cf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:07.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.073+0000 7f92db7fe640 1 -- 172.21.15.46:0/3378524649 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1232206989 0 0) 0x7f92dc0afb60 con 0x7f92dc0b39b0 2023-12-02T00:31:07.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.073+0000 7f92e179f640 1 -- 172.21.15.46:0/3378524649 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92dc0cefc0 con 0x7f92dc0d0020 2023-12-02T00:31:07.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.073+0000 7f92e0f9e640 1 --2- 172.21.15.46:0/3378524649 >> v2:172.21.15.162:3300/0 conn(0x7f92dc160960 0x7f92dc0b3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:31:07.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.074+0000 7f92e179f640 1 --2- 172.21.15.46:0/3378524649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92dc0d0020 0x7f92dc0d0ed0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f92dc06a820 tx=0x7f92d402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c93d9ffeb4f3bc server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:07.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.074+0000 7f92db7fe640 1 -- 172.21.15.46:0/3378524649 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d402ca80 con 0x7f92dc0d0020 2023-12-02T00:31:07.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.074+0000 7f92db7fe640 1 -- 172.21.15.46:0/3378524649 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92d402cbe0 con 0x7f92dc0d0020 2023-12-02T00:31:07.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.074+0000 7f92db7fe640 1 -- 172.21.15.46:0/3378524649 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d40343f0 con 0x7f92dc0d0020 2023-12-02T00:31:07.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.075+0000 7f92e1fa0640 1 -- 172.21.15.46:0/3378524649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92dc0d0020 msgr2=0x7f92dc0d0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:07.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.075+0000 7f92e1fa0640 1 --2- 172.21.15.46:0/3378524649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92dc0d0020 0x7f92dc0d0ed0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f92dc06a820 tx=0x7f92d402bee0 comp rx=0 tx=0).stop 2023-12-02T00:31:07.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.075+0000 7f92e1fa0640 1 -- 172.21.15.46:0/3378524649 shutdown_connections 2023-12-02T00:31:07.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.075+0000 7f92e1fa0640 1 --2- 172.21.15.46:0/3378524649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92dc0d0020 0x7f92dc0d0ed0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:07.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.075+0000 7f92e1fa0640 1 --2- 172.21.15.46:0/3378524649 >> v2:172.21.15.162:3300/0 conn(0x7f92dc160960 0x7f92dc0b3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:07.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.075+0000 7f92e1fa0640 1 -- 172.21.15.46:0/3378524649 >> 172.21.15.46:0/3378524649 conn(0x7f92dc09c300 msgr2=0x7f92dc0d2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:07.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.075+0000 7f92e1fa0640 1 -- 172.21.15.46:0/3378524649 shutdown_connections 2023-12-02T00:31:07.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.075+0000 7f92e1fa0640 1 -- 172.21.15.46:0/3378524649 wait complete. 2023-12-02T00:31:07.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.076+0000 7f92e1fa0640 1 Processor -- start 2023-12-02T00:31:07.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.076+0000 7f92e1fa0640 1 -- start start 2023-12-02T00:31:07.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.077+0000 7f92e1fa0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92dc0b39b0 0x7f92dc14d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:07.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.077+0000 7f92e1fa0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92dc0d0020 0x7f92dc1e5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:07.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.077+0000 7f92e1fa0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92dc160960 0x7f92dc1e6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:07.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.077+0000 7f92e1fa0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92dc14d8e0 con 0x7f92dc0b39b0 2023-12-02T00:31:07.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.077+0000 7f92e1fa0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92dc0c7520 con 0x7f92dc160960 2023-12-02T00:31:07.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.077+0000 7f92e1fa0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92dc0c7800 con 0x7f92dc0d0020 2023-12-02T00:31:07.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.077+0000 7f92e0f9e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92dc0b39b0 0x7f92dc14d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:07.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.077+0000 7f92e0f9e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92dc0b39b0 0x7f92dc14d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59722/0 (socket says 172.21.15.46:59722) 2023-12-02T00:31:07.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.077+0000 7f92e0f9e640 1 -- 172.21.15.46:0/1470361831 learned_addr learned my addr 172.21.15.46:0/1470361831 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:07.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.077+0000 7f92dbfff640 1 --2- 172.21.15.46:0/1470361831 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92dc0d0020 0x7f92dc1e5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:07.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.077+0000 7f92e179f640 1 --2- 172.21.15.46:0/1470361831 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92dc160960 0x7f92dc1e6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:07.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.077+0000 7f92e0f9e640 1 -- 172.21.15.46:0/1470361831 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92dc0d0020 msgr2=0x7f92dc1e5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:07.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.077+0000 7f92e0f9e640 1 --2- 172.21.15.46:0/1470361831 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92dc0d0020 0x7f92dc1e5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:07.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.077+0000 7f92e0f9e640 1 -- 172.21.15.46:0/1470361831 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92dc160960 msgr2=0x7f92dc1e6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:07.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.077+0000 7f92e0f9e640 1 --2- 172.21.15.46:0/1470361831 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92dc160960 0x7f92dc1e6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:07.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.077+0000 7f92e0f9e640 1 -- 172.21.15.46:0/1470361831 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d4009b30 con 0x7f92dc0b39b0 2023-12-02T00:31:07.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.077+0000 7f92dbfff640 1 --2- 172.21.15.46:0/1470361831 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92dc0d0020 0x7f92dc1e5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:31:07.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.077+0000 7f92e179f640 1 --2- 172.21.15.46:0/1470361831 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92dc160960 0x7f92dc1e6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:31:07.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.077+0000 7f92e0f9e640 1 --2- 172.21.15.46:0/1470361831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92dc0b39b0 0x7f92dc14d3a0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f92d000d7d0 tx=0x7f92d000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:07.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.077+0000 7f92d97fa640 1 -- 172.21.15.46:0/1470361831 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d0004030 con 0x7f92dc0b39b0 2023-12-02T00:31:07.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.077+0000 7f92d97fa640 1 -- 172.21.15.46:0/1470361831 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92d0004190 con 0x7f92dc0b39b0 2023-12-02T00:31:07.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.077+0000 7f92e1fa0640 1 -- 172.21.15.46:0/1470361831 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92dc0c7ae0 con 0x7f92dc0b39b0 2023-12-02T00:31:07.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.077+0000 7f92d97fa640 1 -- 172.21.15.46:0/1470361831 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d0004520 con 0x7f92dc0b39b0 2023-12-02T00:31:07.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.077+0000 7f92e1fa0640 1 -- 172.21.15.46:0/1470361831 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92dc1f2590 con 0x7f92dc0b39b0 2023-12-02T00:31:07.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.078+0000 7f92e1fa0640 1 -- 172.21.15.46:0/1470361831 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92dc067940 con 0x7f92dc0b39b0 2023-12-02T00:31:07.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.079+0000 7f92d97fa640 1 -- 172.21.15.46:0/1470361831 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f92d00096a0 con 0x7f92dc0b39b0 2023-12-02T00:31:07.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.079+0000 7f92d97fa640 1 --2- 172.21.15.46:0/1470361831 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f92c006ae40 0x7f92c006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:07.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.080+0000 7f92d97fa640 1 -- 172.21.15.46:0/1470361831 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f92d0089820 con 0x7f92dc0b39b0 2023-12-02T00:31:07.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.080+0000 7f92dbfff640 1 --2- 172.21.15.46:0/1470361831 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f92c006ae40 0x7f92c006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:07.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.080+0000 7f92dbfff640 1 --2- 172.21.15.46:0/1470361831 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f92c006ae40 0x7f92c006d300 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f92dc1604e0 tx=0x7f92c8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:07.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.084+0000 7f92d97fa640 1 -- 172.21.15.46:0/1470361831 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f92d0054850 con 0x7f92dc0b39b0 2023-12-02T00:31:07.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.256+0000 7f92e1fa0640 1 -- 172.21.15.46:0/1470361831 --> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f92dc15c900 con 0x7f92c006ae40 2023-12-02T00:31:07.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.264+0000 7f92d97fa640 1 -- 172.21.15.46:0/1470361831 <== mgr.4108 v2:172.21.15.162:6832/3248644722 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289603 (secure 0 0 0) 0x7f92dc15c900 con 0x7f92c006ae40 2023-12-02T00:31:07.269 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:31:07.270 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-12-02T00:31:07.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.267+0000 7f92e1fa0640 1 -- 172.21.15.46:0/1470361831 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f92c006ae40 msgr2=0x7f92c006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:07.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.267+0000 7f92e1fa0640 1 --2- 172.21.15.46:0/1470361831 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f92c006ae40 0x7f92c006d300 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f92dc1604e0 tx=0x7f92c8009040 comp rx=0 tx=0).stop 2023-12-02T00:31:07.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.267+0000 7f92e1fa0640 1 -- 172.21.15.46:0/1470361831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92dc0b39b0 msgr2=0x7f92dc14d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:07.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.267+0000 7f92e1fa0640 1 --2- 172.21.15.46:0/1470361831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92dc0b39b0 0x7f92dc14d3a0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f92d000d7d0 tx=0x7f92d000dc90 comp rx=0 tx=0).stop 2023-12-02T00:31:07.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.267+0000 7f92e1fa0640 1 -- 172.21.15.46:0/1470361831 shutdown_connections 2023-12-02T00:31:07.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.267+0000 7f92e1fa0640 1 --2- 172.21.15.46:0/1470361831 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f92c006ae40 0x7f92c006d300 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:07.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.267+0000 7f92e1fa0640 1 --2- 172.21.15.46:0/1470361831 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92dc160960 0x7f92dc1e6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:07.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.267+0000 7f92e1fa0640 1 --2- 172.21.15.46:0/1470361831 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92dc0d0020 0x7f92dc1e5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:07.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.267+0000 7f92e1fa0640 1 --2- 172.21.15.46:0/1470361831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92dc0b39b0 0x7f92dc14d3a0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:07.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.267+0000 7f92e1fa0640 1 -- 172.21.15.46:0/1470361831 >> 172.21.15.46:0/1470361831 conn(0x7f92dc09c300 msgr2=0x7f92dc15ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:07.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.267+0000 7f92e1fa0640 1 -- 172.21.15.46:0/1470361831 shutdown_connections 2023-12-02T00:31:07.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:07.267+0000 7f92e1fa0640 1 -- 172.21.15.46:0/1470361831 wait complete. 2023-12-02T00:31:07.282 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":120,"stamp":"2023-12-02T00:31:07.191906+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":156,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238784,"kb_used_data":5952,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735936,"statfs":{"total":773094113280,"available":772849598464,"internally_reserved":0,"allocated":6094848,"data_stored":3200694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.375661"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.902777+0000","last_change":"2023-12-02T00:31:01.843916+0000","last_active":"2023-12-02T00:31:02.902777+0000","last_peered":"2023-12-02T00:31:02.902777+0000","last_clean":"2023-12-02T00:31:02.902777+0000","last_became_active":"2023-12-02T00:31:01.843636+0000","last_became_peered":"2023-12-02T00:31:01.843636+0000","last_unstale":"2023-12-02T00:31:02.902777+0000","last_undegraded":"2023-12-02T00:31:02.902777+0000","last_fullsized":"2023-12-02T00:31:02.902777+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T10:47:45.803692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903186+0000","last_change":"2023-12-02T00:31:01.844324+0000","last_active":"2023-12-02T00:31:02.903186+0000","last_peered":"2023-12-02T00:31:02.903186+0000","last_clean":"2023-12-02T00:31:02.903186+0000","last_became_active":"2023-12-02T00:31:01.844055+0000","last_became_peered":"2023-12-02T00:31:01.844055+0000","last_unstale":"2023-12-02T00:31:02.903186+0000","last_undegraded":"2023-12-02T00:31:02.903186+0000","last_fullsized":"2023-12-02T00:31:02.903186+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T01:54:26.282475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903229+0000","last_change":"2023-12-02T00:31:01.861283+0000","last_active":"2023-12-02T00:31:02.903229+0000","last_peered":"2023-12-02T00:31:02.903229+0000","last_clean":"2023-12-02T00:31:02.903229+0000","last_became_active":"2023-12-02T00:31:01.861007+0000","last_became_peered":"2023-12-02T00:31:01.861007+0000","last_unstale":"2023-12-02T00:31:02.903229+0000","last_undegraded":"2023-12-02T00:31:02.903229+0000","last_fullsized":"2023-12-02T00:31:02.903229+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T02:25:46.050493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903314+0000","last_change":"2023-12-02T00:31:01.848540+0000","last_active":"2023-12-02T00:31:02.903314+0000","last_peered":"2023-12-02T00:31:02.903314+0000","last_clean":"2023-12-02T00:31:02.903314+0000","last_became_active":"2023-12-02T00:31:01.848276+0000","last_became_peered":"2023-12-02T00:31:01.848276+0000","last_unstale":"2023-12-02T00:31:02.903314+0000","last_undegraded":"2023-12-02T00:31:02.903314+0000","last_fullsized":"2023-12-02T00:31:02.903314+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:47:14.572081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.848753+0000","last_change":"2023-12-02T00:31:01.844040+0000","last_active":"2023-12-02T00:31:02.848753+0000","last_peered":"2023-12-02T00:31:02.848753+0000","last_clean":"2023-12-02T00:31:02.848753+0000","last_became_active":"2023-12-02T00:31:01.843778+0000","last_became_peered":"2023-12-02T00:31:01.843778+0000","last_unstale":"2023-12-02T00:31:02.848753+0000","last_undegraded":"2023-12-02T00:31:02.848753+0000","last_fullsized":"2023-12-02T00:31:02.848753+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T10:47:45.803692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903123+0000","last_change":"2023-12-02T00:31:01.844971+0000","last_active":"2023-12-02T00:31:02.903123+0000","last_peered":"2023-12-02T00:31:02.903123+0000","last_clean":"2023-12-02T00:31:02.903123+0000","last_became_active":"2023-12-02T00:31:01.844705+0000","last_became_peered":"2023-12-02T00:31:01.844705+0000","last_unstale":"2023-12-02T00:31:02.903123+0000","last_undegraded":"2023-12-02T00:31:02.903123+0000","last_fullsized":"2023-12-02T00:31:02.903123+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:41:20.970096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903073+0000","last_change":"2023-12-02T00:31:01.863300+0000","last_active":"2023-12-02T00:31:02.903073+0000","last_peered":"2023-12-02T00:31:02.903073+0000","last_clean":"2023-12-02T00:31:02.903073+0000","last_became_active":"2023-12-02T00:31:01.862826+0000","last_became_peered":"2023-12-02T00:31:01.862826+0000","last_unstale":"2023-12-02T00:31:02.903073+0000","last_undegraded":"2023-12-02T00:31:02.903073+0000","last_fullsized":"2023-12-02T00:31:02.903073+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:23:21.917306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.843359+0000","last_change":"2023-12-02T00:31:01.840463+0000","last_active":"2023-12-02T00:31:02.843359+0000","last_peered":"2023-12-02T00:31:02.843359+0000","last_clean":"2023-12-02T00:31:02.843359+0000","last_became_active":"2023-12-02T00:31:01.840123+0000","last_became_peered":"2023-12-02T00:31:01.840123+0000","last_unstale":"2023-12-02T00:31:02.843359+0000","last_undegraded":"2023-12-02T00:31:02.843359+0000","last_fullsized":"2023-12-02T00:31:02.843359+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:15:02.019922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.843484+0000","last_change":"2023-12-02T00:31:01.841005+0000","last_active":"2023-12-02T00:31:02.843484+0000","last_peered":"2023-12-02T00:31:02.843484+0000","last_clean":"2023-12-02T00:31:02.843484+0000","last_became_active":"2023-12-02T00:31:01.840669+0000","last_became_peered":"2023-12-02T00:31:01.840669+0000","last_unstale":"2023-12-02T00:31:02.843484+0000","last_undegraded":"2023-12-02T00:31:02.843484+0000","last_fullsized":"2023-12-02T00:31:02.843484+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:38:10.124023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.902799+0000","last_change":"2023-12-02T00:31:01.852835+0000","last_active":"2023-12-02T00:31:02.902799+0000","last_peered":"2023-12-02T00:31:02.902799+0000","last_clean":"2023-12-02T00:31:02.902799+0000","last_became_active":"2023-12-02T00:31:01.852547+0000","last_became_peered":"2023-12-02T00:31:01.852547+0000","last_unstale":"2023-12-02T00:31:02.902799+0000","last_undegraded":"2023-12-02T00:31:02.902799+0000","last_fullsized":"2023-12-02T00:31:02.902799+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:54:08.916982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.848266+0000","last_change":"2023-12-02T00:31:01.859361+0000","last_active":"2023-12-02T00:31:02.848266+0000","last_peered":"2023-12-02T00:31:02.848266+0000","last_clean":"2023-12-02T00:31:02.848266+0000","last_became_active":"2023-12-02T00:31:01.859054+0000","last_became_peered":"2023-12-02T00:31:01.859054+0000","last_unstale":"2023-12-02T00:31:02.848266+0000","last_undegraded":"2023-12-02T00:31:02.848266+0000","last_fullsized":"2023-12-02T00:31:02.848266+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:09:24.735030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.843299+0000","last_change":"2023-12-02T00:31:01.840823+0000","last_active":"2023-12-02T00:31:02.843299+0000","last_peered":"2023-12-02T00:31:02.843299+0000","last_clean":"2023-12-02T00:31:02.843299+0000","last_became_active":"2023-12-02T00:31:01.840515+0000","last_became_peered":"2023-12-02T00:31:01.840515+0000","last_unstale":"2023-12-02T00:31:02.843299+0000","last_undegraded":"2023-12-02T00:31:02.843299+0000","last_fullsized":"2023-12-02T00:31:02.843299+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:29:47.951230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.902753+0000","last_change":"2023-12-02T00:31:01.848176+0000","last_active":"2023-12-02T00:31:02.902753+0000","last_peered":"2023-12-02T00:31:02.902753+0000","last_clean":"2023-12-02T00:31:02.902753+0000","last_became_active":"2023-12-02T00:31:01.847864+0000","last_became_peered":"2023-12-02T00:31:01.847864+0000","last_unstale":"2023-12-02T00:31:02.902753+0000","last_undegraded":"2023-12-02T00:31:02.902753+0000","last_fullsized":"2023-12-02T00:31:02.902753+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:00:48.239139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903481+0000","last_change":"2023-12-02T00:31:01.860249+0000","last_active":"2023-12-02T00:31:02.903481+0000","last_peered":"2023-12-02T00:31:02.903481+0000","last_clean":"2023-12-02T00:31:02.903481+0000","last_became_active":"2023-12-02T00:31:01.859910+0000","last_became_peered":"2023-12-02T00:31:01.859910+0000","last_unstale":"2023-12-02T00:31:02.903481+0000","last_undegraded":"2023-12-02T00:31:02.903481+0000","last_fullsized":"2023-12-02T00:31:02.903481+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T02:12:42.101295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903141+0000","last_change":"2023-12-02T00:31:01.849636+0000","last_active":"2023-12-02T00:31:02.903141+0000","last_peered":"2023-12-02T00:31:02.903141+0000","last_clean":"2023-12-02T00:31:02.903141+0000","last_became_active":"2023-12-02T00:31:01.849386+0000","last_became_peered":"2023-12-02T00:31:01.849386+0000","last_unstale":"2023-12-02T00:31:02.903141+0000","last_undegraded":"2023-12-02T00:31:02.903141+0000","last_fullsized":"2023-12-02T00:31:02.903141+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T01:48:44.099841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.902625+0000","last_change":"2023-12-02T00:31:01.845720+0000","last_active":"2023-12-02T00:31:02.902625+0000","last_peered":"2023-12-02T00:31:02.902625+0000","last_clean":"2023-12-02T00:31:02.902625+0000","last_became_active":"2023-12-02T00:31:01.845456+0000","last_became_peered":"2023-12-02T00:31:01.845456+0000","last_unstale":"2023-12-02T00:31:02.902625+0000","last_undegraded":"2023-12-02T00:31:02.902625+0000","last_fullsized":"2023-12-02T00:31:02.902625+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:01:27.538047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903671+0000","last_change":"2023-12-02T00:31:01.844680+0000","last_active":"2023-12-02T00:31:02.903671+0000","last_peered":"2023-12-02T00:31:02.903671+0000","last_clean":"2023-12-02T00:31:02.903671+0000","last_became_active":"2023-12-02T00:31:01.842869+0000","last_became_peered":"2023-12-02T00:31:01.842869+0000","last_unstale":"2023-12-02T00:31:02.903671+0000","last_undegraded":"2023-12-02T00:31:02.903671+0000","last_fullsized":"2023-12-02T00:31:02.903671+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:19:31.202411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903303+0000","last_change":"2023-12-02T00:31:01.850644+0000","last_active":"2023-12-02T00:31:02.903303+0000","last_peered":"2023-12-02T00:31:02.903303+0000","last_clean":"2023-12-02T00:31:02.903303+0000","last_became_active":"2023-12-02T00:31:01.850392+0000","last_became_peered":"2023-12-02T00:31:01.850392+0000","last_unstale":"2023-12-02T00:31:02.903303+0000","last_undegraded":"2023-12-02T00:31:02.903303+0000","last_fullsized":"2023-12-02T00:31:02.903303+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T01:53:57.181929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903342+0000","last_change":"2023-12-02T00:31:01.845720+0000","last_active":"2023-12-02T00:31:02.903342+0000","last_peered":"2023-12-02T00:31:02.903342+0000","last_clean":"2023-12-02T00:31:02.903342+0000","last_became_active":"2023-12-02T00:31:01.845415+0000","last_became_peered":"2023-12-02T00:31:01.845415+0000","last_unstale":"2023-12-02T00:31:02.903342+0000","last_undegraded":"2023-12-02T00:31:02.903342+0000","last_fullsized":"2023-12-02T00:31:02.903342+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T12:00:43.351948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.902687+0000","last_change":"2023-12-02T00:31:01.848558+0000","last_active":"2023-12-02T00:31:02.902687+0000","last_peered":"2023-12-02T00:31:02.902687+0000","last_clean":"2023-12-02T00:31:02.902687+0000","last_became_active":"2023-12-02T00:31:01.848274+0000","last_became_peered":"2023-12-02T00:31:01.848274+0000","last_unstale":"2023-12-02T00:31:02.902687+0000","last_undegraded":"2023-12-02T00:31:02.902687+0000","last_fullsized":"2023-12-02T00:31:02.902687+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:56:42.008355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903114+0000","last_change":"2023-12-02T00:31:01.844873+0000","last_active":"2023-12-02T00:31:02.903114+0000","last_peered":"2023-12-02T00:31:02.903114+0000","last_clean":"2023-12-02T00:31:02.903114+0000","last_became_active":"2023-12-02T00:31:01.844607+0000","last_became_peered":"2023-12-02T00:31:01.844607+0000","last_unstale":"2023-12-02T00:31:02.903114+0000","last_undegraded":"2023-12-02T00:31:02.903114+0000","last_fullsized":"2023-12-02T00:31:02.903114+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:00:03.302160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.843116+0000","last_change":"2023-12-02T00:31:01.843039+0000","last_active":"2023-12-02T00:31:02.843116+0000","last_peered":"2023-12-02T00:31:02.843116+0000","last_clean":"2023-12-02T00:31:02.843116+0000","last_became_active":"2023-12-02T00:31:01.842705+0000","last_became_peered":"2023-12-02T00:31:01.842705+0000","last_unstale":"2023-12-02T00:31:02.843116+0000","last_undegraded":"2023-12-02T00:31:02.843116+0000","last_fullsized":"2023-12-02T00:31:02.843116+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:42:36.235273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.848078+0000","last_change":"2023-12-02T00:31:01.859476+0000","last_active":"2023-12-02T00:31:02.848078+0000","last_peered":"2023-12-02T00:31:02.848078+0000","last_clean":"2023-12-02T00:31:02.848078+0000","last_became_active":"2023-12-02T00:31:01.859172+0000","last_became_peered":"2023-12-02T00:31:01.859172+0000","last_unstale":"2023-12-02T00:31:02.848078+0000","last_undegraded":"2023-12-02T00:31:02.848078+0000","last_fullsized":"2023-12-02T00:31:02.848078+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:14:38.642330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.902799+0000","last_change":"2023-12-02T00:31:01.860276+0000","last_active":"2023-12-02T00:31:02.902799+0000","last_peered":"2023-12-02T00:31:02.902799+0000","last_clean":"2023-12-02T00:31:02.902799+0000","last_became_active":"2023-12-02T00:31:01.859967+0000","last_became_peered":"2023-12-02T00:31:01.859967+0000","last_unstale":"2023-12-02T00:31:02.902799+0000","last_undegraded":"2023-12-02T00:31:02.902799+0000","last_fullsized":"2023-12-02T00:31:02.902799+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:34:42.351893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903750+0000","last_change":"2023-12-02T00:31:01.841808+0000","last_active":"2023-12-02T00:31:02.903750+0000","last_peered":"2023-12-02T00:31:02.903750+0000","last_clean":"2023-12-02T00:31:02.903750+0000","last_became_active":"2023-12-02T00:31:01.841492+0000","last_became_peered":"2023-12-02T00:31:01.841492+0000","last_unstale":"2023-12-02T00:31:02.903750+0000","last_undegraded":"2023-12-02T00:31:02.903750+0000","last_fullsized":"2023-12-02T00:31:02.903750+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:47:14.572081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.843376+0000","last_change":"2023-12-02T00:31:01.864012+0000","last_active":"2023-12-02T00:31:02.843376+0000","last_peered":"2023-12-02T00:31:02.843376+0000","last_clean":"2023-12-02T00:31:02.843376+0000","last_became_active":"2023-12-02T00:31:01.863740+0000","last_became_peered":"2023-12-02T00:31:01.863740+0000","last_unstale":"2023-12-02T00:31:02.843376+0000","last_undegraded":"2023-12-02T00:31:02.843376+0000","last_fullsized":"2023-12-02T00:31:02.843376+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:51:36.069816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903332+0000","last_change":"2023-12-02T00:31:01.848266+0000","last_active":"2023-12-02T00:31:02.903332+0000","last_peered":"2023-12-02T00:31:02.903332+0000","last_clean":"2023-12-02T00:31:02.903332+0000","last_became_active":"2023-12-02T00:31:01.847651+0000","last_became_peered":"2023-12-02T00:31:01.847651+0000","last_unstale":"2023-12-02T00:31:02.903332+0000","last_undegraded":"2023-12-02T00:31:02.903332+0000","last_fullsized":"2023-12-02T00:31:02.903332+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:00:04.111155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.902827+0000","last_change":"2023-12-02T00:31:01.859996+0000","last_active":"2023-12-02T00:31:02.902827+0000","last_peered":"2023-12-02T00:31:02.902827+0000","last_clean":"2023-12-02T00:31:02.902827+0000","last_became_active":"2023-12-02T00:31:01.859530+0000","last_became_peered":"2023-12-02T00:31:01.859530+0000","last_unstale":"2023-12-02T00:31:02.902827+0000","last_undegraded":"2023-12-02T00:31:02.902827+0000","last_fullsized":"2023-12-02T00:31:02.902827+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T09:37:49.913977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903077+0000","last_change":"2023-12-02T00:31:01.839428+0000","last_active":"2023-12-02T00:31:02.903077+0000","last_peered":"2023-12-02T00:31:02.903077+0000","last_clean":"2023-12-02T00:31:02.903077+0000","last_became_active":"2023-12-02T00:31:01.839103+0000","last_became_peered":"2023-12-02T00:31:01.839103+0000","last_unstale":"2023-12-02T00:31:02.903077+0000","last_undegraded":"2023-12-02T00:31:02.903077+0000","last_fullsized":"2023-12-02T00:31:02.903077+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:34:40.432062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.902795+0000","last_change":"2023-12-02T00:31:01.857804+0000","last_active":"2023-12-02T00:31:02.902795+0000","last_peered":"2023-12-02T00:31:02.902795+0000","last_clean":"2023-12-02T00:31:02.902795+0000","last_became_active":"2023-12-02T00:31:01.857510+0000","last_became_peered":"2023-12-02T00:31:01.857510+0000","last_unstale":"2023-12-02T00:31:02.902795+0000","last_undegraded":"2023-12-02T00:31:02.902795+0000","last_fullsized":"2023-12-02T00:31:02.902795+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T02:12:42.101295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903207+0000","last_change":"2023-12-02T00:31:01.863101+0000","last_active":"2023-12-02T00:31:02.903207+0000","last_peered":"2023-12-02T00:31:02.903207+0000","last_clean":"2023-12-02T00:31:02.903207+0000","last_became_active":"2023-12-02T00:31:01.862445+0000","last_became_peered":"2023-12-02T00:31:01.862445+0000","last_unstale":"2023-12-02T00:31:02.903207+0000","last_undegraded":"2023-12-02T00:31:02.903207+0000","last_fullsized":"2023-12-02T00:31:02.903207+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:31:09.860031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.902689+0000","last_change":"2023-12-02T00:31:01.850281+0000","last_active":"2023-12-02T00:31:02.902689+0000","last_peered":"2023-12-02T00:31:02.902689+0000","last_clean":"2023-12-02T00:31:02.902689+0000","last_became_active":"2023-12-02T00:31:01.850013+0000","last_became_peered":"2023-12-02T00:31:01.850013+0000","last_unstale":"2023-12-02T00:31:02.902689+0000","last_undegraded":"2023-12-02T00:31:02.902689+0000","last_fullsized":"2023-12-02T00:31:02.902689+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:40:06.079397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.902601+0000","last_change":"2023-12-02T00:31:01.843400+0000","last_active":"2023-12-02T00:31:02.902601+0000","last_peered":"2023-12-02T00:31:02.902601+0000","last_clean":"2023-12-02T00:31:02.902601+0000","last_became_active":"2023-12-02T00:31:01.843151+0000","last_became_peered":"2023-12-02T00:31:01.843151+0000","last_unstale":"2023-12-02T00:31:02.902601+0000","last_undegraded":"2023-12-02T00:31:02.902601+0000","last_fullsized":"2023-12-02T00:31:02.902601+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:28:45.087022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"50'4","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.862288+0000","last_change":"2023-12-02T00:31:00.832088+0000","last_active":"2023-12-02T00:31:01.862288+0000","last_peered":"2023-12-02T00:31:01.862288+0000","last_clean":"2023-12-02T00:31:01.862288+0000","last_became_active":"2023-12-02T00:31:00.830872+0000","last_became_peered":"2023-12-02T00:31:00.830872+0000","last_unstale":"2023-12-02T00:31:01.862288+0000","last_undegraded":"2023-12-02T00:31:01.862288+0000","last_fullsized":"2023-12-02T00:31:01.862288+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:33:58.845396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903341+0000","last_change":"2023-12-02T00:31:01.846929+0000","last_active":"2023-12-02T00:31:02.903341+0000","last_peered":"2023-12-02T00:31:02.903341+0000","last_clean":"2023-12-02T00:31:02.903341+0000","last_became_active":"2023-12-02T00:31:01.846657+0000","last_became_peered":"2023-12-02T00:31:01.846657+0000","last_unstale":"2023-12-02T00:31:02.903341+0000","last_undegraded":"2023-12-02T00:31:02.903341+0000","last_fullsized":"2023-12-02T00:31:02.903341+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:34:02.551214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"50'1","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.864016+0000","last_change":"2023-12-02T00:31:00.834787+0000","last_active":"2023-12-02T00:31:01.864016+0000","last_peered":"2023-12-02T00:31:01.864016+0000","last_clean":"2023-12-02T00:31:01.864016+0000","last_became_active":"2023-12-02T00:31:00.834357+0000","last_became_peered":"2023-12-02T00:31:00.834357+0000","last_unstale":"2023-12-02T00:31:01.864016+0000","last_undegraded":"2023-12-02T00:31:01.864016+0000","last_fullsized":"2023-12-02T00:31:01.864016+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:36:12.890956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.902860+0000","last_change":"2023-12-02T00:31:01.843931+0000","last_active":"2023-12-02T00:31:02.902860+0000","last_peered":"2023-12-02T00:31:02.902860+0000","last_clean":"2023-12-02T00:31:02.902860+0000","last_became_active":"2023-12-02T00:31:01.843625+0000","last_became_peered":"2023-12-02T00:31:01.843625+0000","last_unstale":"2023-12-02T00:31:02.902860+0000","last_undegraded":"2023-12-02T00:31:02.902860+0000","last_fullsized":"2023-12-02T00:31:02.902860+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:36:13.894969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"50'2","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.864846+0000","last_change":"2023-12-02T00:31:00.837792+0000","last_active":"2023-12-02T00:31:01.864846+0000","last_peered":"2023-12-02T00:31:01.864846+0000","last_clean":"2023-12-02T00:31:01.864846+0000","last_became_active":"2023-12-02T00:31:00.837471+0000","last_became_peered":"2023-12-02T00:31:00.837471+0000","last_unstale":"2023-12-02T00:31:01.864846+0000","last_undegraded":"2023-12-02T00:31:01.864846+0000","last_fullsized":"2023-12-02T00:31:01.864846+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T10:02:53.861890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.843174+0000","last_change":"2023-12-02T00:31:01.863073+0000","last_active":"2023-12-02T00:31:02.843174+0000","last_peered":"2023-12-02T00:31:02.843174+0000","last_clean":"2023-12-02T00:31:02.843174+0000","last_became_active":"2023-12-02T00:31:01.861502+0000","last_became_peered":"2023-12-02T00:31:01.861502+0000","last_unstale":"2023-12-02T00:31:02.843174+0000","last_undegraded":"2023-12-02T00:31:02.843174+0000","last_fullsized":"2023-12-02T00:31:02.843174+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T01:54:53.724229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"50'2","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.859901+0000","last_change":"2023-12-02T00:31:00.834209+0000","last_active":"2023-12-02T00:31:01.859901+0000","last_peered":"2023-12-02T00:31:01.859901+0000","last_clean":"2023-12-02T00:31:01.859901+0000","last_became_active":"2023-12-02T00:31:00.833798+0000","last_became_peered":"2023-12-02T00:31:00.833798+0000","last_unstale":"2023-12-02T00:31:01.859901+0000","last_undegraded":"2023-12-02T00:31:01.859901+0000","last_fullsized":"2023-12-02T00:31:01.859901+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T09:26:08.721807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.848706+0000","last_change":"2023-12-02T00:31:01.844842+0000","last_active":"2023-12-02T00:31:02.848706+0000","last_peered":"2023-12-02T00:31:02.848706+0000","last_clean":"2023-12-02T00:31:02.848706+0000","last_became_active":"2023-12-02T00:31:01.844573+0000","last_became_peered":"2023-12-02T00:31:01.844573+0000","last_unstale":"2023-12-02T00:31:02.848706+0000","last_undegraded":"2023-12-02T00:31:02.848706+0000","last_fullsized":"2023-12-02T00:31:02.848706+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:00:48.239139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"50'1","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.864031+0000","last_change":"2023-12-02T00:31:00.821690+0000","last_active":"2023-12-02T00:31:01.864031+0000","last_peered":"2023-12-02T00:31:01.864031+0000","last_clean":"2023-12-02T00:31:01.864031+0000","last_became_active":"2023-12-02T00:31:00.821365+0000","last_became_peered":"2023-12-02T00:31:00.821365+0000","last_unstale":"2023-12-02T00:31:01.864031+0000","last_undegraded":"2023-12-02T00:31:01.864031+0000","last_fullsized":"2023-12-02T00:31:01.864031+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:40:05.075383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903579+0000","last_change":"2023-12-02T00:31:01.847683+0000","last_active":"2023-12-02T00:31:02.903579+0000","last_peered":"2023-12-02T00:31:02.903579+0000","last_clean":"2023-12-02T00:31:02.903579+0000","last_became_active":"2023-12-02T00:31:01.847313+0000","last_became_peered":"2023-12-02T00:31:01.847313+0000","last_unstale":"2023-12-02T00:31:02.903579+0000","last_undegraded":"2023-12-02T00:31:02.903579+0000","last_fullsized":"2023-12-02T00:31:02.903579+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:53:25.157550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"50'1","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.854047+0000","last_change":"2023-12-02T00:31:00.831123+0000","last_active":"2023-12-02T00:31:01.854047+0000","last_peered":"2023-12-02T00:31:01.854047+0000","last_clean":"2023-12-02T00:31:01.854047+0000","last_became_active":"2023-12-02T00:31:00.830870+0000","last_became_peered":"2023-12-02T00:31:00.830870+0000","last_unstale":"2023-12-02T00:31:01.854047+0000","last_undegraded":"2023-12-02T00:31:01.854047+0000","last_fullsized":"2023-12-02T00:31:01.854047+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T10:02:53.861890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.843515+0000","last_change":"2023-12-02T00:31:01.861765+0000","last_active":"2023-12-02T00:31:02.843515+0000","last_peered":"2023-12-02T00:31:02.843515+0000","last_clean":"2023-12-02T00:31:02.843515+0000","last_became_active":"2023-12-02T00:31:01.861462+0000","last_became_peered":"2023-12-02T00:31:01.861462+0000","last_unstale":"2023-12-02T00:31:02.843515+0000","last_undegraded":"2023-12-02T00:31:02.843515+0000","last_fullsized":"2023-12-02T00:31:02.843515+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:48:30.281237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.837604+0000","last_change":"2023-12-02T00:31:00.834261+0000","last_active":"2023-12-02T00:31:01.837604+0000","last_peered":"2023-12-02T00:31:01.837604+0000","last_clean":"2023-12-02T00:31:01.837604+0000","last_became_active":"2023-12-02T00:31:00.833992+0000","last_became_peered":"2023-12-02T00:31:00.833992+0000","last_unstale":"2023-12-02T00:31:01.837604+0000","last_undegraded":"2023-12-02T00:31:01.837604+0000","last_fullsized":"2023-12-02T00:31:01.837604+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T01:12:56.020427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903201+0000","last_change":"2023-12-02T00:31:01.850143+0000","last_active":"2023-12-02T00:31:02.903201+0000","last_peered":"2023-12-02T00:31:02.903201+0000","last_clean":"2023-12-02T00:31:02.903201+0000","last_became_active":"2023-12-02T00:31:01.849883+0000","last_became_peered":"2023-12-02T00:31:01.849883+0000","last_unstale":"2023-12-02T00:31:02.903201+0000","last_undegraded":"2023-12-02T00:31:02.903201+0000","last_fullsized":"2023-12-02T00:31:02.903201+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:19:31.202411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.833108+0000","last_change":"2023-12-02T00:31:00.828451+0000","last_active":"2023-12-02T00:31:01.833108+0000","last_peered":"2023-12-02T00:31:01.833108+0000","last_clean":"2023-12-02T00:31:01.833108+0000","last_became_active":"2023-12-02T00:31:00.828164+0000","last_became_peered":"2023-12-02T00:31:00.828164+0000","last_unstale":"2023-12-02T00:31:01.833108+0000","last_undegraded":"2023-12-02T00:31:01.833108+0000","last_fullsized":"2023-12-02T00:31:01.833108+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:01:26.534034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.848407+0000","last_change":"2023-12-02T00:31:01.841803+0000","last_active":"2023-12-02T00:31:02.848407+0000","last_peered":"2023-12-02T00:31:02.848407+0000","last_clean":"2023-12-02T00:31:02.848407+0000","last_became_active":"2023-12-02T00:31:01.839190+0000","last_became_peered":"2023-12-02T00:31:01.839190+0000","last_unstale":"2023-12-02T00:31:02.848407+0000","last_undegraded":"2023-12-02T00:31:02.848407+0000","last_fullsized":"2023-12-02T00:31:02.848407+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:01:27.538047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.840644+0000","last_change":"2023-12-02T00:31:00.835042+0000","last_active":"2023-12-02T00:31:01.840644+0000","last_peered":"2023-12-02T00:31:01.840644+0000","last_clean":"2023-12-02T00:31:01.840644+0000","last_became_active":"2023-12-02T00:31:00.834764+0000","last_became_peered":"2023-12-02T00:31:00.834764+0000","last_unstale":"2023-12-02T00:31:01.840644+0000","last_undegraded":"2023-12-02T00:31:01.840644+0000","last_fullsized":"2023-12-02T00:31:01.840644+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:34:39.428048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.902661+0000","last_change":"2023-12-02T00:31:01.845726+0000","last_active":"2023-12-02T00:31:02.902661+0000","last_peered":"2023-12-02T00:31:02.902661+0000","last_clean":"2023-12-02T00:31:02.902661+0000","last_became_active":"2023-12-02T00:31:01.845447+0000","last_became_peered":"2023-12-02T00:31:01.845447+0000","last_unstale":"2023-12-02T00:31:02.902661+0000","last_undegraded":"2023-12-02T00:31:02.902661+0000","last_fullsized":"2023-12-02T00:31:02.902661+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:34:42.351893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.838788+0000","last_change":"2023-12-02T00:31:00.821222+0000","last_active":"2023-12-02T00:31:01.838788+0000","last_peered":"2023-12-02T00:31:01.838788+0000","last_clean":"2023-12-02T00:31:01.838788+0000","last_became_active":"2023-12-02T00:31:00.820914+0000","last_became_peered":"2023-12-02T00:31:00.820914+0000","last_unstale":"2023-12-02T00:31:01.838788+0000","last_undegraded":"2023-12-02T00:31:01.838788+0000","last_fullsized":"2023-12-02T00:31:01.838788+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:30:31.052164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903504+0000","last_change":"2023-12-02T00:31:01.837285+0000","last_active":"2023-12-02T00:31:02.903504+0000","last_peered":"2023-12-02T00:31:02.903504+0000","last_clean":"2023-12-02T00:31:02.903504+0000","last_became_active":"2023-12-02T00:31:01.836952+0000","last_became_peered":"2023-12-02T00:31:01.836952+0000","last_unstale":"2023-12-02T00:31:02.903504+0000","last_undegraded":"2023-12-02T00:31:02.903504+0000","last_fullsized":"2023-12-02T00:31:02.903504+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T01:48:44.099841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"50'3","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.865472+0000","last_change":"2023-12-02T00:31:00.837992+0000","last_active":"2023-12-02T00:31:01.865472+0000","last_peered":"2023-12-02T00:31:01.865472+0000","last_clean":"2023-12-02T00:31:01.865472+0000","last_became_active":"2023-12-02T00:31:00.837743+0000","last_became_peered":"2023-12-02T00:31:00.837743+0000","last_unstale":"2023-12-02T00:31:01.865472+0000","last_undegraded":"2023-12-02T00:31:01.865472+0000","last_fullsized":"2023-12-02T00:31:01.865472+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:34:01.547200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.848794+0000","last_change":"2023-12-02T00:31:01.859827+0000","last_active":"2023-12-02T00:31:02.848794+0000","last_peered":"2023-12-02T00:31:02.848794+0000","last_clean":"2023-12-02T00:31:02.848794+0000","last_became_active":"2023-12-02T00:31:01.859565+0000","last_became_peered":"2023-12-02T00:31:01.859565+0000","last_unstale":"2023-12-02T00:31:02.848794+0000","last_undegraded":"2023-12-02T00:31:02.848794+0000","last_fullsized":"2023-12-02T00:31:02.848794+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:54:08.916982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"50'6","reported_seq":18,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.868031+0000","last_change":"2023-12-02T00:31:00.840634+0000","last_active":"2023-12-02T00:31:01.868031+0000","last_peered":"2023-12-02T00:31:01.868031+0000","last_clean":"2023-12-02T00:31:01.868031+0000","last_became_active":"2023-12-02T00:31:00.840359+0000","last_became_peered":"2023-12-02T00:31:00.840359+0000","last_unstale":"2023-12-02T00:31:01.868031+0000","last_undegraded":"2023-12-02T00:31:01.868031+0000","last_fullsized":"2023-12-02T00:31:01.868031+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T02:12:41.097282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903534+0000","last_change":"2023-12-02T00:31:01.847992+0000","last_active":"2023-12-02T00:31:02.903534+0000","last_peered":"2023-12-02T00:31:02.903534+0000","last_clean":"2023-12-02T00:31:02.903534+0000","last_became_active":"2023-12-02T00:31:01.847631+0000","last_became_peered":"2023-12-02T00:31:01.847631+0000","last_unstale":"2023-12-02T00:31:02.903534+0000","last_undegraded":"2023-12-02T00:31:02.903534+0000","last_fullsized":"2023-12-02T00:31:02.903534+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:23:21.917306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"50'2","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.860725+0000","last_change":"2023-12-02T00:31:00.829212+0000","last_active":"2023-12-02T00:31:01.860725+0000","last_peered":"2023-12-02T00:31:01.860725+0000","last_clean":"2023-12-02T00:31:01.860725+0000","last_became_active":"2023-12-02T00:31:00.828848+0000","last_became_peered":"2023-12-02T00:31:00.828848+0000","last_unstale":"2023-12-02T00:31:01.860725+0000","last_undegraded":"2023-12-02T00:31:01.860725+0000","last_fullsized":"2023-12-02T00:31:01.860725+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T10:47:44.799679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903446+0000","last_change":"2023-12-02T00:31:01.846618+0000","last_active":"2023-12-02T00:31:02.903446+0000","last_peered":"2023-12-02T00:31:02.903446+0000","last_clean":"2023-12-02T00:31:02.903446+0000","last_became_active":"2023-12-02T00:31:01.846263+0000","last_became_peered":"2023-12-02T00:31:01.846263+0000","last_unstale":"2023-12-02T00:31:02.903446+0000","last_undegraded":"2023-12-02T00:31:02.903446+0000","last_fullsized":"2023-12-02T00:31:02.903446+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:28:45.087022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.836432+0000","last_change":"2023-12-02T00:31:00.834656+0000","last_active":"2023-12-02T00:31:01.836432+0000","last_peered":"2023-12-02T00:31:01.836432+0000","last_clean":"2023-12-02T00:31:01.836432+0000","last_became_active":"2023-12-02T00:31:00.834359+0000","last_became_peered":"2023-12-02T00:31:00.834359+0000","last_unstale":"2023-12-02T00:31:01.836432+0000","last_undegraded":"2023-12-02T00:31:01.836432+0000","last_fullsized":"2023-12-02T00:31:01.836432+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T09:28:07.117685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.843490+0000","last_change":"2023-12-02T00:31:01.843226+0000","last_active":"2023-12-02T00:31:02.843490+0000","last_peered":"2023-12-02T00:31:02.843490+0000","last_clean":"2023-12-02T00:31:02.843490+0000","last_became_active":"2023-12-02T00:31:01.842940+0000","last_became_peered":"2023-12-02T00:31:01.842940+0000","last_unstale":"2023-12-02T00:31:02.843490+0000","last_undegraded":"2023-12-02T00:31:02.843490+0000","last_fullsized":"2023-12-02T00:31:02.843490+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:05:01.325502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.839978+0000","last_change":"2023-12-02T00:31:00.837228+0000","last_active":"2023-12-02T00:31:01.839978+0000","last_peered":"2023-12-02T00:31:01.839978+0000","last_clean":"2023-12-02T00:31:01.839978+0000","last_became_active":"2023-12-02T00:31:00.836943+0000","last_became_peered":"2023-12-02T00:31:00.836943+0000","last_unstale":"2023-12-02T00:31:01.839978+0000","last_undegraded":"2023-12-02T00:31:01.839978+0000","last_fullsized":"2023-12-02T00:31:01.839978+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:34:41.347880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903680+0000","last_change":"2023-12-02T00:31:01.843905+0000","last_active":"2023-12-02T00:31:02.903680+0000","last_peered":"2023-12-02T00:31:02.903680+0000","last_clean":"2023-12-02T00:31:02.903680+0000","last_became_active":"2023-12-02T00:31:01.842704+0000","last_became_peered":"2023-12-02T00:31:01.842704+0000","last_unstale":"2023-12-02T00:31:02.903680+0000","last_undegraded":"2023-12-02T00:31:02.903680+0000","last_fullsized":"2023-12-02T00:31:02.903680+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T01:53:57.181929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"50'8","reported_seq":20,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.866487+0000","last_change":"2023-12-02T00:31:00.840645+0000","last_active":"2023-12-02T00:31:01.866487+0000","last_peered":"2023-12-02T00:31:01.866487+0000","last_clean":"2023-12-02T00:31:01.866487+0000","last_became_active":"2023-12-02T00:31:00.840295+0000","last_became_peered":"2023-12-02T00:31:00.840295+0000","last_unstale":"2023-12-02T00:31:01.866487+0000","last_undegraded":"2023-12-02T00:31:01.866487+0000","last_fullsized":"2023-12-02T00:31:01.866487+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:13:05.067577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.843226+0000","last_change":"2023-12-02T00:31:01.848869+0000","last_active":"2023-12-02T00:31:02.843226+0000","last_peered":"2023-12-02T00:31:02.843226+0000","last_clean":"2023-12-02T00:31:02.843226+0000","last_became_active":"2023-12-02T00:31:01.848523+0000","last_became_peered":"2023-12-02T00:31:01.848523+0000","last_unstale":"2023-12-02T00:31:02.843226+0000","last_undegraded":"2023-12-02T00:31:02.843226+0000","last_fullsized":"2023-12-02T00:31:02.843226+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:13:02.960439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"50'1","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.859468+0000","last_change":"2023-12-02T00:31:00.839972+0000","last_active":"2023-12-02T00:31:01.859468+0000","last_peered":"2023-12-02T00:31:01.859468+0000","last_clean":"2023-12-02T00:31:01.859468+0000","last_became_active":"2023-12-02T00:31:00.839581+0000","last_became_peered":"2023-12-02T00:31:00.839581+0000","last_unstale":"2023-12-02T00:31:01.859468+0000","last_undegraded":"2023-12-02T00:31:01.859468+0000","last_fullsized":"2023-12-02T00:31:01.859468+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:00:03.107141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.902640+0000","last_change":"2023-12-02T00:31:01.845651+0000","last_active":"2023-12-02T00:31:02.902640+0000","last_peered":"2023-12-02T00:31:02.902640+0000","last_clean":"2023-12-02T00:31:02.902640+0000","last_became_active":"2023-12-02T00:31:01.845364+0000","last_became_peered":"2023-12-02T00:31:01.845364+0000","last_unstale":"2023-12-02T00:31:02.902640+0000","last_undegraded":"2023-12-02T00:31:02.902640+0000","last_fullsized":"2023-12-02T00:31:02.902640+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:51:09.108574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"50'2","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.863224+0000","last_change":"2023-12-02T00:31:00.832413+0000","last_active":"2023-12-02T00:31:01.863224+0000","last_peered":"2023-12-02T00:31:01.863224+0000","last_clean":"2023-12-02T00:31:01.863224+0000","last_became_active":"2023-12-02T00:31:00.832127+0000","last_became_peered":"2023-12-02T00:31:00.832127+0000","last_unstale":"2023-12-02T00:31:01.863224+0000","last_undegraded":"2023-12-02T00:31:01.863224+0000","last_fullsized":"2023-12-02T00:31:01.863224+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:17:18.074493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.848460+0000","last_change":"2023-12-02T00:31:01.844176+0000","last_active":"2023-12-02T00:31:02.848460+0000","last_peered":"2023-12-02T00:31:02.848460+0000","last_clean":"2023-12-02T00:31:02.848460+0000","last_became_active":"2023-12-02T00:31:01.843893+0000","last_became_peered":"2023-12-02T00:31:01.843893+0000","last_unstale":"2023-12-02T00:31:02.848460+0000","last_undegraded":"2023-12-02T00:31:02.848460+0000","last_fullsized":"2023-12-02T00:31:02.848460+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:51:09.108574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"50'1","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.865352+0000","last_change":"2023-12-02T00:31:00.837851+0000","last_active":"2023-12-02T00:31:01.865352+0000","last_peered":"2023-12-02T00:31:01.865352+0000","last_clean":"2023-12-02T00:31:01.865352+0000","last_became_active":"2023-12-02T00:31:00.837581+0000","last_became_peered":"2023-12-02T00:31:00.837581+0000","last_unstale":"2023-12-02T00:31:01.865352+0000","last_undegraded":"2023-12-02T00:31:01.865352+0000","last_fullsized":"2023-12-02T00:31:01.865352+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T10:06:37.837138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.902886+0000","last_change":"2023-12-02T00:31:01.844563+0000","last_active":"2023-12-02T00:31:02.902886+0000","last_peered":"2023-12-02T00:31:02.902886+0000","last_clean":"2023-12-02T00:31:02.902886+0000","last_became_active":"2023-12-02T00:31:01.844264+0000","last_became_peered":"2023-12-02T00:31:01.844264+0000","last_unstale":"2023-12-02T00:31:02.902886+0000","last_undegraded":"2023-12-02T00:31:02.902886+0000","last_fullsized":"2023-12-02T00:31:02.902886+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:44:26.068567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.837441+0000","last_change":"2023-12-02T00:31:00.832751+0000","last_active":"2023-12-02T00:31:01.837441+0000","last_peered":"2023-12-02T00:31:01.837441+0000","last_clean":"2023-12-02T00:31:01.837441+0000","last_became_active":"2023-12-02T00:31:00.832466+0000","last_became_peered":"2023-12-02T00:31:00.832466+0000","last_unstale":"2023-12-02T00:31:01.837441+0000","last_undegraded":"2023-12-02T00:31:01.837441+0000","last_fullsized":"2023-12-02T00:31:01.837441+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:57:02.728754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903534+0000","last_change":"2023-12-02T00:31:01.850937+0000","last_active":"2023-12-02T00:31:02.903534+0000","last_peered":"2023-12-02T00:31:02.903534+0000","last_clean":"2023-12-02T00:31:02.903534+0000","last_became_active":"2023-12-02T00:31:01.850638+0000","last_became_peered":"2023-12-02T00:31:01.850638+0000","last_unstale":"2023-12-02T00:31:02.903534+0000","last_undegraded":"2023-12-02T00:31:02.903534+0000","last_fullsized":"2023-12-02T00:31:02.903534+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:34:40.432062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"50'2","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.864111+0000","last_change":"2023-12-02T00:31:00.833503+0000","last_active":"2023-12-02T00:31:01.864111+0000","last_peered":"2023-12-02T00:31:01.864111+0000","last_clean":"2023-12-02T00:31:01.864111+0000","last_became_active":"2023-12-02T00:31:00.833219+0000","last_became_peered":"2023-12-02T00:31:00.833219+0000","last_unstale":"2023-12-02T00:31:01.864111+0000","last_undegraded":"2023-12-02T00:31:01.864111+0000","last_fullsized":"2023-12-02T00:31:01.864111+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:30:31.052164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.902950+0000","last_change":"2023-12-02T00:31:01.843475+0000","last_active":"2023-12-02T00:31:02.902950+0000","last_peered":"2023-12-02T00:31:02.902950+0000","last_clean":"2023-12-02T00:31:02.902950+0000","last_became_active":"2023-12-02T00:31:01.843185+0000","last_became_peered":"2023-12-02T00:31:01.843185+0000","last_unstale":"2023-12-02T00:31:02.902950+0000","last_undegraded":"2023-12-02T00:31:02.902950+0000","last_fullsized":"2023-12-02T00:31:02.902950+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:23:49.102467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.828451+0000","last_change":"2023-12-02T00:31:00.840501+0000","last_active":"2023-12-02T00:31:01.828451+0000","last_peered":"2023-12-02T00:31:01.828451+0000","last_clean":"2023-12-02T00:31:01.828451+0000","last_became_active":"2023-12-02T00:31:00.840227+0000","last_became_peered":"2023-12-02T00:31:00.840227+0000","last_unstale":"2023-12-02T00:31:01.828451+0000","last_undegraded":"2023-12-02T00:31:01.828451+0000","last_fullsized":"2023-12-02T00:31:01.828451+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:23:20.913293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903584+0000","last_change":"2023-12-02T00:31:01.841976+0000","last_active":"2023-12-02T00:31:02.903584+0000","last_peered":"2023-12-02T00:31:02.903584+0000","last_clean":"2023-12-02T00:31:02.903584+0000","last_became_active":"2023-12-02T00:31:01.838386+0000","last_became_peered":"2023-12-02T00:31:01.838386+0000","last_unstale":"2023-12-02T00:31:02.903584+0000","last_undegraded":"2023-12-02T00:31:02.903584+0000","last_fullsized":"2023-12-02T00:31:02.903584+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T01:54:26.282475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.833290+0000","last_change":"2023-12-02T00:31:00.833798+0000","last_active":"2023-12-02T00:31:01.833290+0000","last_peered":"2023-12-02T00:31:01.833290+0000","last_clean":"2023-12-02T00:31:01.833290+0000","last_became_active":"2023-12-02T00:31:00.833537+0000","last_became_peered":"2023-12-02T00:31:00.833537+0000","last_unstale":"2023-12-02T00:31:01.833290+0000","last_undegraded":"2023-12-02T00:31:01.833290+0000","last_fullsized":"2023-12-02T00:31:01.833290+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T10:02:53.861890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.902901+0000","last_change":"2023-12-02T00:31:01.841890+0000","last_active":"2023-12-02T00:31:02.902901+0000","last_peered":"2023-12-02T00:31:02.902901+0000","last_clean":"2023-12-02T00:31:02.902901+0000","last_became_active":"2023-12-02T00:31:01.841559+0000","last_became_peered":"2023-12-02T00:31:01.841559+0000","last_unstale":"2023-12-02T00:31:02.902901+0000","last_undegraded":"2023-12-02T00:31:02.902901+0000","last_fullsized":"2023-12-02T00:31:02.902901+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T09:26:09.725821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.843299+0000","last_change":"2023-12-02T00:31:00.829323+0000","last_active":"2023-12-02T00:31:01.843299+0000","last_peered":"2023-12-02T00:31:01.843299+0000","last_clean":"2023-12-02T00:31:01.843299+0000","last_became_active":"2023-12-02T00:31:00.829048+0000","last_became_peered":"2023-12-02T00:31:00.829048+0000","last_unstale":"2023-12-02T00:31:01.843299+0000","last_undegraded":"2023-12-02T00:31:01.843299+0000","last_fullsized":"2023-12-02T00:31:01.843299+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:19:14.051539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.902821+0000","last_change":"2023-12-02T00:31:01.849585+0000","last_active":"2023-12-02T00:31:02.902821+0000","last_peered":"2023-12-02T00:31:02.902821+0000","last_clean":"2023-12-02T00:31:02.902821+0000","last_became_active":"2023-12-02T00:31:01.849297+0000","last_became_peered":"2023-12-02T00:31:01.849297+0000","last_unstale":"2023-12-02T00:31:02.902821+0000","last_undegraded":"2023-12-02T00:31:02.902821+0000","last_fullsized":"2023-12-02T00:31:02.902821+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:00:04.111155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"50'2","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.861990+0000","last_change":"2023-12-02T00:31:00.835706+0000","last_active":"2023-12-02T00:31:01.861990+0000","last_peered":"2023-12-02T00:31:01.861990+0000","last_clean":"2023-12-02T00:31:01.861990+0000","last_became_active":"2023-12-02T00:31:00.835381+0000","last_became_peered":"2023-12-02T00:31:00.835381+0000","last_unstale":"2023-12-02T00:31:01.861990+0000","last_undegraded":"2023-12-02T00:31:01.861990+0000","last_fullsized":"2023-12-02T00:31:01.861990+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T02:42:58.558235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903252+0000","last_change":"2023-12-02T00:31:01.861842+0000","last_active":"2023-12-02T00:31:02.903252+0000","last_peered":"2023-12-02T00:31:02.903252+0000","last_clean":"2023-12-02T00:31:02.903252+0000","last_became_active":"2023-12-02T00:31:01.861573+0000","last_became_peered":"2023-12-02T00:31:01.861573+0000","last_unstale":"2023-12-02T00:31:02.903252+0000","last_undegraded":"2023-12-02T00:31:02.903252+0000","last_fullsized":"2023-12-02T00:31:02.903252+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:40:06.079397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.840728+0000","last_change":"2023-12-02T00:31:00.832747+0000","last_active":"2023-12-02T00:31:01.840728+0000","last_peered":"2023-12-02T00:31:01.840728+0000","last_clean":"2023-12-02T00:31:01.840728+0000","last_became_active":"2023-12-02T00:31:00.832477+0000","last_became_peered":"2023-12-02T00:31:00.832477+0000","last_unstale":"2023-12-02T00:31:01.840728+0000","last_undegraded":"2023-12-02T00:31:01.840728+0000","last_fullsized":"2023-12-02T00:31:01.840728+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T02:25:45.046480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.843219+0000","last_change":"2023-12-02T00:31:01.844603+0000","last_active":"2023-12-02T00:31:02.843219+0000","last_peered":"2023-12-02T00:31:02.843219+0000","last_clean":"2023-12-02T00:31:02.843219+0000","last_became_active":"2023-12-02T00:31:01.844329+0000","last_became_peered":"2023-12-02T00:31:01.844329+0000","last_unstale":"2023-12-02T00:31:02.843219+0000","last_undegraded":"2023-12-02T00:31:02.843219+0000","last_fullsized":"2023-12-02T00:31:02.843219+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:21:48.643558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.834121+0000","last_change":"2023-12-02T00:31:00.828668+0000","last_active":"2023-12-02T00:31:01.834121+0000","last_peered":"2023-12-02T00:31:01.834121+0000","last_clean":"2023-12-02T00:31:01.834121+0000","last_became_active":"2023-12-02T00:31:00.828392+0000","last_became_peered":"2023-12-02T00:31:00.828392+0000","last_unstale":"2023-12-02T00:31:01.834121+0000","last_undegraded":"2023-12-02T00:31:01.834121+0000","last_fullsized":"2023-12-02T00:31:01.834121+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:41:19.966082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.902891+0000","last_change":"2023-12-02T00:31:01.839257+0000","last_active":"2023-12-02T00:31:02.902891+0000","last_peered":"2023-12-02T00:31:02.902891+0000","last_clean":"2023-12-02T00:31:02.902891+0000","last_became_active":"2023-12-02T00:31:01.838789+0000","last_became_peered":"2023-12-02T00:31:01.838789+0000","last_unstale":"2023-12-02T00:31:02.902891+0000","last_undegraded":"2023-12-02T00:31:02.902891+0000","last_fullsized":"2023-12-02T00:31:02.902891+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:34:02.551214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.829907+0000","last_change":"2023-12-02T00:31:00.841414+0000","last_active":"2023-12-02T00:31:01.829907+0000","last_peered":"2023-12-02T00:31:01.829907+0000","last_clean":"2023-12-02T00:31:01.829907+0000","last_became_active":"2023-12-02T00:31:00.841049+0000","last_became_peered":"2023-12-02T00:31:00.841049+0000","last_unstale":"2023-12-02T00:31:01.829907+0000","last_undegraded":"2023-12-02T00:31:01.829907+0000","last_fullsized":"2023-12-02T00:31:01.829907+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T01:53:56.177915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903664+0000","last_change":"2023-12-02T00:31:01.843439+0000","last_active":"2023-12-02T00:31:02.903664+0000","last_peered":"2023-12-02T00:31:02.903664+0000","last_clean":"2023-12-02T00:31:02.903664+0000","last_became_active":"2023-12-02T00:31:01.843180+0000","last_became_peered":"2023-12-02T00:31:01.843180+0000","last_unstale":"2023-12-02T00:31:02.903664+0000","last_undegraded":"2023-12-02T00:31:02.903664+0000","last_fullsized":"2023-12-02T00:31:02.903664+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:31:09.860031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.828618+0000","last_change":"2023-12-02T00:31:00.838503+0000","last_active":"2023-12-02T00:31:01.828618+0000","last_peered":"2023-12-02T00:31:01.828618+0000","last_clean":"2023-12-02T00:31:01.828618+0000","last_became_active":"2023-12-02T00:31:00.838125+0000","last_became_peered":"2023-12-02T00:31:00.838125+0000","last_unstale":"2023-12-02T00:31:01.828618+0000","last_undegraded":"2023-12-02T00:31:01.828618+0000","last_fullsized":"2023-12-02T00:31:01.828618+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T02:25:45.046480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903417+0000","last_change":"2023-12-02T00:31:01.860284+0000","last_active":"2023-12-02T00:31:02.903417+0000","last_peered":"2023-12-02T00:31:02.903417+0000","last_clean":"2023-12-02T00:31:02.903417+0000","last_became_active":"2023-12-02T00:31:01.860002+0000","last_became_peered":"2023-12-02T00:31:01.860002+0000","last_unstale":"2023-12-02T00:31:02.903417+0000","last_undegraded":"2023-12-02T00:31:02.903417+0000","last_fullsized":"2023-12-02T00:31:02.903417+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:13:06.071590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.834507+0000","last_change":"2023-12-02T00:31:00.828726+0000","last_active":"2023-12-02T00:31:01.834507+0000","last_peered":"2023-12-02T00:31:01.834507+0000","last_clean":"2023-12-02T00:31:01.834507+0000","last_became_active":"2023-12-02T00:31:00.828424+0000","last_became_peered":"2023-12-02T00:31:00.828424+0000","last_unstale":"2023-12-02T00:31:01.834507+0000","last_undegraded":"2023-12-02T00:31:01.834507+0000","last_fullsized":"2023-12-02T00:31:01.834507+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:06:02.637069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.843494+0000","last_change":"2023-12-02T00:31:01.863549+0000","last_active":"2023-12-02T00:31:02.843494+0000","last_peered":"2023-12-02T00:31:02.843494+0000","last_clean":"2023-12-02T00:31:02.843494+0000","last_became_active":"2023-12-02T00:31:01.861439+0000","last_became_peered":"2023-12-02T00:31:01.861439+0000","last_unstale":"2023-12-02T00:31:02.843494+0000","last_undegraded":"2023-12-02T00:31:02.843494+0000","last_fullsized":"2023-12-02T00:31:02.843494+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T01:24:56.864149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.837957+0000","last_change":"2023-12-02T00:31:00.837338+0000","last_active":"2023-12-02T00:31:01.837957+0000","last_peered":"2023-12-02T00:31:01.837957+0000","last_clean":"2023-12-02T00:31:01.837957+0000","last_became_active":"2023-12-02T00:31:00.837017+0000","last_became_peered":"2023-12-02T00:31:00.837017+0000","last_unstale":"2023-12-02T00:31:01.837957+0000","last_undegraded":"2023-12-02T00:31:01.837957+0000","last_fullsized":"2023-12-02T00:31:01.837957+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:54:07.912968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903112+0000","last_change":"2023-12-02T00:31:01.844332+0000","last_active":"2023-12-02T00:31:02.903112+0000","last_peered":"2023-12-02T00:31:02.903112+0000","last_clean":"2023-12-02T00:31:02.903112+0000","last_became_active":"2023-12-02T00:31:01.843971+0000","last_became_peered":"2023-12-02T00:31:01.843971+0000","last_unstale":"2023-12-02T00:31:02.903112+0000","last_undegraded":"2023-12-02T00:31:02.903112+0000","last_fullsized":"2023-12-02T00:31:02.903112+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T09:04:54.768041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.833964+0000","last_change":"2023-12-02T00:31:00.829255+0000","last_active":"2023-12-02T00:31:01.833964+0000","last_peered":"2023-12-02T00:31:01.833964+0000","last_clean":"2023-12-02T00:31:01.833964+0000","last_became_active":"2023-12-02T00:31:00.828976+0000","last_became_peered":"2023-12-02T00:31:00.828976+0000","last_unstale":"2023-12-02T00:31:01.833964+0000","last_undegraded":"2023-12-02T00:31:01.833964+0000","last_fullsized":"2023-12-02T00:31:01.833964+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T09:26:08.721807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":194,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.839206+0000","last_change":"2023-12-02T00:28:39.379380+0000","last_active":"2023-12-02T00:31:01.839206+0000","last_peered":"2023-12-02T00:31:01.839206+0000","last_clean":"2023-12-02T00:31:01.839206+0000","last_became_active":"2023-12-02T00:28:39.378255+0000","last_became_peered":"2023-12-02T00:28:39.378255+0000","last_unstale":"2023-12-02T00:31:01.839206+0000","last_undegraded":"2023-12-02T00:31:01.839206+0000","last_fullsized":"2023-12-02T00:31:01.839206+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:38.177345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:38.177345+0000","last_clean_scrub_stamp":"2023-12-02T00:28:38.177345+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:21:08.582412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28204,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343636,"statfs":{"total":96636764160,"available":96607883264,"internally_reserved":0,"allocated":1208320,"data_stored":842539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.50900000000000001,"15min":0.499},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.63200000000000001}]},{"osd":1,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53400000000000003,"15min":0.53000000000000003},"min":{"1min":0.40400000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.54700000000000004,"15min":0.54400000000000004},"min":{"1min":0.46100000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.72299999999999998}]},{"osd":2,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54200000000000004,"15min":0.54400000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.67000000000000004,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.54300000000000004,"15min":0.53400000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.78000000000000003}]},{"osd":3,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.53000000000000003,"15min":0.52200000000000002},"min":{"1min":0.48099999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.57299999999999995,"15min":0.57499999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.74399999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.70599999999999996}]},{"osd":4,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.50700000000000001,"15min":0.51100000000000001},"min":{"1min":0.29799999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.72199999999999998,"5min":0.745,"15min":0.745},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.51500000000000001,"15min":0.52100000000000002},"min":{"1min":0.33800000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.61799999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.44700000000000001}]},{"osd":5,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54700000000000004,"15min":0.55000000000000004},"min":{"1min":0.41899999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.78700000000000003,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.56100000000000005,"15min":0.56599999999999995},"min":{"1min":0.39500000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.63900000000000001,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.49399999999999999}]},{"osd":6,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.61799999999999999,"15min":0.627},"min":{"1min":0.40100000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.82899999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.60599999999999998,"15min":0.61299999999999999},"min":{"1min":0.44500000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76600000000000001,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.55200000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27624,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344216,"statfs":{"total":96636764160,"available":96608477184,"internally_reserved":0,"allocated":614400,"data_stored":252699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.49199999999999999,"15min":0.48299999999999998},"min":{"1min":0.36499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.50700000000000001,"15min":0.501},"min":{"1min":0.40000000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.74099999999999999,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.46200000000000002}]},{"osd":1,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.51000000000000001,"15min":0.50600000000000001},"min":{"1min":0.33000000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.72599999999999998,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.53600000000000003,"15min":0.53300000000000003},"min":{"1min":0.39400000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.80900000000000005,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.52200000000000002}]},{"osd":2,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.52000000000000002,"15min":0.51200000000000001},"min":{"1min":0.38,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.628},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55500000000000005,"15min":0.55700000000000005},"min":{"1min":0.40300000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.77400000000000002,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.55400000000000005}]},{"osd":3,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57599999999999996,"15min":0.57299999999999995},"min":{"1min":0.45200000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.76300000000000001,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57199999999999995,"15min":0.56999999999999995},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.53500000000000003}]},{"osd":4,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.35399999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.71299999999999997,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52500000000000002,"15min":0.52300000000000002},"min":{"1min":0.34100000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73699999999999999,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.58299999999999996}]},{"osd":5,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.33400000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58799999999999997,"15min":0.59299999999999997},"min":{"1min":0.42399999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.48399999999999999}]},{"osd":7,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.35999999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.44600000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574882,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27616,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344224,"statfs":{"total":96636764160,"available":96608485376,"internally_reserved":0,"allocated":606208,"data_stored":252047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.51700000000000002,"15min":0.50600000000000001},"min":{"1min":0.29099999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.73899999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.51600000000000001,"15min":0.51100000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.72899999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.47099999999999997}]},{"osd":1,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.51400000000000001,"15min":0.505},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.754,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55300000000000005,"15min":0.55000000000000004},"min":{"1min":0.36199999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.69499999999999995,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.42199999999999999}]},{"osd":2,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.56000000000000005,"15min":0.55100000000000005},"min":{"1min":0.41699999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.77500000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.53600000000000003,"15min":0.53300000000000003},"min":{"1min":0.34399999999999997,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.749,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.503}]},{"osd":3,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58099999999999996,"15min":0.57699999999999996},"min":{"1min":0.39300000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.58299999999999996,"15min":0.58199999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.72499999999999998,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.52800000000000002}]},{"osd":4,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54300000000000004,"15min":0.54600000000000004},"min":{"1min":0.34799999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69899999999999995,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58699999999999997,"15min":0.58799999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.51600000000000001}]},{"osd":6,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.56599999999999995,"15min":0.57099999999999995},"min":{"1min":0.36899999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.72899999999999998,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.59199999999999997,"15min":0.59599999999999997},"min":{"1min":0.435,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.41099999999999998}]},{"osd":7,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.42699999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.82099999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.63100000000000001,"15min":0.63500000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.76700000000000002,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.55300000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":610304,"data_stored":252629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.55100000000000005,"15min":0.54400000000000004},"min":{"1min":0.40699999999999997,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.74099999999999999,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.438,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.79500000000000004,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.82799999999999996}]},{"osd":1,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.57099999999999995,"15min":0.55800000000000005},"min":{"1min":0.50700000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.872},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.55800000000000005,"15min":0.55400000000000005},"min":{"1min":0.38600000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.77300000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.71199999999999997}]},{"osd":2,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.58099999999999996,"15min":0.57699999999999996},"min":{"1min":0.443,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.42799999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.752}]},{"osd":3,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.56999999999999995,"15min":0.56200000000000006},"min":{"1min":0.46600000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60599999999999998,"15min":0.60399999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.80300000000000005}]},{"osd":5,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52600000000000002,"15min":0.52300000000000002},"min":{"1min":0.36699999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.83799999999999997,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53900000000000003,"15min":0.54100000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.46800000000000003}]},{"osd":6,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.54200000000000004,"15min":0.53600000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.57699999999999996,"15min":0.56999999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.57999999999999996}]},{"osd":7,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.377,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.59499999999999997,"15min":0.59099999999999997},"min":{"1min":0.34699999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.54400000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574880,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31796,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340044,"statfs":{"total":96636764160,"available":96604205056,"internally_reserved":0,"allocated":622592,"data_stored":253681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.35699999999999998,"15min":0.35999999999999999},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.50800000000000001,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.38200000000000001,"15min":0.38100000000000001},"min":{"1min":0.27500000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.624,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.36899999999999999}]},{"osd":1,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.379,"15min":0.38300000000000001},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.59999999999999998,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.25800000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.58699999999999997,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.30399999999999999}]},{"osd":2,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.40600000000000003,"15min":0.40999999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.52900000000000003,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.42199999999999999,"15min":0.42099999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.66500000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.34999999999999998}]},{"osd":4,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.60899999999999999,"15min":0.61099999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76600000000000001,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.622,"15min":0.623},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.76300000000000001}]},{"osd":5,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.58299999999999996,"15min":0.58699999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85599999999999998,"5min":1.004,"15min":1.004},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.61299999999999999,"15min":0.61499999999999999},"min":{"1min":0.39300000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.90600000000000003,"5min":1.163,"15min":1.163},"last":0.50600000000000001}]},{"osd":6,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63800000000000001,"15min":0.63900000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.83199999999999996,"5min":1.095,"15min":1.095},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.63200000000000001,"15min":0.63500000000000001},"min":{"1min":0.45100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.96399999999999997,"5min":1.044,"15min":1.044},"last":0.746}]},{"osd":7,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.64700000000000002,"15min":0.65100000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.93899999999999995,"5min":1.177,"15min":1.177},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.64500000000000002,"15min":0.65100000000000002},"min":{"1min":0.45200000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.84799999999999998,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.72299999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31784,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340056,"statfs":{"total":96636764160,"available":96604217344,"internally_reserved":0,"allocated":614400,"data_stored":253040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.377,"15min":0.36299999999999999},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":2.649,"5min":2.649,"15min":2.649},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.42299999999999999,"15min":0.40799999999999997},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":3.173,"5min":3.173,"15min":3.173},"last":0.32200000000000001}]},{"osd":1,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.44800000000000001,"15min":0.41999999999999998},"min":{"1min":0.26300000000000001,"5min":0.246,"15min":0.246},"max":{"1min":4.6639999999999997,"5min":4.6639999999999997,"15min":4.6639999999999997},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.433,"15min":0.39100000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":6.093,"5min":6.093,"15min":6.093},"last":0.35799999999999998}]},{"osd":3,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.40300000000000002,"15min":0.376},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":4.1879999999999997,"5min":4.1879999999999997,"15min":4.1879999999999997},"last":0.37},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.44600000000000001,"15min":0.433},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":2.4159999999999999,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":0.38}]},{"osd":4,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.60999999999999999,"15min":0.59599999999999997},"min":{"1min":0.40999999999999998,"5min":0.38,"15min":0.38},"max":{"1min":2.6840000000000002,"5min":2.6840000000000002,"15min":2.6840000000000002},"last":0.629},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.65800000000000003,"15min":0.64600000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":3.0419999999999998,"5min":3.0419999999999998,"15min":3.0419999999999998},"last":0.62}]},{"osd":5,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.66100000000000003,"15min":0.65300000000000002},"min":{"1min":0.41599999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":3.093,"5min":3.093,"15min":3.093},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.621,"5min":0.64700000000000002,"15min":0.65100000000000002},"min":{"1min":0.41799999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.98499999999999999,"5min":1.111,"15min":1.111},"last":0.67400000000000004}]},{"osd":6,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.64000000000000001,"15min":0.63800000000000001},"min":{"1min":0.42499999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.67800000000000005,"15min":0.67200000000000004},"min":{"1min":0.47799999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":2.6030000000000002,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":0.59099999999999997}]},{"osd":7,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.65800000000000003,"15min":0.64800000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":3.1339999999999999,"5min":3.1339999999999999,"15min":3.1339999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.64200000000000002,"15min":0.64600000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.94199999999999995,"5min":1.2,"15min":1.2},"last":0.60599999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574880,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31784,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340056,"statfs":{"total":96636764160,"available":96604217344,"internally_reserved":0,"allocated":614400,"data_stored":252207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.33900000000000002,"15min":0.34100000000000003},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.436,"5min":0.48099999999999998,"15min":0.48099999999999998},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.39600000000000002,"15min":0.39800000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.58799999999999997,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.35299999999999998}]},{"osd":2,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.35099999999999998,"15min":0.34899999999999998},"min":{"1min":0.249,"5min":0.246,"15min":0.246},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.36299999999999999,"15min":0.36699999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.53500000000000003,"5min":0.626,"15min":0.626},"last":0.29199999999999998}]},{"osd":3,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.40100000000000002,"15min":0.40200000000000002},"min":{"1min":0.30499999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.375},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.371,"15min":0.376},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.5,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.40600000000000003}]},{"osd":4,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.55400000000000005,"15min":0.54800000000000004},"min":{"1min":0.377,"5min":0.376,"15min":0.376},"max":{"1min":0.77100000000000002,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.746},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.55800000000000005,"15min":0.55100000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.78000000000000003,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.54100000000000004}]},{"osd":5,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.58599999999999997,"15min":0.57899999999999996},"min":{"1min":0.45900000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.96799999999999997,"5min":1.107,"15min":1.107},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57299999999999995,"15min":0.56599999999999995},"min":{"1min":0.437,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.70599999999999996}]},{"osd":6,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60399999999999998,"15min":0.60199999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.76600000000000001,"5min":1.044,"15min":1.044},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.61199999999999999,"15min":0.60599999999999998},"min":{"1min":0.51000000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.85399999999999998,"5min":0.998,"15min":0.998},"last":0.77600000000000002}]},{"osd":7,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58999999999999997,"15min":0.58799999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.78900000000000003,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.60599999999999998,"15min":0.60199999999999998},"min":{"1min":0.45300000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.80000000000000004,"5min":1.151,"15min":1.151},"last":0.55700000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32356,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339484,"statfs":{"total":96636764160,"available":96603631616,"internally_reserved":0,"allocated":1204224,"data_stored":841852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Dec 2 00:30:36 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.26000000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.501,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.37,"15min":0.36499999999999999},"min":{"1min":0.28299999999999997,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.58399999999999996,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.32500000000000001}]},{"osd":2,"last update":"Sat Dec 2 00:30:36 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.54500000000000004,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.312},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40200000000000002,"15min":0.40100000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.372}]},{"osd":3,"last update":"Sat Dec 2 00:30:36 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.39300000000000002,"15min":0.39400000000000002},"min":{"1min":0.29999999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.53100000000000003,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.436,"5min":0.42899999999999999,"15min":0.42799999999999999},"min":{"1min":0.32700000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.34999999999999998}]},{"osd":4,"last update":"Sat Dec 2 00:30:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57399999999999995,"15min":0.56899999999999995},"min":{"1min":0.42299999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.622,"15min":0.62},"min":{"1min":0.439,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.93100000000000005,"5min":1.002,"15min":1.002},"last":0.60799999999999998}]},{"osd":5,"last update":"Sat Dec 2 00:30:36 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.628,"15min":0.626},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.64700000000000002,"15min":0.64600000000000002},"min":{"1min":0.48099999999999998,"5min":0.433,"15min":0.433},"max":{"1min":0.91100000000000003,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.57399999999999995}]},{"osd":6,"last update":"Sat Dec 2 00:30:36 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.60799999999999998,"15min":0.59999999999999998},"min":{"1min":0.435,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.89400000000000002,"5min":1.05,"15min":1.05},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.626,"5min":0.63800000000000001,"15min":0.64000000000000001},"min":{"1min":0.45300000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.94499999999999995,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.47099999999999997}]},{"osd":7,"last update":"Sat Dec 2 00:30:36 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.64300000000000002,"15min":0.63900000000000001},"min":{"1min":0.501,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.97699999999999998,"5min":1.028,"15min":1.028},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.626,"15min":0.626},"min":{"1min":0.46200000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.78000000000000003,"5min":0.874,"15min":0.874},"last":0.51500000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-02T00:31:10.288 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-02T00:31:10.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.406+0000 7fe320f97640 1 Processor -- start 2023-12-02T00:31:10.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.406+0000 7fe320f97640 1 -- start start 2023-12-02T00:31:10.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.407+0000 7fe320f97640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe31c143490 0x7fe31c0b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:10.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.407+0000 7fe320f97640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe31c15e730 0x7fe31c0b1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:10.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.407+0000 7fe320f97640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe31c15a4e0 0x7fe31c15a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:10.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.407+0000 7fe320f97640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe31c15b520 con 0x7fe31c143490 2023-12-02T00:31:10.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.407+0000 7fe320f97640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe31c143870 con 0x7fe31c15e730 2023-12-02T00:31:10.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.407+0000 7fe320f97640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c15e2a0 con 0x7fe31c15a4e0 2023-12-02T00:31:10.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.407+0000 7fe31a59c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe31c143490 0x7fe31c0b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:10.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.407+0000 7fe31a59c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe31c143490 0x7fe31c0b0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59738/0 (socket says 172.21.15.46:59738) 2023-12-02T00:31:10.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.407+0000 7fe31a59c640 1 -- 172.21.15.46:0/4037671117 learned_addr learned my addr 172.21.15.46:0/4037671117 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:10.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.407+0000 7fe31ad9d640 1 --2- 172.21.15.46:0/4037671117 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe31c15a4e0 0x7fe31c15a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:10.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.407+0000 7fe319d9b640 1 --2- 172.21.15.46:0/4037671117 >> v2:172.21.15.162:3300/0 conn(0x7fe31c15e730 0x7fe31c0b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:10.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.407+0000 7fe31a59c640 1 -- 172.21.15.46:0/4037671117 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe31c15a4e0 msgr2=0x7fe31c15a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:10.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.407+0000 7fe31a59c640 1 --2- 172.21.15.46:0/4037671117 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe31c15a4e0 0x7fe31c15a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:10.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.407+0000 7fe31a59c640 1 -- 172.21.15.46:0/4037671117 >> v2:172.21.15.162:3300/0 conn(0x7fe31c15e730 msgr2=0x7fe31c0b1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:10.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.407+0000 7fe31a59c640 1 --2- 172.21.15.46:0/4037671117 >> v2:172.21.15.162:3300/0 conn(0x7fe31c15e730 0x7fe31c0b1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:10.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.407+0000 7fe31a59c640 1 -- 172.21.15.46:0/4037671117 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe31c15b240 con 0x7fe31c143490 2023-12-02T00:31:10.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.407+0000 7fe31ad9d640 1 --2- 172.21.15.46:0/4037671117 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe31c15a4e0 0x7fe31c15a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:31:10.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.407+0000 7fe319d9b640 1 --2- 172.21.15.46:0/4037671117 >> v2:172.21.15.162:3300/0 conn(0x7fe31c15e730 0x7fe31c0b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:31:10.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.408+0000 7fe31a59c640 1 --2- 172.21.15.46:0/4037671117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe31c143490 0x7fe31c0b0cf0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fe31c0aeca0 tx=0x7fe31402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4082196338c7edd server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:10.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.408+0000 7fe31959a640 1 -- 172.21.15.46:0/4037671117 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe314031870 con 0x7fe31c143490 2023-12-02T00:31:10.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.409+0000 7fe31959a640 1 -- 172.21.15.46:0/4037671117 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3140319d0 con 0x7fe31c143490 2023-12-02T00:31:10.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.409+0000 7fe31959a640 1 -- 172.21.15.46:0/4037671117 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe314031d60 con 0x7fe31c143490 2023-12-02T00:31:10.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.409+0000 7fe320f97640 1 -- 172.21.15.46:0/4037671117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe31c143490 msgr2=0x7fe31c0b0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:10.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.409+0000 7fe320f97640 1 --2- 172.21.15.46:0/4037671117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe31c143490 0x7fe31c0b0cf0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fe31c0aeca0 tx=0x7fe31402b910 comp rx=0 tx=0).stop 2023-12-02T00:31:10.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.409+0000 7fe320f97640 1 -- 172.21.15.46:0/4037671117 shutdown_connections 2023-12-02T00:31:10.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.409+0000 7fe320f97640 1 --2- 172.21.15.46:0/4037671117 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe31c15a4e0 0x7fe31c15a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:10.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.409+0000 7fe320f97640 1 --2- 172.21.15.46:0/4037671117 >> v2:172.21.15.162:3300/0 conn(0x7fe31c15e730 0x7fe31c0b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:10.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.409+0000 7fe320f97640 1 --2- 172.21.15.46:0/4037671117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe31c143490 0x7fe31c0b0cf0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:10.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.409+0000 7fe320f97640 1 -- 172.21.15.46:0/4037671117 >> 172.21.15.46:0/4037671117 conn(0x7fe31c06a4d0 msgr2=0x7fe31c06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:10.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.409+0000 7fe320f97640 1 -- 172.21.15.46:0/4037671117 shutdown_connections 2023-12-02T00:31:10.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.409+0000 7fe320f97640 1 -- 172.21.15.46:0/4037671117 wait complete. 2023-12-02T00:31:10.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.411+0000 7fe320f97640 1 Processor -- start 2023-12-02T00:31:10.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.411+0000 7fe320f97640 1 -- start start 2023-12-02T00:31:10.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.411+0000 7fe320f97640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe31c143490 0x7fe31c0aba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:10.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.411+0000 7fe320f97640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe31c15a4e0 0x7fe31c0ae190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:10.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.411+0000 7fe320f97640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe31c15e730 0x7fe31c0afd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:10.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.411+0000 7fe320f97640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c0ad480 con 0x7fe31c15a4e0 2023-12-02T00:31:10.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.411+0000 7fe320f97640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c1f06d0 con 0x7fe31c15e730 2023-12-02T00:31:10.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.411+0000 7fe320f97640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c1f03f0 con 0x7fe31c143490 2023-12-02T00:31:10.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.411+0000 7fe319d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe31c15a4e0 0x7fe31c0ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:10.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.412+0000 7fe319d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe31c15a4e0 0x7fe31c0ae190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59752/0 (socket says 172.21.15.46:59752) 2023-12-02T00:31:10.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.412+0000 7fe319d9b640 1 -- 172.21.15.46:0/2555055944 learned_addr learned my addr 172.21.15.46:0/2555055944 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:10.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.412+0000 7fe31a59c640 1 --2- 172.21.15.46:0/2555055944 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe31c143490 0x7fe31c0aba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:10.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.412+0000 7fe31ad9d640 1 --2- 172.21.15.46:0/2555055944 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe31c15e730 0x7fe31c0afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:10.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.412+0000 7fe319d9b640 1 -- 172.21.15.46:0/2555055944 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe31c143490 msgr2=0x7fe31c0aba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:10.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.412+0000 7fe319d9b640 1 --2- 172.21.15.46:0/2555055944 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe31c143490 0x7fe31c0aba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:10.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.412+0000 7fe319d9b640 1 -- 172.21.15.46:0/2555055944 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe31c15e730 msgr2=0x7fe31c0afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:10.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.412+0000 7fe319d9b640 1 --2- 172.21.15.46:0/2555055944 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe31c15e730 0x7fe31c0afd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:10.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.412+0000 7fe319d9b640 1 -- 172.21.15.46:0/2555055944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3140094d0 con 0x7fe31c15a4e0 2023-12-02T00:31:10.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.412+0000 7fe319d9b640 1 --2- 172.21.15.46:0/2555055944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe31c15a4e0 0x7fe31c0ae190 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fe31000c7f0 tx=0x7fe31000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:10.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.412+0000 7fe30affd640 1 -- 172.21.15.46:0/2555055944 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe310007820 con 0x7fe31c15a4e0 2023-12-02T00:31:10.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.412+0000 7fe30affd640 1 -- 172.21.15.46:0/2555055944 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe310007980 con 0x7fe31c15a4e0 2023-12-02T00:31:10.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.412+0000 7fe320f97640 1 -- 172.21.15.46:0/2555055944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe31c0ad6b0 con 0x7fe31c15a4e0 2023-12-02T00:31:10.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.412+0000 7fe30affd640 1 -- 172.21.15.46:0/2555055944 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe310007cf0 con 0x7fe31c15a4e0 2023-12-02T00:31:10.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.412+0000 7fe320f97640 1 -- 172.21.15.46:0/2555055944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe31c0adc70 con 0x7fe31c15a4e0 2023-12-02T00:31:10.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.413+0000 7fe320f97640 1 -- 172.21.15.46:0/2555055944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe31c069400 con 0x7fe31c15a4e0 2023-12-02T00:31:10.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.414+0000 7fe30affd640 1 -- 172.21.15.46:0/2555055944 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe3100113f0 con 0x7fe31c15a4e0 2023-12-02T00:31:10.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.414+0000 7fe30affd640 1 --2- 172.21.15.46:0/2555055944 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe2fc06ae40 0x7fe2fc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:10.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.415+0000 7fe30affd640 1 -- 172.21.15.46:0/2555055944 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe310089860 con 0x7fe31c15a4e0 2023-12-02T00:31:10.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.415+0000 7fe31a59c640 1 --2- 172.21.15.46:0/2555055944 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe2fc06ae40 0x7fe2fc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:10.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.415+0000 7fe31a59c640 1 --2- 172.21.15.46:0/2555055944 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe2fc06ae40 0x7fe2fc06d300 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe314036000 tx=0x7fe31403c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:10.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.419+0000 7fe30affd640 1 -- 172.21.15.46:0/2555055944 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe3100549b0 con 0x7fe31c15a4e0 2023-12-02T00:31:10.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.591+0000 7fe320f97640 1 -- 172.21.15.46:0/2555055944 --> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe31c15aeb0 con 0x7fe2fc06ae40 2023-12-02T00:31:10.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.599+0000 7fe30affd640 1 -- 172.21.15.46:0/2555055944 <== mgr.4108 v2:172.21.15.162:6832/3248644722 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289603 (secure 0 0 0) 0x7fe314044400 con 0x7fe2fc06ae40 2023-12-02T00:31:10.603 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:31:10.605 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-12-02T00:31:10.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.602+0000 7fe320f97640 1 -- 172.21.15.46:0/2555055944 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe2fc06ae40 msgr2=0x7fe2fc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:10.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.602+0000 7fe320f97640 1 --2- 172.21.15.46:0/2555055944 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe2fc06ae40 0x7fe2fc06d300 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe314036000 tx=0x7fe31403c040 comp rx=0 tx=0).stop 2023-12-02T00:31:10.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.602+0000 7fe320f97640 1 -- 172.21.15.46:0/2555055944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe31c15a4e0 msgr2=0x7fe31c0ae190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:10.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.602+0000 7fe320f97640 1 --2- 172.21.15.46:0/2555055944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe31c15a4e0 0x7fe31c0ae190 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fe31000c7f0 tx=0x7fe31000ccb0 comp rx=0 tx=0).stop 2023-12-02T00:31:10.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.602+0000 7fe320f97640 1 -- 172.21.15.46:0/2555055944 shutdown_connections 2023-12-02T00:31:10.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.602+0000 7fe320f97640 1 --2- 172.21.15.46:0/2555055944 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe2fc06ae40 0x7fe2fc06d300 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:10.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.602+0000 7fe320f97640 1 --2- 172.21.15.46:0/2555055944 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe31c15e730 0x7fe31c0afd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:10.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.602+0000 7fe320f97640 1 --2- 172.21.15.46:0/2555055944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe31c15a4e0 0x7fe31c0ae190 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:10.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.602+0000 7fe320f97640 1 --2- 172.21.15.46:0/2555055944 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe31c143490 0x7fe31c0aba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:10.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.602+0000 7fe320f97640 1 -- 172.21.15.46:0/2555055944 >> 172.21.15.46:0/2555055944 conn(0x7fe31c06a4d0 msgr2=0x7fe31c0cd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:10.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.602+0000 7fe320f97640 1 -- 172.21.15.46:0/2555055944 shutdown_connections 2023-12-02T00:31:10.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.602+0000 7fe320f97640 1 -- 172.21.15.46:0/2555055944 wait complete. 2023-12-02T00:31:10.615 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":121,"stamp":"2023-12-02T00:31:09.192622+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":156,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238784,"kb_used_data":5952,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735936,"statfs":{"total":773094113280,"available":772849598464,"internally_reserved":0,"allocated":6094848,"data_stored":3200694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002216"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.902777+0000","last_change":"2023-12-02T00:31:01.843916+0000","last_active":"2023-12-02T00:31:02.902777+0000","last_peered":"2023-12-02T00:31:02.902777+0000","last_clean":"2023-12-02T00:31:02.902777+0000","last_became_active":"2023-12-02T00:31:01.843636+0000","last_became_peered":"2023-12-02T00:31:01.843636+0000","last_unstale":"2023-12-02T00:31:02.902777+0000","last_undegraded":"2023-12-02T00:31:02.902777+0000","last_fullsized":"2023-12-02T00:31:02.902777+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T10:47:45.803692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903186+0000","last_change":"2023-12-02T00:31:01.844324+0000","last_active":"2023-12-02T00:31:02.903186+0000","last_peered":"2023-12-02T00:31:02.903186+0000","last_clean":"2023-12-02T00:31:02.903186+0000","last_became_active":"2023-12-02T00:31:01.844055+0000","last_became_peered":"2023-12-02T00:31:01.844055+0000","last_unstale":"2023-12-02T00:31:02.903186+0000","last_undegraded":"2023-12-02T00:31:02.903186+0000","last_fullsized":"2023-12-02T00:31:02.903186+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T01:54:26.282475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903229+0000","last_change":"2023-12-02T00:31:01.861283+0000","last_active":"2023-12-02T00:31:02.903229+0000","last_peered":"2023-12-02T00:31:02.903229+0000","last_clean":"2023-12-02T00:31:02.903229+0000","last_became_active":"2023-12-02T00:31:01.861007+0000","last_became_peered":"2023-12-02T00:31:01.861007+0000","last_unstale":"2023-12-02T00:31:02.903229+0000","last_undegraded":"2023-12-02T00:31:02.903229+0000","last_fullsized":"2023-12-02T00:31:02.903229+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T02:25:46.050493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903314+0000","last_change":"2023-12-02T00:31:01.848540+0000","last_active":"2023-12-02T00:31:02.903314+0000","last_peered":"2023-12-02T00:31:02.903314+0000","last_clean":"2023-12-02T00:31:02.903314+0000","last_became_active":"2023-12-02T00:31:01.848276+0000","last_became_peered":"2023-12-02T00:31:01.848276+0000","last_unstale":"2023-12-02T00:31:02.903314+0000","last_undegraded":"2023-12-02T00:31:02.903314+0000","last_fullsized":"2023-12-02T00:31:02.903314+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:47:14.572081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.848753+0000","last_change":"2023-12-02T00:31:01.844040+0000","last_active":"2023-12-02T00:31:02.848753+0000","last_peered":"2023-12-02T00:31:02.848753+0000","last_clean":"2023-12-02T00:31:02.848753+0000","last_became_active":"2023-12-02T00:31:01.843778+0000","last_became_peered":"2023-12-02T00:31:01.843778+0000","last_unstale":"2023-12-02T00:31:02.848753+0000","last_undegraded":"2023-12-02T00:31:02.848753+0000","last_fullsized":"2023-12-02T00:31:02.848753+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T10:47:45.803692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903123+0000","last_change":"2023-12-02T00:31:01.844971+0000","last_active":"2023-12-02T00:31:02.903123+0000","last_peered":"2023-12-02T00:31:02.903123+0000","last_clean":"2023-12-02T00:31:02.903123+0000","last_became_active":"2023-12-02T00:31:01.844705+0000","last_became_peered":"2023-12-02T00:31:01.844705+0000","last_unstale":"2023-12-02T00:31:02.903123+0000","last_undegraded":"2023-12-02T00:31:02.903123+0000","last_fullsized":"2023-12-02T00:31:02.903123+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:41:20.970096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903073+0000","last_change":"2023-12-02T00:31:01.863300+0000","last_active":"2023-12-02T00:31:02.903073+0000","last_peered":"2023-12-02T00:31:02.903073+0000","last_clean":"2023-12-02T00:31:02.903073+0000","last_became_active":"2023-12-02T00:31:01.862826+0000","last_became_peered":"2023-12-02T00:31:01.862826+0000","last_unstale":"2023-12-02T00:31:02.903073+0000","last_undegraded":"2023-12-02T00:31:02.903073+0000","last_fullsized":"2023-12-02T00:31:02.903073+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:23:21.917306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.843359+0000","last_change":"2023-12-02T00:31:01.840463+0000","last_active":"2023-12-02T00:31:02.843359+0000","last_peered":"2023-12-02T00:31:02.843359+0000","last_clean":"2023-12-02T00:31:02.843359+0000","last_became_active":"2023-12-02T00:31:01.840123+0000","last_became_peered":"2023-12-02T00:31:01.840123+0000","last_unstale":"2023-12-02T00:31:02.843359+0000","last_undegraded":"2023-12-02T00:31:02.843359+0000","last_fullsized":"2023-12-02T00:31:02.843359+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:15:02.019922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.843484+0000","last_change":"2023-12-02T00:31:01.841005+0000","last_active":"2023-12-02T00:31:02.843484+0000","last_peered":"2023-12-02T00:31:02.843484+0000","last_clean":"2023-12-02T00:31:02.843484+0000","last_became_active":"2023-12-02T00:31:01.840669+0000","last_became_peered":"2023-12-02T00:31:01.840669+0000","last_unstale":"2023-12-02T00:31:02.843484+0000","last_undegraded":"2023-12-02T00:31:02.843484+0000","last_fullsized":"2023-12-02T00:31:02.843484+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:38:10.124023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.902799+0000","last_change":"2023-12-02T00:31:01.852835+0000","last_active":"2023-12-02T00:31:02.902799+0000","last_peered":"2023-12-02T00:31:02.902799+0000","last_clean":"2023-12-02T00:31:02.902799+0000","last_became_active":"2023-12-02T00:31:01.852547+0000","last_became_peered":"2023-12-02T00:31:01.852547+0000","last_unstale":"2023-12-02T00:31:02.902799+0000","last_undegraded":"2023-12-02T00:31:02.902799+0000","last_fullsized":"2023-12-02T00:31:02.902799+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:54:08.916982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.848266+0000","last_change":"2023-12-02T00:31:01.859361+0000","last_active":"2023-12-02T00:31:02.848266+0000","last_peered":"2023-12-02T00:31:02.848266+0000","last_clean":"2023-12-02T00:31:02.848266+0000","last_became_active":"2023-12-02T00:31:01.859054+0000","last_became_peered":"2023-12-02T00:31:01.859054+0000","last_unstale":"2023-12-02T00:31:02.848266+0000","last_undegraded":"2023-12-02T00:31:02.848266+0000","last_fullsized":"2023-12-02T00:31:02.848266+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:09:24.735030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.843299+0000","last_change":"2023-12-02T00:31:01.840823+0000","last_active":"2023-12-02T00:31:02.843299+0000","last_peered":"2023-12-02T00:31:02.843299+0000","last_clean":"2023-12-02T00:31:02.843299+0000","last_became_active":"2023-12-02T00:31:01.840515+0000","last_became_peered":"2023-12-02T00:31:01.840515+0000","last_unstale":"2023-12-02T00:31:02.843299+0000","last_undegraded":"2023-12-02T00:31:02.843299+0000","last_fullsized":"2023-12-02T00:31:02.843299+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:29:47.951230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.902753+0000","last_change":"2023-12-02T00:31:01.848176+0000","last_active":"2023-12-02T00:31:02.902753+0000","last_peered":"2023-12-02T00:31:02.902753+0000","last_clean":"2023-12-02T00:31:02.902753+0000","last_became_active":"2023-12-02T00:31:01.847864+0000","last_became_peered":"2023-12-02T00:31:01.847864+0000","last_unstale":"2023-12-02T00:31:02.902753+0000","last_undegraded":"2023-12-02T00:31:02.902753+0000","last_fullsized":"2023-12-02T00:31:02.902753+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:00:48.239139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903481+0000","last_change":"2023-12-02T00:31:01.860249+0000","last_active":"2023-12-02T00:31:02.903481+0000","last_peered":"2023-12-02T00:31:02.903481+0000","last_clean":"2023-12-02T00:31:02.903481+0000","last_became_active":"2023-12-02T00:31:01.859910+0000","last_became_peered":"2023-12-02T00:31:01.859910+0000","last_unstale":"2023-12-02T00:31:02.903481+0000","last_undegraded":"2023-12-02T00:31:02.903481+0000","last_fullsized":"2023-12-02T00:31:02.903481+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T02:12:42.101295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903141+0000","last_change":"2023-12-02T00:31:01.849636+0000","last_active":"2023-12-02T00:31:02.903141+0000","last_peered":"2023-12-02T00:31:02.903141+0000","last_clean":"2023-12-02T00:31:02.903141+0000","last_became_active":"2023-12-02T00:31:01.849386+0000","last_became_peered":"2023-12-02T00:31:01.849386+0000","last_unstale":"2023-12-02T00:31:02.903141+0000","last_undegraded":"2023-12-02T00:31:02.903141+0000","last_fullsized":"2023-12-02T00:31:02.903141+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T01:48:44.099841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.902625+0000","last_change":"2023-12-02T00:31:01.845720+0000","last_active":"2023-12-02T00:31:02.902625+0000","last_peered":"2023-12-02T00:31:02.902625+0000","last_clean":"2023-12-02T00:31:02.902625+0000","last_became_active":"2023-12-02T00:31:01.845456+0000","last_became_peered":"2023-12-02T00:31:01.845456+0000","last_unstale":"2023-12-02T00:31:02.902625+0000","last_undegraded":"2023-12-02T00:31:02.902625+0000","last_fullsized":"2023-12-02T00:31:02.902625+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:01:27.538047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903671+0000","last_change":"2023-12-02T00:31:01.844680+0000","last_active":"2023-12-02T00:31:02.903671+0000","last_peered":"2023-12-02T00:31:02.903671+0000","last_clean":"2023-12-02T00:31:02.903671+0000","last_became_active":"2023-12-02T00:31:01.842869+0000","last_became_peered":"2023-12-02T00:31:01.842869+0000","last_unstale":"2023-12-02T00:31:02.903671+0000","last_undegraded":"2023-12-02T00:31:02.903671+0000","last_fullsized":"2023-12-02T00:31:02.903671+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:19:31.202411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903303+0000","last_change":"2023-12-02T00:31:01.850644+0000","last_active":"2023-12-02T00:31:02.903303+0000","last_peered":"2023-12-02T00:31:02.903303+0000","last_clean":"2023-12-02T00:31:02.903303+0000","last_became_active":"2023-12-02T00:31:01.850392+0000","last_became_peered":"2023-12-02T00:31:01.850392+0000","last_unstale":"2023-12-02T00:31:02.903303+0000","last_undegraded":"2023-12-02T00:31:02.903303+0000","last_fullsized":"2023-12-02T00:31:02.903303+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T01:53:57.181929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903342+0000","last_change":"2023-12-02T00:31:01.845720+0000","last_active":"2023-12-02T00:31:02.903342+0000","last_peered":"2023-12-02T00:31:02.903342+0000","last_clean":"2023-12-02T00:31:02.903342+0000","last_became_active":"2023-12-02T00:31:01.845415+0000","last_became_peered":"2023-12-02T00:31:01.845415+0000","last_unstale":"2023-12-02T00:31:02.903342+0000","last_undegraded":"2023-12-02T00:31:02.903342+0000","last_fullsized":"2023-12-02T00:31:02.903342+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T12:00:43.351948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.902687+0000","last_change":"2023-12-02T00:31:01.848558+0000","last_active":"2023-12-02T00:31:02.902687+0000","last_peered":"2023-12-02T00:31:02.902687+0000","last_clean":"2023-12-02T00:31:02.902687+0000","last_became_active":"2023-12-02T00:31:01.848274+0000","last_became_peered":"2023-12-02T00:31:01.848274+0000","last_unstale":"2023-12-02T00:31:02.902687+0000","last_undegraded":"2023-12-02T00:31:02.902687+0000","last_fullsized":"2023-12-02T00:31:02.902687+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:56:42.008355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903114+0000","last_change":"2023-12-02T00:31:01.844873+0000","last_active":"2023-12-02T00:31:02.903114+0000","last_peered":"2023-12-02T00:31:02.903114+0000","last_clean":"2023-12-02T00:31:02.903114+0000","last_became_active":"2023-12-02T00:31:01.844607+0000","last_became_peered":"2023-12-02T00:31:01.844607+0000","last_unstale":"2023-12-02T00:31:02.903114+0000","last_undegraded":"2023-12-02T00:31:02.903114+0000","last_fullsized":"2023-12-02T00:31:02.903114+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:00:03.302160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.843116+0000","last_change":"2023-12-02T00:31:01.843039+0000","last_active":"2023-12-02T00:31:02.843116+0000","last_peered":"2023-12-02T00:31:02.843116+0000","last_clean":"2023-12-02T00:31:02.843116+0000","last_became_active":"2023-12-02T00:31:01.842705+0000","last_became_peered":"2023-12-02T00:31:01.842705+0000","last_unstale":"2023-12-02T00:31:02.843116+0000","last_undegraded":"2023-12-02T00:31:02.843116+0000","last_fullsized":"2023-12-02T00:31:02.843116+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:42:36.235273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.848078+0000","last_change":"2023-12-02T00:31:01.859476+0000","last_active":"2023-12-02T00:31:02.848078+0000","last_peered":"2023-12-02T00:31:02.848078+0000","last_clean":"2023-12-02T00:31:02.848078+0000","last_became_active":"2023-12-02T00:31:01.859172+0000","last_became_peered":"2023-12-02T00:31:01.859172+0000","last_unstale":"2023-12-02T00:31:02.848078+0000","last_undegraded":"2023-12-02T00:31:02.848078+0000","last_fullsized":"2023-12-02T00:31:02.848078+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:14:38.642330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.902799+0000","last_change":"2023-12-02T00:31:01.860276+0000","last_active":"2023-12-02T00:31:02.902799+0000","last_peered":"2023-12-02T00:31:02.902799+0000","last_clean":"2023-12-02T00:31:02.902799+0000","last_became_active":"2023-12-02T00:31:01.859967+0000","last_became_peered":"2023-12-02T00:31:01.859967+0000","last_unstale":"2023-12-02T00:31:02.902799+0000","last_undegraded":"2023-12-02T00:31:02.902799+0000","last_fullsized":"2023-12-02T00:31:02.902799+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:34:42.351893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903750+0000","last_change":"2023-12-02T00:31:01.841808+0000","last_active":"2023-12-02T00:31:02.903750+0000","last_peered":"2023-12-02T00:31:02.903750+0000","last_clean":"2023-12-02T00:31:02.903750+0000","last_became_active":"2023-12-02T00:31:01.841492+0000","last_became_peered":"2023-12-02T00:31:01.841492+0000","last_unstale":"2023-12-02T00:31:02.903750+0000","last_undegraded":"2023-12-02T00:31:02.903750+0000","last_fullsized":"2023-12-02T00:31:02.903750+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:47:14.572081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.843376+0000","last_change":"2023-12-02T00:31:01.864012+0000","last_active":"2023-12-02T00:31:02.843376+0000","last_peered":"2023-12-02T00:31:02.843376+0000","last_clean":"2023-12-02T00:31:02.843376+0000","last_became_active":"2023-12-02T00:31:01.863740+0000","last_became_peered":"2023-12-02T00:31:01.863740+0000","last_unstale":"2023-12-02T00:31:02.843376+0000","last_undegraded":"2023-12-02T00:31:02.843376+0000","last_fullsized":"2023-12-02T00:31:02.843376+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:51:36.069816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903332+0000","last_change":"2023-12-02T00:31:01.848266+0000","last_active":"2023-12-02T00:31:02.903332+0000","last_peered":"2023-12-02T00:31:02.903332+0000","last_clean":"2023-12-02T00:31:02.903332+0000","last_became_active":"2023-12-02T00:31:01.847651+0000","last_became_peered":"2023-12-02T00:31:01.847651+0000","last_unstale":"2023-12-02T00:31:02.903332+0000","last_undegraded":"2023-12-02T00:31:02.903332+0000","last_fullsized":"2023-12-02T00:31:02.903332+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:00:04.111155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.902827+0000","last_change":"2023-12-02T00:31:01.859996+0000","last_active":"2023-12-02T00:31:02.902827+0000","last_peered":"2023-12-02T00:31:02.902827+0000","last_clean":"2023-12-02T00:31:02.902827+0000","last_became_active":"2023-12-02T00:31:01.859530+0000","last_became_peered":"2023-12-02T00:31:01.859530+0000","last_unstale":"2023-12-02T00:31:02.902827+0000","last_undegraded":"2023-12-02T00:31:02.902827+0000","last_fullsized":"2023-12-02T00:31:02.902827+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T09:37:49.913977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903077+0000","last_change":"2023-12-02T00:31:01.839428+0000","last_active":"2023-12-02T00:31:02.903077+0000","last_peered":"2023-12-02T00:31:02.903077+0000","last_clean":"2023-12-02T00:31:02.903077+0000","last_became_active":"2023-12-02T00:31:01.839103+0000","last_became_peered":"2023-12-02T00:31:01.839103+0000","last_unstale":"2023-12-02T00:31:02.903077+0000","last_undegraded":"2023-12-02T00:31:02.903077+0000","last_fullsized":"2023-12-02T00:31:02.903077+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:34:40.432062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.902795+0000","last_change":"2023-12-02T00:31:01.857804+0000","last_active":"2023-12-02T00:31:02.902795+0000","last_peered":"2023-12-02T00:31:02.902795+0000","last_clean":"2023-12-02T00:31:02.902795+0000","last_became_active":"2023-12-02T00:31:01.857510+0000","last_became_peered":"2023-12-02T00:31:01.857510+0000","last_unstale":"2023-12-02T00:31:02.902795+0000","last_undegraded":"2023-12-02T00:31:02.902795+0000","last_fullsized":"2023-12-02T00:31:02.902795+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T02:12:42.101295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903207+0000","last_change":"2023-12-02T00:31:01.863101+0000","last_active":"2023-12-02T00:31:02.903207+0000","last_peered":"2023-12-02T00:31:02.903207+0000","last_clean":"2023-12-02T00:31:02.903207+0000","last_became_active":"2023-12-02T00:31:01.862445+0000","last_became_peered":"2023-12-02T00:31:01.862445+0000","last_unstale":"2023-12-02T00:31:02.903207+0000","last_undegraded":"2023-12-02T00:31:02.903207+0000","last_fullsized":"2023-12-02T00:31:02.903207+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:31:09.860031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.902689+0000","last_change":"2023-12-02T00:31:01.850281+0000","last_active":"2023-12-02T00:31:02.902689+0000","last_peered":"2023-12-02T00:31:02.902689+0000","last_clean":"2023-12-02T00:31:02.902689+0000","last_became_active":"2023-12-02T00:31:01.850013+0000","last_became_peered":"2023-12-02T00:31:01.850013+0000","last_unstale":"2023-12-02T00:31:02.902689+0000","last_undegraded":"2023-12-02T00:31:02.902689+0000","last_fullsized":"2023-12-02T00:31:02.902689+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:40:06.079397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.902601+0000","last_change":"2023-12-02T00:31:01.843400+0000","last_active":"2023-12-02T00:31:02.902601+0000","last_peered":"2023-12-02T00:31:02.902601+0000","last_clean":"2023-12-02T00:31:02.902601+0000","last_became_active":"2023-12-02T00:31:01.843151+0000","last_became_peered":"2023-12-02T00:31:01.843151+0000","last_unstale":"2023-12-02T00:31:02.902601+0000","last_undegraded":"2023-12-02T00:31:02.902601+0000","last_fullsized":"2023-12-02T00:31:02.902601+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:28:45.087022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"50'4","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.862288+0000","last_change":"2023-12-02T00:31:00.832088+0000","last_active":"2023-12-02T00:31:01.862288+0000","last_peered":"2023-12-02T00:31:01.862288+0000","last_clean":"2023-12-02T00:31:01.862288+0000","last_became_active":"2023-12-02T00:31:00.830872+0000","last_became_peered":"2023-12-02T00:31:00.830872+0000","last_unstale":"2023-12-02T00:31:01.862288+0000","last_undegraded":"2023-12-02T00:31:01.862288+0000","last_fullsized":"2023-12-02T00:31:01.862288+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:33:58.845396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903341+0000","last_change":"2023-12-02T00:31:01.846929+0000","last_active":"2023-12-02T00:31:02.903341+0000","last_peered":"2023-12-02T00:31:02.903341+0000","last_clean":"2023-12-02T00:31:02.903341+0000","last_became_active":"2023-12-02T00:31:01.846657+0000","last_became_peered":"2023-12-02T00:31:01.846657+0000","last_unstale":"2023-12-02T00:31:02.903341+0000","last_undegraded":"2023-12-02T00:31:02.903341+0000","last_fullsized":"2023-12-02T00:31:02.903341+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:34:02.551214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"50'1","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.864016+0000","last_change":"2023-12-02T00:31:00.834787+0000","last_active":"2023-12-02T00:31:01.864016+0000","last_peered":"2023-12-02T00:31:01.864016+0000","last_clean":"2023-12-02T00:31:01.864016+0000","last_became_active":"2023-12-02T00:31:00.834357+0000","last_became_peered":"2023-12-02T00:31:00.834357+0000","last_unstale":"2023-12-02T00:31:01.864016+0000","last_undegraded":"2023-12-02T00:31:01.864016+0000","last_fullsized":"2023-12-02T00:31:01.864016+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:36:12.890956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.902860+0000","last_change":"2023-12-02T00:31:01.843931+0000","last_active":"2023-12-02T00:31:02.902860+0000","last_peered":"2023-12-02T00:31:02.902860+0000","last_clean":"2023-12-02T00:31:02.902860+0000","last_became_active":"2023-12-02T00:31:01.843625+0000","last_became_peered":"2023-12-02T00:31:01.843625+0000","last_unstale":"2023-12-02T00:31:02.902860+0000","last_undegraded":"2023-12-02T00:31:02.902860+0000","last_fullsized":"2023-12-02T00:31:02.902860+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:36:13.894969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"50'2","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.864846+0000","last_change":"2023-12-02T00:31:00.837792+0000","last_active":"2023-12-02T00:31:01.864846+0000","last_peered":"2023-12-02T00:31:01.864846+0000","last_clean":"2023-12-02T00:31:01.864846+0000","last_became_active":"2023-12-02T00:31:00.837471+0000","last_became_peered":"2023-12-02T00:31:00.837471+0000","last_unstale":"2023-12-02T00:31:01.864846+0000","last_undegraded":"2023-12-02T00:31:01.864846+0000","last_fullsized":"2023-12-02T00:31:01.864846+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T10:02:53.861890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.843174+0000","last_change":"2023-12-02T00:31:01.863073+0000","last_active":"2023-12-02T00:31:02.843174+0000","last_peered":"2023-12-02T00:31:02.843174+0000","last_clean":"2023-12-02T00:31:02.843174+0000","last_became_active":"2023-12-02T00:31:01.861502+0000","last_became_peered":"2023-12-02T00:31:01.861502+0000","last_unstale":"2023-12-02T00:31:02.843174+0000","last_undegraded":"2023-12-02T00:31:02.843174+0000","last_fullsized":"2023-12-02T00:31:02.843174+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T01:54:53.724229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"50'2","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.859901+0000","last_change":"2023-12-02T00:31:00.834209+0000","last_active":"2023-12-02T00:31:01.859901+0000","last_peered":"2023-12-02T00:31:01.859901+0000","last_clean":"2023-12-02T00:31:01.859901+0000","last_became_active":"2023-12-02T00:31:00.833798+0000","last_became_peered":"2023-12-02T00:31:00.833798+0000","last_unstale":"2023-12-02T00:31:01.859901+0000","last_undegraded":"2023-12-02T00:31:01.859901+0000","last_fullsized":"2023-12-02T00:31:01.859901+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T09:26:08.721807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.848706+0000","last_change":"2023-12-02T00:31:01.844842+0000","last_active":"2023-12-02T00:31:02.848706+0000","last_peered":"2023-12-02T00:31:02.848706+0000","last_clean":"2023-12-02T00:31:02.848706+0000","last_became_active":"2023-12-02T00:31:01.844573+0000","last_became_peered":"2023-12-02T00:31:01.844573+0000","last_unstale":"2023-12-02T00:31:02.848706+0000","last_undegraded":"2023-12-02T00:31:02.848706+0000","last_fullsized":"2023-12-02T00:31:02.848706+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:00:48.239139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"50'1","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.864031+0000","last_change":"2023-12-02T00:31:00.821690+0000","last_active":"2023-12-02T00:31:01.864031+0000","last_peered":"2023-12-02T00:31:01.864031+0000","last_clean":"2023-12-02T00:31:01.864031+0000","last_became_active":"2023-12-02T00:31:00.821365+0000","last_became_peered":"2023-12-02T00:31:00.821365+0000","last_unstale":"2023-12-02T00:31:01.864031+0000","last_undegraded":"2023-12-02T00:31:01.864031+0000","last_fullsized":"2023-12-02T00:31:01.864031+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:40:05.075383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903579+0000","last_change":"2023-12-02T00:31:01.847683+0000","last_active":"2023-12-02T00:31:02.903579+0000","last_peered":"2023-12-02T00:31:02.903579+0000","last_clean":"2023-12-02T00:31:02.903579+0000","last_became_active":"2023-12-02T00:31:01.847313+0000","last_became_peered":"2023-12-02T00:31:01.847313+0000","last_unstale":"2023-12-02T00:31:02.903579+0000","last_undegraded":"2023-12-02T00:31:02.903579+0000","last_fullsized":"2023-12-02T00:31:02.903579+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:53:25.157550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"50'1","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.854047+0000","last_change":"2023-12-02T00:31:00.831123+0000","last_active":"2023-12-02T00:31:01.854047+0000","last_peered":"2023-12-02T00:31:01.854047+0000","last_clean":"2023-12-02T00:31:01.854047+0000","last_became_active":"2023-12-02T00:31:00.830870+0000","last_became_peered":"2023-12-02T00:31:00.830870+0000","last_unstale":"2023-12-02T00:31:01.854047+0000","last_undegraded":"2023-12-02T00:31:01.854047+0000","last_fullsized":"2023-12-02T00:31:01.854047+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T10:02:53.861890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.843515+0000","last_change":"2023-12-02T00:31:01.861765+0000","last_active":"2023-12-02T00:31:02.843515+0000","last_peered":"2023-12-02T00:31:02.843515+0000","last_clean":"2023-12-02T00:31:02.843515+0000","last_became_active":"2023-12-02T00:31:01.861462+0000","last_became_peered":"2023-12-02T00:31:01.861462+0000","last_unstale":"2023-12-02T00:31:02.843515+0000","last_undegraded":"2023-12-02T00:31:02.843515+0000","last_fullsized":"2023-12-02T00:31:02.843515+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:48:30.281237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.837604+0000","last_change":"2023-12-02T00:31:00.834261+0000","last_active":"2023-12-02T00:31:01.837604+0000","last_peered":"2023-12-02T00:31:01.837604+0000","last_clean":"2023-12-02T00:31:01.837604+0000","last_became_active":"2023-12-02T00:31:00.833992+0000","last_became_peered":"2023-12-02T00:31:00.833992+0000","last_unstale":"2023-12-02T00:31:01.837604+0000","last_undegraded":"2023-12-02T00:31:01.837604+0000","last_fullsized":"2023-12-02T00:31:01.837604+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T01:12:56.020427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903201+0000","last_change":"2023-12-02T00:31:01.850143+0000","last_active":"2023-12-02T00:31:02.903201+0000","last_peered":"2023-12-02T00:31:02.903201+0000","last_clean":"2023-12-02T00:31:02.903201+0000","last_became_active":"2023-12-02T00:31:01.849883+0000","last_became_peered":"2023-12-02T00:31:01.849883+0000","last_unstale":"2023-12-02T00:31:02.903201+0000","last_undegraded":"2023-12-02T00:31:02.903201+0000","last_fullsized":"2023-12-02T00:31:02.903201+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:19:31.202411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.833108+0000","last_change":"2023-12-02T00:31:00.828451+0000","last_active":"2023-12-02T00:31:01.833108+0000","last_peered":"2023-12-02T00:31:01.833108+0000","last_clean":"2023-12-02T00:31:01.833108+0000","last_became_active":"2023-12-02T00:31:00.828164+0000","last_became_peered":"2023-12-02T00:31:00.828164+0000","last_unstale":"2023-12-02T00:31:01.833108+0000","last_undegraded":"2023-12-02T00:31:01.833108+0000","last_fullsized":"2023-12-02T00:31:01.833108+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:01:26.534034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.848407+0000","last_change":"2023-12-02T00:31:01.841803+0000","last_active":"2023-12-02T00:31:02.848407+0000","last_peered":"2023-12-02T00:31:02.848407+0000","last_clean":"2023-12-02T00:31:02.848407+0000","last_became_active":"2023-12-02T00:31:01.839190+0000","last_became_peered":"2023-12-02T00:31:01.839190+0000","last_unstale":"2023-12-02T00:31:02.848407+0000","last_undegraded":"2023-12-02T00:31:02.848407+0000","last_fullsized":"2023-12-02T00:31:02.848407+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:01:27.538047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.840644+0000","last_change":"2023-12-02T00:31:00.835042+0000","last_active":"2023-12-02T00:31:01.840644+0000","last_peered":"2023-12-02T00:31:01.840644+0000","last_clean":"2023-12-02T00:31:01.840644+0000","last_became_active":"2023-12-02T00:31:00.834764+0000","last_became_peered":"2023-12-02T00:31:00.834764+0000","last_unstale":"2023-12-02T00:31:01.840644+0000","last_undegraded":"2023-12-02T00:31:01.840644+0000","last_fullsized":"2023-12-02T00:31:01.840644+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:34:39.428048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.902661+0000","last_change":"2023-12-02T00:31:01.845726+0000","last_active":"2023-12-02T00:31:02.902661+0000","last_peered":"2023-12-02T00:31:02.902661+0000","last_clean":"2023-12-02T00:31:02.902661+0000","last_became_active":"2023-12-02T00:31:01.845447+0000","last_became_peered":"2023-12-02T00:31:01.845447+0000","last_unstale":"2023-12-02T00:31:02.902661+0000","last_undegraded":"2023-12-02T00:31:02.902661+0000","last_fullsized":"2023-12-02T00:31:02.902661+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:34:42.351893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.838788+0000","last_change":"2023-12-02T00:31:00.821222+0000","last_active":"2023-12-02T00:31:01.838788+0000","last_peered":"2023-12-02T00:31:01.838788+0000","last_clean":"2023-12-02T00:31:01.838788+0000","last_became_active":"2023-12-02T00:31:00.820914+0000","last_became_peered":"2023-12-02T00:31:00.820914+0000","last_unstale":"2023-12-02T00:31:01.838788+0000","last_undegraded":"2023-12-02T00:31:01.838788+0000","last_fullsized":"2023-12-02T00:31:01.838788+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:30:31.052164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903504+0000","last_change":"2023-12-02T00:31:01.837285+0000","last_active":"2023-12-02T00:31:02.903504+0000","last_peered":"2023-12-02T00:31:02.903504+0000","last_clean":"2023-12-02T00:31:02.903504+0000","last_became_active":"2023-12-02T00:31:01.836952+0000","last_became_peered":"2023-12-02T00:31:01.836952+0000","last_unstale":"2023-12-02T00:31:02.903504+0000","last_undegraded":"2023-12-02T00:31:02.903504+0000","last_fullsized":"2023-12-02T00:31:02.903504+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T01:48:44.099841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"50'3","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.865472+0000","last_change":"2023-12-02T00:31:00.837992+0000","last_active":"2023-12-02T00:31:01.865472+0000","last_peered":"2023-12-02T00:31:01.865472+0000","last_clean":"2023-12-02T00:31:01.865472+0000","last_became_active":"2023-12-02T00:31:00.837743+0000","last_became_peered":"2023-12-02T00:31:00.837743+0000","last_unstale":"2023-12-02T00:31:01.865472+0000","last_undegraded":"2023-12-02T00:31:01.865472+0000","last_fullsized":"2023-12-02T00:31:01.865472+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:34:01.547200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.848794+0000","last_change":"2023-12-02T00:31:01.859827+0000","last_active":"2023-12-02T00:31:02.848794+0000","last_peered":"2023-12-02T00:31:02.848794+0000","last_clean":"2023-12-02T00:31:02.848794+0000","last_became_active":"2023-12-02T00:31:01.859565+0000","last_became_peered":"2023-12-02T00:31:01.859565+0000","last_unstale":"2023-12-02T00:31:02.848794+0000","last_undegraded":"2023-12-02T00:31:02.848794+0000","last_fullsized":"2023-12-02T00:31:02.848794+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:54:08.916982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"50'6","reported_seq":18,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.868031+0000","last_change":"2023-12-02T00:31:00.840634+0000","last_active":"2023-12-02T00:31:01.868031+0000","last_peered":"2023-12-02T00:31:01.868031+0000","last_clean":"2023-12-02T00:31:01.868031+0000","last_became_active":"2023-12-02T00:31:00.840359+0000","last_became_peered":"2023-12-02T00:31:00.840359+0000","last_unstale":"2023-12-02T00:31:01.868031+0000","last_undegraded":"2023-12-02T00:31:01.868031+0000","last_fullsized":"2023-12-02T00:31:01.868031+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T02:12:41.097282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903534+0000","last_change":"2023-12-02T00:31:01.847992+0000","last_active":"2023-12-02T00:31:02.903534+0000","last_peered":"2023-12-02T00:31:02.903534+0000","last_clean":"2023-12-02T00:31:02.903534+0000","last_became_active":"2023-12-02T00:31:01.847631+0000","last_became_peered":"2023-12-02T00:31:01.847631+0000","last_unstale":"2023-12-02T00:31:02.903534+0000","last_undegraded":"2023-12-02T00:31:02.903534+0000","last_fullsized":"2023-12-02T00:31:02.903534+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:23:21.917306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"50'2","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.860725+0000","last_change":"2023-12-02T00:31:00.829212+0000","last_active":"2023-12-02T00:31:01.860725+0000","last_peered":"2023-12-02T00:31:01.860725+0000","last_clean":"2023-12-02T00:31:01.860725+0000","last_became_active":"2023-12-02T00:31:00.828848+0000","last_became_peered":"2023-12-02T00:31:00.828848+0000","last_unstale":"2023-12-02T00:31:01.860725+0000","last_undegraded":"2023-12-02T00:31:01.860725+0000","last_fullsized":"2023-12-02T00:31:01.860725+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T10:47:44.799679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903446+0000","last_change":"2023-12-02T00:31:01.846618+0000","last_active":"2023-12-02T00:31:02.903446+0000","last_peered":"2023-12-02T00:31:02.903446+0000","last_clean":"2023-12-02T00:31:02.903446+0000","last_became_active":"2023-12-02T00:31:01.846263+0000","last_became_peered":"2023-12-02T00:31:01.846263+0000","last_unstale":"2023-12-02T00:31:02.903446+0000","last_undegraded":"2023-12-02T00:31:02.903446+0000","last_fullsized":"2023-12-02T00:31:02.903446+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:28:45.087022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.836432+0000","last_change":"2023-12-02T00:31:00.834656+0000","last_active":"2023-12-02T00:31:01.836432+0000","last_peered":"2023-12-02T00:31:01.836432+0000","last_clean":"2023-12-02T00:31:01.836432+0000","last_became_active":"2023-12-02T00:31:00.834359+0000","last_became_peered":"2023-12-02T00:31:00.834359+0000","last_unstale":"2023-12-02T00:31:01.836432+0000","last_undegraded":"2023-12-02T00:31:01.836432+0000","last_fullsized":"2023-12-02T00:31:01.836432+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T09:28:07.117685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.843490+0000","last_change":"2023-12-02T00:31:01.843226+0000","last_active":"2023-12-02T00:31:02.843490+0000","last_peered":"2023-12-02T00:31:02.843490+0000","last_clean":"2023-12-02T00:31:02.843490+0000","last_became_active":"2023-12-02T00:31:01.842940+0000","last_became_peered":"2023-12-02T00:31:01.842940+0000","last_unstale":"2023-12-02T00:31:02.843490+0000","last_undegraded":"2023-12-02T00:31:02.843490+0000","last_fullsized":"2023-12-02T00:31:02.843490+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:05:01.325502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.839978+0000","last_change":"2023-12-02T00:31:00.837228+0000","last_active":"2023-12-02T00:31:01.839978+0000","last_peered":"2023-12-02T00:31:01.839978+0000","last_clean":"2023-12-02T00:31:01.839978+0000","last_became_active":"2023-12-02T00:31:00.836943+0000","last_became_peered":"2023-12-02T00:31:00.836943+0000","last_unstale":"2023-12-02T00:31:01.839978+0000","last_undegraded":"2023-12-02T00:31:01.839978+0000","last_fullsized":"2023-12-02T00:31:01.839978+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T00:34:41.347880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903680+0000","last_change":"2023-12-02T00:31:01.843905+0000","last_active":"2023-12-02T00:31:02.903680+0000","last_peered":"2023-12-02T00:31:02.903680+0000","last_clean":"2023-12-02T00:31:02.903680+0000","last_became_active":"2023-12-02T00:31:01.842704+0000","last_became_peered":"2023-12-02T00:31:01.842704+0000","last_unstale":"2023-12-02T00:31:02.903680+0000","last_undegraded":"2023-12-02T00:31:02.903680+0000","last_fullsized":"2023-12-02T00:31:02.903680+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T01:53:57.181929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"50'8","reported_seq":20,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.866487+0000","last_change":"2023-12-02T00:31:00.840645+0000","last_active":"2023-12-02T00:31:01.866487+0000","last_peered":"2023-12-02T00:31:01.866487+0000","last_clean":"2023-12-02T00:31:01.866487+0000","last_became_active":"2023-12-02T00:31:00.840295+0000","last_became_peered":"2023-12-02T00:31:00.840295+0000","last_unstale":"2023-12-02T00:31:01.866487+0000","last_undegraded":"2023-12-02T00:31:01.866487+0000","last_fullsized":"2023-12-02T00:31:01.866487+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:13:05.067577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.843226+0000","last_change":"2023-12-02T00:31:01.848869+0000","last_active":"2023-12-02T00:31:02.843226+0000","last_peered":"2023-12-02T00:31:02.843226+0000","last_clean":"2023-12-02T00:31:02.843226+0000","last_became_active":"2023-12-02T00:31:01.848523+0000","last_became_peered":"2023-12-02T00:31:01.848523+0000","last_unstale":"2023-12-02T00:31:02.843226+0000","last_undegraded":"2023-12-02T00:31:02.843226+0000","last_fullsized":"2023-12-02T00:31:02.843226+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:13:02.960439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"50'1","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.859468+0000","last_change":"2023-12-02T00:31:00.839972+0000","last_active":"2023-12-02T00:31:01.859468+0000","last_peered":"2023-12-02T00:31:01.859468+0000","last_clean":"2023-12-02T00:31:01.859468+0000","last_became_active":"2023-12-02T00:31:00.839581+0000","last_became_peered":"2023-12-02T00:31:00.839581+0000","last_unstale":"2023-12-02T00:31:01.859468+0000","last_undegraded":"2023-12-02T00:31:01.859468+0000","last_fullsized":"2023-12-02T00:31:01.859468+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:00:03.107141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.902640+0000","last_change":"2023-12-02T00:31:01.845651+0000","last_active":"2023-12-02T00:31:02.902640+0000","last_peered":"2023-12-02T00:31:02.902640+0000","last_clean":"2023-12-02T00:31:02.902640+0000","last_became_active":"2023-12-02T00:31:01.845364+0000","last_became_peered":"2023-12-02T00:31:01.845364+0000","last_unstale":"2023-12-02T00:31:02.902640+0000","last_undegraded":"2023-12-02T00:31:02.902640+0000","last_fullsized":"2023-12-02T00:31:02.902640+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:51:09.108574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"50'2","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.863224+0000","last_change":"2023-12-02T00:31:00.832413+0000","last_active":"2023-12-02T00:31:01.863224+0000","last_peered":"2023-12-02T00:31:01.863224+0000","last_clean":"2023-12-02T00:31:01.863224+0000","last_became_active":"2023-12-02T00:31:00.832127+0000","last_became_peered":"2023-12-02T00:31:00.832127+0000","last_unstale":"2023-12-02T00:31:01.863224+0000","last_undegraded":"2023-12-02T00:31:01.863224+0000","last_fullsized":"2023-12-02T00:31:01.863224+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:17:18.074493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.848460+0000","last_change":"2023-12-02T00:31:01.844176+0000","last_active":"2023-12-02T00:31:02.848460+0000","last_peered":"2023-12-02T00:31:02.848460+0000","last_clean":"2023-12-02T00:31:02.848460+0000","last_became_active":"2023-12-02T00:31:01.843893+0000","last_became_peered":"2023-12-02T00:31:01.843893+0000","last_unstale":"2023-12-02T00:31:02.848460+0000","last_undegraded":"2023-12-02T00:31:02.848460+0000","last_fullsized":"2023-12-02T00:31:02.848460+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:51:09.108574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"50'1","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.865352+0000","last_change":"2023-12-02T00:31:00.837851+0000","last_active":"2023-12-02T00:31:01.865352+0000","last_peered":"2023-12-02T00:31:01.865352+0000","last_clean":"2023-12-02T00:31:01.865352+0000","last_became_active":"2023-12-02T00:31:00.837581+0000","last_became_peered":"2023-12-02T00:31:00.837581+0000","last_unstale":"2023-12-02T00:31:01.865352+0000","last_undegraded":"2023-12-02T00:31:01.865352+0000","last_fullsized":"2023-12-02T00:31:01.865352+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T10:06:37.837138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.902886+0000","last_change":"2023-12-02T00:31:01.844563+0000","last_active":"2023-12-02T00:31:02.902886+0000","last_peered":"2023-12-02T00:31:02.902886+0000","last_clean":"2023-12-02T00:31:02.902886+0000","last_became_active":"2023-12-02T00:31:01.844264+0000","last_became_peered":"2023-12-02T00:31:01.844264+0000","last_unstale":"2023-12-02T00:31:02.902886+0000","last_undegraded":"2023-12-02T00:31:02.902886+0000","last_fullsized":"2023-12-02T00:31:02.902886+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:44:26.068567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.837441+0000","last_change":"2023-12-02T00:31:00.832751+0000","last_active":"2023-12-02T00:31:01.837441+0000","last_peered":"2023-12-02T00:31:01.837441+0000","last_clean":"2023-12-02T00:31:01.837441+0000","last_became_active":"2023-12-02T00:31:00.832466+0000","last_became_peered":"2023-12-02T00:31:00.832466+0000","last_unstale":"2023-12-02T00:31:01.837441+0000","last_undegraded":"2023-12-02T00:31:01.837441+0000","last_fullsized":"2023-12-02T00:31:01.837441+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:57:02.728754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903534+0000","last_change":"2023-12-02T00:31:01.850937+0000","last_active":"2023-12-02T00:31:02.903534+0000","last_peered":"2023-12-02T00:31:02.903534+0000","last_clean":"2023-12-02T00:31:02.903534+0000","last_became_active":"2023-12-02T00:31:01.850638+0000","last_became_peered":"2023-12-02T00:31:01.850638+0000","last_unstale":"2023-12-02T00:31:02.903534+0000","last_undegraded":"2023-12-02T00:31:02.903534+0000","last_fullsized":"2023-12-02T00:31:02.903534+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:34:40.432062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"50'2","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.864111+0000","last_change":"2023-12-02T00:31:00.833503+0000","last_active":"2023-12-02T00:31:01.864111+0000","last_peered":"2023-12-02T00:31:01.864111+0000","last_clean":"2023-12-02T00:31:01.864111+0000","last_became_active":"2023-12-02T00:31:00.833219+0000","last_became_peered":"2023-12-02T00:31:00.833219+0000","last_unstale":"2023-12-02T00:31:01.864111+0000","last_undegraded":"2023-12-02T00:31:01.864111+0000","last_fullsized":"2023-12-02T00:31:01.864111+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:30:31.052164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.902950+0000","last_change":"2023-12-02T00:31:01.843475+0000","last_active":"2023-12-02T00:31:02.902950+0000","last_peered":"2023-12-02T00:31:02.902950+0000","last_clean":"2023-12-02T00:31:02.902950+0000","last_became_active":"2023-12-02T00:31:01.843185+0000","last_became_peered":"2023-12-02T00:31:01.843185+0000","last_unstale":"2023-12-02T00:31:02.902950+0000","last_undegraded":"2023-12-02T00:31:02.902950+0000","last_fullsized":"2023-12-02T00:31:02.902950+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:23:49.102467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.828451+0000","last_change":"2023-12-02T00:31:00.840501+0000","last_active":"2023-12-02T00:31:01.828451+0000","last_peered":"2023-12-02T00:31:01.828451+0000","last_clean":"2023-12-02T00:31:01.828451+0000","last_became_active":"2023-12-02T00:31:00.840227+0000","last_became_peered":"2023-12-02T00:31:00.840227+0000","last_unstale":"2023-12-02T00:31:01.828451+0000","last_undegraded":"2023-12-02T00:31:01.828451+0000","last_fullsized":"2023-12-02T00:31:01.828451+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:23:20.913293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903584+0000","last_change":"2023-12-02T00:31:01.841976+0000","last_active":"2023-12-02T00:31:02.903584+0000","last_peered":"2023-12-02T00:31:02.903584+0000","last_clean":"2023-12-02T00:31:02.903584+0000","last_became_active":"2023-12-02T00:31:01.838386+0000","last_became_peered":"2023-12-02T00:31:01.838386+0000","last_unstale":"2023-12-02T00:31:02.903584+0000","last_undegraded":"2023-12-02T00:31:02.903584+0000","last_fullsized":"2023-12-02T00:31:02.903584+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T01:54:26.282475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.833290+0000","last_change":"2023-12-02T00:31:00.833798+0000","last_active":"2023-12-02T00:31:01.833290+0000","last_peered":"2023-12-02T00:31:01.833290+0000","last_clean":"2023-12-02T00:31:01.833290+0000","last_became_active":"2023-12-02T00:31:00.833537+0000","last_became_peered":"2023-12-02T00:31:00.833537+0000","last_unstale":"2023-12-02T00:31:01.833290+0000","last_undegraded":"2023-12-02T00:31:01.833290+0000","last_fullsized":"2023-12-02T00:31:01.833290+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T10:02:53.861890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.902901+0000","last_change":"2023-12-02T00:31:01.841890+0000","last_active":"2023-12-02T00:31:02.902901+0000","last_peered":"2023-12-02T00:31:02.902901+0000","last_clean":"2023-12-02T00:31:02.902901+0000","last_became_active":"2023-12-02T00:31:01.841559+0000","last_became_peered":"2023-12-02T00:31:01.841559+0000","last_unstale":"2023-12-02T00:31:02.902901+0000","last_undegraded":"2023-12-02T00:31:02.902901+0000","last_fullsized":"2023-12-02T00:31:02.902901+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T09:26:09.725821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.843299+0000","last_change":"2023-12-02T00:31:00.829323+0000","last_active":"2023-12-02T00:31:01.843299+0000","last_peered":"2023-12-02T00:31:01.843299+0000","last_clean":"2023-12-02T00:31:01.843299+0000","last_became_active":"2023-12-02T00:31:00.829048+0000","last_became_peered":"2023-12-02T00:31:00.829048+0000","last_unstale":"2023-12-02T00:31:01.843299+0000","last_undegraded":"2023-12-02T00:31:01.843299+0000","last_fullsized":"2023-12-02T00:31:01.843299+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T07:19:14.051539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.902821+0000","last_change":"2023-12-02T00:31:01.849585+0000","last_active":"2023-12-02T00:31:02.902821+0000","last_peered":"2023-12-02T00:31:02.902821+0000","last_clean":"2023-12-02T00:31:02.902821+0000","last_became_active":"2023-12-02T00:31:01.849297+0000","last_became_peered":"2023-12-02T00:31:01.849297+0000","last_unstale":"2023-12-02T00:31:02.902821+0000","last_undegraded":"2023-12-02T00:31:02.902821+0000","last_fullsized":"2023-12-02T00:31:02.902821+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:00:04.111155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"50'2","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.861990+0000","last_change":"2023-12-02T00:31:00.835706+0000","last_active":"2023-12-02T00:31:01.861990+0000","last_peered":"2023-12-02T00:31:01.861990+0000","last_clean":"2023-12-02T00:31:01.861990+0000","last_became_active":"2023-12-02T00:31:00.835381+0000","last_became_peered":"2023-12-02T00:31:00.835381+0000","last_unstale":"2023-12-02T00:31:01.861990+0000","last_undegraded":"2023-12-02T00:31:01.861990+0000","last_fullsized":"2023-12-02T00:31:01.861990+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T02:42:58.558235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903252+0000","last_change":"2023-12-02T00:31:01.861842+0000","last_active":"2023-12-02T00:31:02.903252+0000","last_peered":"2023-12-02T00:31:02.903252+0000","last_clean":"2023-12-02T00:31:02.903252+0000","last_became_active":"2023-12-02T00:31:01.861573+0000","last_became_peered":"2023-12-02T00:31:01.861573+0000","last_unstale":"2023-12-02T00:31:02.903252+0000","last_undegraded":"2023-12-02T00:31:02.903252+0000","last_fullsized":"2023-12-02T00:31:02.903252+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:40:06.079397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.840728+0000","last_change":"2023-12-02T00:31:00.832747+0000","last_active":"2023-12-02T00:31:01.840728+0000","last_peered":"2023-12-02T00:31:01.840728+0000","last_clean":"2023-12-02T00:31:01.840728+0000","last_became_active":"2023-12-02T00:31:00.832477+0000","last_became_peered":"2023-12-02T00:31:00.832477+0000","last_unstale":"2023-12-02T00:31:01.840728+0000","last_undegraded":"2023-12-02T00:31:01.840728+0000","last_fullsized":"2023-12-02T00:31:01.840728+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T02:25:45.046480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.843219+0000","last_change":"2023-12-02T00:31:01.844603+0000","last_active":"2023-12-02T00:31:02.843219+0000","last_peered":"2023-12-02T00:31:02.843219+0000","last_clean":"2023-12-02T00:31:02.843219+0000","last_became_active":"2023-12-02T00:31:01.844329+0000","last_became_peered":"2023-12-02T00:31:01.844329+0000","last_unstale":"2023-12-02T00:31:02.843219+0000","last_undegraded":"2023-12-02T00:31:02.843219+0000","last_fullsized":"2023-12-02T00:31:02.843219+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:21:48.643558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.834121+0000","last_change":"2023-12-02T00:31:00.828668+0000","last_active":"2023-12-02T00:31:01.834121+0000","last_peered":"2023-12-02T00:31:01.834121+0000","last_clean":"2023-12-02T00:31:01.834121+0000","last_became_active":"2023-12-02T00:31:00.828392+0000","last_became_peered":"2023-12-02T00:31:00.828392+0000","last_unstale":"2023-12-02T00:31:01.834121+0000","last_undegraded":"2023-12-02T00:31:01.834121+0000","last_fullsized":"2023-12-02T00:31:01.834121+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T08:41:19.966082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.902891+0000","last_change":"2023-12-02T00:31:01.839257+0000","last_active":"2023-12-02T00:31:02.902891+0000","last_peered":"2023-12-02T00:31:02.902891+0000","last_clean":"2023-12-02T00:31:02.902891+0000","last_became_active":"2023-12-02T00:31:01.838789+0000","last_became_peered":"2023-12-02T00:31:01.838789+0000","last_unstale":"2023-12-02T00:31:02.902891+0000","last_undegraded":"2023-12-02T00:31:02.902891+0000","last_fullsized":"2023-12-02T00:31:02.902891+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T06:34:02.551214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.829907+0000","last_change":"2023-12-02T00:31:00.841414+0000","last_active":"2023-12-02T00:31:01.829907+0000","last_peered":"2023-12-02T00:31:01.829907+0000","last_clean":"2023-12-02T00:31:01.829907+0000","last_became_active":"2023-12-02T00:31:00.841049+0000","last_became_peered":"2023-12-02T00:31:00.841049+0000","last_unstale":"2023-12-02T00:31:01.829907+0000","last_undegraded":"2023-12-02T00:31:01.829907+0000","last_fullsized":"2023-12-02T00:31:01.829907+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T01:53:56.177915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903664+0000","last_change":"2023-12-02T00:31:01.843439+0000","last_active":"2023-12-02T00:31:02.903664+0000","last_peered":"2023-12-02T00:31:02.903664+0000","last_clean":"2023-12-02T00:31:02.903664+0000","last_became_active":"2023-12-02T00:31:01.843180+0000","last_became_peered":"2023-12-02T00:31:01.843180+0000","last_unstale":"2023-12-02T00:31:02.903664+0000","last_undegraded":"2023-12-02T00:31:02.903664+0000","last_fullsized":"2023-12-02T00:31:02.903664+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:31:09.860031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.828618+0000","last_change":"2023-12-02T00:31:00.838503+0000","last_active":"2023-12-02T00:31:01.828618+0000","last_peered":"2023-12-02T00:31:01.828618+0000","last_clean":"2023-12-02T00:31:01.828618+0000","last_became_active":"2023-12-02T00:31:00.838125+0000","last_became_peered":"2023-12-02T00:31:00.838125+0000","last_unstale":"2023-12-02T00:31:01.828618+0000","last_undegraded":"2023-12-02T00:31:01.828618+0000","last_fullsized":"2023-12-02T00:31:01.828618+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T02:25:45.046480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903417+0000","last_change":"2023-12-02T00:31:01.860284+0000","last_active":"2023-12-02T00:31:02.903417+0000","last_peered":"2023-12-02T00:31:02.903417+0000","last_clean":"2023-12-02T00:31:02.903417+0000","last_became_active":"2023-12-02T00:31:01.860002+0000","last_became_peered":"2023-12-02T00:31:01.860002+0000","last_unstale":"2023-12-02T00:31:02.903417+0000","last_undegraded":"2023-12-02T00:31:02.903417+0000","last_fullsized":"2023-12-02T00:31:02.903417+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:13:06.071590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.834507+0000","last_change":"2023-12-02T00:31:00.828726+0000","last_active":"2023-12-02T00:31:01.834507+0000","last_peered":"2023-12-02T00:31:01.834507+0000","last_clean":"2023-12-02T00:31:01.834507+0000","last_became_active":"2023-12-02T00:31:00.828424+0000","last_became_peered":"2023-12-02T00:31:00.828424+0000","last_unstale":"2023-12-02T00:31:01.834507+0000","last_undegraded":"2023-12-02T00:31:01.834507+0000","last_fullsized":"2023-12-02T00:31:01.834507+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T03:06:02.637069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.843494+0000","last_change":"2023-12-02T00:31:01.863549+0000","last_active":"2023-12-02T00:31:02.843494+0000","last_peered":"2023-12-02T00:31:02.843494+0000","last_clean":"2023-12-02T00:31:02.843494+0000","last_became_active":"2023-12-02T00:31:01.861439+0000","last_became_peered":"2023-12-02T00:31:01.861439+0000","last_unstale":"2023-12-02T00:31:02.843494+0000","last_undegraded":"2023-12-02T00:31:02.843494+0000","last_fullsized":"2023-12-02T00:31:02.843494+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T01:24:56.864149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.837957+0000","last_change":"2023-12-02T00:31:00.837338+0000","last_active":"2023-12-02T00:31:01.837957+0000","last_peered":"2023-12-02T00:31:01.837957+0000","last_clean":"2023-12-02T00:31:01.837957+0000","last_became_active":"2023-12-02T00:31:00.837017+0000","last_became_peered":"2023-12-02T00:31:00.837017+0000","last_unstale":"2023-12-02T00:31:01.837957+0000","last_undegraded":"2023-12-02T00:31:01.837957+0000","last_fullsized":"2023-12-02T00:31:01.837957+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T05:54:07.912968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-12-02T00:31:02.903112+0000","last_change":"2023-12-02T00:31:01.844332+0000","last_active":"2023-12-02T00:31:02.903112+0000","last_peered":"2023-12-02T00:31:02.903112+0000","last_clean":"2023-12-02T00:31:02.903112+0000","last_became_active":"2023-12-02T00:31:01.843971+0000","last_became_peered":"2023-12-02T00:31:01.843971+0000","last_unstale":"2023-12-02T00:31:02.903112+0000","last_undegraded":"2023-12-02T00:31:02.903112+0000","last_fullsized":"2023-12-02T00:31:02.903112+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:31:00.812881+0000","last_clean_scrub_stamp":"2023-12-02T00:31:00.812881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T09:04:54.768041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.833964+0000","last_change":"2023-12-02T00:31:00.829255+0000","last_active":"2023-12-02T00:31:01.833964+0000","last_peered":"2023-12-02T00:31:01.833964+0000","last_clean":"2023-12-02T00:31:01.833964+0000","last_became_active":"2023-12-02T00:31:00.828976+0000","last_became_peered":"2023-12-02T00:31:00.828976+0000","last_unstale":"2023-12-02T00:31:01.833964+0000","last_undegraded":"2023-12-02T00:31:01.833964+0000","last_fullsized":"2023-12-02T00:31:01.833964+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:30:59.808868+0000","last_clean_scrub_stamp":"2023-12-02T00:30:59.808868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T09:26:08.721807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":194,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-02T00:31:01.839206+0000","last_change":"2023-12-02T00:28:39.379380+0000","last_active":"2023-12-02T00:31:01.839206+0000","last_peered":"2023-12-02T00:31:01.839206+0000","last_clean":"2023-12-02T00:31:01.839206+0000","last_became_active":"2023-12-02T00:28:39.378255+0000","last_became_peered":"2023-12-02T00:28:39.378255+0000","last_unstale":"2023-12-02T00:31:01.839206+0000","last_undegraded":"2023-12-02T00:31:01.839206+0000","last_fullsized":"2023-12-02T00:31:01.839206+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:38.177345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:38.177345+0000","last_clean_scrub_stamp":"2023-12-02T00:28:38.177345+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:21:08.582412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28204,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343636,"statfs":{"total":96636764160,"available":96607883264,"internally_reserved":0,"allocated":1208320,"data_stored":842539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.50900000000000001,"15min":0.499},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.63200000000000001}]},{"osd":1,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53400000000000003,"15min":0.53000000000000003},"min":{"1min":0.40400000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.54700000000000004,"15min":0.54400000000000004},"min":{"1min":0.46100000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.72299999999999998}]},{"osd":2,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54200000000000004,"15min":0.54400000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.67000000000000004,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.54300000000000004,"15min":0.53400000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.78000000000000003}]},{"osd":3,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.53000000000000003,"15min":0.52200000000000002},"min":{"1min":0.48099999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.57299999999999995,"15min":0.57499999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.74399999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.70599999999999996}]},{"osd":4,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.50700000000000001,"15min":0.51100000000000001},"min":{"1min":0.29799999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.72199999999999998,"5min":0.745,"15min":0.745},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.51500000000000001,"15min":0.52100000000000002},"min":{"1min":0.33800000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.61799999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.44700000000000001}]},{"osd":5,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54700000000000004,"15min":0.55000000000000004},"min":{"1min":0.41899999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.78700000000000003,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.56100000000000005,"15min":0.56599999999999995},"min":{"1min":0.39500000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.63900000000000001,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.49399999999999999}]},{"osd":6,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.61799999999999999,"15min":0.627},"min":{"1min":0.40100000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.82899999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.60599999999999998,"15min":0.61299999999999999},"min":{"1min":0.44500000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76600000000000001,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.55200000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27624,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344216,"statfs":{"total":96636764160,"available":96608477184,"internally_reserved":0,"allocated":614400,"data_stored":252699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.49199999999999999,"15min":0.48299999999999998},"min":{"1min":0.36499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.50700000000000001,"15min":0.501},"min":{"1min":0.40000000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.74099999999999999,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.46200000000000002}]},{"osd":1,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.51000000000000001,"15min":0.50600000000000001},"min":{"1min":0.33000000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.72599999999999998,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.53600000000000003,"15min":0.53300000000000003},"min":{"1min":0.39400000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.80900000000000005,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.52200000000000002}]},{"osd":2,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.52000000000000002,"15min":0.51200000000000001},"min":{"1min":0.38,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.628},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55500000000000005,"15min":0.55700000000000005},"min":{"1min":0.40300000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.77400000000000002,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.55400000000000005}]},{"osd":3,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57599999999999996,"15min":0.57299999999999995},"min":{"1min":0.45200000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.76300000000000001,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57199999999999995,"15min":0.56999999999999995},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.53500000000000003}]},{"osd":4,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.35399999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.71299999999999997,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52500000000000002,"15min":0.52300000000000002},"min":{"1min":0.34100000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73699999999999999,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.58299999999999996}]},{"osd":5,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.33400000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58799999999999997,"15min":0.59299999999999997},"min":{"1min":0.42399999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.48399999999999999}]},{"osd":7,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.35999999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.44600000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574882,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27616,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344224,"statfs":{"total":96636764160,"available":96608485376,"internally_reserved":0,"allocated":606208,"data_stored":252047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.51700000000000002,"15min":0.50600000000000001},"min":{"1min":0.29099999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.73899999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.51600000000000001,"15min":0.51100000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.72899999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.47099999999999997}]},{"osd":1,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.51400000000000001,"15min":0.505},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.754,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55300000000000005,"15min":0.55000000000000004},"min":{"1min":0.36199999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.69499999999999995,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.42199999999999999}]},{"osd":2,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.56000000000000005,"15min":0.55100000000000005},"min":{"1min":0.41699999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.77500000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.53600000000000003,"15min":0.53300000000000003},"min":{"1min":0.34399999999999997,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.749,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.503}]},{"osd":3,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58099999999999996,"15min":0.57699999999999996},"min":{"1min":0.39300000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.58299999999999996,"15min":0.58199999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.72499999999999998,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.52800000000000002}]},{"osd":4,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54300000000000004,"15min":0.54600000000000004},"min":{"1min":0.34799999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69899999999999995,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58699999999999997,"15min":0.58799999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.51600000000000001}]},{"osd":6,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.56599999999999995,"15min":0.57099999999999995},"min":{"1min":0.36899999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.72899999999999998,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.59199999999999997,"15min":0.59599999999999997},"min":{"1min":0.435,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.41099999999999998}]},{"osd":7,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.42699999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.82099999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.63100000000000001,"15min":0.63500000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.76700000000000002,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.55300000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":610304,"data_stored":252629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.55100000000000005,"15min":0.54400000000000004},"min":{"1min":0.40699999999999997,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.74099999999999999,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.438,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.79500000000000004,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.82799999999999996}]},{"osd":1,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.57099999999999995,"15min":0.55800000000000005},"min":{"1min":0.50700000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.872},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.55800000000000005,"15min":0.55400000000000005},"min":{"1min":0.38600000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.77300000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.71199999999999997}]},{"osd":2,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.58099999999999996,"15min":0.57699999999999996},"min":{"1min":0.443,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.42799999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.752}]},{"osd":3,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.56999999999999995,"15min":0.56200000000000006},"min":{"1min":0.46600000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60599999999999998,"15min":0.60399999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.80300000000000005}]},{"osd":5,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52600000000000002,"15min":0.52300000000000002},"min":{"1min":0.36699999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.83799999999999997,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53900000000000003,"15min":0.54100000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.46800000000000003}]},{"osd":6,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.54200000000000004,"15min":0.53600000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.57699999999999996,"15min":0.56999999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.57999999999999996}]},{"osd":7,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.377,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.59499999999999997,"15min":0.59099999999999997},"min":{"1min":0.34699999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.54400000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574880,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31796,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340044,"statfs":{"total":96636764160,"available":96604205056,"internally_reserved":0,"allocated":622592,"data_stored":253681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.35699999999999998,"15min":0.35999999999999999},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.50800000000000001,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.38200000000000001,"15min":0.38100000000000001},"min":{"1min":0.27500000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.624,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.36899999999999999}]},{"osd":1,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.379,"15min":0.38300000000000001},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.59999999999999998,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.25800000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.58699999999999997,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.30399999999999999}]},{"osd":2,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.40600000000000003,"15min":0.40999999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.52900000000000003,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.42199999999999999,"15min":0.42099999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.66500000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.34999999999999998}]},{"osd":4,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.60899999999999999,"15min":0.61099999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76600000000000001,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.622,"15min":0.623},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.76300000000000001}]},{"osd":5,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.58299999999999996,"15min":0.58699999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85599999999999998,"5min":1.004,"15min":1.004},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.61299999999999999,"15min":0.61499999999999999},"min":{"1min":0.39300000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.90600000000000003,"5min":1.163,"15min":1.163},"last":0.50600000000000001}]},{"osd":6,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63800000000000001,"15min":0.63900000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.83199999999999996,"5min":1.095,"15min":1.095},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.63200000000000001,"15min":0.63500000000000001},"min":{"1min":0.45100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.96399999999999997,"5min":1.044,"15min":1.044},"last":0.746}]},{"osd":7,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.64700000000000002,"15min":0.65100000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.93899999999999995,"5min":1.177,"15min":1.177},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.64500000000000002,"15min":0.65100000000000002},"min":{"1min":0.45200000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.84799999999999998,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.72299999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31784,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340056,"statfs":{"total":96636764160,"available":96604217344,"internally_reserved":0,"allocated":614400,"data_stored":253040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.377,"15min":0.36299999999999999},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":2.649,"5min":2.649,"15min":2.649},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.42299999999999999,"15min":0.40799999999999997},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":3.173,"5min":3.173,"15min":3.173},"last":0.32200000000000001}]},{"osd":1,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.44800000000000001,"15min":0.41999999999999998},"min":{"1min":0.26300000000000001,"5min":0.246,"15min":0.246},"max":{"1min":4.6639999999999997,"5min":4.6639999999999997,"15min":4.6639999999999997},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.433,"15min":0.39100000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":6.093,"5min":6.093,"15min":6.093},"last":0.35799999999999998}]},{"osd":3,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.40300000000000002,"15min":0.376},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":4.1879999999999997,"5min":4.1879999999999997,"15min":4.1879999999999997},"last":0.37},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.44600000000000001,"15min":0.433},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":2.4159999999999999,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":0.38}]},{"osd":4,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.60999999999999999,"15min":0.59599999999999997},"min":{"1min":0.40999999999999998,"5min":0.38,"15min":0.38},"max":{"1min":2.6840000000000002,"5min":2.6840000000000002,"15min":2.6840000000000002},"last":0.629},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.65800000000000003,"15min":0.64600000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":3.0419999999999998,"5min":3.0419999999999998,"15min":3.0419999999999998},"last":0.62}]},{"osd":5,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.66100000000000003,"15min":0.65300000000000002},"min":{"1min":0.41599999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":3.093,"5min":3.093,"15min":3.093},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.621,"5min":0.64700000000000002,"15min":0.65100000000000002},"min":{"1min":0.41799999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.98499999999999999,"5min":1.111,"15min":1.111},"last":0.67400000000000004}]},{"osd":6,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.64000000000000001,"15min":0.63800000000000001},"min":{"1min":0.42499999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.67800000000000005,"15min":0.67200000000000004},"min":{"1min":0.47799999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":2.6030000000000002,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":0.59099999999999997}]},{"osd":7,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.65800000000000003,"15min":0.64800000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":3.1339999999999999,"5min":3.1339999999999999,"15min":3.1339999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.64200000000000002,"15min":0.64600000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.94199999999999995,"5min":1.2,"15min":1.2},"last":0.60599999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574880,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31784,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340056,"statfs":{"total":96636764160,"available":96604217344,"internally_reserved":0,"allocated":614400,"data_stored":252207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.33900000000000002,"15min":0.34100000000000003},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.436,"5min":0.48099999999999998,"15min":0.48099999999999998},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.39600000000000002,"15min":0.39800000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.58799999999999997,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.35299999999999998}]},{"osd":2,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.35099999999999998,"15min":0.34899999999999998},"min":{"1min":0.249,"5min":0.246,"15min":0.246},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.36299999999999999,"15min":0.36699999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.53500000000000003,"5min":0.626,"15min":0.626},"last":0.29199999999999998}]},{"osd":3,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.40100000000000002,"15min":0.40200000000000002},"min":{"1min":0.30499999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.375},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.371,"15min":0.376},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.5,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.40600000000000003}]},{"osd":4,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.55400000000000005,"15min":0.54800000000000004},"min":{"1min":0.377,"5min":0.376,"15min":0.376},"max":{"1min":0.77100000000000002,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.746},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.55800000000000005,"15min":0.55100000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.78000000000000003,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.54100000000000004}]},{"osd":5,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.58599999999999997,"15min":0.57899999999999996},"min":{"1min":0.45900000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.96799999999999997,"5min":1.107,"15min":1.107},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57299999999999995,"15min":0.56599999999999995},"min":{"1min":0.437,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.70599999999999996}]},{"osd":6,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60399999999999998,"15min":0.60199999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.76600000000000001,"5min":1.044,"15min":1.044},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.61199999999999999,"15min":0.60599999999999998},"min":{"1min":0.51000000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.85399999999999998,"5min":0.998,"15min":0.998},"last":0.77600000000000002}]},{"osd":7,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58999999999999997,"15min":0.58799999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.78900000000000003,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.60599999999999998,"15min":0.60199999999999998},"min":{"1min":0.45300000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.80000000000000004,"5min":1.151,"15min":1.151},"last":0.55700000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32356,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339484,"statfs":{"total":96636764160,"available":96603631616,"internally_reserved":0,"allocated":1204224,"data_stored":841852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Dec 2 00:30:36 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.26000000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.501,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.37,"15min":0.36499999999999999},"min":{"1min":0.28299999999999997,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.58399999999999996,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.32500000000000001}]},{"osd":2,"last update":"Sat Dec 2 00:30:36 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.54500000000000004,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.312},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40200000000000002,"15min":0.40100000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.372}]},{"osd":3,"last update":"Sat Dec 2 00:30:36 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.39300000000000002,"15min":0.39400000000000002},"min":{"1min":0.29999999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.53100000000000003,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.436,"5min":0.42899999999999999,"15min":0.42799999999999999},"min":{"1min":0.32700000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.34999999999999998}]},{"osd":4,"last update":"Sat Dec 2 00:30:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57399999999999995,"15min":0.56899999999999995},"min":{"1min":0.42299999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.622,"15min":0.62},"min":{"1min":0.439,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.93100000000000005,"5min":1.002,"15min":1.002},"last":0.60799999999999998}]},{"osd":5,"last update":"Sat Dec 2 00:30:36 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.628,"15min":0.626},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.64700000000000002,"15min":0.64600000000000002},"min":{"1min":0.48099999999999998,"5min":0.433,"15min":0.433},"max":{"1min":0.91100000000000003,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.57399999999999995}]},{"osd":6,"last update":"Sat Dec 2 00:30:36 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.60799999999999998,"15min":0.59999999999999998},"min":{"1min":0.435,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.89400000000000002,"5min":1.05,"15min":1.05},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.626,"5min":0.63800000000000001,"15min":0.64000000000000001},"min":{"1min":0.45300000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.94499999999999995,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.47099999999999997}]},{"osd":7,"last update":"Sat Dec 2 00:30:36 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.64300000000000002,"15min":0.63900000000000001},"min":{"1min":0.501,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.97699999999999998,"5min":1.028,"15min":1.028},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.626,"15min":0.626},"min":{"1min":0.46200000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.78000000000000003,"5min":0.874,"15min":0.874},"last":0.51500000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-02T00:31:10.620 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-02T00:31:10.621 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-02T00:31:10.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.768+0000 7f8873b6f640 1 Processor -- start 2023-12-02T00:31:10.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.768+0000 7f8873b6f640 1 -- start start 2023-12-02T00:31:10.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.769+0000 7f8873b6f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f886c0d3850 0x7f886c0b2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:10.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.769+0000 7f8873b6f640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f886c0d00f0 con 0x7f886c0b43a0 2023-12-02T00:31:10.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.769+0000 7f8873b6f640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f886c0d03d0 con 0x7f886c15d420 2023-12-02T00:31:10.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.769+0000 7f8873b6f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f886c0d1420 con 0x7f886c0d3850 2023-12-02T00:31:10.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.769+0000 7f887210c640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f886c0b43a0 0x7f886c0ce320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41810/0 (socket says 172.21.15.46:41810) 2023-12-02T00:31:10.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.769+0000 7f887210c640 1 -- 172.21.15.46:0/3395835348 learned_addr learned my addr 172.21.15.46:0/3395835348 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:31:10.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.770+0000 7f887110a640 1 --2- 172.21.15.46:0/3395835348 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f886c0d3850 0x7f886c0b2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:10.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.770+0000 7f8870909640 1 -- 172.21.15.46:0/3395835348 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3571863868 0 0) 0x7f886c0d00f0 con 0x7f886c0b43a0 2023-12-02T00:31:10.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.770+0000 7f8870909640 1 -- 172.21.15.46:0/3395835348 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8854003520 con 0x7f886c0b43a0 2023-12-02T00:31:10.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.770+0000 7f8870909640 1 -- 172.21.15.46:0/3395835348 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3308932058 0 0) 0x7f886c0d03d0 con 0x7f886c15d420 2023-12-02T00:31:10.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.770+0000 7f8870909640 1 -- 172.21.15.46:0/3395835348 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f886c0d00f0 con 0x7f886c15d420 2023-12-02T00:31:10.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.770+0000 7f8870909640 1 -- 172.21.15.46:0/3395835348 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 413909571 0 0) 0x7f8854003520 con 0x7f886c0b43a0 2023-12-02T00:31:10.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.770+0000 7f8870909640 1 -- 172.21.15.46:0/3395835348 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f886c0d3850 msgr2=0x7f886c0b2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:10.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.770+0000 7f8870909640 1 --2- 172.21.15.46:0/3395835348 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f886c0d3850 0x7f886c0b2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:10.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.770+0000 7f8870909640 1 -- 172.21.15.46:0/3395835348 >> v1:172.21.15.162:6789/0 conn(0x7f886c15d420 legacy=0x7f886c0d3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:10.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.770+0000 7f8870909640 1 -- 172.21.15.46:0/3395835348 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f886c0d06b0 con 0x7f886c0b43a0 2023-12-02T00:31:10.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.770+0000 7f8870909640 1 -- 172.21.15.46:0/3395835348 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7f885c001970 con 0x7f886c0b43a0 2023-12-02T00:31:10.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.770+0000 7f8870909640 1 -- 172.21.15.46:0/3395835348 >> v1:172.21.15.46:6789/0 conn(0x7f886c0b43a0 legacy=0x7f886c0ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:10.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.770+0000 7f8870909640 1 --2- 172.21.15.46:0/3395835348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8854003b30 0x7f8854003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:10.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.770+0000 7f8870909640 1 --2- 172.21.15.46:0/3395835348 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f88540047c0 0x7f8854004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:10.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.770+0000 7f8870909640 1 --2- 172.21.15.46:0/3395835348 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f886c0d3850 0x7f8854005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:10.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.770+0000 7f8870909640 1 -- 172.21.15.46:0/3395835348 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f886c0d1420 con 0x7f8854003b30 2023-12-02T00:31:10.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.770+0000 7f8870909640 1 -- 172.21.15.46:0/3395835348 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885400ba60 con 0x7f88540047c0 2023-12-02T00:31:10.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.770+0000 7f8870909640 1 -- 172.21.15.46:0/3395835348 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f885400bc20 con 0x7f886c0d3850 2023-12-02T00:31:10.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.770+0000 7f887190b640 1 --2- 172.21.15.46:0/3395835348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8854003b30 0x7f8854003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:10.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.770+0000 7f887110a640 1 --2- 172.21.15.46:0/3395835348 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f88540047c0 0x7f8854004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:10.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.770+0000 7f887210c640 1 --2- 172.21.15.46:0/3395835348 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f886c0d3850 0x7f8854005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:10.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.771+0000 7f887190b640 1 -- 172.21.15.46:0/3395835348 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f886c0d3850 msgr2=0x7f8854005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:10.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.771+0000 7f887190b640 1 --2- 172.21.15.46:0/3395835348 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f886c0d3850 0x7f8854005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:10.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.771+0000 7f887190b640 1 -- 172.21.15.46:0/3395835348 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f88540047c0 msgr2=0x7f8854004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:10.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.771+0000 7f887190b640 1 --2- 172.21.15.46:0/3395835348 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f88540047c0 0x7f8854004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:10.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.771+0000 7f887190b640 1 -- 172.21.15.46:0/3395835348 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8854003520 con 0x7f8854003b30 2023-12-02T00:31:10.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.771+0000 7f887190b640 1 --2- 172.21.15.46:0/3395835348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8854003b30 0x7f8854003f30 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f88640018d0 tx=0x7f88540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78a27c0a0472bce0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:10.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.771+0000 7f8870909640 1 -- 172.21.15.46:0/3395835348 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88640052d0 con 0x7f8854003b30 2023-12-02T00:31:10.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.771+0000 7f8870909640 1 -- 172.21.15.46:0/3395835348 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8864005430 con 0x7f8854003b30 2023-12-02T00:31:10.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.771+0000 7f8870909640 1 -- 172.21.15.46:0/3395835348 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8864003810 con 0x7f8854003b30 2023-12-02T00:31:10.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.772+0000 7f8873b6f640 1 -- 172.21.15.46:0/3395835348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8854003b30 msgr2=0x7f8854003f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:10.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.772+0000 7f8873b6f640 1 --2- 172.21.15.46:0/3395835348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8854003b30 0x7f8854003f30 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f88640018d0 tx=0x7f88540057f0 comp rx=0 tx=0).stop 2023-12-02T00:31:10.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.772+0000 7f887210c640 1 -- 172.21.15.46:0/3395835348 reap_dead start 2023-12-02T00:31:10.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.772+0000 7f8873b6f640 1 -- 172.21.15.46:0/3395835348 shutdown_connections 2023-12-02T00:31:10.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.772+0000 7f8873b6f640 1 -- 172.21.15.46:0/3395835348 >> 172.21.15.46:0/3395835348 conn(0x7f886c08b000 msgr2=0x7f886c09c580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:10.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.772+0000 7f8873b6f640 1 -- 172.21.15.46:0/3395835348 shutdown_connections 2023-12-02T00:31:10.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.772+0000 7f8873b6f640 1 -- 172.21.15.46:0/3395835348 wait complete. 2023-12-02T00:31:10.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.773+0000 7f8873b6f640 1 Processor -- start 2023-12-02T00:31:10.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.773+0000 7f8873b6f640 1 -- start start 2023-12-02T00:31:10.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.774+0000 7f8873b6f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8854003b30 0x7f886c14d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:10.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.774+0000 7f8873b6f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f886c0cb330 0x7f886c0cc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:10.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.774+0000 7f8873b6f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f886c0b5dd0 0x7f886c0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:10.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.774+0000 7f8873b6f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f886c0c7cc0 con 0x7f886c0b5dd0 2023-12-02T00:31:10.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.774+0000 7f8873b6f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f886c0b6790 con 0x7f886c0cb330 2023-12-02T00:31:10.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.774+0000 7f8873b6f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f886c0c73b0 con 0x7f8854003b30 2023-12-02T00:31:10.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.774+0000 7f887210c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f886c0b5dd0 0x7f886c0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:10.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.774+0000 7f887210c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f886c0b5dd0 0x7f886c0b6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59778/0 (socket says 172.21.15.46:59778) 2023-12-02T00:31:10.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.774+0000 7f887210c640 1 -- 172.21.15.46:0/1960100562 learned_addr learned my addr 172.21.15.46:0/1960100562 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:10.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.774+0000 7f887110a640 1 --2- 172.21.15.46:0/1960100562 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f886c0cb330 0x7f886c0cc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:10.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.774+0000 7f887190b640 1 --2- 172.21.15.46:0/1960100562 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8854003b30 0x7f886c14d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:10.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.774+0000 7f887210c640 1 -- 172.21.15.46:0/1960100562 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8854003b30 msgr2=0x7f886c14d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:10.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.774+0000 7f887210c640 1 --2- 172.21.15.46:0/1960100562 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8854003b30 0x7f886c14d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:10.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.774+0000 7f887210c640 1 -- 172.21.15.46:0/1960100562 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f886c0cb330 msgr2=0x7f886c0cc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:10.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.774+0000 7f887210c640 1 --2- 172.21.15.46:0/1960100562 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f886c0cb330 0x7f886c0cc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:10.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.774+0000 7f887210c640 1 -- 172.21.15.46:0/1960100562 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f886c14db90 con 0x7f886c0b5dd0 2023-12-02T00:31:10.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.774+0000 7f887190b640 1 --2- 172.21.15.46:0/1960100562 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8854003b30 0x7f886c14d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:31:10.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.775+0000 7f887210c640 1 --2- 172.21.15.46:0/1960100562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f886c0b5dd0 0x7f886c0b6250 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f886c08c6b0 tx=0x7f885c00f5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:10.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.775+0000 7f886a7fc640 1 -- 172.21.15.46:0/1960100562 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885c00fba0 con 0x7f886c0b5dd0 2023-12-02T00:31:10.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.775+0000 7f886a7fc640 1 -- 172.21.15.46:0/1960100562 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f885c00fd00 con 0x7f886c0b5dd0 2023-12-02T00:31:10.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.775+0000 7f8873b6f640 1 -- 172.21.15.46:0/1960100562 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f886c0ccaa0 con 0x7f886c0b5dd0 2023-12-02T00:31:10.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.775+0000 7f886a7fc640 1 -- 172.21.15.46:0/1960100562 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885c013630 con 0x7f886c0b5dd0 2023-12-02T00:31:10.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.775+0000 7f8873b6f640 1 -- 172.21.15.46:0/1960100562 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f886c0c78e0 con 0x7f886c0b5dd0 2023-12-02T00:31:10.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.776+0000 7f8873b6f640 1 -- 172.21.15.46:0/1960100562 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f886c0d2090 con 0x7f886c0b5dd0 2023-12-02T00:31:10.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.776+0000 7f886a7fc640 1 -- 172.21.15.46:0/1960100562 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f885c013790 con 0x7f886c0b5dd0 2023-12-02T00:31:10.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.777+0000 7f886a7fc640 1 --2- 172.21.15.46:0/1960100562 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f885806ae40 0x7f885806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:10.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.777+0000 7f886a7fc640 1 -- 172.21.15.46:0/1960100562 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f885c089920 con 0x7f886c0b5dd0 2023-12-02T00:31:10.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.777+0000 7f887190b640 1 --2- 172.21.15.46:0/1960100562 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f885806ae40 0x7f885806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:10.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.778+0000 7f887190b640 1 --2- 172.21.15.46:0/1960100562 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f885806ae40 0x7f885806d300 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8864034000 tx=0x7f8864031000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:10.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:10.781+0000 7f886a7fc640 1 -- 172.21.15.46:0/1960100562 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f885c0549d0 con 0x7f886c0b5dd0 2023-12-02T00:31:11.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.027+0000 7f8873b6f640 1 -- 172.21.15.46:0/1960100562 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f886c069400 con 0x7f886c0b5dd0 2023-12-02T00:31:11.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.029+0000 7f886a7fc640 1 -- 172.21.15.46:0/1960100562 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+147 (secure 0 0 0) 0x7f885c057ff0 con 0x7f886c0b5dd0 2023-12-02T00:31:11.030 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2023-12-02T00:31:11.030 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAJempl5Lx1DhAAjpK6ajoQ5jQiUr/EMpFeLg== 2023-12-02T00:31:11.030 INFO:teuthology.orchestra.run.smithi046.stdout: caps mds = "allow" 2023-12-02T00:31:11.030 INFO:teuthology.orchestra.run.smithi046.stdout: caps mon = "allow r" 2023-12-02T00:31:11.030 INFO:teuthology.orchestra.run.smithi046.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-12-02T00:31:11.030 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2023-12-02T00:31:11.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.031+0000 7f8873b6f640 1 -- 172.21.15.46:0/1960100562 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f885806ae40 msgr2=0x7f885806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:11.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.031+0000 7f8873b6f640 1 --2- 172.21.15.46:0/1960100562 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f885806ae40 0x7f885806d300 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8864034000 tx=0x7f8864031000 comp rx=0 tx=0).stop 2023-12-02T00:31:11.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.031+0000 7f8873b6f640 1 -- 172.21.15.46:0/1960100562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f886c0b5dd0 msgr2=0x7f886c0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:11.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.031+0000 7f8873b6f640 1 --2- 172.21.15.46:0/1960100562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f886c0b5dd0 0x7f886c0b6250 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f886c08c6b0 tx=0x7f885c00f5a0 comp rx=0 tx=0).stop 2023-12-02T00:31:11.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.031+0000 7f8873b6f640 1 -- 172.21.15.46:0/1960100562 shutdown_connections 2023-12-02T00:31:11.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.031+0000 7f8873b6f640 1 --2- 172.21.15.46:0/1960100562 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f885806ae40 0x7f885806d300 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:11.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.031+0000 7f8873b6f640 1 --2- 172.21.15.46:0/1960100562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f886c0b5dd0 0x7f886c0b6250 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:11.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.031+0000 7f8873b6f640 1 --2- 172.21.15.46:0/1960100562 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f886c0cb330 0x7f886c0cc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:11.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.031+0000 7f8873b6f640 1 --2- 172.21.15.46:0/1960100562 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8854003b30 0x7f886c14d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:11.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.031+0000 7f8873b6f640 1 -- 172.21.15.46:0/1960100562 >> 172.21.15.46:0/1960100562 conn(0x7f886c08b000 msgr2=0x7f886c06b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:11.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.031+0000 7f8873b6f640 1 -- 172.21.15.46:0/1960100562 shutdown_connections 2023-12-02T00:31:11.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.031+0000 7f8873b6f640 1 -- 172.21.15.46:0/1960100562 wait complete. 2023-12-02T00:31:11.043 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-02T00:31:11.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.148+0000 7f9b1a1d0640 1 Processor -- start 2023-12-02T00:31:11.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.148+0000 7f9b1a1d0640 1 -- start start 2023-12-02T00:31:11.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.149+0000 7f9b1a1d0640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9b1415d330 0x7f9b140b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:11.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.149+0000 7f9b1a1d0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b140d2910 0x7f9b140d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:11.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.149+0000 7f9b1a1d0640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b140b2b10 con 0x7f9b140d3730 2023-12-02T00:31:11.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.149+0000 7f9b1a1d0640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b140d3b50 con 0x7f9b1415d330 2023-12-02T00:31:11.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.149+0000 7f9b1a1d0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b140d1590 con 0x7f9b140d2910 2023-12-02T00:31:11.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.149+0000 7f9b12ffd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b140d2910 0x7f9b140d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:11.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.149+0000 7f9b137fe640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9b1415d330 0x7f9b140b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:11.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.150+0000 7f9b12ffd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b140d2910 0x7f9b140d31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.46:47454/0 (socket says 172.21.15.46:47454) 2023-12-02T00:31:11.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.150+0000 7f9b12ffd640 1 -- 172.21.15.46:0/1927655992 learned_addr learned my addr 172.21.15.46:0/1927655992 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:11.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.150+0000 7f9b127fc640 1 -- 172.21.15.46:0/1927655992 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2986092445 0 0) 0x7f9b140b2b10 con 0x7f9b140d3730 2023-12-02T00:31:11.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.150+0000 7f9b12ffd640 1 -- 172.21.15.46:0/1927655992 >> v2:172.21.15.162:3300/0 conn(0x7f9b1415d330 msgr2=0x7f9b140b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:11.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.150+0000 7f9b12ffd640 1 --2- 172.21.15.46:0/1927655992 >> v2:172.21.15.162:3300/0 conn(0x7f9b1415d330 0x7f9b140b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:11.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.150+0000 7f9b12ffd640 1 -- 172.21.15.46:0/1927655992 >> v1:172.21.15.162:6789/0 conn(0x7f9b140d3730 legacy=0x7f9b140b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:11.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.150+0000 7f9b12ffd640 1 -- 172.21.15.46:0/1927655992 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b140d1870 con 0x7f9b140d2910 2023-12-02T00:31:11.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.151+0000 7f9b12ffd640 1 --2- 172.21.15.46:0/1927655992 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b140d2910 0x7f9b140d31f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9b08009870 tx=0x7f9b0802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5a8198f67ae4772 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:11.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.151+0000 7f9b127fc640 1 -- 172.21.15.46:0/1927655992 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b08031640 con 0x7f9b140d2910 2023-12-02T00:31:11.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.151+0000 7f9b127fc640 1 -- 172.21.15.46:0/1927655992 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b080317a0 con 0x7f9b140d2910 2023-12-02T00:31:11.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.151+0000 7f9b127fc640 1 -- 172.21.15.46:0/1927655992 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b08031b30 con 0x7f9b140d2910 2023-12-02T00:31:11.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.151+0000 7f9b1a1d0640 1 -- 172.21.15.46:0/1927655992 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b140d2910 msgr2=0x7f9b140d31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:11.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.151+0000 7f9b1a1d0640 1 --2- 172.21.15.46:0/1927655992 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b140d2910 0x7f9b140d31f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9b08009870 tx=0x7f9b0802b8f0 comp rx=0 tx=0).stop 2023-12-02T00:31:11.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.151+0000 7f9b1a1d0640 1 -- 172.21.15.46:0/1927655992 shutdown_connections 2023-12-02T00:31:11.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.151+0000 7f9b1a1d0640 1 --2- 172.21.15.46:0/1927655992 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b140d2910 0x7f9b140d31f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:11.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.151+0000 7f9b1a1d0640 1 --2- 172.21.15.46:0/1927655992 >> v2:172.21.15.162:3300/0 conn(0x7f9b1415d330 0x7f9b140b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:11.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.151+0000 7f9b1a1d0640 1 -- 172.21.15.46:0/1927655992 >> 172.21.15.46:0/1927655992 conn(0x7f9b14072020 msgr2=0x7f9b14072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:11.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.151+0000 7f9b1a1d0640 1 -- 172.21.15.46:0/1927655992 shutdown_connections 2023-12-02T00:31:11.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.151+0000 7f9b1a1d0640 1 -- 172.21.15.46:0/1927655992 wait complete. 2023-12-02T00:31:11.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.152+0000 7f9b1a1d0640 1 Processor -- start 2023-12-02T00:31:11.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.153+0000 7f9b1a1d0640 1 -- start start 2023-12-02T00:31:11.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.153+0000 7f9b1a1d0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b140d2910 0x7f9b140c9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:11.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.153+0000 7f9b1a1d0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b140d3730 0x7f9b1415d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:11.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.153+0000 7f9b1a1d0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b1415d330 0x7f9b141537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:11.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.153+0000 7f9b1a1d0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b140c8150 con 0x7f9b140d3730 2023-12-02T00:31:11.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.153+0000 7f9b1a1d0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1415dc50 con 0x7f9b140d2910 2023-12-02T00:31:11.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.153+0000 7f9b1a1d0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1415def0 con 0x7f9b1415d330 2023-12-02T00:31:11.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.154+0000 7f9b12ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b140d3730 0x7f9b1415d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:11.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.154+0000 7f9b12ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b140d3730 0x7f9b1415d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59790/0 (socket says 172.21.15.46:59790) 2023-12-02T00:31:11.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.154+0000 7f9b12ffd640 1 -- 172.21.15.46:0/3542339618 learned_addr learned my addr 172.21.15.46:0/3542339618 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:11.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.154+0000 7f9b137fe640 1 --2- 172.21.15.46:0/3542339618 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b140d2910 0x7f9b140c9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:11.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.154+0000 7f9b13fff640 1 --2- 172.21.15.46:0/3542339618 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b1415d330 0x7f9b141537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:11.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.154+0000 7f9b12ffd640 1 -- 172.21.15.46:0/3542339618 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b1415d330 msgr2=0x7f9b141537e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:11.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.154+0000 7f9b12ffd640 1 --2- 172.21.15.46:0/3542339618 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b1415d330 0x7f9b141537e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:11.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.154+0000 7f9b12ffd640 1 -- 172.21.15.46:0/3542339618 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b140d2910 msgr2=0x7f9b140c9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:11.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.154+0000 7f9b12ffd640 1 --2- 172.21.15.46:0/3542339618 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b140d2910 0x7f9b140c9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:11.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.154+0000 7f9b12ffd640 1 -- 172.21.15.46:0/3542339618 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b080094d0 con 0x7f9b140d3730 2023-12-02T00:31:11.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.154+0000 7f9b12ffd640 1 --2- 172.21.15.46:0/3542339618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b140d3730 0x7f9b1415d710 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f9b08034040 tx=0x7f9b080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:11.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.154+0000 7f9afbfff640 1 -- 172.21.15.46:0/3542339618 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b08004230 con 0x7f9b140d3730 2023-12-02T00:31:11.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.154+0000 7f9afbfff640 1 -- 172.21.15.46:0/3542339618 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b080043b0 con 0x7f9b140d3730 2023-12-02T00:31:11.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.154+0000 7f9b1a1d0640 1 -- 172.21.15.46:0/3542339618 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b1415e1d0 con 0x7f9b140d3730 2023-12-02T00:31:11.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.154+0000 7f9afbfff640 1 -- 172.21.15.46:0/3542339618 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b08004520 con 0x7f9b140d3730 2023-12-02T00:31:11.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.154+0000 7f9b1a1d0640 1 -- 172.21.15.46:0/3542339618 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b1415e790 con 0x7f9b140d3730 2023-12-02T00:31:11.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.155+0000 7f9b1a1d0640 1 -- 172.21.15.46:0/3542339618 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b1415f000 con 0x7f9b140d3730 2023-12-02T00:31:11.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.156+0000 7f9afbfff640 1 -- 172.21.15.46:0/3542339618 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9b08042070 con 0x7f9b140d3730 2023-12-02T00:31:11.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.156+0000 7f9afbfff640 1 --2- 172.21.15.46:0/3542339618 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9b0006ae40 0x7f9b0006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:11.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.156+0000 7f9afbfff640 1 -- 172.21.15.46:0/3542339618 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9b080ab1d0 con 0x7f9b140d3730 2023-12-02T00:31:11.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.156+0000 7f9b137fe640 1 --2- 172.21.15.46:0/3542339618 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9b0006ae40 0x7f9b0006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:11.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.157+0000 7f9b137fe640 1 --2- 172.21.15.46:0/3542339618 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9b0006ae40 0x7f9b0006d300 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9b04009b40 tx=0x7f9b04009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:11.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.160+0000 7f9afbfff640 1 -- 172.21.15.46:0/3542339618 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9b1415f000 con 0x7f9b140d3730 2023-12-02T00:31:11.374 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.373+0000 7f9b1a1d0640 1 -- 172.21.15.46:0/3542339618 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9b1415f000 con 0x7f9b140d3730 2023-12-02T00:31:11.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.373+0000 7f9afbfff640 1 -- 172.21.15.46:0/3542339618 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f9b1415f000 con 0x7f9b140d3730 2023-12-02T00:31:11.375 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:31:11.375 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5189,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/4047806587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":4047806587},{"type":"v1","addr":"172.21.15.162:6837","nonce":4047806587}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5620,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/1418783715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1418783715},{"type":"v1","addr":"172.21.15.162:6836","nonce":1418783715}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5979,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/452915985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":452915985},{"type":"v1","addr":"172.21.15.46:6833","nonce":452915985}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-02T00:31:01.187535+0000","modified":"2023-12-02T00:31:04.211234+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5994":{"gid":5994,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/903694874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":903694874},{"type":"v1","addr":"172.21.15.46:6835","nonce":903694874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-12-02T00:31:11.375 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 24 2023-12-02T00:31:11.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.375+0000 7f9b1a1d0640 1 -- 172.21.15.46:0/3542339618 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9b0006ae40 msgr2=0x7f9b0006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:11.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.375+0000 7f9b1a1d0640 1 --2- 172.21.15.46:0/3542339618 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9b0006ae40 0x7f9b0006d300 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9b04009b40 tx=0x7f9b04009190 comp rx=0 tx=0).stop 2023-12-02T00:31:11.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.375+0000 7f9b1a1d0640 1 -- 172.21.15.46:0/3542339618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b140d3730 msgr2=0x7f9b1415d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:11.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.375+0000 7f9b1a1d0640 1 --2- 172.21.15.46:0/3542339618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b140d3730 0x7f9b1415d710 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f9b08034040 tx=0x7f9b080046c0 comp rx=0 tx=0).stop 2023-12-02T00:31:11.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.375+0000 7f9b1a1d0640 1 -- 172.21.15.46:0/3542339618 shutdown_connections 2023-12-02T00:31:11.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.375+0000 7f9b1a1d0640 1 --2- 172.21.15.46:0/3542339618 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9b0006ae40 0x7f9b0006d300 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:11.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.375+0000 7f9b1a1d0640 1 --2- 172.21.15.46:0/3542339618 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b1415d330 0x7f9b141537e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:11.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.375+0000 7f9b1a1d0640 1 --2- 172.21.15.46:0/3542339618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b140d3730 0x7f9b1415d710 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:11.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.375+0000 7f9b1a1d0640 1 --2- 172.21.15.46:0/3542339618 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b140d2910 0x7f9b140c9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:11.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.375+0000 7f9b1a1d0640 1 -- 172.21.15.46:0/3542339618 >> 172.21.15.46:0/3542339618 conn(0x7f9b14072020 msgr2=0x7f9b14072400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:11.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.375+0000 7f9b1a1d0640 1 -- 172.21.15.46:0/3542339618 shutdown_connections 2023-12-02T00:31:11.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.375+0000 7f9b1a1d0640 1 -- 172.21.15.46:0/3542339618 wait complete. 2023-12-02T00:31:11.790 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-02T00:31:01.187535+0000', 'modified': '2023-12-02T00:31:04.211234+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5994}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5994': {'gid': 5994, 'name': 'a', 'rank': 0, 'incarnation': 20, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.46:6835/903694874', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6834', 'nonce': 903694874}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 903694874}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-02T00:31:11.791 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-02T00:31:11.791 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-12-02T00:31:11.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.901+0000 7f87d2875640 1 Processor -- start 2023-12-02T00:31:11.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.902+0000 7f87d2875640 1 -- start start 2023-12-02T00:31:11.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.902+0000 7f87d2875640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f87cc0cf320 0x7f87cc0d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:11.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.902+0000 7f87d2875640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f87cc15f760 0x7f87cc0cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:11.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.902+0000 7f87d2875640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87cc0b3f10 con 0x7f87cc15b130 2023-12-02T00:31:11.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.902+0000 7f87d2875640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87cc15fd20 con 0x7f87cc0cf320 2023-12-02T00:31:11.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.902+0000 7f87d2875640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87cc0d0730 con 0x7f87cc15f760 2023-12-02T00:31:11.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.902+0000 7f87cb7fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f87cc0cf320 0x7f87cc0d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:11.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.902+0000 7f87cbfff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f87cc15b130 0x7f87cc0cec10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41814/0 (socket says 172.21.15.46:41814) 2023-12-02T00:31:11.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.902+0000 7f87cbfff640 1 -- 172.21.15.46:0/2159717056 learned_addr learned my addr 172.21.15.46:0/2159717056 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:31:11.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.903+0000 7f87cb7fe640 1 -- 172.21.15.46:0/2159717056 >> v2:172.21.15.162:3300/0 conn(0x7f87cc15f760 msgr2=0x7f87cc0cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:11.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.903+0000 7f87cb7fe640 1 --2- 172.21.15.46:0/2159717056 >> v2:172.21.15.162:3300/0 conn(0x7f87cc15f760 0x7f87cc0cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:11.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.903+0000 7f87cb7fe640 1 -- 172.21.15.46:0/2159717056 >> v1:172.21.15.46:6789/0 conn(0x7f87cc15b130 legacy=0x7f87cc0cec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:11.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.903+0000 7f87caffd640 1 -- 172.21.15.46:0/2159717056 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1493126198 0 0) 0x7f87cc0b3f10 con 0x7f87cc15b130 2023-12-02T00:31:11.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.903+0000 7f87cb7fe640 1 -- 172.21.15.46:0/2159717056 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87cc0d0a50 con 0x7f87cc0cf320 2023-12-02T00:31:11.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.903+0000 7f87cb7fe640 1 --2- 172.21.15.46:0/2159717056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87cc0cf320 0x7f87cc0d01f0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f87cc06eb70 tx=0x7f87c002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a56489824191d61 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:11.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.904+0000 7f87caffd640 1 -- 172.21.15.46:0/2159717056 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c0031830 con 0x7f87cc0cf320 2023-12-02T00:31:11.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.904+0000 7f87caffd640 1 -- 172.21.15.46:0/2159717056 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87c0031990 con 0x7f87cc0cf320 2023-12-02T00:31:11.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.904+0000 7f87caffd640 1 -- 172.21.15.46:0/2159717056 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c0031d20 con 0x7f87cc0cf320 2023-12-02T00:31:11.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.904+0000 7f87d2875640 1 -- 172.21.15.46:0/2159717056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87cc0cf320 msgr2=0x7f87cc0d01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:11.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.904+0000 7f87d2875640 1 --2- 172.21.15.46:0/2159717056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87cc0cf320 0x7f87cc0d01f0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f87cc06eb70 tx=0x7f87c002b8d0 comp rx=0 tx=0).stop 2023-12-02T00:31:11.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.904+0000 7f87d2875640 1 -- 172.21.15.46:0/2159717056 shutdown_connections 2023-12-02T00:31:11.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.904+0000 7f87d2875640 1 --2- 172.21.15.46:0/2159717056 >> v2:172.21.15.162:3300/0 conn(0x7f87cc15f760 0x7f87cc0cd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:11.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.904+0000 7f87d2875640 1 --2- 172.21.15.46:0/2159717056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87cc0cf320 0x7f87cc0d01f0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:11.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.904+0000 7f87d2875640 1 -- 172.21.15.46:0/2159717056 >> 172.21.15.46:0/2159717056 conn(0x7f87cc0b2c10 msgr2=0x7f87cc0b2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:11.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.904+0000 7f87d2875640 1 -- 172.21.15.46:0/2159717056 shutdown_connections 2023-12-02T00:31:11.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.904+0000 7f87d2875640 1 -- 172.21.15.46:0/2159717056 wait complete. 2023-12-02T00:31:11.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.905+0000 7f87d2875640 1 Processor -- start 2023-12-02T00:31:11.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.906+0000 7f87d2875640 1 -- start start 2023-12-02T00:31:11.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.906+0000 7f87d2875640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f87cc0cf320 0x7f87cc1f3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:11.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.906+0000 7f87d2875640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87cc15b130 0x7f87cc0ca680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:11.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.906+0000 7f87d2875640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87cc15f760 0x7f87cc0c74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:11.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.906+0000 7f87d2875640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc0ca1f0 con 0x7f87cc15f760 2023-12-02T00:31:11.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.906+0000 7f87d2875640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc0cabc0 con 0x7f87cc15b130 2023-12-02T00:31:11.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.906+0000 7f87d2875640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc0c95e0 con 0x7f87cc0cf320 2023-12-02T00:31:11.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.906+0000 7f87d0e12640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87cc15f760 0x7f87cc0c74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:11.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.906+0000 7f87d0e12640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87cc15f760 0x7f87cc0c74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59806/0 (socket says 172.21.15.46:59806) 2023-12-02T00:31:11.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.906+0000 7f87d0e12640 1 -- 172.21.15.46:0/3467990216 learned_addr learned my addr 172.21.15.46:0/3467990216 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:11.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.907+0000 7f87cb7fe640 1 --2- 172.21.15.46:0/3467990216 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87cc15b130 0x7f87cc0ca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:11.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.907+0000 7f87cbfff640 1 --2- 172.21.15.46:0/3467990216 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f87cc0cf320 0x7f87cc1f3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:11.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.907+0000 7f87d0e12640 1 -- 172.21.15.46:0/3467990216 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f87cc0cf320 msgr2=0x7f87cc1f3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:11.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.907+0000 7f87d0e12640 1 --2- 172.21.15.46:0/3467990216 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f87cc0cf320 0x7f87cc1f3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:11.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.907+0000 7f87d0e12640 1 -- 172.21.15.46:0/3467990216 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87cc15b130 msgr2=0x7f87cc0ca680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:11.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.907+0000 7f87d0e12640 1 --2- 172.21.15.46:0/3467990216 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87cc15b130 0x7f87cc0ca680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:11.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.907+0000 7f87d0e12640 1 -- 172.21.15.46:0/3467990216 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c0009400 con 0x7f87cc15f760 2023-12-02T00:31:11.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.907+0000 7f87d0e12640 1 --2- 172.21.15.46:0/3467990216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87cc15f760 0x7f87cc0c74e0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f87c400b970 tx=0x7f87c400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:11.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.907+0000 7f87c8ff9640 1 -- 172.21.15.46:0/3467990216 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c400cb40 con 0x7f87cc15f760 2023-12-02T00:31:11.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.907+0000 7f87c8ff9640 1 -- 172.21.15.46:0/3467990216 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87c400ccc0 con 0x7f87cc15f760 2023-12-02T00:31:11.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.907+0000 7f87d2875640 1 -- 172.21.15.46:0/3467990216 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87cc0c98c0 con 0x7f87cc15f760 2023-12-02T00:31:11.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.907+0000 7f87c8ff9640 1 -- 172.21.15.46:0/3467990216 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c4014680 con 0x7f87cc15f760 2023-12-02T00:31:11.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.907+0000 7f87d2875640 1 -- 172.21.15.46:0/3467990216 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87cc0c9e80 con 0x7f87cc15f760 2023-12-02T00:31:11.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.909+0000 7f87c8ff9640 1 -- 172.21.15.46:0/3467990216 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f87c401d3b0 con 0x7f87cc15f760 2023-12-02T00:31:11.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.909+0000 7f87c8ff9640 1 --2- 172.21.15.46:0/3467990216 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f87b806ae40 0x7f87b806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:11.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.909+0000 7f87c8ff9640 1 -- 172.21.15.46:0/3467990216 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f87c40894a0 con 0x7f87cc15f760 2023-12-02T00:31:11.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.909+0000 7f87cbfff640 1 --2- 172.21.15.46:0/3467990216 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f87b806ae40 0x7f87b806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:11.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.910+0000 7f87cbfff640 1 --2- 172.21.15.46:0/3467990216 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f87b806ae40 0x7f87b806d300 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f87bc000f60 tx=0x7f87bc007d50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:11.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.910+0000 7f87d5406740 1 Processor -- start 2023-12-02T00:31:11.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.910+0000 7f87d5406740 1 -- start start 2023-12-02T00:31:11.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.911+0000 7f87d5406740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555a7a02fc50 0x555a7a002bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:11.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.911+0000 7f87d5406740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555a7a030030 0x555a7a0329e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:11.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.911+0000 7f87d5406740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a7a003130 0x555a7a032f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:11.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.911+0000 7f87d5406740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a79fc3980 con 0x555a7a003130 2023-12-02T00:31:11.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.911+0000 7f87d5406740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a79f75b90 con 0x555a7a02fc50 2023-12-02T00:31:11.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.911+0000 7f87d5406740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a79f73280 con 0x555a7a030030 2023-12-02T00:31:11.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.911+0000 7f87d0e12640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a7a003130 0x555a7a032f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:11.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.911+0000 7f87d0e12640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a7a003130 0x555a7a032f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59816/0 (socket says 172.21.15.46:59816) 2023-12-02T00:31:11.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.911+0000 7f87d0e12640 1 -- 172.21.15.46:0/676795318 learned_addr learned my addr 172.21.15.46:0/676795318 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:11.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.911+0000 7f87cb7fe640 1 --2- 172.21.15.46:0/676795318 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555a7a030030 0x555a7a0329e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:11.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.911+0000 7f87cbfff640 1 --2- 172.21.15.46:0/676795318 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555a7a02fc50 0x555a7a002bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:11.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.911+0000 7f87d0e12640 1 -- 172.21.15.46:0/676795318 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555a7a030030 msgr2=0x555a7a0329e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:11.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.911+0000 7f87d0e12640 1 --2- 172.21.15.46:0/676795318 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555a7a030030 0x555a7a0329e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:11.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.911+0000 7f87d0e12640 1 -- 172.21.15.46:0/676795318 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555a7a02fc50 msgr2=0x555a7a002bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:11.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.911+0000 7f87d0e12640 1 --2- 172.21.15.46:0/676795318 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555a7a02fc50 0x555a7a002bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:11.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.911+0000 7f87d0e12640 1 -- 172.21.15.46:0/676795318 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a79f34500 con 0x555a7a003130 2023-12-02T00:31:11.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.911+0000 7f87cbfff640 1 --2- 172.21.15.46:0/676795318 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555a7a02fc50 0x555a7a002bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:31:11.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.911+0000 7f87d0e12640 1 --2- 172.21.15.46:0/676795318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a7a003130 0x555a7a032f20 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f87c4087970 tx=0x7f87c400cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cebbda2fe2f7066c server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:11.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.911+0000 7f87b1ffb640 1 -- 172.21.15.46:0/676795318 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c405c120 con 0x555a7a003130 2023-12-02T00:31:11.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.911+0000 7f87b1ffb640 1 -- 172.21.15.46:0/676795318 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87c4060510 con 0x555a7a003130 2023-12-02T00:31:11.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.911+0000 7f87b1ffb640 1 -- 172.21.15.46:0/676795318 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c4056660 con 0x555a7a003130 2023-12-02T00:31:11.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.912+0000 7f87d5406740 1 -- 172.21.15.46:0/676795318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a7a003130 msgr2=0x555a7a032f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:11.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.912+0000 7f87d5406740 1 --2- 172.21.15.46:0/676795318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a7a003130 0x555a7a032f20 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f87c4087970 tx=0x7f87c400cf70 comp rx=0 tx=0).stop 2023-12-02T00:31:11.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.912+0000 7f87d5406740 1 -- 172.21.15.46:0/676795318 shutdown_connections 2023-12-02T00:31:11.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.912+0000 7f87d5406740 1 --2- 172.21.15.46:0/676795318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a7a003130 0x555a7a032f20 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:11.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.912+0000 7f87d5406740 1 --2- 172.21.15.46:0/676795318 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555a7a030030 0x555a7a0329e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:11.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.912+0000 7f87d5406740 1 --2- 172.21.15.46:0/676795318 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555a7a02fc50 0x555a7a002bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:11.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.912+0000 7f87d5406740 1 -- 172.21.15.46:0/676795318 >> 172.21.15.46:0/676795318 conn(0x555a7a004e60 msgr2=0x555a7a04d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:11.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.912+0000 7f87d5406740 1 -- 172.21.15.46:0/676795318 shutdown_connections 2023-12-02T00:31:11.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.912+0000 7f87d5406740 1 -- 172.21.15.46:0/676795318 wait complete. 2023-12-02T00:31:11.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.913+0000 7f87d5406740 1 -- start start 2023-12-02T00:31:11.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.913+0000 7f87d5406740 1 Processor -- start 2023-12-02T00:31:11.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.914+0000 7f87d5406740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555a7a02fc50 0x555a7a02b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:11.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.914+0000 7f87d5406740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a7a030030 0x555a7a061c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:11.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.914+0000 7f87d5406740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555a7a05e980 0x555a7a02e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:11.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.914+0000 7f87d5406740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a79f6eb70 con 0x555a7a030030 2023-12-02T00:31:11.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.914+0000 7f87d5406740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a79f28e00 con 0x555a7a02fc50 2023-12-02T00:31:11.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.914+0000 7f87d5406740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a79f28aa0 con 0x555a7a05e980 2023-12-02T00:31:11.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.914+0000 7f87d5406740 10 client.0 _ll_register_callbacks cb 0x555a79c80ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-02T00:31:11.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.914+0000 7f87cb7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a7a030030 0x555a7a061c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:11.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.914+0000 7f87cb7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a7a030030 0x555a7a061c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59824/0 (socket says 172.21.15.46:59824) 2023-12-02T00:31:11.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.914+0000 7f87cb7fe640 1 -- 172.21.15.46:0/4073737538 learned_addr learned my addr 172.21.15.46:0/4073737538 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:11.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.914+0000 7f87d0e12640 1 --2- 172.21.15.46:0/4073737538 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555a7a05e980 0x555a7a02e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:11.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.914+0000 7f87cbfff640 1 --2- 172.21.15.46:0/4073737538 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555a7a02fc50 0x555a7a02b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:11.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.914+0000 7f87cb7fe640 1 -- 172.21.15.46:0/4073737538 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555a7a05e980 msgr2=0x555a7a02e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:11.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.914+0000 7f87cb7fe640 1 --2- 172.21.15.46:0/4073737538 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555a7a05e980 0x555a7a02e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:11.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.914+0000 7f87cb7fe640 1 -- 172.21.15.46:0/4073737538 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555a7a02fc50 msgr2=0x555a7a02b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:11.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.914+0000 7f87cb7fe640 1 --2- 172.21.15.46:0/4073737538 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555a7a02fc50 0x555a7a02b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:11.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.914+0000 7f87cb7fe640 1 -- 172.21.15.46:0/4073737538 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87cc0c9e80 con 0x555a7a030030 2023-12-02T00:31:11.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.914+0000 7f87cb7fe640 1 --2- 172.21.15.46:0/4073737538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a7a030030 0x555a7a061c90 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f87c00098c0 tx=0x7f87c0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71d74fd9961b1e63 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:11.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.914+0000 7f87aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2023-12-02T00:31:11.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.914+0000 7f87aaffd640 1 -- 172.21.15.46:0/4073737538 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c0041040 con 0x555a7a030030 2023-12-02T00:31:11.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.914+0000 7f87aaffd640 1 -- 172.21.15.46:0/4073737538 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f87c0003610 con 0x555a7a030030 2023-12-02T00:31:11.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.914+0000 7f87d5406740 1 -- 172.21.15.46:0/4073737538 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555a79f67b50 con 0x555a7a030030 2023-12-02T00:31:11.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.914+0000 7f87aaffd640 1 -- 172.21.15.46:0/4073737538 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87c0004510 con 0x555a7a030030 2023-12-02T00:31:11.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.914+0000 7f87d5406740 10 client.6156 fetch_fsmap learned FSMap version 24 2023-12-02T00:31:11.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.915+0000 7f87d5406740 1 -- 172.21.15.46:0/4073737538 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x555a79f6d1f0 con 0x555a7a030030 2023-12-02T00:31:11.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.915+0000 7f87aaffd640 1 -- 172.21.15.46:0/4073737538 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555a79f67b50 con 0x555a7a030030 2023-12-02T00:31:11.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.915+0000 7f87aaffd640 1 -- 172.21.15.46:0/4073737538 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f87c00037d0 con 0x555a7a030030 2023-12-02T00:31:11.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.915+0000 7f87d5406740 10 client.6156 fetch_fsmap finished waiting for FSMap version 24 2023-12-02T00:31:11.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.915+0000 7f87d5406740 10 client.6156 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-02T00:31:11.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.915+0000 7f87d5406740 20 client.6156 populate_metadata read hostname 'smithi046' 2023-12-02T00:31:11.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.915+0000 7f87d5406740 1 --2- 172.21.15.46:0/4073737538 >> [v2:172.21.15.46:6834/903694874,v1:172.21.15.46:6835/903694874] conn(0x555a7a05f6b0 0x555a7a031a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:11.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.915+0000 7f87d5406740 4 client.6156 mds_command: new command op to 5994 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-02T00:31:11.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.915+0000 7f87d5406740 1 -- 172.21.15.46:0/4073737538 --> [v2:172.21.15.46:6834/903694874,v1:172.21.15.46:6835/903694874] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555a79f67870 con 0x555a7a05f6b0 2023-12-02T00:31:11.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.915+0000 7f87cbfff640 1 --2- 172.21.15.46:0/4073737538 >> [v2:172.21.15.46:6834/903694874,v1:172.21.15.46:6835/903694874] conn(0x555a7a05f6b0 0x555a7a031a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:11.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.915+0000 7f87aaffd640 10 client.6156 ms_handle_connect on v2:172.21.15.46:6834/903694874 2023-12-02T00:31:11.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.915+0000 7f87cbfff640 1 --2- 172.21.15.46:0/4073737538 >> [v2:172.21.15.46:6834/903694874,v1:172.21.15.46:6835/903694874] conn(0x555a7a05f6b0 0x555a7a031a10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a82dc198077057e server_cookie=33a66d7698200008 in_seq=0 out_seq=0 2023-12-02T00:31:11.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.916+0000 7f87aaffd640 1 -- 172.21.15.46:0/4073737538 <== mds.0 v2:172.21.15.46:6834/903694874 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555a79f67870 con 0x555a7a05f6b0 2023-12-02T00:31:11.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.916+0000 7f87aaffd640 10 client.6156 handle_command_reply: tid=0 2023-12-02T00:31:11.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.916+0000 7f87d5406740 1 client.6156 shutdown 2023-12-02T00:31:11.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.916+0000 7f87d5406740 1 -- 172.21.15.46:0/4073737538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a7a030030 msgr2=0x555a7a061c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:11.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.916+0000 7f87d5406740 1 --2- 172.21.15.46:0/4073737538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a7a030030 0x555a7a061c90 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f87c00098c0 tx=0x7f87c0004730 comp rx=0 tx=0).stop 2023-12-02T00:31:11.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.916+0000 7f87d5406740 1 -- 172.21.15.46:0/4073737538 shutdown_connections 2023-12-02T00:31:11.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.916+0000 7f87d5406740 1 --2- 172.21.15.46:0/4073737538 >> [v2:172.21.15.46:6834/903694874,v1:172.21.15.46:6835/903694874] conn(0x555a7a05f6b0 0x555a7a031a10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:11.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.916+0000 7f87d5406740 1 --2- 172.21.15.46:0/4073737538 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555a7a05e980 0x555a7a02e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:11.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.916+0000 7f87d5406740 1 --2- 172.21.15.46:0/4073737538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a7a030030 0x555a7a061c90 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:11.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.916+0000 7f87aaffd640 0 client.6156 ms_handle_reset on v2:172.21.15.46:6834/903694874 2023-12-02T00:31:11.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.916+0000 7f87d5406740 1 --2- 172.21.15.46:0/4073737538 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555a7a02fc50 0x555a7a02b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:11.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.916+0000 7f87d5406740 1 -- 172.21.15.46:0/4073737538 >> 172.21.15.46:0/4073737538 conn(0x555a7a003130 msgr2=0x555a7a050ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:11.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.916+0000 7f87d5406740 1 -- 172.21.15.46:0/4073737538 shutdown_connections 2023-12-02T00:31:11.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.916+0000 7f87d5406740 1 -- 172.21.15.46:0/4073737538 wait complete. 2023-12-02T00:31:11.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.916+0000 7f87d5406740 20 client.6156 trim_cache size 0 max 16384 2023-12-02T00:31:11.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.932+0000 7f87d5406740 1 Processor -- start 2023-12-02T00:31:11.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.932+0000 7f87d5406740 1 -- start start 2023-12-02T00:31:11.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.933+0000 7f87d5406740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a7a02fc50 0x555a7a0089f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:11.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.933+0000 7f87d5406740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555a7a030030 0x555a7a02df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:11.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.933+0000 7f87d5406740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555a7a05e980 0x555a7a02e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:11.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.933+0000 7f87d5406740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a79eee270 con 0x555a7a02fc50 2023-12-02T00:31:11.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.933+0000 7f87d5406740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a7a05df10 con 0x555a7a030030 2023-12-02T00:31:11.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.933+0000 7f87d5406740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a7a051900 con 0x555a7a05e980 2023-12-02T00:31:11.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.933+0000 7f87cbfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a7a02fc50 0x555a7a0089f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:11.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.933+0000 7f87cbfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a7a02fc50 0x555a7a0089f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59828/0 (socket says 172.21.15.46:59828) 2023-12-02T00:31:11.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.933+0000 7f87cbfff640 1 -- 172.21.15.46:0/1870946916 learned_addr learned my addr 172.21.15.46:0/1870946916 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:11.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.933+0000 7f87d0e12640 1 --2- 172.21.15.46:0/1870946916 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555a7a05e980 0x555a7a02e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:11.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.933+0000 7f87cb7fe640 1 --2- 172.21.15.46:0/1870946916 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555a7a030030 0x555a7a02df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:11.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.933+0000 7f87cbfff640 1 -- 172.21.15.46:0/1870946916 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555a7a05e980 msgr2=0x555a7a02e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:11.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.933+0000 7f87cbfff640 1 --2- 172.21.15.46:0/1870946916 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555a7a05e980 0x555a7a02e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:11.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.933+0000 7f87cbfff640 1 -- 172.21.15.46:0/1870946916 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555a7a030030 msgr2=0x555a7a02df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:11.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.933+0000 7f87cbfff640 1 --2- 172.21.15.46:0/1870946916 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555a7a030030 0x555a7a02df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:11.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.933+0000 7f87cbfff640 1 -- 172.21.15.46:0/1870946916 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a79f0ad00 con 0x555a7a02fc50 2023-12-02T00:31:11.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.933+0000 7f87cbfff640 1 --2- 172.21.15.46:0/1870946916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a7a02fc50 0x555a7a0089f0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x555a7a044980 tx=0x7f87bc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18aeb3af88731a33 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:11.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.934+0000 7f87b27fc640 1 -- 172.21.15.46:0/1870946916 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87bc001cb0 con 0x555a7a02fc50 2023-12-02T00:31:11.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.934+0000 7f87b27fc640 1 -- 172.21.15.46:0/1870946916 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87bc0260b0 con 0x555a7a02fc50 2023-12-02T00:31:11.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.934+0000 7f87b27fc640 1 -- 172.21.15.46:0/1870946916 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87bc026220 con 0x555a7a02fc50 2023-12-02T00:31:11.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.934+0000 7f87d5406740 1 -- 172.21.15.46:0/1870946916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a7a02fc50 msgr2=0x555a7a0089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:11.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.934+0000 7f87d5406740 1 --2- 172.21.15.46:0/1870946916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a7a02fc50 0x555a7a0089f0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x555a7a044980 tx=0x7f87bc013000 comp rx=0 tx=0).stop 2023-12-02T00:31:11.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.934+0000 7f87d5406740 1 -- 172.21.15.46:0/1870946916 shutdown_connections 2023-12-02T00:31:11.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.934+0000 7f87d5406740 1 --2- 172.21.15.46:0/1870946916 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555a7a05e980 0x555a7a02e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:11.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.934+0000 7f87d5406740 1 --2- 172.21.15.46:0/1870946916 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555a7a030030 0x555a7a02df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:11.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.934+0000 7f87d5406740 1 --2- 172.21.15.46:0/1870946916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a7a02fc50 0x555a7a0089f0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:11.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.934+0000 7f87d5406740 1 -- 172.21.15.46:0/1870946916 >> 172.21.15.46:0/1870946916 conn(0x555a7a003130 msgr2=0x555a7a070a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:11.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.934+0000 7f87d5406740 1 -- 172.21.15.46:0/1870946916 shutdown_connections 2023-12-02T00:31:11.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.934+0000 7f87d5406740 1 -- 172.21.15.46:0/1870946916 wait complete. 2023-12-02T00:31:11.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.935+0000 7f87d5406740 1 -- start start 2023-12-02T00:31:11.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.935+0000 7f87d5406740 1 Processor -- start 2023-12-02T00:31:11.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.936+0000 7f87d5406740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555a7a02fc50 0x555a7a0bcaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:11.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.936+0000 7f87d5406740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555a7a030030 0x555a7a0089f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:11.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.936+0000 7f87d5406740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a7a05e980 0x555a7a008f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:11.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.936+0000 7f87d5406740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a7a02e870 con 0x555a7a05e980 2023-12-02T00:31:11.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.936+0000 7f87d5406740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a7a05c360 con 0x555a7a02fc50 2023-12-02T00:31:11.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.936+0000 7f87d5406740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a7a044050 con 0x555a7a030030 2023-12-02T00:31:11.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.936+0000 7f87d5406740 10 client.0 _ll_register_callbacks cb 0x7f87bc007bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-02T00:31:11.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.936+0000 7f87d0e12640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a7a05e980 0x555a7a008f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:11.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.936+0000 7f87d0e12640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a7a05e980 0x555a7a008f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59830/0 (socket says 172.21.15.46:59830) 2023-12-02T00:31:11.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.936+0000 7f87d0e12640 1 -- 172.21.15.46:0/2542268699 learned_addr learned my addr 172.21.15.46:0/2542268699 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:11.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.936+0000 7f87cb7fe640 1 --2- 172.21.15.46:0/2542268699 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555a7a030030 0x555a7a0089f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:11.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.936+0000 7f87cbfff640 1 --2- 172.21.15.46:0/2542268699 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555a7a02fc50 0x555a7a0bcaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:11.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.936+0000 7f87d0e12640 1 -- 172.21.15.46:0/2542268699 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555a7a030030 msgr2=0x555a7a0089f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:11.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.936+0000 7f87d0e12640 1 --2- 172.21.15.46:0/2542268699 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555a7a030030 0x555a7a0089f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:11.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.936+0000 7f87d0e12640 1 -- 172.21.15.46:0/2542268699 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555a7a02fc50 msgr2=0x555a7a0bcaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:11.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.936+0000 7f87d0e12640 1 --2- 172.21.15.46:0/2542268699 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555a7a02fc50 0x555a7a0bcaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:11.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.936+0000 7f87d0e12640 1 -- 172.21.15.46:0/2542268699 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87bc001920 con 0x555a7a05e980 2023-12-02T00:31:11.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.936+0000 7f87d0e12640 1 --2- 172.21.15.46:0/2542268699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a7a05e980 0x555a7a008f30 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f87c4057fc0 tx=0x7f87c400e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1115c54c944180ca server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:11.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.936+0000 7f87aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2023-12-02T00:31:11.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.936+0000 7f87aaffd640 1 -- 172.21.15.46:0/2542268699 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c405c120 con 0x555a7a05e980 2023-12-02T00:31:11.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.937+0000 7f87aaffd640 1 -- 172.21.15.46:0/2542268699 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f87c4086040 con 0x555a7a05e980 2023-12-02T00:31:11.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.937+0000 7f87d5406740 1 -- 172.21.15.46:0/2542268699 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f87c4060510 con 0x555a7a05e980 2023-12-02T00:31:11.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.937+0000 7f87aaffd640 1 -- 172.21.15.46:0/2542268699 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87c40890c0 con 0x555a7a05e980 2023-12-02T00:31:11.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.937+0000 7f87d5406740 10 client.6162 fetch_fsmap learned FSMap version 24 2023-12-02T00:31:11.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.937+0000 7f87d5406740 1 -- 172.21.15.46:0/2542268699 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x555a79f304f0 con 0x555a7a05e980 2023-12-02T00:31:11.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.937+0000 7f87aaffd640 1 -- 172.21.15.46:0/2542268699 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87c4060510 con 0x555a7a05e980 2023-12-02T00:31:11.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.937+0000 7f87aaffd640 1 -- 172.21.15.46:0/2542268699 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f87c40577b0 con 0x555a7a05e980 2023-12-02T00:31:11.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.937+0000 7f87d5406740 10 client.6162 fetch_fsmap finished waiting for FSMap version 24 2023-12-02T00:31:11.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.937+0000 7f87d5406740 10 client.6162 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-02T00:31:11.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.937+0000 7f87d5406740 20 client.6162 populate_metadata read hostname 'smithi046' 2023-12-02T00:31:11.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.937+0000 7f87d5406740 1 --2- 172.21.15.46:0/2542268699 >> [v2:172.21.15.46:6834/903694874,v1:172.21.15.46:6835/903694874] conn(0x555a7a036330 0x555a7a036710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:11.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.937+0000 7f87d5406740 4 client.6162 mds_command: new command op to 5994 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-02T00:31:11.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.937+0000 7f87d5406740 1 -- 172.21.15.46:0/2542268699 --> [v2:172.21.15.46:6834/903694874,v1:172.21.15.46:6835/903694874] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555a79c6f6c0 con 0x555a7a036330 2023-12-02T00:31:11.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.937+0000 7f87cbfff640 1 --2- 172.21.15.46:0/2542268699 >> [v2:172.21.15.46:6834/903694874,v1:172.21.15.46:6835/903694874] conn(0x555a7a036330 0x555a7a036710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:11.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.937+0000 7f87cbfff640 1 --2- 172.21.15.46:0/2542268699 >> [v2:172.21.15.46:6834/903694874,v1:172.21.15.46:6835/903694874] conn(0x555a7a036330 0x555a7a036710 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8907cb2a9939d1e3 server_cookie=5112841268f0fe9b in_seq=0 out_seq=0 2023-12-02T00:31:11.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.937+0000 7f87aaffd640 10 client.6162 ms_handle_connect on v2:172.21.15.46:6834/903694874 2023-12-02T00:31:11.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.937+0000 7f87aaffd640 1 -- 172.21.15.46:0/2542268699 <== mds.0 v2:172.21.15.46:6834/903694874 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x555a79c6f6c0 con 0x555a7a036330 2023-12-02T00:31:11.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.938+0000 7f87aaffd640 10 client.6162 handle_command_reply: tid=0 2023-12-02T00:31:11.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.938+0000 7f87d5406740 1 client.6162 shutdown 2023-12-02T00:31:11.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.938+0000 7f87d5406740 1 -- 172.21.15.46:0/2542268699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a7a05e980 msgr2=0x555a7a008f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:11.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.938+0000 7f87d5406740 1 --2- 172.21.15.46:0/2542268699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a7a05e980 0x555a7a008f30 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f87c4057fc0 tx=0x7f87c400e040 comp rx=0 tx=0).stop 2023-12-02T00:31:11.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.938+0000 7f87d5406740 1 -- 172.21.15.46:0/2542268699 shutdown_connections 2023-12-02T00:31:11.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.938+0000 7f87d5406740 1 --2- 172.21.15.46:0/2542268699 >> [v2:172.21.15.46:6834/903694874,v1:172.21.15.46:6835/903694874] conn(0x555a7a036330 0x555a7a036710 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:11.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.938+0000 7f87d5406740 1 --2- 172.21.15.46:0/2542268699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a7a05e980 0x555a7a008f30 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:11.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.938+0000 7f87d5406740 1 --2- 172.21.15.46:0/2542268699 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555a7a030030 0x555a7a0089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:11.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.938+0000 7f87aaffd640 0 client.6162 ms_handle_reset on v2:172.21.15.46:6834/903694874 2023-12-02T00:31:11.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.938+0000 7f87d5406740 1 --2- 172.21.15.46:0/2542268699 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555a7a02fc50 0x555a7a0bcaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:11.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.938+0000 7f87d5406740 1 -- 172.21.15.46:0/2542268699 >> 172.21.15.46:0/2542268699 conn(0x555a7a003130 msgr2=0x555a7a033040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:11.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.938+0000 7f87d5406740 1 -- 172.21.15.46:0/2542268699 shutdown_connections 2023-12-02T00:31:11.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.938+0000 7f87d5406740 1 -- 172.21.15.46:0/2542268699 wait complete. 2023-12-02T00:31:11.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.938+0000 7f87d5406740 20 client.6162 trim_cache size 0 max 16384 2023-12-02T00:31:11.945 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-12-02T00:31:11.945 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "6d588070-5443-40f5-81f6-108a1e3f56c3", 2023-12-02T00:31:11.945 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2023-12-02T00:31:11.945 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 5994, 2023-12-02T00:31:11.945 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2023-12-02T00:31:11.945 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2023-12-02T00:31:11.945 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2023-12-02T00:31:11.945 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 10.100862597000001, 2023-12-02T00:31:11.945 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 24, 2023-12-02T00:31:11.945 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 51, 2023-12-02T00:31:11.946 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 51, 2023-12-02T00:31:11.946 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 15.183048170999999 2023-12-02T00:31:11.946 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-12-02T00:31:11.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.938+0000 7f87d2875640 1 -- 172.21.15.46:0/3467990216 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f87b806ae40 msgr2=0x7f87b806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:11.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.938+0000 7f87d2875640 1 --2- 172.21.15.46:0/3467990216 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f87b806ae40 0x7f87b806d300 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f87bc000f60 tx=0x7f87bc007d50 comp rx=0 tx=0).stop 2023-12-02T00:31:11.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.938+0000 7f87d2875640 1 -- 172.21.15.46:0/3467990216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87cc15f760 msgr2=0x7f87cc0c74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:11.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.938+0000 7f87d2875640 1 --2- 172.21.15.46:0/3467990216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87cc15f760 0x7f87cc0c74e0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f87c400b970 tx=0x7f87c400be30 comp rx=0 tx=0).stop 2023-12-02T00:31:11.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.938+0000 7f87d2875640 1 -- 172.21.15.46:0/3467990216 shutdown_connections 2023-12-02T00:31:11.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.938+0000 7f87d2875640 1 --2- 172.21.15.46:0/3467990216 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f87b806ae40 0x7f87b806d300 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:11.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.938+0000 7f87d2875640 1 --2- 172.21.15.46:0/3467990216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87cc15f760 0x7f87cc0c74e0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:11.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.938+0000 7f87d2875640 1 --2- 172.21.15.46:0/3467990216 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87cc15b130 0x7f87cc0ca680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:11.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.938+0000 7f87d2875640 1 --2- 172.21.15.46:0/3467990216 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f87cc0cf320 0x7f87cc1f3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:11.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.938+0000 7f87d2875640 1 -- 172.21.15.46:0/3467990216 >> 172.21.15.46:0/3467990216 conn(0x7f87cc0b2c10 msgr2=0x7f87cc15a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:11.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.939+0000 7f87d2875640 1 -- 172.21.15.46:0/3467990216 shutdown_connections 2023-12-02T00:31:11.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:11.939+0000 7f87d2875640 1 -- 172.21.15.46:0/3467990216 wait complete. 2023-12-02T00:31:11.950 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-02T00:31:12.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.056+0000 7f0152e10640 1 Processor -- start 2023-12-02T00:31:12.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.056+0000 7f0152e10640 1 -- start start 2023-12-02T00:31:12.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.057+0000 7f0152e10640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f014c13d080 0x7f014c15afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:12.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.057+0000 7f0152e10640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f014c0cf570 0x7f014c06b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:12.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.057+0000 7f0152e10640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f014c15b500 con 0x7f014c06b4e0 2023-12-02T00:31:12.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.057+0000 7f0152e10640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f014c13d460 con 0x7f014c0cf570 2023-12-02T00:31:12.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.057+0000 7f0152e10640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f014c06bea0 con 0x7f014c13d080 2023-12-02T00:31:12.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.057+0000 7f0150bac640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f014c13d080 0x7f014c15afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:12.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.057+0000 7f01513ad640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f014c0cf570 0x7f014c06b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:12.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.057+0000 7f0150bac640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f014c13d080 0x7f014c15afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.46:47498/0 (socket says 172.21.15.46:47498) 2023-12-02T00:31:12.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.057+0000 7f01513ad640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f014c0cf570 0x7f014c06b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.46:45250/0 (socket says 172.21.15.46:45250) 2023-12-02T00:31:12.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.057+0000 7f0150bac640 1 -- 172.21.15.46:0/4208129706 learned_addr learned my addr 172.21.15.46:0/4208129706 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:12.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.057+0000 7f0150bac640 1 -- 172.21.15.46:0/4208129706 >> v2:172.21.15.162:3300/0 conn(0x7f014c0cf570 msgr2=0x7f014c06b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:12.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.057+0000 7f0150bac640 1 --2- 172.21.15.46:0/4208129706 >> v2:172.21.15.162:3300/0 conn(0x7f014c0cf570 0x7f014c06b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:12.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.057+0000 7f0150bac640 1 -- 172.21.15.46:0/4208129706 >> v1:172.21.15.162:6789/0 conn(0x7f014c06b4e0 legacy=0x7f014c0b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:12.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.057+0000 7f0150bac640 1 -- 172.21.15.46:0/4208129706 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f014c0cc0a0 con 0x7f014c13d080 2023-12-02T00:31:12.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.058+0000 7f0150bac640 1 --2- 172.21.15.46:0/4208129706 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f014c13d080 0x7f014c15afc0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f014800bb10 tx=0x7f014802e150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb89911dbb096c4b server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:12.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.059+0000 7f01437fe640 1 -- 172.21.15.46:0/4208129706 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0148036440 con 0x7f014c13d080 2023-12-02T00:31:12.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.059+0000 7f01437fe640 1 -- 172.21.15.46:0/4208129706 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01480365a0 con 0x7f014c13d080 2023-12-02T00:31:12.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.059+0000 7f01437fe640 1 -- 172.21.15.46:0/4208129706 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0148036930 con 0x7f014c13d080 2023-12-02T00:31:12.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.059+0000 7f0152e10640 1 -- 172.21.15.46:0/4208129706 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f014c13d080 msgr2=0x7f014c15afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:12.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.059+0000 7f0152e10640 1 --2- 172.21.15.46:0/4208129706 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f014c13d080 0x7f014c15afc0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f014800bb10 tx=0x7f014802e150 comp rx=0 tx=0).stop 2023-12-02T00:31:12.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.059+0000 7f0152e10640 1 -- 172.21.15.46:0/4208129706 shutdown_connections 2023-12-02T00:31:12.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.059+0000 7f0152e10640 1 --2- 172.21.15.46:0/4208129706 >> v2:172.21.15.162:3300/0 conn(0x7f014c0cf570 0x7f014c06b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:12.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.059+0000 7f0152e10640 1 --2- 172.21.15.46:0/4208129706 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f014c13d080 0x7f014c15afc0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:12.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.059+0000 7f0152e10640 1 -- 172.21.15.46:0/4208129706 >> 172.21.15.46:0/4208129706 conn(0x7f014c070e70 msgr2=0x7f014c071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:12.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.059+0000 7f0152e10640 1 -- 172.21.15.46:0/4208129706 shutdown_connections 2023-12-02T00:31:12.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.059+0000 7f0152e10640 1 -- 172.21.15.46:0/4208129706 wait complete. 2023-12-02T00:31:12.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.060+0000 7f0152e10640 1 Processor -- start 2023-12-02T00:31:12.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.060+0000 7f0152e10640 1 -- start start 2023-12-02T00:31:12.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.061+0000 7f0152e10640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f014c06b4e0 0x7f014c154c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:12.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.061+0000 7f0152e10640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f014c0cf570 0x7f014c134eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:12.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.061+0000 7f0152e10640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f014c13d080 0x7f014c1f18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:12.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.061+0000 7f0152e10640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014c155e70 con 0x7f014c0cf570 2023-12-02T00:31:12.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.061+0000 7f0152e10640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014c1353f0 con 0x7f014c13d080 2023-12-02T00:31:12.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.061+0000 7f0152e10640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f014c1551f0 con 0x7f014c06b4e0 2023-12-02T00:31:12.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.061+0000 7f0143fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f014c0cf570 0x7f014c134eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:12.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.061+0000 7f0143fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f014c0cf570 0x7f014c134eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59846/0 (socket says 172.21.15.46:59846) 2023-12-02T00:31:12.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.061+0000 7f0143fff640 1 -- 172.21.15.46:0/1598795170 learned_addr learned my addr 172.21.15.46:0/1598795170 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:12.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.061+0000 7f0150bac640 1 --2- 172.21.15.46:0/1598795170 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f014c06b4e0 0x7f014c154c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:12.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.061+0000 7f01513ad640 1 --2- 172.21.15.46:0/1598795170 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f014c13d080 0x7f014c1f18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:12.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.062+0000 7f0143fff640 1 -- 172.21.15.46:0/1598795170 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f014c06b4e0 msgr2=0x7f014c154c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:12.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.062+0000 7f0143fff640 1 --2- 172.21.15.46:0/1598795170 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f014c06b4e0 0x7f014c154c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:12.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.062+0000 7f0143fff640 1 -- 172.21.15.46:0/1598795170 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f014c13d080 msgr2=0x7f014c1f18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:12.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.062+0000 7f0143fff640 1 --2- 172.21.15.46:0/1598795170 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f014c13d080 0x7f014c1f18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:12.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.062+0000 7f0143fff640 1 -- 172.21.15.46:0/1598795170 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f014800b770 con 0x7f014c0cf570 2023-12-02T00:31:12.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.062+0000 7f0143fff640 1 --2- 172.21.15.46:0/1598795170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f014c0cf570 0x7f014c134eb0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f013c00c400 tx=0x7f013c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:12.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.062+0000 7f01417fa640 1 -- 172.21.15.46:0/1598795170 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013c0052d0 con 0x7f014c0cf570 2023-12-02T00:31:12.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.062+0000 7f01417fa640 1 -- 172.21.15.46:0/1598795170 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f013c005430 con 0x7f014c0cf570 2023-12-02T00:31:12.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.062+0000 7f01417fa640 1 -- 172.21.15.46:0/1598795170 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013c003830 con 0x7f014c0cf570 2023-12-02T00:31:12.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.062+0000 7f0152e10640 1 -- 172.21.15.46:0/1598795170 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f014c1554d0 con 0x7f014c0cf570 2023-12-02T00:31:12.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.062+0000 7f0152e10640 1 -- 172.21.15.46:0/1598795170 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f014c155a90 con 0x7f014c0cf570 2023-12-02T00:31:12.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.063+0000 7f0152e10640 1 -- 172.21.15.46:0/1598795170 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f014c155fb0 con 0x7f014c0cf570 2023-12-02T00:31:12.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.063+0000 7f01417fa640 1 -- 172.21.15.46:0/1598795170 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f013c003990 con 0x7f014c0cf570 2023-12-02T00:31:12.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.064+0000 7f01417fa640 1 --2- 172.21.15.46:0/1598795170 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f013806ae40 0x7f013806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:12.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.064+0000 7f01417fa640 1 -- 172.21.15.46:0/1598795170 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f013c088a30 con 0x7f014c0cf570 2023-12-02T00:31:12.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.064+0000 7f0150bac640 1 --2- 172.21.15.46:0/1598795170 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f013806ae40 0x7f013806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:12.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.065+0000 7f0150bac640 1 --2- 172.21.15.46:0/1598795170 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f013806ae40 0x7f013806d300 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0148038000 tx=0x7f0148032000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:12.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.068+0000 7f01417fa640 1 -- 172.21.15.46:0/1598795170 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f014c155fb0 con 0x7f014c0cf570 2023-12-02T00:31:12.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.280+0000 7f0152e10640 1 -- 172.21.15.46:0/1598795170 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f014c155fb0 con 0x7f014c0cf570 2023-12-02T00:31:12.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.281+0000 7f01417fa640 1 -- 172.21.15.46:0/1598795170 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+73 (secure 0 0 0) 0x7f014c155fb0 con 0x7f014c0cf570 2023-12-02T00:31:12.282 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-02T00:31:12.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.282+0000 7f0152e10640 1 -- 172.21.15.46:0/1598795170 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f013806ae40 msgr2=0x7f013806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:12.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.282+0000 7f0152e10640 1 --2- 172.21.15.46:0/1598795170 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f013806ae40 0x7f013806d300 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0148038000 tx=0x7f0148032000 comp rx=0 tx=0).stop 2023-12-02T00:31:12.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.282+0000 7f0152e10640 1 -- 172.21.15.46:0/1598795170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f014c0cf570 msgr2=0x7f014c134eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:12.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.282+0000 7f0152e10640 1 --2- 172.21.15.46:0/1598795170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f014c0cf570 0x7f014c134eb0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f013c00c400 tx=0x7f013c00c8c0 comp rx=0 tx=0).stop 2023-12-02T00:31:12.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.283+0000 7f0152e10640 1 -- 172.21.15.46:0/1598795170 shutdown_connections 2023-12-02T00:31:12.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.283+0000 7f0152e10640 1 --2- 172.21.15.46:0/1598795170 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f013806ae40 0x7f013806d300 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:12.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.283+0000 7f0152e10640 1 --2- 172.21.15.46:0/1598795170 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f014c13d080 0x7f014c1f18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:12.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.283+0000 7f0152e10640 1 --2- 172.21.15.46:0/1598795170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f014c0cf570 0x7f014c134eb0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:12.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.283+0000 7f0152e10640 1 --2- 172.21.15.46:0/1598795170 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f014c06b4e0 0x7f014c154c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:12.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.283+0000 7f0152e10640 1 -- 172.21.15.46:0/1598795170 >> 172.21.15.46:0/1598795170 conn(0x7f014c070e70 msgr2=0x7f014c15c710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:12.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.283+0000 7f0152e10640 1 -- 172.21.15.46:0/1598795170 shutdown_connections 2023-12-02T00:31:12.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:12.283+0000 7f0152e10640 1 -- 172.21.15.46:0/1598795170 wait complete. 2023-12-02T00:31:12.294 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-02T00:31:12.295 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-02T00:31:12.295 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2023-12-02T00:31:12.295 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-02T00:31:12.295 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-02T00:31:12.295 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-02T00:31:12.295 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-02T00:31:12.295 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-02T00:31:12.295 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-02T00:31:12.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:31:12.295 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2023-12-02T00:31:12.312 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-02T00:31:12.312 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-02T00:31:12.312 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2023-12-02T00:31:12.312 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:31:12.312 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2023-12-02T00:31:12.312 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:31:12.312 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-02T00:31:12.312 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2023-12-02T00:31:12.313 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2023-12-02T00:31:12.313 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-02T00:31:12.313 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2023-12-02T00:31:12.313 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2023-12-02T00:31:12.313 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-02T00:31:12.313 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2023-12-02T00:31:12.313 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2023-12-02T00:31:12.313 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-12-02T00:31:12.313 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42507sec preferred_lft 42507sec 2023-12-02T00:31:12.313 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link noprefixroute 2023-12-02T00:31:12.314 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:31:12.314 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-02T00:31:12.315 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2023-12-02T00:31:12.315 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2023-12-02T00:31:12.315 INFO:teuthology.orchestra.run.smithi046.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-02T00:31:12.315 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether b6:1a:8b:80:15:f1 brd ff:ff:ff:ff:ff:ff 2023-12-02T00:31:12.315 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-02T00:31:12.315 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:31:12.315 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::eca8:9ff:feb3:1726/64 scope link 2023-12-02T00:31:12.315 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:31:12.315 INFO:teuthology.orchestra.run.smithi046.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-02T00:31:12.315 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether b6:1a:8b:80:15:f1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-02T00:31:12.315 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::b41a:8bff:fe80:15f1/64 scope link 2023-12-02T00:31:12.315 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:31:12.316 INFO:teuthology.orchestra.run.smithi046.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-02T00:31:12.316 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether c6:93:a5:c0:e2:af brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-02T00:31:12.316 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c493:a5ff:fec0:e2af/64 scope link 2023-12-02T00:31:12.316 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:31:12.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:31:12.316 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-02T00:31:12.382 INFO:teuthology.orchestra.run.smithi046.stdout:1 2023-12-02T00:31:12.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:31:12.383 DEBUG:teuthology.orchestra.run.smithi046:> route 2023-12-02T00:31:12.440 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2023-12-02T00:31:12.440 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-02T00:31:12.440 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-12-02T00:31:12.440 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-12-02T00:31:12.440 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-02T00:31:12.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:31:12.441 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2023-12-02T00:31:12.441 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2023-12-02T00:31:12.441 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-02T00:31:12.441 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-02T00:31:12.441 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-02T00:31:12.441 DEBUG:teuthology.orchestra.run.smithi046:> ') 2023-12-02T00:31:12.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:31:12.546 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2023-12-02T00:31:12.561 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-02T00:31:12.561 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-02T00:31:12.561 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-02T00:31:12.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:31:12.561 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2023-12-02T00:31:12.632 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-12-02T00:31:12.632 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-02T00:31:12.632 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-02T00:31:12.687 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-02T00:31:12.687 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-02T00:31:12.687 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-02T00:31:12.742 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2023-12-02T00:31:12.813 DEBUG:teuthology.orchestra.run.smithi046:> cat /proc/self/mounts | awk '{print $2}' 2023-12-02T00:31:12.868 INFO:teuthology.orchestra.run.smithi046.stdout:/proc 2023-12-02T00:31:12.869 INFO:teuthology.orchestra.run.smithi046.stdout:/sys 2023-12-02T00:31:12.869 INFO:teuthology.orchestra.run.smithi046.stdout:/dev 2023-12-02T00:31:12.869 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/security 2023-12-02T00:31:12.869 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/shm 2023-12-02T00:31:12.869 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/pts 2023-12-02T00:31:12.869 INFO:teuthology.orchestra.run.smithi046.stdout:/run 2023-12-02T00:31:12.869 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/cgroup 2023-12-02T00:31:12.869 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/pstore 2023-12-02T00:31:12.869 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/bpf 2023-12-02T00:31:12.869 INFO:teuthology.orchestra.run.smithi046.stdout:/ 2023-12-02T00:31:12.869 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/selinux 2023-12-02T00:31:12.869 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2023-12-02T00:31:12.869 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/mqueue 2023-12-02T00:31:12.869 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/hugepages 2023-12-02T00:31:12.869 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/debug 2023-12-02T00:31:12.870 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/tracing 2023-12-02T00:31:12.870 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/fuse/connections 2023-12-02T00:31:12.870 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/config 2023-12-02T00:31:12.870 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-sysctl.service 2023-12-02T00:31:12.870 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-02T00:31:12.870 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-02T00:31:12.870 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/nfs/rpc_pipefs 2023-12-02T00:31:12.870 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/1000 2023-12-02T00:31:12.870 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph 2023-12-02T00:31:12.870 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-0 2023-12-02T00:31:12.870 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-1 2023-12-02T00:31:12.870 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-2 2023-12-02T00:31:12.870 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-3 2023-12-02T00:31:12.870 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns 2023-12-02T00:31:12.870 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-02T00:31:12.871 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-02T00:31:12.871 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-02T00:31:12.871 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-02T00:31:12.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:31:12.871 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2023-12-02T00:31:12.925 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-12-02T00:31:12.925 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-12-02T00:31:12.968 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2023-12-02T00:31:12.998 DEBUG:teuthology.orchestra.run.smithi046:> cat /proc/self/mounts | awk '{print $2}' 2023-12-02T00:31:13.055 INFO:teuthology.orchestra.run.smithi046.stdout:/proc 2023-12-02T00:31:13.055 INFO:teuthology.orchestra.run.smithi046.stdout:/sys 2023-12-02T00:31:13.055 INFO:teuthology.orchestra.run.smithi046.stdout:/dev 2023-12-02T00:31:13.055 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/security 2023-12-02T00:31:13.055 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/shm 2023-12-02T00:31:13.055 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/pts 2023-12-02T00:31:13.055 INFO:teuthology.orchestra.run.smithi046.stdout:/run 2023-12-02T00:31:13.055 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/cgroup 2023-12-02T00:31:13.056 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/pstore 2023-12-02T00:31:13.056 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/bpf 2023-12-02T00:31:13.056 INFO:teuthology.orchestra.run.smithi046.stdout:/ 2023-12-02T00:31:13.056 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/selinux 2023-12-02T00:31:13.056 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2023-12-02T00:31:13.056 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/mqueue 2023-12-02T00:31:13.056 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/hugepages 2023-12-02T00:31:13.056 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/debug 2023-12-02T00:31:13.056 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/tracing 2023-12-02T00:31:13.056 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/fuse/connections 2023-12-02T00:31:13.056 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/config 2023-12-02T00:31:13.056 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-sysctl.service 2023-12-02T00:31:13.056 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-02T00:31:13.056 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-02T00:31:13.057 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/nfs/rpc_pipefs 2023-12-02T00:31:13.057 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/1000 2023-12-02T00:31:13.057 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph 2023-12-02T00:31:13.057 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-0 2023-12-02T00:31:13.057 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-1 2023-12-02T00:31:13.057 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-2 2023-12-02T00:31:13.057 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-3 2023-12-02T00:31:13.057 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns 2023-12-02T00:31:13.057 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-02T00:31:13.057 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-02T00:31:13.057 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-02T00:31:13.057 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-02T00:31:13.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:31:13.058 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2023-12-02T00:31:13.061 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:2023-12-02T00:31:13.060+0000 7f0a2503f480 -1 init, newargv = 0x555df75cf420 newargc=15 2023-12-02T00:31:13.061 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[110046]: starting ceph client 2023-12-02T00:31:13.065 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[110046]: starting fuse 2023-12-02T00:31:13.076 INFO:teuthology.orchestra.run.smithi046.stdout:41 2023-12-02T00:31:13.076 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-12-02T00:31:13.076 DEBUG:teuthology.orchestra.run.smithi046:> sudo stdin-killer -- python3 -c ' 2023-12-02T00:31:13.076 DEBUG:teuthology.orchestra.run.smithi046:> import glob 2023-12-02T00:31:13.077 DEBUG:teuthology.orchestra.run.smithi046:> import re 2023-12-02T00:31:13.077 DEBUG:teuthology.orchestra.run.smithi046:> import os 2023-12-02T00:31:13.077 DEBUG:teuthology.orchestra.run.smithi046:> import subprocess 2023-12-02T00:31:13.077 DEBUG:teuthology.orchestra.run.smithi046:> 2023-12-02T00:31:13.077 DEBUG:teuthology.orchestra.run.smithi046:> def _find_admin_socket(client_name): 2023-12-02T00:31:13.077 DEBUG:teuthology.orchestra.run.smithi046:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-12-02T00:31:13.077 DEBUG:teuthology.orchestra.run.smithi046:> files = glob.glob(asok_path) 2023-12-02T00:31:13.077 DEBUG:teuthology.orchestra.run.smithi046:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-12-02T00:31:13.077 DEBUG:teuthology.orchestra.run.smithi046:> 2023-12-02T00:31:13.077 DEBUG:teuthology.orchestra.run.smithi046:> # Given a non-glob path, it better be there 2023-12-02T00:31:13.077 DEBUG:teuthology.orchestra.run.smithi046:> if "*" not in asok_path: 2023-12-02T00:31:13.077 DEBUG:teuthology.orchestra.run.smithi046:> assert(len(files) == 1) 2023-12-02T00:31:13.077 DEBUG:teuthology.orchestra.run.smithi046:> return files[0] 2023-12-02T00:31:13.077 DEBUG:teuthology.orchestra.run.smithi046:> 2023-12-02T00:31:13.077 DEBUG:teuthology.orchestra.run.smithi046:> for f in files: 2023-12-02T00:31:13.078 DEBUG:teuthology.orchestra.run.smithi046:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-02T00:31:13.078 DEBUG:teuthology.orchestra.run.smithi046:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-02T00:31:13.078 DEBUG:teuthology.orchestra.run.smithi046:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-02T00:31:13.078 DEBUG:teuthology.orchestra.run.smithi046:> contents = proc_f.read() 2023-12-02T00:31:13.078 DEBUG:teuthology.orchestra.run.smithi046:> if mountpoint in contents: 2023-12-02T00:31:13.078 DEBUG:teuthology.orchestra.run.smithi046:> return f 2023-12-02T00:31:13.078 DEBUG:teuthology.orchestra.run.smithi046:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-02T00:31:13.078 DEBUG:teuthology.orchestra.run.smithi046:> 2023-12-02T00:31:13.078 DEBUG:teuthology.orchestra.run.smithi046:> print(_find_admin_socket("client.0")) 2023-12-02T00:31:13.078 DEBUG:teuthology.orchestra.run.smithi046:> ' 2023-12-02T00:31:13.201 INFO:teuthology.orchestra.run.smithi046.stdout:/var/run/ceph/ceph-client.0.110046.asok 2023-12-02T00:31:13.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-12-02T00:31:13.211 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.110046.asok 2023-12-02T00:31:13.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:31:13.211 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.110046.asok status 2023-12-02T00:31:13.352 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-12-02T00:31:13.352 INFO:teuthology.orchestra.run.smithi046.stdout: "metadata": { 2023-12-02T00:31:13.352 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_sha1": "bed7681480ccb0192b91a35e691658867e1e96b7", 2023-12-02T00:31:13.352 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_version": "ceph version 18.0.0-7631-gbed76814 (bed7681480ccb0192b91a35e691658867e1e96b7) reef (dev)", 2023-12-02T00:31:13.352 INFO:teuthology.orchestra.run.smithi046.stdout: "entity_id": "0", 2023-12-02T00:31:13.353 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": "smithi046", 2023-12-02T00:31:13.353 INFO:teuthology.orchestra.run.smithi046.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-12-02T00:31:13.353 INFO:teuthology.orchestra.run.smithi046.stdout: "pid": "110046", 2023-12-02T00:31:13.353 INFO:teuthology.orchestra.run.smithi046.stdout: "root": "/" 2023-12-02T00:31:13.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:31:13.353 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_count": 0, 2023-12-02T00:31:13.353 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_pinned_count": 0, 2023-12-02T00:31:13.353 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 6171, 2023-12-02T00:31:13.353 INFO:teuthology.orchestra.run.smithi046.stdout: "inst": { 2023-12-02T00:31:13.353 INFO:teuthology.orchestra.run.smithi046.stdout: "name": { 2023-12-02T00:31:13.353 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "client", 2023-12-02T00:31:13.353 INFO:teuthology.orchestra.run.smithi046.stdout: "num": 6171 2023-12-02T00:31:13.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:31:13.353 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2023-12-02T00:31:13.353 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2023-12-02T00:31:13.354 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2023-12-02T00:31:13.354 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2649366082 2023-12-02T00:31:13.354 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:31:13.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:31:13.354 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2023-12-02T00:31:13.354 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2023-12-02T00:31:13.354 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2023-12-02T00:31:13.354 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2649366082 2023-12-02T00:31:13.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:31:13.354 INFO:teuthology.orchestra.run.smithi046.stdout: "inst_str": "client.6171 192.168.0.1:0/2649366082", 2023-12-02T00:31:13.354 INFO:teuthology.orchestra.run.smithi046.stdout: "addr_str": "192.168.0.1:0/2649366082", 2023-12-02T00:31:13.354 INFO:teuthology.orchestra.run.smithi046.stdout: "inode_count": 1, 2023-12-02T00:31:13.354 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_epoch": 24, 2023-12-02T00:31:13.354 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch": 51, 2023-12-02T00:31:13.355 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch_barrier": 0, 2023-12-02T00:31:13.355 INFO:teuthology.orchestra.run.smithi046.stdout: "blocklisted": false, 2023-12-02T00:31:13.355 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs" 2023-12-02T00:31:13.355 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-12-02T00:31:13.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:31:13.360 DEBUG:teuthology.orchestra.run.smithi046:> stat --file-system '--printf=%T 2023-12-02T00:31:13.361 DEBUG:teuthology.orchestra.run.smithi046:> ' -- /home/ubuntu/cephtest/mnt.0 2023-12-02T00:31:13.376 INFO:teuthology.orchestra.run.smithi046.stdout:fuseblk 2023-12-02T00:31:13.376 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-12-02T00:31:13.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:31:13.377 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-02T00:31:13.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:31:13.448 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch test_file) 2023-12-02T00:31:13.509 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow r pool=cephfs_data' 2023-12-02T00:31:13.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.662+0000 7fccbb55a640 1 Processor -- start 2023-12-02T00:31:13.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.662+0000 7fccbb55a640 1 -- start start 2023-12-02T00:31:13.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.663+0000 7fccbb55a640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fccb4146010 0x7fccb40cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:13.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.663+0000 7fccbb55a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fccb4146d70 0x7fccb40b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:13.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.663+0000 7fccbb55a640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccb4161170 con 0x7fccb40b4060 2023-12-02T00:31:13.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.663+0000 7fccbb55a640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccb4161450 con 0x7fccb4146010 2023-12-02T00:31:13.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.663+0000 7fccbb55a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccb4161730 con 0x7fccb4146d70 2023-12-02T00:31:13.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.663+0000 7fccb92f6640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fccb4146010 0x7fccb40cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:13.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.663+0000 7fccb92f6640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fccb4146010 0x7fccb40cef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39554/0 (socket says 172.21.15.46:39554) 2023-12-02T00:31:13.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.663+0000 7fccb92f6640 1 -- 172.21.15.46:0/1715674685 learned_addr learned my addr 172.21.15.46:0/1715674685 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:13.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.663+0000 7fccb8af5640 1 --2- 172.21.15.46:0/1715674685 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fccb4146d70 0x7fccb40b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:13.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.663+0000 7fccb92f6640 1 -- 172.21.15.46:0/1715674685 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fccb4146d70 msgr2=0x7fccb40b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:13.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.663+0000 7fccb92f6640 1 --2- 172.21.15.46:0/1715674685 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fccb4146d70 0x7fccb40b2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:13.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.664+0000 7fccb92f6640 1 -- 172.21.15.46:0/1715674685 >> v1:172.21.15.162:6789/0 conn(0x7fccb40b4060 legacy=0x7fccb40d0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:13.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.664+0000 7fccb92f6640 1 -- 172.21.15.46:0/1715674685 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb4161a10 con 0x7fccb4146010 2023-12-02T00:31:13.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.664+0000 7fccb92f6640 1 --2- 172.21.15.46:0/1715674685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccb4146010 0x7fccb40cef50 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fccac009870 tx=0x7fccac02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56a5aea0d4423d04 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:13.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.664+0000 7fccb3fff640 1 -- 172.21.15.46:0/1715674685 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccac030840 con 0x7fccb4146010 2023-12-02T00:31:13.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.664+0000 7fccb3fff640 1 -- 172.21.15.46:0/1715674685 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccac0309a0 con 0x7fccb4146010 2023-12-02T00:31:13.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.664+0000 7fccb3fff640 1 -- 172.21.15.46:0/1715674685 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccac030d30 con 0x7fccb4146010 2023-12-02T00:31:13.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.665+0000 7fccbb55a640 1 -- 172.21.15.46:0/1715674685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccb4146010 msgr2=0x7fccb40cef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:13.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.665+0000 7fccbb55a640 1 --2- 172.21.15.46:0/1715674685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccb4146010 0x7fccb40cef50 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fccac009870 tx=0x7fccac02b980 comp rx=0 tx=0).stop 2023-12-02T00:31:13.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.665+0000 7fccbb55a640 1 -- 172.21.15.46:0/1715674685 shutdown_connections 2023-12-02T00:31:13.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.665+0000 7fccbb55a640 1 --2- 172.21.15.46:0/1715674685 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fccb4146d70 0x7fccb40b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:13.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.665+0000 7fccbb55a640 1 --2- 172.21.15.46:0/1715674685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccb4146010 0x7fccb40cef50 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:13.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.665+0000 7fccbb55a640 1 -- 172.21.15.46:0/1715674685 >> 172.21.15.46:0/1715674685 conn(0x7fccb406a4d0 msgr2=0x7fccb406a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:13.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.665+0000 7fccbb55a640 1 -- 172.21.15.46:0/1715674685 shutdown_connections 2023-12-02T00:31:13.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.665+0000 7fccbb55a640 1 -- 172.21.15.46:0/1715674685 wait complete. 2023-12-02T00:31:13.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.666+0000 7fccbb55a640 1 Processor -- start 2023-12-02T00:31:13.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.666+0000 7fccbb55a640 1 -- start start 2023-12-02T00:31:13.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.667+0000 7fccbb55a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fccb40b4060 0x7fccb413a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:13.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.667+0000 7fccbb55a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fccb4146010 0x7fccb40a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:13.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.667+0000 7fccbb55a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccb4146d70 0x7fccb40cd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:13.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.667+0000 7fccbb55a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb40a9830 con 0x7fccb4146d70 2023-12-02T00:31:13.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.667+0000 7fccbb55a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb40a8f90 con 0x7fccb40b4060 2023-12-02T00:31:13.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.667+0000 7fccbb55a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccb40a9270 con 0x7fccb4146010 2023-12-02T00:31:13.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.667+0000 7fccb9af7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccb4146d70 0x7fccb40cd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:13.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.667+0000 7fccb9af7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccb4146d70 0x7fccb40cd7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39568/0 (socket says 172.21.15.46:39568) 2023-12-02T00:31:13.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.667+0000 7fccb9af7640 1 -- 172.21.15.46:0/3356114577 learned_addr learned my addr 172.21.15.46:0/3356114577 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:13.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.667+0000 7fccb8af5640 1 --2- 172.21.15.46:0/3356114577 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fccb4146010 0x7fccb40a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:13.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.667+0000 7fccb92f6640 1 --2- 172.21.15.46:0/3356114577 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fccb40b4060 0x7fccb413a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:13.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.667+0000 7fccb9af7640 1 -- 172.21.15.46:0/3356114577 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fccb4146010 msgr2=0x7fccb40a84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:13.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.667+0000 7fccb9af7640 1 --2- 172.21.15.46:0/3356114577 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fccb4146010 0x7fccb40a84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:13.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.667+0000 7fccb9af7640 1 -- 172.21.15.46:0/3356114577 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fccb40b4060 msgr2=0x7fccb413a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:13.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.667+0000 7fccb9af7640 1 --2- 172.21.15.46:0/3356114577 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fccb40b4060 0x7fccb413a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:13.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.667+0000 7fccb9af7640 1 -- 172.21.15.46:0/3356114577 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccac0094d0 con 0x7fccb4146d70 2023-12-02T00:31:13.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.667+0000 7fccb8af5640 1 --2- 172.21.15.46:0/3356114577 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fccb4146010 0x7fccb40a84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:31:13.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.667+0000 7fccb92f6640 1 --2- 172.21.15.46:0/3356114577 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fccb40b4060 0x7fccb413a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:31:13.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.668+0000 7fccb9af7640 1 --2- 172.21.15.46:0/3356114577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccb4146d70 0x7fccb40cd7d0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fcca400c680 tx=0x7fcca400cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:13.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.668+0000 7fccb1ffb640 1 -- 172.21.15.46:0/3356114577 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca4003b70 con 0x7fccb4146d70 2023-12-02T00:31:13.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.668+0000 7fccb1ffb640 1 -- 172.21.15.46:0/3356114577 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcca4003cd0 con 0x7fccb4146d70 2023-12-02T00:31:13.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.668+0000 7fccb1ffb640 1 -- 172.21.15.46:0/3356114577 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca4005310 con 0x7fccb4146d70 2023-12-02T00:31:13.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.668+0000 7fccbb55a640 1 -- 172.21.15.46:0/3356114577 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccb4146570 con 0x7fccb4146d70 2023-12-02T00:31:13.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.668+0000 7fccbb55a640 1 -- 172.21.15.46:0/3356114577 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccb40cb540 con 0x7fccb4146d70 2023-12-02T00:31:13.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.669+0000 7fccbb55a640 1 -- 172.21.15.46:0/3356114577 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccb40cb760 con 0x7fccb4146d70 2023-12-02T00:31:13.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.669+0000 7fccb1ffb640 1 -- 172.21.15.46:0/3356114577 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcca4005470 con 0x7fccb4146d70 2023-12-02T00:31:13.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.670+0000 7fccb1ffb640 1 --2- 172.21.15.46:0/3356114577 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fcca006ae40 0x7fcca006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:13.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.670+0000 7fccb1ffb640 1 -- 172.21.15.46:0/3356114577 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcca40893b0 con 0x7fccb4146d70 2023-12-02T00:31:13.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.670+0000 7fccb92f6640 1 --2- 172.21.15.46:0/3356114577 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fcca006ae40 0x7fcca006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:13.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.671+0000 7fccb92f6640 1 --2- 172.21.15.46:0/3356114577 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fcca006ae40 0x7fcca006d300 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fccac036000 tx=0x7fccac03b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:13.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.674+0000 7fccb1ffb640 1 -- 172.21.15.46:0/3356114577 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcca4054460 con 0x7fccb4146d70 2023-12-02T00:31:13.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.917+0000 7fccbb55a640 1 -- 172.21.15.46:0/3356114577 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow r pool=cephfs_data"]} v 0) v1 -- 0x7fccb4070600 con 0x7fccb4146d70 2023-12-02T00:31:13.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.919+0000 7fccb1ffb640 1 -- 172.21.15.46:0/3356114577 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow r pool=cephfs_data"]}]=0 updated caps for client.0 v18) v1 ==== 183+0+140 (secure 0 0 0) 0x7fcca4057a80 con 0x7fccb4146d70 2023-12-02T00:31:13.921 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2023-12-02T00:31:13.921 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAJempl5Lx1DhAAjpK6ajoQ5jQiUr/EMpFeLg== 2023-12-02T00:31:13.921 INFO:teuthology.orchestra.run.smithi046.stdout: caps mds = "allow" 2023-12-02T00:31:13.921 INFO:teuthology.orchestra.run.smithi046.stdout: caps mon = "allow r" 2023-12-02T00:31:13.921 INFO:teuthology.orchestra.run.smithi046.stdout: caps osd = "allow r pool=cephfs_data" 2023-12-02T00:31:13.921 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2023-12-02T00:31:13.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.922+0000 7fccbb55a640 1 -- 172.21.15.46:0/3356114577 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fcca006ae40 msgr2=0x7fcca006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:13.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.922+0000 7fccbb55a640 1 --2- 172.21.15.46:0/3356114577 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fcca006ae40 0x7fcca006d300 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fccac036000 tx=0x7fccac03b040 comp rx=0 tx=0).stop 2023-12-02T00:31:13.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.922+0000 7fccbb55a640 1 -- 172.21.15.46:0/3356114577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccb4146d70 msgr2=0x7fccb40cd7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:13.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.922+0000 7fccbb55a640 1 --2- 172.21.15.46:0/3356114577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccb4146d70 0x7fccb40cd7d0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fcca400c680 tx=0x7fcca400cb40 comp rx=0 tx=0).stop 2023-12-02T00:31:13.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.922+0000 7fccbb55a640 1 -- 172.21.15.46:0/3356114577 shutdown_connections 2023-12-02T00:31:13.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.922+0000 7fccbb55a640 1 --2- 172.21.15.46:0/3356114577 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fcca006ae40 0x7fcca006d300 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:13.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.922+0000 7fccbb55a640 1 --2- 172.21.15.46:0/3356114577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccb4146d70 0x7fccb40cd7d0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:13.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.922+0000 7fccbb55a640 1 --2- 172.21.15.46:0/3356114577 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fccb4146010 0x7fccb40a84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:13.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.922+0000 7fccbb55a640 1 --2- 172.21.15.46:0/3356114577 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fccb40b4060 0x7fccb413a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:13.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.922+0000 7fccbb55a640 1 -- 172.21.15.46:0/3356114577 >> 172.21.15.46:0/3356114577 conn(0x7fccb406a4d0 msgr2=0x7fccb40b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:13.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.922+0000 7fccbb55a640 1 -- 172.21.15.46:0/3356114577 shutdown_connections 2023-12-02T00:31:13.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:13.922+0000 7fccbb55a640 1 -- 172.21.15.46:0/3356114577 wait complete. 2023-12-02T00:31:13.934 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-02T00:31:13.934 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-02T00:31:13.951 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:31:14.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.097+0000 7fd8cf7d0640 1 Processor -- start 2023-12-02T00:31:14.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.097+0000 7fd8cf7d0640 1 -- start start 2023-12-02T00:31:14.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.098+0000 7fd8cf7d0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8c815d1d0 0x7fd8c806b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:14.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.098+0000 7fd8cf7d0640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd8c80ad640 0x7fd8c806ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:14.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.098+0000 7fd8cf7d0640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd8c80aeb80 0x7fd8c8160ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:14.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.098+0000 7fd8cf7d0640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8c80adae0 con 0x7fd8c80ad640 2023-12-02T00:31:14.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.098+0000 7fd8cf7d0640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8c80addc0 con 0x7fd8c80aeb80 2023-12-02T00:31:14.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.098+0000 7fd8cf7d0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c80ae0a0 con 0x7fd8c815d1d0 2023-12-02T00:31:14.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.098+0000 7fd8ccd6b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd8c80ad640 0x7fd8c806ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:14.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.098+0000 7fd8ccd6b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd8c80ad640 0x7fd8c806ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39582/0 (socket says 172.21.15.46:39582) 2023-12-02T00:31:14.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.098+0000 7fd8ccd6b640 1 -- 172.21.15.46:0/1319879425 learned_addr learned my addr 172.21.15.46:0/1319879425 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:14.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.098+0000 7fd8ccd6b640 1 -- 172.21.15.46:0/1319879425 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8c815d1d0 msgr2=0x7fd8c806b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:14.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.098+0000 7fd8cd56c640 1 --2- 172.21.15.46:0/1319879425 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8c815d1d0 0x7fd8c806b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:14.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.098+0000 7fd8cdd6d640 1 --2- 172.21.15.46:0/1319879425 >> v2:172.21.15.162:3300/0 conn(0x7fd8c80aeb80 0x7fd8c8160ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:14.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.098+0000 7fd8ccd6b640 1 --2- 172.21.15.46:0/1319879425 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8c815d1d0 0x7fd8c806b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:14.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.098+0000 7fd8ccd6b640 1 -- 172.21.15.46:0/1319879425 >> v2:172.21.15.162:3300/0 conn(0x7fd8c80aeb80 msgr2=0x7fd8c8160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:14.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.098+0000 7fd8ccd6b640 1 --2- 172.21.15.46:0/1319879425 >> v2:172.21.15.162:3300/0 conn(0x7fd8c80aeb80 0x7fd8c8160ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:14.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.098+0000 7fd8ccd6b640 1 -- 172.21.15.46:0/1319879425 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c80ae380 con 0x7fd8c80ad640 2023-12-02T00:31:14.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.099+0000 7fd8ccd6b640 1 --2- 172.21.15.46:0/1319879425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8c80ad640 0x7fd8c806ba20 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fd8c806f410 tx=0x7fd8bc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16aaf1d387297a8a server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:14.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.099+0000 7fd8c7fff640 1 -- 172.21.15.46:0/1319879425 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc031870 con 0x7fd8c80ad640 2023-12-02T00:31:14.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.099+0000 7fd8c7fff640 1 -- 172.21.15.46:0/1319879425 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8bc0319d0 con 0x7fd8c80ad640 2023-12-02T00:31:14.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.099+0000 7fd8c7fff640 1 -- 172.21.15.46:0/1319879425 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc031d60 con 0x7fd8c80ad640 2023-12-02T00:31:14.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.100+0000 7fd8cf7d0640 1 -- 172.21.15.46:0/1319879425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8c80ad640 msgr2=0x7fd8c806ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:14.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.100+0000 7fd8cf7d0640 1 --2- 172.21.15.46:0/1319879425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8c80ad640 0x7fd8c806ba20 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fd8c806f410 tx=0x7fd8bc02b910 comp rx=0 tx=0).stop 2023-12-02T00:31:14.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.100+0000 7fd8cf7d0640 1 -- 172.21.15.46:0/1319879425 shutdown_connections 2023-12-02T00:31:14.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.100+0000 7fd8cf7d0640 1 --2- 172.21.15.46:0/1319879425 >> v2:172.21.15.162:3300/0 conn(0x7fd8c80aeb80 0x7fd8c8160ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:14.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.100+0000 7fd8cf7d0640 1 --2- 172.21.15.46:0/1319879425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8c80ad640 0x7fd8c806ba20 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:14.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.100+0000 7fd8cf7d0640 1 --2- 172.21.15.46:0/1319879425 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8c815d1d0 0x7fd8c806b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:14.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.100+0000 7fd8cf7d0640 1 -- 172.21.15.46:0/1319879425 >> 172.21.15.46:0/1319879425 conn(0x7fd8c8070e70 msgr2=0x7fd8c8071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:14.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.100+0000 7fd8cf7d0640 1 -- 172.21.15.46:0/1319879425 shutdown_connections 2023-12-02T00:31:14.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.100+0000 7fd8cf7d0640 1 -- 172.21.15.46:0/1319879425 wait complete. 2023-12-02T00:31:14.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.101+0000 7fd8cf7d0640 1 Processor -- start 2023-12-02T00:31:14.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.101+0000 7fd8cf7d0640 1 -- start start 2023-12-02T00:31:14.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.102+0000 7fd8cf7d0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8c80ad640 0x7fd8c80cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:14.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.102+0000 7fd8cf7d0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8c80aeb80 0x7fd8c81602d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:14.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.102+0000 7fd8cf7d0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8c815d1d0 0x7fd8c806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:14.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.102+0000 7fd8cf7d0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c80cca30 con 0x7fd8c80aeb80 2023-12-02T00:31:14.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.102+0000 7fd8cf7d0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c815f230 con 0x7fd8c80ad640 2023-12-02T00:31:14.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.102+0000 7fd8cf7d0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c815f510 con 0x7fd8c815d1d0 2023-12-02T00:31:14.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.102+0000 7fd8ccd6b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8c80aeb80 0x7fd8c81602d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:14.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.102+0000 7fd8ccd6b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8c80aeb80 0x7fd8c81602d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39598/0 (socket says 172.21.15.46:39598) 2023-12-02T00:31:14.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.102+0000 7fd8ccd6b640 1 -- 172.21.15.46:0/3976367403 learned_addr learned my addr 172.21.15.46:0/3976367403 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:14.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.102+0000 7fd8cdd6d640 1 --2- 172.21.15.46:0/3976367403 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8c815d1d0 0x7fd8c806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:14.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.102+0000 7fd8ccd6b640 1 -- 172.21.15.46:0/3976367403 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8c815d1d0 msgr2=0x7fd8c806e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:14.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.102+0000 7fd8ccd6b640 1 --2- 172.21.15.46:0/3976367403 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8c815d1d0 0x7fd8c806e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:14.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.102+0000 7fd8cd56c640 1 --2- 172.21.15.46:0/3976367403 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8c80ad640 0x7fd8c80cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:14.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.102+0000 7fd8ccd6b640 1 -- 172.21.15.46:0/3976367403 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8c80ad640 msgr2=0x7fd8c80cc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:14.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.102+0000 7fd8ccd6b640 1 --2- 172.21.15.46:0/3976367403 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8c80ad640 0x7fd8c80cc4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:14.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.102+0000 7fd8ccd6b640 1 -- 172.21.15.46:0/3976367403 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8bc0094d0 con 0x7fd8c80aeb80 2023-12-02T00:31:14.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.103+0000 7fd8ccd6b640 1 --2- 172.21.15.46:0/3976367403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8c80aeb80 0x7fd8c81602d0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc02b950 tx=0x7fd8bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:14.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.103+0000 7fd8c5ffb640 1 -- 172.21.15.46:0/3976367403 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc031860 con 0x7fd8c80aeb80 2023-12-02T00:31:14.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.103+0000 7fd8c5ffb640 1 -- 172.21.15.46:0/3976367403 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8bc0443f0 con 0x7fd8c80aeb80 2023-12-02T00:31:14.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.103+0000 7fd8cf7d0640 1 -- 172.21.15.46:0/3976367403 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8c815f7f0 con 0x7fd8c80aeb80 2023-12-02T00:31:14.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.103+0000 7fd8c5ffb640 1 -- 172.21.15.46:0/3976367403 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc0447a0 con 0x7fd8c80aeb80 2023-12-02T00:31:14.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.103+0000 7fd8cf7d0640 1 -- 172.21.15.46:0/3976367403 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8c815fdb0 con 0x7fd8c80aeb80 2023-12-02T00:31:14.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.104+0000 7fd8cf7d0640 1 -- 172.21.15.46:0/3976367403 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8c815ffa0 con 0x7fd8c80aeb80 2023-12-02T00:31:14.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.105+0000 7fd8c5ffb640 1 -- 172.21.15.46:0/3976367403 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd8bc04d450 con 0x7fd8c80aeb80 2023-12-02T00:31:14.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.105+0000 7fd8c5ffb640 1 --2- 172.21.15.46:0/3976367403 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd8b406ae40 0x7fd8b406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:14.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.105+0000 7fd8c5ffb640 1 -- 172.21.15.46:0/3976367403 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd8bc02d040 con 0x7fd8c80aeb80 2023-12-02T00:31:14.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.105+0000 7fd8cd56c640 1 --2- 172.21.15.46:0/3976367403 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd8b406ae40 0x7fd8b406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:14.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.106+0000 7fd8cd56c640 1 --2- 172.21.15.46:0/3976367403 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd8b406ae40 0x7fd8b406d300 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd8c815ee00 tx=0x7fd8c00083c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:14.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.109+0000 7fd8c5ffb640 1 -- 172.21.15.46:0/3976367403 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd8bc076ea0 con 0x7fd8c80aeb80 2023-12-02T00:31:14.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.286+0000 7fd8cf7d0640 1 -- 172.21.15.46:0/3976367403 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd8c8069400 con 0x7fd8c80aeb80 2023-12-02T00:31:14.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.287+0000 7fd8c5ffb640 1 -- 172.21.15.46:0/3976367403 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+16545 (secure 0 0 0) 0x7fd8bc035020 con 0x7fd8c80aeb80 2023-12-02T00:31:14.289 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:31:14.289 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":51,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:31:02.837102+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-12-02T00:30:59.469070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-12-02T00:31:00.174610+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:31:14.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.289+0000 7fd8cf7d0640 1 -- 172.21.15.46:0/3976367403 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd8b406ae40 msgr2=0x7fd8b406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:14.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.289+0000 7fd8cf7d0640 1 --2- 172.21.15.46:0/3976367403 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd8b406ae40 0x7fd8b406d300 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd8c815ee00 tx=0x7fd8c00083c0 comp rx=0 tx=0).stop 2023-12-02T00:31:14.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.289+0000 7fd8cf7d0640 1 -- 172.21.15.46:0/3976367403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8c80aeb80 msgr2=0x7fd8c81602d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:14.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.289+0000 7fd8cf7d0640 1 --2- 172.21.15.46:0/3976367403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8c80aeb80 0x7fd8c81602d0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc02b950 tx=0x7fd8bc0046c0 comp rx=0 tx=0).stop 2023-12-02T00:31:14.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.289+0000 7fd8cf7d0640 1 -- 172.21.15.46:0/3976367403 shutdown_connections 2023-12-02T00:31:14.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.289+0000 7fd8cf7d0640 1 --2- 172.21.15.46:0/3976367403 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd8b406ae40 0x7fd8b406d300 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:14.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.289+0000 7fd8cf7d0640 1 --2- 172.21.15.46:0/3976367403 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8c815d1d0 0x7fd8c806e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:14.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.289+0000 7fd8cf7d0640 1 --2- 172.21.15.46:0/3976367403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8c80aeb80 0x7fd8c81602d0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:14.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.289+0000 7fd8cf7d0640 1 --2- 172.21.15.46:0/3976367403 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8c80ad640 0x7fd8c80cc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:14.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.289+0000 7fd8cf7d0640 1 -- 172.21.15.46:0/3976367403 >> 172.21.15.46:0/3976367403 conn(0x7fd8c8070e70 msgr2=0x7fd8c806a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:14.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.289+0000 7fd8cf7d0640 1 -- 172.21.15.46:0/3976367403 shutdown_connections 2023-12-02T00:31:14.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.289+0000 7fd8cf7d0640 1 -- 172.21.15.46:0/3976367403 wait complete. 2023-12-02T00:31:14.301 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-02T00:28:37.213645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-02T00:30:59.469070+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-12-02T00:31:00.174610+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-02T00:31:14.302 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-02T00:31:14.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.410+0000 7ff2ae826640 1 Processor -- start 2023-12-02T00:31:14.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.410+0000 7ff2ae826640 1 -- start start 2023-12-02T00:31:14.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.411+0000 7ff2ae826640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff2a8146460 0x7ff2a80b49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:14.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.411+0000 7ff2ae826640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff2a806f1c0 0x7ff2a806baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:14.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.411+0000 7ff2ae826640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2a8146fd0 con 0x7ff2a8146840 2023-12-02T00:31:14.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.411+0000 7ff2ae826640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2a806c180 con 0x7ff2a8146460 2023-12-02T00:31:14.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.411+0000 7ff2ae826640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2a806c460 con 0x7ff2a806f1c0 2023-12-02T00:31:14.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.411+0000 7ff2a7fff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff2a8146460 0x7ff2a80b49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:14.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.411+0000 7ff2a77fe640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff2a8146840 0x7ff2a806eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57140/0 (socket says 172.21.15.46:57140) 2023-12-02T00:31:14.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.411+0000 7ff2a77fe640 1 -- 172.21.15.46:0/1735499985 learned_addr learned my addr 172.21.15.46:0/1735499985 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:31:14.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.411+0000 7ff2a7fff640 1 -- 172.21.15.46:0/1735499985 >> v2:172.21.15.162:3300/0 conn(0x7ff2a806f1c0 msgr2=0x7ff2a806baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:14.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.411+0000 7ff2a7fff640 1 --2- 172.21.15.46:0/1735499985 >> v2:172.21.15.162:3300/0 conn(0x7ff2a806f1c0 0x7ff2a806baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:14.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.411+0000 7ff2a7fff640 1 -- 172.21.15.46:0/1735499985 >> v1:172.21.15.46:6789/0 conn(0x7ff2a8146840 legacy=0x7ff2a806eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:14.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.411+0000 7ff2a7fff640 1 -- 172.21.15.46:0/1735499985 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a806c740 con 0x7ff2a8146460 2023-12-02T00:31:14.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.411+0000 7ff2a6ffd640 1 -- 172.21.15.46:0/1735499985 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1263552038 0 0) 0x7ff2a8146fd0 con 0x7ff2a8146840 2023-12-02T00:31:14.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.412+0000 7ff2a7fff640 1 --2- 172.21.15.46:0/1735499985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2a8146460 0x7ff2a80b49d0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7ff2a80b4fe0 tx=0x7ff29c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3366ba926d2b9ddf server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:14.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.412+0000 7ff2a6ffd640 1 -- 172.21.15.46:0/1735499985 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c031640 con 0x7ff2a8146460 2023-12-02T00:31:14.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.412+0000 7ff2a6ffd640 1 -- 172.21.15.46:0/1735499985 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff29c0317a0 con 0x7ff2a8146460 2023-12-02T00:31:14.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.412+0000 7ff2a6ffd640 1 -- 172.21.15.46:0/1735499985 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c031b30 con 0x7ff2a8146460 2023-12-02T00:31:14.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.413+0000 7ff2ae826640 1 -- 172.21.15.46:0/1735499985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2a8146460 msgr2=0x7ff2a80b49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:14.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.413+0000 7ff2ae826640 1 --2- 172.21.15.46:0/1735499985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2a8146460 0x7ff2a80b49d0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7ff2a80b4fe0 tx=0x7ff29c02b8f0 comp rx=0 tx=0).stop 2023-12-02T00:31:14.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.413+0000 7ff2ae826640 1 -- 172.21.15.46:0/1735499985 shutdown_connections 2023-12-02T00:31:14.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.413+0000 7ff2ae826640 1 --2- 172.21.15.46:0/1735499985 >> v2:172.21.15.162:3300/0 conn(0x7ff2a806f1c0 0x7ff2a806baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:14.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.413+0000 7ff2ae826640 1 --2- 172.21.15.46:0/1735499985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2a8146460 0x7ff2a80b49d0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:14.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.413+0000 7ff2ae826640 1 -- 172.21.15.46:0/1735499985 >> 172.21.15.46:0/1735499985 conn(0x7ff2a806a6b0 msgr2=0x7ff2a806aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:14.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.413+0000 7ff2ae826640 1 -- 172.21.15.46:0/1735499985 shutdown_connections 2023-12-02T00:31:14.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.413+0000 7ff2ae826640 1 -- 172.21.15.46:0/1735499985 wait complete. 2023-12-02T00:31:14.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.414+0000 7ff2ae826640 1 Processor -- start 2023-12-02T00:31:14.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.414+0000 7ff2ae826640 1 -- start start 2023-12-02T00:31:14.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.415+0000 7ff2ae826640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2a806f1c0 0x7ff2a815d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:14.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.415+0000 7ff2ae826640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2a8146460 0x7ff2a80cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:14.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.415+0000 7ff2ae826640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2a8146840 0x7ff2a80a9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:14.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.415+0000 7ff2ae826640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a813a8d0 con 0x7ff2a8146460 2023-12-02T00:31:14.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.415+0000 7ff2ae826640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a815d560 con 0x7ff2a8146840 2023-12-02T00:31:14.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.415+0000 7ff2ae826640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a81f7240 con 0x7ff2a806f1c0 2023-12-02T00:31:14.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.415+0000 7ff2a77fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2a8146460 0x7ff2a80cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:14.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.415+0000 7ff2a77fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2a8146460 0x7ff2a80cd3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39614/0 (socket says 172.21.15.46:39614) 2023-12-02T00:31:14.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.415+0000 7ff2a77fe640 1 -- 172.21.15.46:0/2879147865 learned_addr learned my addr 172.21.15.46:0/2879147865 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:14.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.415+0000 7ff2a77fe640 1 -- 172.21.15.46:0/2879147865 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2a806f1c0 msgr2=0x7ff2a815d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:14.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.415+0000 7ff2a77fe640 1 --2- 172.21.15.46:0/2879147865 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2a806f1c0 0x7ff2a815d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:14.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.415+0000 7ff2a77fe640 1 -- 172.21.15.46:0/2879147865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2a8146840 msgr2=0x7ff2a80a9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:14.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.415+0000 7ff2a77fe640 1 --2- 172.21.15.46:0/2879147865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2a8146840 0x7ff2a80a9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:14.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.415+0000 7ff2a77fe640 1 -- 172.21.15.46:0/2879147865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff29c009400 con 0x7ff2a8146460 2023-12-02T00:31:14.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.415+0000 7ff2a77fe640 1 --2- 172.21.15.46:0/2879147865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2a8146460 0x7ff2a80cd3d0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7ff29800c440 tx=0x7ff298001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:14.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.416+0000 7ff2a4ff9640 1 -- 172.21.15.46:0/2879147865 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2980074e0 con 0x7ff2a8146460 2023-12-02T00:31:14.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.416+0000 7ff2a4ff9640 1 -- 172.21.15.46:0/2879147865 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff298007660 con 0x7ff2a8146460 2023-12-02T00:31:14.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.416+0000 7ff2ae826640 1 -- 172.21.15.46:0/2879147865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2a81f6f60 con 0x7ff2a8146460 2023-12-02T00:31:14.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.416+0000 7ff2a4ff9640 1 -- 172.21.15.46:0/2879147865 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff298004a20 con 0x7ff2a8146460 2023-12-02T00:31:14.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.416+0000 7ff2ae826640 1 -- 172.21.15.46:0/2879147865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2a80cff40 con 0x7ff2a8146460 2023-12-02T00:31:14.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.417+0000 7ff2ae826640 1 -- 172.21.15.46:0/2879147865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2a8069400 con 0x7ff2a8146460 2023-12-02T00:31:14.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.417+0000 7ff2a4ff9640 1 -- 172.21.15.46:0/2879147865 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff298004b80 con 0x7ff2a8146460 2023-12-02T00:31:14.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.418+0000 7ff2a4ff9640 1 --2- 172.21.15.46:0/2879147865 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff29406ae40 0x7ff29406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:14.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.418+0000 7ff2a4ff9640 1 -- 172.21.15.46:0/2879147865 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff298089320 con 0x7ff2a8146460 2023-12-02T00:31:14.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.418+0000 7ff2a7fff640 1 --2- 172.21.15.46:0/2879147865 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff29406ae40 0x7ff29406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:14.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.418+0000 7ff2a7fff640 1 --2- 172.21.15.46:0/2879147865 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff29406ae40 0x7ff29406d300 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff29c036000 tx=0x7ff29c030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:14.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.422+0000 7ff2a4ff9640 1 -- 172.21.15.46:0/2879147865 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff298057ef0 con 0x7ff2a8146460 2023-12-02T00:31:14.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.598+0000 7ff2ae826640 1 -- 172.21.15.46:0/2879147865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff2a8070460 con 0x7ff2a8146460 2023-12-02T00:31:14.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.598+0000 7ff2a4ff9640 1 -- 172.21.15.46:0/2879147865 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff298057a80 con 0x7ff2a8146460 2023-12-02T00:31:14.600 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-12-02T00:31:14.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.600+0000 7ff2ae826640 1 -- 172.21.15.46:0/2879147865 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff29406ae40 msgr2=0x7ff29406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:14.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.600+0000 7ff2ae826640 1 --2- 172.21.15.46:0/2879147865 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff29406ae40 0x7ff29406d300 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff29c036000 tx=0x7ff29c030000 comp rx=0 tx=0).stop 2023-12-02T00:31:14.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.600+0000 7ff2ae826640 1 -- 172.21.15.46:0/2879147865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2a8146460 msgr2=0x7ff2a80cd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:14.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.600+0000 7ff2ae826640 1 --2- 172.21.15.46:0/2879147865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2a8146460 0x7ff2a80cd3d0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7ff29800c440 tx=0x7ff298001e70 comp rx=0 tx=0).stop 2023-12-02T00:31:14.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.600+0000 7ff2ae826640 1 -- 172.21.15.46:0/2879147865 shutdown_connections 2023-12-02T00:31:14.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.600+0000 7ff2ae826640 1 --2- 172.21.15.46:0/2879147865 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff29406ae40 0x7ff29406d300 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:14.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.600+0000 7ff2ae826640 1 --2- 172.21.15.46:0/2879147865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2a8146840 0x7ff2a80a9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:14.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.600+0000 7ff2ae826640 1 --2- 172.21.15.46:0/2879147865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2a8146460 0x7ff2a80cd3d0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:14.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.600+0000 7ff2ae826640 1 --2- 172.21.15.46:0/2879147865 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2a806f1c0 0x7ff2a815d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:14.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.600+0000 7ff2ae826640 1 -- 172.21.15.46:0/2879147865 >> 172.21.15.46:0/2879147865 conn(0x7ff2a806a6b0 msgr2=0x7ff2a8145a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:14.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.600+0000 7ff2ae826640 1 -- 172.21.15.46:0/2879147865 shutdown_connections 2023-12-02T00:31:14.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.600+0000 7ff2ae826640 1 -- 172.21.15.46:0/2879147865 wait complete. 2023-12-02T00:31:14.612 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-02T00:31:14.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.720+0000 7f85ed905640 1 Processor -- start 2023-12-02T00:31:14.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.720+0000 7f85ed905640 1 -- start start 2023-12-02T00:31:14.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.721+0000 7f85ed905640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85e815d2e0 0x7f85e80d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:14.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.721+0000 7f85ed905640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85e806ce00 con 0x7f85e806bbd0 2023-12-02T00:31:14.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.721+0000 7f85ed905640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85e806c1f0 con 0x7f85e815d6c0 2023-12-02T00:31:14.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.721+0000 7f85ed905640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85e806c4d0 con 0x7f85e815d2e0 2023-12-02T00:31:14.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.721+0000 7f85e77fe640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f85e806bbd0 0x7f85e80afee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57154/0 (socket says 172.21.15.46:57154) 2023-12-02T00:31:14.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.721+0000 7f85e77fe640 1 -- 172.21.15.46:0/357636911 learned_addr learned my addr 172.21.15.46:0/357636911 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:31:14.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.721+0000 7f85e6ffd640 1 --2- 172.21.15.46:0/357636911 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85e815d2e0 0x7f85e80d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:14.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.722+0000 7f85e5ffb640 1 -- 172.21.15.46:0/357636911 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3957191798 0 0) 0x7f85e806ce00 con 0x7f85e806bbd0 2023-12-02T00:31:14.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.722+0000 7f85e5ffb640 1 -- 172.21.15.46:0/357636911 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85d4003520 con 0x7f85e806bbd0 2023-12-02T00:31:14.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.722+0000 7f85e5ffb640 1 -- 172.21.15.46:0/357636911 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2884680220 0 0) 0x7f85e806c1f0 con 0x7f85e815d6c0 2023-12-02T00:31:14.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.722+0000 7f85e5ffb640 1 -- 172.21.15.46:0/357636911 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85e806ce00 con 0x7f85e815d6c0 2023-12-02T00:31:14.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.722+0000 7f85e5ffb640 1 -- 172.21.15.46:0/357636911 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1932385253 0 0) 0x7f85d4003520 con 0x7f85e806bbd0 2023-12-02T00:31:14.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.722+0000 7f85e5ffb640 1 -- 172.21.15.46:0/357636911 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85e815d2e0 msgr2=0x7f85e80d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:14.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.722+0000 7f85e5ffb640 1 --2- 172.21.15.46:0/357636911 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85e815d2e0 0x7f85e80d1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:14.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.722+0000 7f85e5ffb640 1 -- 172.21.15.46:0/357636911 >> v1:172.21.15.162:6789/0 conn(0x7f85e815d6c0 legacy=0x7f85e806b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:14.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.722+0000 7f85e5ffb640 1 -- 172.21.15.46:0/357636911 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e806c7b0 con 0x7f85e806bbd0 2023-12-02T00:31:14.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.722+0000 7f85e5ffb640 1 -- 172.21.15.46:0/357636911 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7f85e0001670 con 0x7f85e806bbd0 2023-12-02T00:31:14.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.722+0000 7f85e6ffd640 1 --2- 172.21.15.46:0/357636911 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85e815d2e0 0x7f85e80d1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:31:14.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.722+0000 7f85e5ffb640 1 -- 172.21.15.46:0/357636911 >> v1:172.21.15.46:6789/0 conn(0x7f85e806bbd0 legacy=0x7f85e80afee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:14.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.722+0000 7f85e5ffb640 1 --2- 172.21.15.46:0/357636911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85d4003b30 0x7f85d4003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:14.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.722+0000 7f85e5ffb640 1 --2- 172.21.15.46:0/357636911 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85e815d2e0 0x7f85d40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:14.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.722+0000 7f85e5ffb640 1 --2- 172.21.15.46:0/357636911 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85d4004d00 0x7f85d4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:14.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.722+0000 7f85e5ffb640 1 -- 172.21.15.46:0/357636911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e806c4d0 con 0x7f85d4003b30 2023-12-02T00:31:14.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.722+0000 7f85e5ffb640 1 -- 172.21.15.46:0/357636911 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d400ba60 con 0x7f85d4004d00 2023-12-02T00:31:14.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.722+0000 7f85e5ffb640 1 -- 172.21.15.46:0/357636911 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85d400bc20 con 0x7f85e815d2e0 2023-12-02T00:31:14.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.722+0000 7f85e6ffd640 1 --2- 172.21.15.46:0/357636911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85d4003b30 0x7f85d4003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:14.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.722+0000 7f85e77fe640 1 --2- 172.21.15.46:0/357636911 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85d4004d00 0x7f85d4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:14.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.722+0000 7f85e67fc640 1 --2- 172.21.15.46:0/357636911 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85e815d2e0 0x7f85d40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:14.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.722+0000 7f85e6ffd640 1 -- 172.21.15.46:0/357636911 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85e815d2e0 msgr2=0x7f85d40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:14.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.722+0000 7f85e6ffd640 1 --2- 172.21.15.46:0/357636911 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85e815d2e0 0x7f85d40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:14.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.722+0000 7f85e6ffd640 1 -- 172.21.15.46:0/357636911 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85d4004d00 msgr2=0x7f85d4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:14.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.722+0000 7f85e6ffd640 1 --2- 172.21.15.46:0/357636911 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85d4004d00 0x7f85d4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:14.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.722+0000 7f85e6ffd640 1 -- 172.21.15.46:0/357636911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85d8009a70 con 0x7f85d4003b30 2023-12-02T00:31:14.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.722+0000 7f85e77fe640 1 --2- 172.21.15.46:0/357636911 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85d4004d00 0x7f85d4005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:31:14.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.723+0000 7f85e6ffd640 1 --2- 172.21.15.46:0/357636911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85d4003b30 0x7f85d4003f30 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f85d800e350 tx=0x7f85d40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48e6f0ff40157816 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:14.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.723+0000 7f85e5ffb640 1 -- 172.21.15.46:0/357636911 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d802cd70 con 0x7f85d4003b30 2023-12-02T00:31:14.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.724+0000 7f85e5ffb640 1 -- 172.21.15.46:0/357636911 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85d800f040 con 0x7f85d4003b30 2023-12-02T00:31:14.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.724+0000 7f85e5ffb640 1 -- 172.21.15.46:0/357636911 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d8034640 con 0x7f85d4003b30 2023-12-02T00:31:14.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.724+0000 7f85ed905640 1 -- 172.21.15.46:0/357636911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85d4003b30 msgr2=0x7f85d4003f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:14.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.724+0000 7f85ed905640 1 --2- 172.21.15.46:0/357636911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85d4003b30 0x7f85d4003f30 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f85d800e350 tx=0x7f85d40057f0 comp rx=0 tx=0).stop 2023-12-02T00:31:14.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.724+0000 7f85e77fe640 1 -- 172.21.15.46:0/357636911 reap_dead start 2023-12-02T00:31:14.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.724+0000 7f85ed905640 1 -- 172.21.15.46:0/357636911 shutdown_connections 2023-12-02T00:31:14.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.724+0000 7f85ed905640 1 -- 172.21.15.46:0/357636911 >> 172.21.15.46:0/357636911 conn(0x7f85e809c580 msgr2=0x7f85e80d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:14.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.724+0000 7f85ed905640 1 -- 172.21.15.46:0/357636911 shutdown_connections 2023-12-02T00:31:14.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.724+0000 7f85ed905640 1 -- 172.21.15.46:0/357636911 wait complete. 2023-12-02T00:31:14.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.725+0000 7f85ed905640 1 Processor -- start 2023-12-02T00:31:14.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.726+0000 7f85ed905640 1 -- start start 2023-12-02T00:31:14.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.726+0000 7f85ed905640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85d4003b30 0x7f85e81f6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:14.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.726+0000 7f85ed905640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85e80ca2c0 0x7f85e80ca720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:14.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.726+0000 7f85ed905640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85e80cc550 0x7f85e80cca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:14.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.726+0000 7f85ed905640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e80cac60 con 0x7f85e80ca2c0 2023-12-02T00:31:14.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.726+0000 7f85ed905640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e80b8900 con 0x7f85d4003b30 2023-12-02T00:31:14.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.726+0000 7f85ed905640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85e815e0c0 con 0x7f85e80cc550 2023-12-02T00:31:14.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.726+0000 7f85e67fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85e80ca2c0 0x7f85e80ca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:14.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.726+0000 7f85e67fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85e80ca2c0 0x7f85e80ca720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39638/0 (socket says 172.21.15.46:39638) 2023-12-02T00:31:14.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.726+0000 7f85e67fc640 1 -- 172.21.15.46:0/1833480274 learned_addr learned my addr 172.21.15.46:0/1833480274 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:14.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.726+0000 7f85e6ffd640 1 --2- 172.21.15.46:0/1833480274 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85d4003b30 0x7f85e81f6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:14.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.727+0000 7f85e77fe640 1 --2- 172.21.15.46:0/1833480274 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85e80cc550 0x7f85e80cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:14.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.727+0000 7f85e67fc640 1 -- 172.21.15.46:0/1833480274 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85e80cc550 msgr2=0x7f85e80cca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:14.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.727+0000 7f85e67fc640 1 --2- 172.21.15.46:0/1833480274 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85e80cc550 0x7f85e80cca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:14.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.727+0000 7f85e67fc640 1 -- 172.21.15.46:0/1833480274 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85d4003b30 msgr2=0x7f85e81f6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:14.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.727+0000 7f85e67fc640 1 --2- 172.21.15.46:0/1833480274 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85d4003b30 0x7f85e81f6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:14.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.727+0000 7f85e67fc640 1 -- 172.21.15.46:0/1833480274 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e815e3a0 con 0x7f85e80ca2c0 2023-12-02T00:31:14.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.727+0000 7f85e67fc640 1 --2- 172.21.15.46:0/1833480274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85e80ca2c0 0x7f85e80ca720 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f85d000e7e0 tx=0x7f85d000ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:14.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.727+0000 7f85dffff640 1 -- 172.21.15.46:0/1833480274 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d00050d0 con 0x7f85e80ca2c0 2023-12-02T00:31:14.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.727+0000 7f85dffff640 1 -- 172.21.15.46:0/1833480274 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85d0005230 con 0x7f85e80ca2c0 2023-12-02T00:31:14.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.727+0000 7f85dffff640 1 -- 172.21.15.46:0/1833480274 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d000c6c0 con 0x7f85e80ca2c0 2023-12-02T00:31:14.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.727+0000 7f85ed905640 1 -- 172.21.15.46:0/1833480274 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85e815e680 con 0x7f85e80ca2c0 2023-12-02T00:31:14.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.727+0000 7f85ed905640 1 -- 172.21.15.46:0/1833480274 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85e815ec40 con 0x7f85e80ca2c0 2023-12-02T00:31:14.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.728+0000 7f85ed905640 1 -- 172.21.15.46:0/1833480274 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85e80ced30 con 0x7f85e80ca2c0 2023-12-02T00:31:14.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.729+0000 7f85dffff640 1 -- 172.21.15.46:0/1833480274 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f85d000c820 con 0x7f85e80ca2c0 2023-12-02T00:31:14.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.729+0000 7f85dffff640 1 --2- 172.21.15.46:0/1833480274 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f85c806ae40 0x7f85c806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:14.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.729+0000 7f85dffff640 1 -- 172.21.15.46:0/1833480274 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f85d00893b0 con 0x7f85e80ca2c0 2023-12-02T00:31:14.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.729+0000 7f85e6ffd640 1 --2- 172.21.15.46:0/1833480274 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f85c806ae40 0x7f85c806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:14.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.730+0000 7f85e6ffd640 1 --2- 172.21.15.46:0/1833480274 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f85c806ae40 0x7f85c806d300 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f85d8036000 tx=0x7f85d802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:14.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.734+0000 7f85dffff640 1 -- 172.21.15.46:0/1833480274 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f85d0054460 con 0x7f85e80ca2c0 2023-12-02T00:31:14.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.909+0000 7f85ed905640 1 -- 172.21.15.46:0/1833480274 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f85e806c1f0 con 0x7f85e80ca2c0 2023-12-02T00:31:14.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.909+0000 7f85dffff640 1 -- 172.21.15.46:0/1833480274 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v51) v1 ==== 104+0+11 (secure 0 0 0) 0x7f85d0057a80 con 0x7f85e80ca2c0 2023-12-02T00:31:14.911 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-12-02T00:31:14.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.911+0000 7f85ed905640 1 -- 172.21.15.46:0/1833480274 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f85c806ae40 msgr2=0x7f85c806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:14.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.911+0000 7f85ed905640 1 --2- 172.21.15.46:0/1833480274 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f85c806ae40 0x7f85c806d300 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f85d8036000 tx=0x7f85d802e000 comp rx=0 tx=0).stop 2023-12-02T00:31:14.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.911+0000 7f85ed905640 1 -- 172.21.15.46:0/1833480274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85e80ca2c0 msgr2=0x7f85e80ca720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:14.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.911+0000 7f85ed905640 1 --2- 172.21.15.46:0/1833480274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85e80ca2c0 0x7f85e80ca720 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f85d000e7e0 tx=0x7f85d000ebe0 comp rx=0 tx=0).stop 2023-12-02T00:31:14.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.911+0000 7f85ed905640 1 -- 172.21.15.46:0/1833480274 shutdown_connections 2023-12-02T00:31:14.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.911+0000 7f85ed905640 1 --2- 172.21.15.46:0/1833480274 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f85c806ae40 0x7f85c806d300 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:14.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.911+0000 7f85ed905640 1 --2- 172.21.15.46:0/1833480274 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85e80cc550 0x7f85e80cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:14.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.911+0000 7f85ed905640 1 --2- 172.21.15.46:0/1833480274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85e80ca2c0 0x7f85e80ca720 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:14.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.911+0000 7f85ed905640 1 --2- 172.21.15.46:0/1833480274 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85d4003b30 0x7f85e81f6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:14.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.911+0000 7f85ed905640 1 -- 172.21.15.46:0/1833480274 >> 172.21.15.46:0/1833480274 conn(0x7f85e809c580 msgr2=0x7f85e815b270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:14.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.912+0000 7f85ed905640 1 -- 172.21.15.46:0/1833480274 shutdown_connections 2023-12-02T00:31:14.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:14.912+0000 7f85ed905640 1 -- 172.21.15.46:0/1833480274 wait complete. 2023-12-02T00:31:14.923 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-02T00:31:15.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.032+0000 7ff143116640 1 Processor -- start 2023-12-02T00:31:15.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.032+0000 7ff143116640 1 -- start start 2023-12-02T00:31:15.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.033+0000 7ff143116640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff13c160fb0 0x7ff13c161440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:15.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.033+0000 7ff143116640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff13c0d0ed0 0x7ff13c0d1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:15.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.033+0000 7ff143116640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff13c161d20 con 0x7ff13c06ea50 2023-12-02T00:31:15.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.033+0000 7ff143116640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff13c15e760 con 0x7ff13c160fb0 2023-12-02T00:31:15.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.033+0000 7ff143116640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff13c15d9a0 con 0x7ff13c0d0ed0 2023-12-02T00:31:15.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.033+0000 7ff13bfff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff13c160fb0 0x7ff13c161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:15.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.033+0000 7ff140eb2640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff13c06ea50 0x7ff13c06ee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57162/0 (socket says 172.21.15.46:57162) 2023-12-02T00:31:15.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.033+0000 7ff140eb2640 1 -- 172.21.15.46:0/3149384248 learned_addr learned my addr 172.21.15.46:0/3149384248 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:31:15.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.033+0000 7ff13bfff640 1 --2- 172.21.15.46:0/3149384248 >> v2:172.21.15.46:3300/0 conn(0x7ff13c160fb0 0x7ff13c161440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39648/0 (socket says 172.21.15.46:39648) 2023-12-02T00:31:15.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.033+0000 7ff1416b3640 1 --2- 172.21.15.46:0/3149384248 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff13c0d0ed0 0x7ff13c0d1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:15.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.033+0000 7ff13bfff640 1 -- 172.21.15.46:0/3149384248 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff13c0d0ed0 msgr2=0x7ff13c0d1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:15.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.033+0000 7ff13bfff640 1 --2- 172.21.15.46:0/3149384248 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff13c0d0ed0 0x7ff13c0d1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:15.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.033+0000 7ff13bfff640 1 -- 172.21.15.46:0/3149384248 >> v1:172.21.15.46:6789/0 conn(0x7ff13c06ea50 legacy=0x7ff13c06ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:15.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.033+0000 7ff13bfff640 1 -- 172.21.15.46:0/3149384248 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff13c15dc80 con 0x7ff13c160fb0 2023-12-02T00:31:15.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.033+0000 7ff13b7fe640 1 -- 172.21.15.46:0/3149384248 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 317371506 0 0) 0x7ff13c161d20 con 0x7ff13c06ea50 2023-12-02T00:31:15.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.034+0000 7ff13bfff640 1 --2- 172.21.15.46:0/3149384248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff13c160fb0 0x7ff13c161440 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7ff12c0097a0 tx=0x7ff12c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42836e957b95c28a server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:15.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.034+0000 7ff13b7fe640 1 -- 172.21.15.46:0/3149384248 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c031830 con 0x7ff13c160fb0 2023-12-02T00:31:15.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.034+0000 7ff13b7fe640 1 -- 172.21.15.46:0/3149384248 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff12c031990 con 0x7ff13c160fb0 2023-12-02T00:31:15.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.034+0000 7ff13b7fe640 1 -- 172.21.15.46:0/3149384248 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c031d20 con 0x7ff13c160fb0 2023-12-02T00:31:15.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.035+0000 7ff143116640 1 -- 172.21.15.46:0/3149384248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff13c160fb0 msgr2=0x7ff13c161440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:15.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.035+0000 7ff143116640 1 --2- 172.21.15.46:0/3149384248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff13c160fb0 0x7ff13c161440 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7ff12c0097a0 tx=0x7ff12c02b8d0 comp rx=0 tx=0).stop 2023-12-02T00:31:15.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.035+0000 7ff143116640 1 -- 172.21.15.46:0/3149384248 shutdown_connections 2023-12-02T00:31:15.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.035+0000 7ff143116640 1 --2- 172.21.15.46:0/3149384248 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff13c0d0ed0 0x7ff13c0d1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:15.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.035+0000 7ff143116640 1 --2- 172.21.15.46:0/3149384248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff13c160fb0 0x7ff13c161440 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:15.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.035+0000 7ff143116640 1 -- 172.21.15.46:0/3149384248 >> 172.21.15.46:0/3149384248 conn(0x7ff13c06a6b0 msgr2=0x7ff13c06aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:15.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.035+0000 7ff143116640 1 -- 172.21.15.46:0/3149384248 shutdown_connections 2023-12-02T00:31:15.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.035+0000 7ff143116640 1 -- 172.21.15.46:0/3149384248 wait complete. 2023-12-02T00:31:15.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.036+0000 7ff143116640 1 Processor -- start 2023-12-02T00:31:15.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.036+0000 7ff143116640 1 -- start start 2023-12-02T00:31:15.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.037+0000 7ff143116640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff13c06ea50 0x7ff13c156360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:15.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.037+0000 7ff143116640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff13c0d0ed0 0x7ff13c159ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:15.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.037+0000 7ff143116640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff13c160fb0 0x7ff13c15c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:15.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.037+0000 7ff143116640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13c156180 con 0x7ff13c0d0ed0 2023-12-02T00:31:15.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.037+0000 7ff143116640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13c15a0e0 con 0x7ff13c160fb0 2023-12-02T00:31:15.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.037+0000 7ff143116640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff13c13e390 con 0x7ff13c06ea50 2023-12-02T00:31:15.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.037+0000 7ff13bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff13c0d0ed0 0x7ff13c159ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:15.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.037+0000 7ff13bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff13c0d0ed0 0x7ff13c159ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39652/0 (socket says 172.21.15.46:39652) 2023-12-02T00:31:15.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.037+0000 7ff13bfff640 1 -- 172.21.15.46:0/3480697980 learned_addr learned my addr 172.21.15.46:0/3480697980 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:15.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.037+0000 7ff140eb2640 1 --2- 172.21.15.46:0/3480697980 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff13c06ea50 0x7ff13c156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:15.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.037+0000 7ff1416b3640 1 --2- 172.21.15.46:0/3480697980 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff13c160fb0 0x7ff13c15c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:15.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.037+0000 7ff13bfff640 1 -- 172.21.15.46:0/3480697980 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff13c06ea50 msgr2=0x7ff13c156360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:15.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.037+0000 7ff13bfff640 1 --2- 172.21.15.46:0/3480697980 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff13c06ea50 0x7ff13c156360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:15.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.037+0000 7ff13bfff640 1 -- 172.21.15.46:0/3480697980 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff13c160fb0 msgr2=0x7ff13c15c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:15.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.037+0000 7ff13bfff640 1 --2- 172.21.15.46:0/3480697980 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff13c160fb0 0x7ff13c15c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:15.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.037+0000 7ff13bfff640 1 -- 172.21.15.46:0/3480697980 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff12c009400 con 0x7ff13c0d0ed0 2023-12-02T00:31:15.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.038+0000 7ff13bfff640 1 --2- 172.21.15.46:0/3480697980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff13c0d0ed0 0x7ff13c159ba0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7ff12c0098c0 tx=0x7ff12c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:15.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.038+0000 7ff1397fa640 1 -- 172.21.15.46:0/3480697980 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c0317d0 con 0x7ff13c0d0ed0 2023-12-02T00:31:15.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.038+0000 7ff1397fa640 1 -- 172.21.15.46:0/3480697980 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff12c03ba70 con 0x7ff13c0d0ed0 2023-12-02T00:31:15.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.038+0000 7ff143116640 1 -- 172.21.15.46:0/3480697980 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff13c158b70 con 0x7ff13c0d0ed0 2023-12-02T00:31:15.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.038+0000 7ff1397fa640 1 -- 172.21.15.46:0/3480697980 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c03be20 con 0x7ff13c0d0ed0 2023-12-02T00:31:15.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.038+0000 7ff143116640 1 -- 172.21.15.46:0/3480697980 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff13c158240 con 0x7ff13c0d0ed0 2023-12-02T00:31:15.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.039+0000 7ff143116640 1 -- 172.21.15.46:0/3480697980 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff13c069400 con 0x7ff13c0d0ed0 2023-12-02T00:31:15.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.039+0000 7ff1397fa640 1 -- 172.21.15.46:0/3480697980 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff12c0453f0 con 0x7ff13c0d0ed0 2023-12-02T00:31:15.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.040+0000 7ff1397fa640 1 --2- 172.21.15.46:0/3480697980 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff11c06ae40 0x7ff11c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:15.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.040+0000 7ff1397fa640 1 -- 172.21.15.46:0/3480697980 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff12c0ab920 con 0x7ff13c0d0ed0 2023-12-02T00:31:15.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.040+0000 7ff140eb2640 1 --2- 172.21.15.46:0/3480697980 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff11c06ae40 0x7ff11c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:15.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.041+0000 7ff140eb2640 1 --2- 172.21.15.46:0/3480697980 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff11c06ae40 0x7ff11c06d300 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff134007ed0 tx=0x7ff134007e60 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:15.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.044+0000 7ff1397fa640 1 -- 172.21.15.46:0/3480697980 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff12c076af0 con 0x7ff13c0d0ed0 2023-12-02T00:31:15.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.222+0000 7ff143116640 1 -- 172.21.15.46:0/3480697980 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff13c15a280 con 0x7ff13c0d0ed0 2023-12-02T00:31:15.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.222+0000 7ff1397fa640 1 -- 172.21.15.46:0/3480697980 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v51) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff12c035020 con 0x7ff13c0d0ed0 2023-12-02T00:31:15.223 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-12-02T00:31:15.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.224+0000 7ff143116640 1 -- 172.21.15.46:0/3480697980 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff11c06ae40 msgr2=0x7ff11c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:15.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.224+0000 7ff143116640 1 --2- 172.21.15.46:0/3480697980 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff11c06ae40 0x7ff11c06d300 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff134007ed0 tx=0x7ff134007e60 comp rx=0 tx=0).stop 2023-12-02T00:31:15.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.224+0000 7ff143116640 1 -- 172.21.15.46:0/3480697980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff13c0d0ed0 msgr2=0x7ff13c159ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:15.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.224+0000 7ff143116640 1 --2- 172.21.15.46:0/3480697980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff13c0d0ed0 0x7ff13c159ba0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7ff12c0098c0 tx=0x7ff12c0046c0 comp rx=0 tx=0).stop 2023-12-02T00:31:15.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.224+0000 7ff143116640 1 -- 172.21.15.46:0/3480697980 shutdown_connections 2023-12-02T00:31:15.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.224+0000 7ff143116640 1 --2- 172.21.15.46:0/3480697980 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff11c06ae40 0x7ff11c06d300 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:15.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.224+0000 7ff143116640 1 --2- 172.21.15.46:0/3480697980 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff13c160fb0 0x7ff13c15c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:15.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.224+0000 7ff143116640 1 --2- 172.21.15.46:0/3480697980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff13c0d0ed0 0x7ff13c159ba0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:15.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.224+0000 7ff143116640 1 --2- 172.21.15.46:0/3480697980 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff13c06ea50 0x7ff13c156360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:15.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.224+0000 7ff143116640 1 -- 172.21.15.46:0/3480697980 >> 172.21.15.46:0/3480697980 conn(0x7ff13c06a6b0 msgr2=0x7ff13c06aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:15.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.224+0000 7ff143116640 1 -- 172.21.15.46:0/3480697980 shutdown_connections 2023-12-02T00:31:15.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.224+0000 7ff143116640 1 -- 172.21.15.46:0/3480697980 wait complete. 2023-12-02T00:31:15.236 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-02T00:31:15.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.342+0000 7f954384f640 1 Processor -- start 2023-12-02T00:31:15.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.342+0000 7f954384f640 1 -- start start 2023-12-02T00:31:15.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.343+0000 7f954384f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f953c06ea50 0x7f953c06ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:15.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.343+0000 7f954384f640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f953c0d0ed0 0x7f953c0d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:15.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.343+0000 7f954384f640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f953c161d20 con 0x7f953c160fb0 2023-12-02T00:31:15.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.343+0000 7f954384f640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f953c161b20 con 0x7f953c0d0ed0 2023-12-02T00:31:15.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.343+0000 7f954384f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f953c06f370 con 0x7f953c06ea50 2023-12-02T00:31:15.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.343+0000 7f9540dea640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f953c160fb0 0x7f953c161410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57164/0 (socket says 172.21.15.46:57164) 2023-12-02T00:31:15.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.343+0000 7f9540dea640 1 -- 172.21.15.46:0/254519460 learned_addr learned my addr 172.21.15.46:0/254519460 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:31:15.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.343+0000 7f95415eb640 1 --2- 172.21.15.46:0/254519460 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f953c06ea50 0x7f953c06ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:15.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.344+0000 7f9541dec640 1 --2- 172.21.15.46:0/254519460 >> v2:172.21.15.162:3300/0 conn(0x7f953c0d0ed0 0x7f953c0d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:15.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.344+0000 7f953bfff640 1 -- 172.21.15.46:0/254519460 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2361707327 0 0) 0x7f953c161d20 con 0x7f953c160fb0 2023-12-02T00:31:15.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.344+0000 7f953bfff640 1 -- 172.21.15.46:0/254519460 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9524003520 con 0x7f953c160fb0 2023-12-02T00:31:15.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.344+0000 7f95415eb640 1 -- 172.21.15.46:0/254519460 >> v2:172.21.15.162:3300/0 conn(0x7f953c0d0ed0 msgr2=0x7f953c0d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:15.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.344+0000 7f95415eb640 1 --2- 172.21.15.46:0/254519460 >> v2:172.21.15.162:3300/0 conn(0x7f953c0d0ed0 0x7f953c0d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:15.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.344+0000 7f95415eb640 1 -- 172.21.15.46:0/254519460 >> v1:172.21.15.46:6789/0 conn(0x7f953c160fb0 legacy=0x7f953c161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:15.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.344+0000 7f95415eb640 1 -- 172.21.15.46:0/254519460 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f953c15e760 con 0x7f953c06ea50 2023-12-02T00:31:15.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.344+0000 7f9541dec640 1 --2- 172.21.15.46:0/254519460 >> v2:172.21.15.162:3300/0 conn(0x7f953c0d0ed0 0x7f953c0d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:31:15.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.345+0000 7f95415eb640 1 --2- 172.21.15.46:0/254519460 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f953c06ea50 0x7f953c06ee30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f952c009d90 tx=0x7f952c02bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebc0c437ca266c7 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:15.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.345+0000 7f953bfff640 1 -- 172.21.15.46:0/254519460 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952c02cc50 con 0x7f953c06ea50 2023-12-02T00:31:15.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.345+0000 7f953bfff640 1 -- 172.21.15.46:0/254519460 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f952c02cdb0 con 0x7f953c06ea50 2023-12-02T00:31:15.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.345+0000 7f953bfff640 1 -- 172.21.15.46:0/254519460 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952c034660 con 0x7f953c06ea50 2023-12-02T00:31:15.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.345+0000 7f954384f640 1 -- 172.21.15.46:0/254519460 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f953c06ea50 msgr2=0x7f953c06ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:15.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.345+0000 7f954384f640 1 --2- 172.21.15.46:0/254519460 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f953c06ea50 0x7f953c06ee30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f952c009d90 tx=0x7f952c02bea0 comp rx=0 tx=0).stop 2023-12-02T00:31:15.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.346+0000 7f954384f640 1 -- 172.21.15.46:0/254519460 shutdown_connections 2023-12-02T00:31:15.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.346+0000 7f954384f640 1 --2- 172.21.15.46:0/254519460 >> v2:172.21.15.162:3300/0 conn(0x7f953c0d0ed0 0x7f953c0d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:15.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.346+0000 7f954384f640 1 --2- 172.21.15.46:0/254519460 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f953c06ea50 0x7f953c06ee30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:15.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.346+0000 7f954384f640 1 -- 172.21.15.46:0/254519460 >> 172.21.15.46:0/254519460 conn(0x7f953c06a6b0 msgr2=0x7f953c06aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:15.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.346+0000 7f954384f640 1 -- 172.21.15.46:0/254519460 shutdown_connections 2023-12-02T00:31:15.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.346+0000 7f954384f640 1 -- 172.21.15.46:0/254519460 wait complete. 2023-12-02T00:31:15.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.347+0000 7f954384f640 1 Processor -- start 2023-12-02T00:31:15.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.347+0000 7f954384f640 1 -- start start 2023-12-02T00:31:15.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.348+0000 7f954384f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f953c06ea50 0x7f953c1566e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:15.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.348+0000 7f954384f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f953c0d0ed0 0x7f953c0cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:15.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.348+0000 7f954384f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f953c160fb0 0x7f953c0cb940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:15.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.348+0000 7f954384f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f953c157ec0 con 0x7f953c160fb0 2023-12-02T00:31:15.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.348+0000 7f954384f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f953c158110 con 0x7f953c0d0ed0 2023-12-02T00:31:15.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.348+0000 7f954384f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f953c1583f0 con 0x7f953c06ea50 2023-12-02T00:31:15.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.348+0000 7f9541dec640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f953c160fb0 0x7f953c0cb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:15.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.348+0000 7f9541dec640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f953c160fb0 0x7f953c0cb940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39658/0 (socket says 172.21.15.46:39658) 2023-12-02T00:31:15.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.348+0000 7f9541dec640 1 -- 172.21.15.46:0/69901795 learned_addr learned my addr 172.21.15.46:0/69901795 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:15.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.348+0000 7f95415eb640 1 --2- 172.21.15.46:0/69901795 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f953c06ea50 0x7f953c1566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:15.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.348+0000 7f9540dea640 1 --2- 172.21.15.46:0/69901795 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f953c0d0ed0 0x7f953c0cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:15.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.348+0000 7f9541dec640 1 -- 172.21.15.46:0/69901795 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f953c06ea50 msgr2=0x7f953c1566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:15.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.348+0000 7f9541dec640 1 --2- 172.21.15.46:0/69901795 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f953c06ea50 0x7f953c1566e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:15.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.348+0000 7f9541dec640 1 -- 172.21.15.46:0/69901795 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f953c0d0ed0 msgr2=0x7f953c0cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:15.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.348+0000 7f9541dec640 1 --2- 172.21.15.46:0/69901795 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f953c0d0ed0 0x7f953c0cdbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:15.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.348+0000 7f9541dec640 1 -- 172.21.15.46:0/69901795 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9534009400 con 0x7f953c160fb0 2023-12-02T00:31:15.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.348+0000 7f9541dec640 1 --2- 172.21.15.46:0/69901795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f953c160fb0 0x7f953c0cb940 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f953400e880 tx=0x7f953400ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:15.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.348+0000 7f9539ffb640 1 -- 172.21.15.46:0/69901795 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9534009650 con 0x7f953c160fb0 2023-12-02T00:31:15.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.348+0000 7f9539ffb640 1 -- 172.21.15.46:0/69901795 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9534004020 con 0x7f953c160fb0 2023-12-02T00:31:15.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.348+0000 7f954384f640 1 -- 172.21.15.46:0/69901795 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f952c009a70 con 0x7f953c160fb0 2023-12-02T00:31:15.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.348+0000 7f9539ffb640 1 -- 172.21.15.46:0/69901795 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95340043b0 con 0x7f953c160fb0 2023-12-02T00:31:15.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.349+0000 7f954384f640 1 -- 172.21.15.46:0/69901795 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f953c1589b0 con 0x7f953c160fb0 2023-12-02T00:31:15.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.350+0000 7f954384f640 1 -- 172.21.15.46:0/69901795 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f953c158bd0 con 0x7f953c160fb0 2023-12-02T00:31:15.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.350+0000 7f9539ffb640 1 -- 172.21.15.46:0/69901795 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f953400c940 con 0x7f953c160fb0 2023-12-02T00:31:15.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.351+0000 7f9539ffb640 1 --2- 172.21.15.46:0/69901795 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f952806ae40 0x7f952806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:15.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.351+0000 7f9539ffb640 1 -- 172.21.15.46:0/69901795 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9534089320 con 0x7f953c160fb0 2023-12-02T00:31:15.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.351+0000 7f95415eb640 1 --2- 172.21.15.46:0/69901795 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f952806ae40 0x7f952806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:15.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.351+0000 7f95415eb640 1 --2- 172.21.15.46:0/69901795 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f952806ae40 0x7f952806d300 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f952c036000 tx=0x7f952c030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:15.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.355+0000 7f9539ffb640 1 -- 172.21.15.46:0/69901795 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9534054510 con 0x7f953c160fb0 2023-12-02T00:31:15.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.544+0000 7f954384f640 1 -- 172.21.15.46:0/69901795 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f953c069400 con 0x7f953c160fb0 2023-12-02T00:31:15.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.545+0000 7f9539ffb640 1 -- 172.21.15.46:0/69901795 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v51) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9534057b30 con 0x7f953c160fb0 2023-12-02T00:31:15.547 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2023-12-02T00:31:15.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.547+0000 7f954384f640 1 -- 172.21.15.46:0/69901795 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f952806ae40 msgr2=0x7f952806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:15.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.547+0000 7f954384f640 1 --2- 172.21.15.46:0/69901795 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f952806ae40 0x7f952806d300 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f952c036000 tx=0x7f952c030000 comp rx=0 tx=0).stop 2023-12-02T00:31:15.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.547+0000 7f954384f640 1 -- 172.21.15.46:0/69901795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f953c160fb0 msgr2=0x7f953c0cb940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:15.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.547+0000 7f954384f640 1 --2- 172.21.15.46:0/69901795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f953c160fb0 0x7f953c0cb940 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f953400e880 tx=0x7f953400ed40 comp rx=0 tx=0).stop 2023-12-02T00:31:15.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.547+0000 7f954384f640 1 -- 172.21.15.46:0/69901795 shutdown_connections 2023-12-02T00:31:15.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.547+0000 7f954384f640 1 --2- 172.21.15.46:0/69901795 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f952806ae40 0x7f952806d300 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:15.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.547+0000 7f954384f640 1 --2- 172.21.15.46:0/69901795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f953c160fb0 0x7f953c0cb940 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:15.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.547+0000 7f954384f640 1 --2- 172.21.15.46:0/69901795 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f953c0d0ed0 0x7f953c0cdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:15.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.547+0000 7f954384f640 1 --2- 172.21.15.46:0/69901795 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f953c06ea50 0x7f953c1566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:15.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.547+0000 7f954384f640 1 -- 172.21.15.46:0/69901795 >> 172.21.15.46:0/69901795 conn(0x7f953c06a6b0 msgr2=0x7f953c161410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:15.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.547+0000 7f954384f640 1 -- 172.21.15.46:0/69901795 shutdown_connections 2023-12-02T00:31:15.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.547+0000 7f954384f640 1 -- 172.21.15.46:0/69901795 wait complete. 2023-12-02T00:31:15.559 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-02T00:31:15.560 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-02T00:31:15.576 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:31:15.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.721+0000 7fb1363e5640 1 Processor -- start 2023-12-02T00:31:15.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.721+0000 7fb1363e5640 1 -- start start 2023-12-02T00:31:15.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.721+0000 7fb1363e5640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb13015d1d0 0x7fb13006b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:15.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.721+0000 7fb1363e5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb1300ad640 0x7fb13006ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:15.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.721+0000 7fb1363e5640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1300adae0 con 0x7fb1300aeb80 2023-12-02T00:31:15.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.721+0000 7fb1363e5640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1300addc0 con 0x7fb13015d1d0 2023-12-02T00:31:15.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.721+0000 7fb1363e5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1300ae0a0 con 0x7fb1300ad640 2023-12-02T00:31:15.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.722+0000 7fb12ffff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb13015d1d0 0x7fb13006b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:15.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.722+0000 7fb12ffff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb13015d1d0 0x7fb13006b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39674/0 (socket says 172.21.15.46:39674) 2023-12-02T00:31:15.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.722+0000 7fb12ffff640 1 -- 172.21.15.46:0/3017770768 learned_addr learned my addr 172.21.15.46:0/3017770768 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:15.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.722+0000 7fb12f7fe640 1 --2- 172.21.15.46:0/3017770768 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb1300ad640 0x7fb13006ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:15.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.722+0000 7fb12ffff640 1 -- 172.21.15.46:0/3017770768 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb1300ad640 msgr2=0x7fb13006ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:15.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.722+0000 7fb12ffff640 1 --2- 172.21.15.46:0/3017770768 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb1300ad640 0x7fb13006ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:15.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.722+0000 7fb12ffff640 1 -- 172.21.15.46:0/3017770768 >> v1:172.21.15.162:6789/0 conn(0x7fb1300aeb80 legacy=0x7fb130160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:15.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.722+0000 7fb12ffff640 1 -- 172.21.15.46:0/3017770768 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1300ae380 con 0x7fb13015d1d0 2023-12-02T00:31:15.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.723+0000 7fb12ffff640 1 --2- 172.21.15.46:0/3017770768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb13015d1d0 0x7fb13006b4e0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fb120009870 tx=0x7fb12002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d8644f8de1c0cf9 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:15.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.723+0000 7fb12effd640 1 -- 172.21.15.46:0/3017770768 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb120030840 con 0x7fb13015d1d0 2023-12-02T00:31:15.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.723+0000 7fb12effd640 1 -- 172.21.15.46:0/3017770768 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1200309a0 con 0x7fb13015d1d0 2023-12-02T00:31:15.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.723+0000 7fb12effd640 1 -- 172.21.15.46:0/3017770768 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb120030d30 con 0x7fb13015d1d0 2023-12-02T00:31:15.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.723+0000 7fb1363e5640 1 -- 172.21.15.46:0/3017770768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb13015d1d0 msgr2=0x7fb13006b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:15.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.723+0000 7fb1363e5640 1 --2- 172.21.15.46:0/3017770768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb13015d1d0 0x7fb13006b4e0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fb120009870 tx=0x7fb12002b980 comp rx=0 tx=0).stop 2023-12-02T00:31:15.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.723+0000 7fb1363e5640 1 -- 172.21.15.46:0/3017770768 shutdown_connections 2023-12-02T00:31:15.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.723+0000 7fb1363e5640 1 --2- 172.21.15.46:0/3017770768 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb1300ad640 0x7fb13006ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:15.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.723+0000 7fb1363e5640 1 --2- 172.21.15.46:0/3017770768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb13015d1d0 0x7fb13006b4e0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:15.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.723+0000 7fb1363e5640 1 -- 172.21.15.46:0/3017770768 >> 172.21.15.46:0/3017770768 conn(0x7fb130070e70 msgr2=0x7fb130071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:15.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.723+0000 7fb1363e5640 1 -- 172.21.15.46:0/3017770768 shutdown_connections 2023-12-02T00:31:15.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.723+0000 7fb1363e5640 1 -- 172.21.15.46:0/3017770768 wait complete. 2023-12-02T00:31:15.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.725+0000 7fb1363e5640 1 Processor -- start 2023-12-02T00:31:15.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.725+0000 7fb1363e5640 1 -- start start 2023-12-02T00:31:15.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.725+0000 7fb1363e5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb1300ad640 0x7fb1301503c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:15.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.725+0000 7fb1363e5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb1300aeb80 0x7fb1300ced50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:15.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.725+0000 7fb1363e5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb13015d1d0 0x7fb1300cf290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:15.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.725+0000 7fb1363e5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13015a120 con 0x7fb13015d1d0 2023-12-02T00:31:15.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.725+0000 7fb1363e5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13015a5b0 con 0x7fb1300aeb80 2023-12-02T00:31:15.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.725+0000 7fb1363e5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb130159510 con 0x7fb1300ad640 2023-12-02T00:31:15.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.726+0000 7fb134982640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb13015d1d0 0x7fb1300cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:15.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.726+0000 7fb134982640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb13015d1d0 0x7fb1300cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39680/0 (socket says 172.21.15.46:39680) 2023-12-02T00:31:15.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.726+0000 7fb134982640 1 -- 172.21.15.46:0/2000941380 learned_addr learned my addr 172.21.15.46:0/2000941380 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:15.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.726+0000 7fb12ffff640 1 --2- 172.21.15.46:0/2000941380 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb1300ad640 0x7fb1301503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:15.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.726+0000 7fb12f7fe640 1 --2- 172.21.15.46:0/2000941380 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb1300aeb80 0x7fb1300ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:15.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.726+0000 7fb134982640 1 -- 172.21.15.46:0/2000941380 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb1300ad640 msgr2=0x7fb1301503c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:15.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.726+0000 7fb134982640 1 --2- 172.21.15.46:0/2000941380 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb1300ad640 0x7fb1301503c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:15.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.726+0000 7fb134982640 1 -- 172.21.15.46:0/2000941380 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb1300aeb80 msgr2=0x7fb1300ced50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:15.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.726+0000 7fb134982640 1 --2- 172.21.15.46:0/2000941380 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb1300aeb80 0x7fb1300ced50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:15.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.726+0000 7fb134982640 1 -- 172.21.15.46:0/2000941380 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1200094d0 con 0x7fb13015d1d0 2023-12-02T00:31:15.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.726+0000 7fb134982640 1 --2- 172.21.15.46:0/2000941380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb13015d1d0 0x7fb1300cf290 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fb12800c7a0 tx=0x7fb12800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:15.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.726+0000 7fb12cff9640 1 -- 172.21.15.46:0/2000941380 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb128003b70 con 0x7fb13015d1d0 2023-12-02T00:31:15.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.726+0000 7fb12cff9640 1 -- 172.21.15.46:0/2000941380 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb128003cf0 con 0x7fb13015d1d0 2023-12-02T00:31:15.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.726+0000 7fb1363e5640 1 -- 172.21.15.46:0/2000941380 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1301597f0 con 0x7fb13015d1d0 2023-12-02T00:31:15.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.726+0000 7fb12cff9640 1 -- 172.21.15.46:0/2000941380 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb128003ea0 con 0x7fb13015d1d0 2023-12-02T00:31:15.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.726+0000 7fb1363e5640 1 -- 172.21.15.46:0/2000941380 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb130159db0 con 0x7fb13015d1d0 2023-12-02T00:31:15.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.727+0000 7fb1363e5640 1 -- 172.21.15.46:0/2000941380 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb13015a290 con 0x7fb13015d1d0 2023-12-02T00:31:15.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.728+0000 7fb12cff9640 1 -- 172.21.15.46:0/2000941380 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb12800aa90 con 0x7fb13015d1d0 2023-12-02T00:31:15.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.728+0000 7fb12cff9640 1 --2- 172.21.15.46:0/2000941380 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb11c06ae40 0x7fb11c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:15.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.728+0000 7fb12cff9640 1 -- 172.21.15.46:0/2000941380 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb1280892e0 con 0x7fb13015d1d0 2023-12-02T00:31:15.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.728+0000 7fb12ffff640 1 --2- 172.21.15.46:0/2000941380 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb11c06ae40 0x7fb11c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:15.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.729+0000 7fb12ffff640 1 --2- 172.21.15.46:0/2000941380 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb11c06ae40 0x7fb11c06d300 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb120036000 tx=0x7fb120008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:15.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.732+0000 7fb12cff9640 1 -- 172.21.15.46:0/2000941380 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb128054440 con 0x7fb13015d1d0 2023-12-02T00:31:15.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.911+0000 7fb1363e5640 1 -- 172.21.15.46:0/2000941380 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb130069400 con 0x7fb13015d1d0 2023-12-02T00:31:15.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.912+0000 7fb12cff9640 1 -- 172.21.15.46:0/2000941380 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+16545 (secure 0 0 0) 0x7fb128057a60 con 0x7fb13015d1d0 2023-12-02T00:31:15.913 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:31:15.913 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":51,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:31:02.837102+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-12-02T00:30:59.469070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-12-02T00:31:00.174610+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:31:15.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.914+0000 7fb1363e5640 1 -- 172.21.15.46:0/2000941380 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb11c06ae40 msgr2=0x7fb11c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:15.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.914+0000 7fb1363e5640 1 --2- 172.21.15.46:0/2000941380 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb11c06ae40 0x7fb11c06d300 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb120036000 tx=0x7fb120008030 comp rx=0 tx=0).stop 2023-12-02T00:31:15.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.914+0000 7fb1363e5640 1 -- 172.21.15.46:0/2000941380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb13015d1d0 msgr2=0x7fb1300cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:15.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.914+0000 7fb1363e5640 1 --2- 172.21.15.46:0/2000941380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb13015d1d0 0x7fb1300cf290 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fb12800c7a0 tx=0x7fb12800cc60 comp rx=0 tx=0).stop 2023-12-02T00:31:15.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.914+0000 7fb1363e5640 1 -- 172.21.15.46:0/2000941380 shutdown_connections 2023-12-02T00:31:15.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.914+0000 7fb1363e5640 1 --2- 172.21.15.46:0/2000941380 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb11c06ae40 0x7fb11c06d300 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:15.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.914+0000 7fb1363e5640 1 --2- 172.21.15.46:0/2000941380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb13015d1d0 0x7fb1300cf290 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:15.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.914+0000 7fb1363e5640 1 --2- 172.21.15.46:0/2000941380 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb1300aeb80 0x7fb1300ced50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:15.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.914+0000 7fb1363e5640 1 --2- 172.21.15.46:0/2000941380 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb1300ad640 0x7fb1301503c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:15.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.914+0000 7fb1363e5640 1 -- 172.21.15.46:0/2000941380 >> 172.21.15.46:0/2000941380 conn(0x7fb130070e70 msgr2=0x7fb13006a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:15.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.914+0000 7fb1363e5640 1 -- 172.21.15.46:0/2000941380 shutdown_connections 2023-12-02T00:31:15.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:15.914+0000 7fb1363e5640 1 -- 172.21.15.46:0/2000941380 wait complete. 2023-12-02T00:31:15.926 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-02T00:28:37.213645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-02T00:30:59.469070+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-12-02T00:31:00.174610+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-02T00:31:15.927 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-02T00:31:16.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.035+0000 7fc94279e640 1 Processor -- start 2023-12-02T00:31:16.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.035+0000 7fc94279e640 1 -- start start 2023-12-02T00:31:16.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.036+0000 7fc94279e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc93c15d0f0 0x7fc93c0d3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:16.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.036+0000 7fc94279e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc93c15d4d0 0x7fc93c0d0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:16.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.036+0000 7fc94279e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc93c0d1320 con 0x7fc93c06aff0 2023-12-02T00:31:16.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.036+0000 7fc94279e640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc93c0cddb0 con 0x7fc93c15d0f0 2023-12-02T00:31:16.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.036+0000 7fc94279e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc93c0d0040 con 0x7fc93c15d4d0 2023-12-02T00:31:16.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.036+0000 7fc94179c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc93c15d0f0 0x7fc93c0d3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:16.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.036+0000 7fc94179c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc93c15d0f0 0x7fc93c0d3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39682/0 (socket says 172.21.15.46:39682) 2023-12-02T00:31:16.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.036+0000 7fc94179c640 1 -- 172.21.15.46:0/1619185605 learned_addr learned my addr 172.21.15.46:0/1619185605 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:16.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.037+0000 7fc940f9b640 1 --2- 172.21.15.46:0/1619185605 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc93c15d4d0 0x7fc93c0d0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:16.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.037+0000 7fc94179c640 1 -- 172.21.15.46:0/1619185605 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc93c15d4d0 msgr2=0x7fc93c0d0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:16.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.037+0000 7fc94179c640 1 --2- 172.21.15.46:0/1619185605 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc93c15d4d0 0x7fc93c0d0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:16.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.037+0000 7fc94179c640 1 -- 172.21.15.46:0/1619185605 >> v1:172.21.15.162:6789/0 conn(0x7fc93c06aff0 legacy=0x7fc93c0d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:16.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.037+0000 7fc94179c640 1 -- 172.21.15.46:0/1619185605 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc93c0d0320 con 0x7fc93c15d0f0 2023-12-02T00:31:16.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.037+0000 7fc94179c640 1 --2- 172.21.15.46:0/1619185605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc93c15d0f0 0x7fc93c0d3b60 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fc93c08bed0 tx=0x7fc93402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27f6c189d25dfa89 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:16.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.038+0000 7fc93bfff640 1 -- 172.21.15.46:0/1619185605 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc934030610 con 0x7fc93c15d0f0 2023-12-02T00:31:16.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.038+0000 7fc93bfff640 1 -- 172.21.15.46:0/1619185605 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc934030770 con 0x7fc93c15d0f0 2023-12-02T00:31:16.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.038+0000 7fc93bfff640 1 -- 172.21.15.46:0/1619185605 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc934030b00 con 0x7fc93c15d0f0 2023-12-02T00:31:16.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.038+0000 7fc94279e640 1 -- 172.21.15.46:0/1619185605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc93c15d0f0 msgr2=0x7fc93c0d3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:16.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.038+0000 7fc94279e640 1 --2- 172.21.15.46:0/1619185605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc93c15d0f0 0x7fc93c0d3b60 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fc93c08bed0 tx=0x7fc93402b960 comp rx=0 tx=0).stop 2023-12-02T00:31:16.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.038+0000 7fc94279e640 1 -- 172.21.15.46:0/1619185605 shutdown_connections 2023-12-02T00:31:16.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.038+0000 7fc94279e640 1 --2- 172.21.15.46:0/1619185605 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc93c15d4d0 0x7fc93c0d0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:16.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.038+0000 7fc94279e640 1 --2- 172.21.15.46:0/1619185605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc93c15d0f0 0x7fc93c0d3b60 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:16.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.038+0000 7fc94279e640 1 -- 172.21.15.46:0/1619185605 >> 172.21.15.46:0/1619185605 conn(0x7fc93c09c320 msgr2=0x7fc93c0d3000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:16.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.038+0000 7fc94279e640 1 -- 172.21.15.46:0/1619185605 shutdown_connections 2023-12-02T00:31:16.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.038+0000 7fc94279e640 1 -- 172.21.15.46:0/1619185605 wait complete. 2023-12-02T00:31:16.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.040+0000 7fc94279e640 1 Processor -- start 2023-12-02T00:31:16.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.040+0000 7fc94279e640 1 -- start start 2023-12-02T00:31:16.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.040+0000 7fc94279e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc93c06aff0 0x7fc93c0b7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:16.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.040+0000 7fc94279e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc93c15d0f0 0x7fc93c0ca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:16.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.040+0000 7fc94279e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc93c15d4d0 0x7fc93c0b7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:16.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.040+0000 7fc94279e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93c0ca610 con 0x7fc93c06aff0 2023-12-02T00:31:16.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.040+0000 7fc94279e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93c0b84c0 con 0x7fc93c15d4d0 2023-12-02T00:31:16.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.040+0000 7fc94279e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc93c0b87a0 con 0x7fc93c15d0f0 2023-12-02T00:31:16.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.040+0000 7fc94179c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc93c06aff0 0x7fc93c0b7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:16.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.040+0000 7fc94179c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc93c06aff0 0x7fc93c0b7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39696/0 (socket says 172.21.15.46:39696) 2023-12-02T00:31:16.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.040+0000 7fc94179c640 1 -- 172.21.15.46:0/3157547324 learned_addr learned my addr 172.21.15.46:0/3157547324 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:16.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.041+0000 7fc940f9b640 1 --2- 172.21.15.46:0/3157547324 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc93c15d0f0 0x7fc93c0ca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:16.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.041+0000 7fc941f9d640 1 --2- 172.21.15.46:0/3157547324 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc93c15d4d0 0x7fc93c0b7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:16.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.041+0000 7fc94179c640 1 -- 172.21.15.46:0/3157547324 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc93c15d0f0 msgr2=0x7fc93c0ca0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:16.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.041+0000 7fc94179c640 1 --2- 172.21.15.46:0/3157547324 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc93c15d0f0 0x7fc93c0ca0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:16.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.041+0000 7fc94179c640 1 -- 172.21.15.46:0/3157547324 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc93c15d4d0 msgr2=0x7fc93c0b7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:16.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.041+0000 7fc94179c640 1 --2- 172.21.15.46:0/3157547324 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc93c15d4d0 0x7fc93c0b7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:16.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.041+0000 7fc94179c640 1 -- 172.21.15.46:0/3157547324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9340094d0 con 0x7fc93c06aff0 2023-12-02T00:31:16.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.041+0000 7fc94179c640 1 --2- 172.21.15.46:0/3157547324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc93c06aff0 0x7fc93c0b7e30 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fc934000c00 tx=0x7fc934004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:16.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.041+0000 7fc939ffb640 1 -- 172.21.15.46:0/3157547324 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc934039c90 con 0x7fc93c06aff0 2023-12-02T00:31:16.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.041+0000 7fc939ffb640 1 -- 172.21.15.46:0/3157547324 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc934039df0 con 0x7fc93c06aff0 2023-12-02T00:31:16.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.041+0000 7fc94279e640 1 -- 172.21.15.46:0/3157547324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc93c0b8a80 con 0x7fc93c06aff0 2023-12-02T00:31:16.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.041+0000 7fc939ffb640 1 -- 172.21.15.46:0/3157547324 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc934043620 con 0x7fc93c06aff0 2023-12-02T00:31:16.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.041+0000 7fc94279e640 1 -- 172.21.15.46:0/3157547324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc93c0cbf00 con 0x7fc93c06aff0 2023-12-02T00:31:16.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.043+0000 7fc94279e640 1 -- 172.21.15.46:0/3157547324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc93c067940 con 0x7fc93c06aff0 2023-12-02T00:31:16.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.043+0000 7fc939ffb640 1 -- 172.21.15.46:0/3157547324 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc934043780 con 0x7fc93c06aff0 2023-12-02T00:31:16.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.043+0000 7fc939ffb640 1 --2- 172.21.15.46:0/3157547324 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc92806ae40 0x7fc92806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:16.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.043+0000 7fc939ffb640 1 -- 172.21.15.46:0/3157547324 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc93403e070 con 0x7fc93c06aff0 2023-12-02T00:31:16.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.043+0000 7fc940f9b640 1 --2- 172.21.15.46:0/3157547324 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc92806ae40 0x7fc92806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:16.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.044+0000 7fc940f9b640 1 --2- 172.21.15.46:0/3157547324 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc92806ae40 0x7fc92806d300 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc930006fd0 tx=0x7fc930008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:16.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.048+0000 7fc939ffb640 1 -- 172.21.15.46:0/3157547324 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc934077520 con 0x7fc93c06aff0 2023-12-02T00:31:16.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.223+0000 7fc94279e640 1 -- 172.21.15.46:0/3157547324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc93c069320 con 0x7fc93c06aff0 2023-12-02T00:31:16.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.223+0000 7fc939ffb640 1 -- 172.21.15.46:0/3157547324 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc93407ab40 con 0x7fc93c06aff0 2023-12-02T00:31:16.224 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-12-02T00:31:16.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.225+0000 7fc94279e640 1 -- 172.21.15.46:0/3157547324 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc92806ae40 msgr2=0x7fc92806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:16.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.225+0000 7fc94279e640 1 --2- 172.21.15.46:0/3157547324 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc92806ae40 0x7fc92806d300 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc930006fd0 tx=0x7fc930008040 comp rx=0 tx=0).stop 2023-12-02T00:31:16.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.225+0000 7fc94279e640 1 -- 172.21.15.46:0/3157547324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc93c06aff0 msgr2=0x7fc93c0b7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:16.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.225+0000 7fc94279e640 1 --2- 172.21.15.46:0/3157547324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc93c06aff0 0x7fc93c0b7e30 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fc934000c00 tx=0x7fc934004730 comp rx=0 tx=0).stop 2023-12-02T00:31:16.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.225+0000 7fc94279e640 1 -- 172.21.15.46:0/3157547324 shutdown_connections 2023-12-02T00:31:16.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.225+0000 7fc94279e640 1 --2- 172.21.15.46:0/3157547324 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc92806ae40 0x7fc92806d300 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:16.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.225+0000 7fc94279e640 1 --2- 172.21.15.46:0/3157547324 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc93c15d4d0 0x7fc93c0b7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:16.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.225+0000 7fc94279e640 1 --2- 172.21.15.46:0/3157547324 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc93c15d0f0 0x7fc93c0ca0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:16.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.225+0000 7fc94279e640 1 --2- 172.21.15.46:0/3157547324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc93c06aff0 0x7fc93c0b7e30 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:16.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.225+0000 7fc94279e640 1 -- 172.21.15.46:0/3157547324 >> 172.21.15.46:0/3157547324 conn(0x7fc93c09c320 msgr2=0x7fc93c0ce570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:16.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.225+0000 7fc94279e640 1 -- 172.21.15.46:0/3157547324 shutdown_connections 2023-12-02T00:31:16.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.225+0000 7fc94279e640 1 -- 172.21.15.46:0/3157547324 wait complete. 2023-12-02T00:31:16.237 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-02T00:31:16.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.345+0000 7f1844f8a640 1 Processor -- start 2023-12-02T00:31:16.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.345+0000 7f1844f8a640 1 -- start start 2023-12-02T00:31:16.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.346+0000 7f1844f8a640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1840144980 0x7f184015c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:16.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.346+0000 7f1844f8a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f18400b4370 0x7f18400b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:16.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.346+0000 7f1844f8a640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1840071770 con 0x7f18400ceb70 2023-12-02T00:31:16.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.346+0000 7f1844f8a640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18400ccb60 con 0x7f1840144980 2023-12-02T00:31:16.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.346+0000 7f1844f8a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18400b4ce0 con 0x7f18400b4370 2023-12-02T00:31:16.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.346+0000 7f183e59c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1840144980 0x7f184015c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:16.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.346+0000 7f183e59c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1840144980 0x7f184015c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39710/0 (socket says 172.21.15.46:39710) 2023-12-02T00:31:16.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.346+0000 7f183e59c640 1 -- 172.21.15.46:0/1731295288 learned_addr learned my addr 172.21.15.46:0/1731295288 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:16.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.346+0000 7f183ed9d640 1 --2- 172.21.15.46:0/1731295288 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f18400b4370 0x7f18400b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:16.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.346+0000 7f183e59c640 1 -- 172.21.15.46:0/1731295288 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f18400b4370 msgr2=0x7f18400b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:16.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.346+0000 7f183e59c640 1 --2- 172.21.15.46:0/1731295288 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f18400b4370 0x7f18400b47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:16.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.346+0000 7f183e59c640 1 -- 172.21.15.46:0/1731295288 >> v1:172.21.15.162:6789/0 conn(0x7f18400ceb70 legacy=0x7f18400cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:16.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.346+0000 7f183e59c640 1 -- 172.21.15.46:0/1731295288 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18400ccdc0 con 0x7f1840144980 2023-12-02T00:31:16.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.347+0000 7f183e59c640 1 --2- 172.21.15.46:0/1731295288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1840144980 0x7f184015c7c0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f1830009870 tx=0x7f183002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9d745a4c0e424e server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:16.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.347+0000 7f183dd9b640 1 -- 172.21.15.46:0/1731295288 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1830030840 con 0x7f1840144980 2023-12-02T00:31:16.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.347+0000 7f183dd9b640 1 -- 172.21.15.46:0/1731295288 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18300309a0 con 0x7f1840144980 2023-12-02T00:31:16.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.347+0000 7f183dd9b640 1 -- 172.21.15.46:0/1731295288 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1830030d30 con 0x7f1840144980 2023-12-02T00:31:16.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.347+0000 7f1844f8a640 1 -- 172.21.15.46:0/1731295288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1840144980 msgr2=0x7f184015c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:16.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.347+0000 7f1844f8a640 1 --2- 172.21.15.46:0/1731295288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1840144980 0x7f184015c7c0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f1830009870 tx=0x7f183002b980 comp rx=0 tx=0).stop 2023-12-02T00:31:16.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.347+0000 7f1844f8a640 1 -- 172.21.15.46:0/1731295288 shutdown_connections 2023-12-02T00:31:16.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.347+0000 7f1844f8a640 1 --2- 172.21.15.46:0/1731295288 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f18400b4370 0x7f18400b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:16.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.347+0000 7f1844f8a640 1 --2- 172.21.15.46:0/1731295288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1840144980 0x7f184015c7c0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:16.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.347+0000 7f1844f8a640 1 -- 172.21.15.46:0/1731295288 >> 172.21.15.46:0/1731295288 conn(0x7f184006a4d0 msgr2=0x7f184006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:16.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.348+0000 7f1844f8a640 1 -- 172.21.15.46:0/1731295288 shutdown_connections 2023-12-02T00:31:16.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.348+0000 7f1844f8a640 1 -- 172.21.15.46:0/1731295288 wait complete. 2023-12-02T00:31:16.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.349+0000 7f1844f8a640 1 Processor -- start 2023-12-02T00:31:16.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.349+0000 7f1844f8a640 1 -- start start 2023-12-02T00:31:16.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.350+0000 7f1844f8a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18400b4370 0x7f18400ae460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:16.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.350+0000 7f1844f8a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f18400ceb70 0x7f18400b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:16.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.350+0000 7f1844f8a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1840144980 0x7f1840140230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:16.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.350+0000 7f1844f8a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1840140770 con 0x7f1840144980 2023-12-02T00:31:16.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.350+0000 7f1844f8a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18400afb30 con 0x7f18400b4370 2023-12-02T00:31:16.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.350+0000 7f1844f8a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1840140c20 con 0x7f18400ceb70 2023-12-02T00:31:16.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.350+0000 7f183ed9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1840144980 0x7f1840140230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:16.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.350+0000 7f183ed9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1840144980 0x7f1840140230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39714/0 (socket says 172.21.15.46:39714) 2023-12-02T00:31:16.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.350+0000 7f183ed9d640 1 -- 172.21.15.46:0/747173182 learned_addr learned my addr 172.21.15.46:0/747173182 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:16.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.350+0000 7f183e59c640 1 --2- 172.21.15.46:0/747173182 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18400b4370 0x7f18400ae460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:16.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.350+0000 7f1835d9b640 1 --2- 172.21.15.46:0/747173182 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f18400ceb70 0x7f18400b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:16.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.350+0000 7f183ed9d640 1 -- 172.21.15.46:0/747173182 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f18400ceb70 msgr2=0x7f18400b3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:16.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.350+0000 7f183ed9d640 1 --2- 172.21.15.46:0/747173182 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f18400ceb70 0x7f18400b3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:16.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.350+0000 7f183ed9d640 1 -- 172.21.15.46:0/747173182 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18400b4370 msgr2=0x7f18400ae460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:16.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.350+0000 7f183ed9d640 1 --2- 172.21.15.46:0/747173182 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18400b4370 0x7f18400ae460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:16.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.350+0000 7f183ed9d640 1 -- 172.21.15.46:0/747173182 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18300094d0 con 0x7f1840144980 2023-12-02T00:31:16.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.350+0000 7f183ed9d640 1 --2- 172.21.15.46:0/747173182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1840144980 0x7f1840140230 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f183800efc0 tx=0x7f183800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:16.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.351+0000 7f18377fe640 1 -- 172.21.15.46:0/747173182 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1838009ba0 con 0x7f1840144980 2023-12-02T00:31:16.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.351+0000 7f18377fe640 1 -- 172.21.15.46:0/747173182 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f183800f040 con 0x7f1840144980 2023-12-02T00:31:16.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.351+0000 7f18377fe640 1 -- 172.21.15.46:0/747173182 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18380146c0 con 0x7f1840144980 2023-12-02T00:31:16.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.351+0000 7f1844f8a640 1 -- 172.21.15.46:0/747173182 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18400b4140 con 0x7f1840144980 2023-12-02T00:31:16.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.351+0000 7f1844f8a640 1 -- 172.21.15.46:0/747173182 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18400af050 con 0x7f1840144980 2023-12-02T00:31:16.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.352+0000 7f1844f8a640 1 -- 172.21.15.46:0/747173182 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1840069400 con 0x7f1840144980 2023-12-02T00:31:16.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.352+0000 7f18377fe640 1 -- 172.21.15.46:0/747173182 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1838014820 con 0x7f1840144980 2023-12-02T00:31:16.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.352+0000 7f18377fe640 1 --2- 172.21.15.46:0/747173182 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f182006ae40 0x7f182006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:16.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.353+0000 7f18377fe640 1 -- 172.21.15.46:0/747173182 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f183808a2e0 con 0x7f1840144980 2023-12-02T00:31:16.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.353+0000 7f183e59c640 1 --2- 172.21.15.46:0/747173182 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f182006ae40 0x7f182006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:16.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.353+0000 7f183e59c640 1 --2- 172.21.15.46:0/747173182 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f182006ae40 0x7f182006d300 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1830036000 tx=0x7f1830008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:16.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.357+0000 7f18377fe640 1 -- 172.21.15.46:0/747173182 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f18380553c0 con 0x7f1840144980 2023-12-02T00:31:16.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.532+0000 7f1844f8a640 1 -- 172.21.15.46:0/747173182 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f184015d9c0 con 0x7f1840144980 2023-12-02T00:31:16.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.532+0000 7f18377fe640 1 -- 172.21.15.46:0/747173182 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v51) v1 ==== 104+0+11 (secure 0 0 0) 0x7f183805de60 con 0x7f1840144980 2023-12-02T00:31:16.534 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-12-02T00:31:16.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.534+0000 7f1844f8a640 1 -- 172.21.15.46:0/747173182 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f182006ae40 msgr2=0x7f182006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:16.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.534+0000 7f1844f8a640 1 --2- 172.21.15.46:0/747173182 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f182006ae40 0x7f182006d300 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1830036000 tx=0x7f1830008030 comp rx=0 tx=0).stop 2023-12-02T00:31:16.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.534+0000 7f1844f8a640 1 -- 172.21.15.46:0/747173182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1840144980 msgr2=0x7f1840140230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:16.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.534+0000 7f1844f8a640 1 --2- 172.21.15.46:0/747173182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1840144980 0x7f1840140230 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f183800efc0 tx=0x7f183800c490 comp rx=0 tx=0).stop 2023-12-02T00:31:16.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.534+0000 7f1844f8a640 1 -- 172.21.15.46:0/747173182 shutdown_connections 2023-12-02T00:31:16.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.534+0000 7f1844f8a640 1 --2- 172.21.15.46:0/747173182 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f182006ae40 0x7f182006d300 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:16.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.534+0000 7f1844f8a640 1 --2- 172.21.15.46:0/747173182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1840144980 0x7f1840140230 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:16.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.534+0000 7f1844f8a640 1 --2- 172.21.15.46:0/747173182 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f18400ceb70 0x7f18400b3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:16.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.534+0000 7f1844f8a640 1 --2- 172.21.15.46:0/747173182 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18400b4370 0x7f18400ae460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:16.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.534+0000 7f1844f8a640 1 -- 172.21.15.46:0/747173182 >> 172.21.15.46:0/747173182 conn(0x7f184006a4d0 msgr2=0x7f184015a780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:16.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.534+0000 7f1844f8a640 1 -- 172.21.15.46:0/747173182 shutdown_connections 2023-12-02T00:31:16.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.534+0000 7f1844f8a640 1 -- 172.21.15.46:0/747173182 wait complete. 2023-12-02T00:31:16.546 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-02T00:31:16.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.654+0000 7f62445d7640 1 Processor -- start 2023-12-02T00:31:16.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.655+0000 7f62445d7640 1 -- start start 2023-12-02T00:31:16.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.655+0000 7f62445d7640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f623c160fb0 0x7f623c161440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:16.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.655+0000 7f62445d7640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f623c161d20 con 0x7f623c0d0ed0 2023-12-02T00:31:16.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.655+0000 7f62445d7640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f623c15e760 con 0x7f623c06ea50 2023-12-02T00:31:16.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.655+0000 7f62445d7640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f623c15d9a0 con 0x7f623c160fb0 2023-12-02T00:31:16.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.656+0000 7f6241b72640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f623c160fb0 0x7f623c161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:16.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.656+0000 7f6242b74640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f623c0d0ed0 0x7f623c0d1320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57168/0 (socket says 172.21.15.46:57168) 2023-12-02T00:31:16.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.656+0000 7f6242b74640 1 -- 172.21.15.46:0/1444532831 learned_addr learned my addr 172.21.15.46:0/1444532831 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:31:16.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.656+0000 7f6241b72640 1 -- 172.21.15.46:0/1444532831 >> v1:172.21.15.162:6789/0 conn(0x7f623c06ea50 legacy=0x7f623c06ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:16.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.656+0000 7f6241b72640 1 -- 172.21.15.46:0/1444532831 >> v1:172.21.15.46:6789/0 conn(0x7f623c0d0ed0 legacy=0x7f623c0d1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:16.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.656+0000 7f6241371640 1 -- 172.21.15.46:0/1444532831 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1214163375 0 0) 0x7f623c161d20 con 0x7f623c0d0ed0 2023-12-02T00:31:16.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.656+0000 7f6241b72640 1 -- 172.21.15.46:0/1444532831 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f623c15dc80 con 0x7f623c160fb0 2023-12-02T00:31:16.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.657+0000 7f6241b72640 1 --2- 172.21.15.46:0/1444532831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f623c160fb0 0x7f623c161440 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f622c0097a0 tx=0x7f622c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c35d60bac180be4b server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:16.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.657+0000 7f6241371640 1 -- 172.21.15.46:0/1444532831 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622c031830 con 0x7f623c160fb0 2023-12-02T00:31:16.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.657+0000 7f6241371640 1 -- 172.21.15.46:0/1444532831 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f622c031990 con 0x7f623c160fb0 2023-12-02T00:31:16.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.657+0000 7f6241371640 1 -- 172.21.15.46:0/1444532831 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622c031d20 con 0x7f623c160fb0 2023-12-02T00:31:16.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.657+0000 7f62445d7640 1 -- 172.21.15.46:0/1444532831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f623c160fb0 msgr2=0x7f623c161440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:16.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.657+0000 7f62445d7640 1 --2- 172.21.15.46:0/1444532831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f623c160fb0 0x7f623c161440 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f622c0097a0 tx=0x7f622c02b8d0 comp rx=0 tx=0).stop 2023-12-02T00:31:16.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.658+0000 7f62445d7640 1 -- 172.21.15.46:0/1444532831 shutdown_connections 2023-12-02T00:31:16.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.658+0000 7f62445d7640 1 --2- 172.21.15.46:0/1444532831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f623c160fb0 0x7f623c161440 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:16.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.658+0000 7f62445d7640 1 -- 172.21.15.46:0/1444532831 >> 172.21.15.46:0/1444532831 conn(0x7f623c06a6b0 msgr2=0x7f623c06aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:16.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.658+0000 7f62445d7640 1 -- 172.21.15.46:0/1444532831 shutdown_connections 2023-12-02T00:31:16.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.658+0000 7f62445d7640 1 -- 172.21.15.46:0/1444532831 wait complete. 2023-12-02T00:31:16.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.659+0000 7f62445d7640 1 Processor -- start 2023-12-02T00:31:16.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.659+0000 7f62445d7640 1 -- start start 2023-12-02T00:31:16.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.659+0000 7f62445d7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f623c06ea50 0x7f623c156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:16.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.659+0000 7f62445d7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f623c0d0ed0 0x7f623c1e8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:16.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.659+0000 7f62445d7640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f623c160fb0 0x7f623c15a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:16.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.659+0000 7f62445d7640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623c1e7e00 con 0x7f623c06ea50 2023-12-02T00:31:16.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.660+0000 7f62445d7640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623c1e7180 con 0x7f623c160fb0 2023-12-02T00:31:16.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.660+0000 7f62445d7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f623c1e7460 con 0x7f623c0d0ed0 2023-12-02T00:31:16.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.660+0000 7f6242373640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f623c06ea50 0x7f623c156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:16.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.660+0000 7f6242373640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f623c06ea50 0x7f623c156b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39726/0 (socket says 172.21.15.46:39726) 2023-12-02T00:31:16.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.660+0000 7f6242373640 1 -- 172.21.15.46:0/1455667155 learned_addr learned my addr 172.21.15.46:0/1455667155 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:16.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.660+0000 7f6241b72640 1 --2- 172.21.15.46:0/1455667155 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f623c0d0ed0 0x7f623c1e8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:16.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.660+0000 7f6242b74640 1 --2- 172.21.15.46:0/1455667155 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f623c160fb0 0x7f623c15a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:16.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.660+0000 7f6242373640 1 -- 172.21.15.46:0/1455667155 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f623c0d0ed0 msgr2=0x7f623c1e8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:16.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.660+0000 7f6242373640 1 --2- 172.21.15.46:0/1455667155 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f623c0d0ed0 0x7f623c1e8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:16.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.660+0000 7f6242373640 1 -- 172.21.15.46:0/1455667155 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f623c160fb0 msgr2=0x7f623c15a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:16.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.660+0000 7f6242373640 1 --2- 172.21.15.46:0/1455667155 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f623c160fb0 0x7f623c15a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:16.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.660+0000 7f6242373640 1 -- 172.21.15.46:0/1455667155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f622c009400 con 0x7f623c06ea50 2023-12-02T00:31:16.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.660+0000 7f6242373640 1 --2- 172.21.15.46:0/1455667155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f623c06ea50 0x7f623c156b90 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f623800c400 tx=0x7f623800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:16.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.660+0000 7f6232ffd640 1 -- 172.21.15.46:0/1455667155 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62380052d0 con 0x7f623c06ea50 2023-12-02T00:31:16.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.660+0000 7f6232ffd640 1 -- 172.21.15.46:0/1455667155 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6238005430 con 0x7f623c06ea50 2023-12-02T00:31:16.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.660+0000 7f62445d7640 1 -- 172.21.15.46:0/1455667155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f623c1e7740 con 0x7f623c06ea50 2023-12-02T00:31:16.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.660+0000 7f6232ffd640 1 -- 172.21.15.46:0/1455667155 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6238003830 con 0x7f623c06ea50 2023-12-02T00:31:16.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.660+0000 7f62445d7640 1 -- 172.21.15.46:0/1455667155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f623c1e6de0 con 0x7f623c06ea50 2023-12-02T00:31:16.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.661+0000 7f62445d7640 1 -- 172.21.15.46:0/1455667155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f623c069400 con 0x7f623c06ea50 2023-12-02T00:31:16.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.662+0000 7f6232ffd640 1 -- 172.21.15.46:0/1455667155 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6238003990 con 0x7f623c06ea50 2023-12-02T00:31:16.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.662+0000 7f6232ffd640 1 --2- 172.21.15.46:0/1455667155 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f622006ae70 0x7f622006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:16.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.662+0000 7f6232ffd640 1 -- 172.21.15.46:0/1455667155 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6238088ae0 con 0x7f623c06ea50 2023-12-02T00:31:16.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.662+0000 7f6241b72640 1 --2- 172.21.15.46:0/1455667155 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f622006ae70 0x7f622006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:16.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.663+0000 7f6241b72640 1 --2- 172.21.15.46:0/1455667155 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f622006ae70 0x7f622006d330 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f622c036000 tx=0x7f622c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:16.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.666+0000 7f6232ffd640 1 -- 172.21.15.46:0/1455667155 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6238053c20 con 0x7f623c06ea50 2023-12-02T00:31:16.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.846+0000 7f62445d7640 1 -- 172.21.15.46:0/1455667155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f623c070d30 con 0x7f623c06ea50 2023-12-02T00:31:16.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.847+0000 7f6232ffd640 1 -- 172.21.15.46:0/1455667155 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v51) v1 ==== 100+0+11 (secure 0 0 0) 0x7f62380170a0 con 0x7f623c06ea50 2023-12-02T00:31:16.848 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-12-02T00:31:16.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.849+0000 7f62445d7640 1 -- 172.21.15.46:0/1455667155 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f622006ae70 msgr2=0x7f622006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:16.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.849+0000 7f62445d7640 1 --2- 172.21.15.46:0/1455667155 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f622006ae70 0x7f622006d330 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f622c036000 tx=0x7f622c008030 comp rx=0 tx=0).stop 2023-12-02T00:31:16.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.849+0000 7f62445d7640 1 -- 172.21.15.46:0/1455667155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f623c06ea50 msgr2=0x7f623c156b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:16.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.849+0000 7f62445d7640 1 --2- 172.21.15.46:0/1455667155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f623c06ea50 0x7f623c156b90 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f623800c400 tx=0x7f623800c8c0 comp rx=0 tx=0).stop 2023-12-02T00:31:16.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.849+0000 7f62445d7640 1 -- 172.21.15.46:0/1455667155 shutdown_connections 2023-12-02T00:31:16.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.849+0000 7f62445d7640 1 --2- 172.21.15.46:0/1455667155 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f622006ae70 0x7f622006d330 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:16.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.849+0000 7f62445d7640 1 --2- 172.21.15.46:0/1455667155 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f623c160fb0 0x7f623c15a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:16.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.849+0000 7f62445d7640 1 --2- 172.21.15.46:0/1455667155 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f623c0d0ed0 0x7f623c1e8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:16.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.849+0000 7f62445d7640 1 --2- 172.21.15.46:0/1455667155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f623c06ea50 0x7f623c156b90 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:16.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.849+0000 7f62445d7640 1 -- 172.21.15.46:0/1455667155 >> 172.21.15.46:0/1455667155 conn(0x7f623c06a6b0 msgr2=0x7f623c13fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:16.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.849+0000 7f62445d7640 1 -- 172.21.15.46:0/1455667155 shutdown_connections 2023-12-02T00:31:16.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.849+0000 7f62445d7640 1 -- 172.21.15.46:0/1455667155 wait complete. 2023-12-02T00:31:16.861 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-02T00:31:16.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.967+0000 7f3c4ef5a640 1 Processor -- start 2023-12-02T00:31:16.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.968+0000 7f3c4ef5a640 1 -- start start 2023-12-02T00:31:16.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.968+0000 7f3c4ef5a640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3c4815d330 0x7f3c480b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:16.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.968+0000 7f3c4ef5a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3c480d2910 0x7f3c480d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:16.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.968+0000 7f3c4ef5a640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3c480d3730 0x7f3c480b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:16.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.968+0000 7f3c4ef5a640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c480b2b10 con 0x7f3c480d3730 2023-12-02T00:31:16.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.968+0000 7f3c4ef5a640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c480d3b50 con 0x7f3c4815d330 2023-12-02T00:31:16.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.968+0000 7f3c4ef5a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c480d1590 con 0x7f3c480d2910 2023-12-02T00:31:16.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.969+0000 7f3c4d4f7640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3c480d3730 0x7f3c480b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:16.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.969+0000 7f3c4d4f7640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3c480d3730 0x7f3c480b6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39728/0 (socket says 172.21.15.46:39728) 2023-12-02T00:31:16.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.969+0000 7f3c4d4f7640 1 -- 172.21.15.46:0/2183299734 learned_addr learned my addr 172.21.15.46:0/2183299734 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:16.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.969+0000 7f3c3ffff640 1 --2- 172.21.15.46:0/2183299734 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3c480d2910 0x7f3c480d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:16.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.969+0000 7f3c4ccf6640 1 --2- 172.21.15.46:0/2183299734 >> v2:172.21.15.162:3300/0 conn(0x7f3c4815d330 0x7f3c480b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:16.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.969+0000 7f3c4d4f7640 1 -- 172.21.15.46:0/2183299734 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3c480d2910 msgr2=0x7f3c480d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:16.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.969+0000 7f3c4d4f7640 1 --2- 172.21.15.46:0/2183299734 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3c480d2910 0x7f3c480d31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:16.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.969+0000 7f3c4d4f7640 1 -- 172.21.15.46:0/2183299734 >> v2:172.21.15.162:3300/0 conn(0x7f3c4815d330 msgr2=0x7f3c480b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:16.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.969+0000 7f3c4d4f7640 1 --2- 172.21.15.46:0/2183299734 >> v2:172.21.15.162:3300/0 conn(0x7f3c4815d330 0x7f3c480b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:16.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.969+0000 7f3c4d4f7640 1 -- 172.21.15.46:0/2183299734 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c480d1870 con 0x7f3c480d3730 2023-12-02T00:31:16.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.970+0000 7f3c4d4f7640 1 --2- 172.21.15.46:0/2183299734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c480d3730 0x7f3c480b6390 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f3c480d0d10 tx=0x7f3c4002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b0dd137440ff913 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:16.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.970+0000 7f3c3f7fe640 1 -- 172.21.15.46:0/2183299734 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4002c9e0 con 0x7f3c480d3730 2023-12-02T00:31:16.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.970+0000 7f3c3f7fe640 1 -- 172.21.15.46:0/2183299734 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c4002cb40 con 0x7f3c480d3730 2023-12-02T00:31:16.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.970+0000 7f3c3f7fe640 1 -- 172.21.15.46:0/2183299734 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c400343f0 con 0x7f3c480d3730 2023-12-02T00:31:16.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.970+0000 7f3c4ef5a640 1 -- 172.21.15.46:0/2183299734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c480d3730 msgr2=0x7f3c480b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:16.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.970+0000 7f3c4ef5a640 1 --2- 172.21.15.46:0/2183299734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c480d3730 0x7f3c480b6390 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f3c480d0d10 tx=0x7f3c4002be40 comp rx=0 tx=0).stop 2023-12-02T00:31:16.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.970+0000 7f3c4ef5a640 1 -- 172.21.15.46:0/2183299734 shutdown_connections 2023-12-02T00:31:16.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.970+0000 7f3c4ef5a640 1 --2- 172.21.15.46:0/2183299734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c480d3730 0x7f3c480b6390 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:16.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.970+0000 7f3c4ef5a640 1 --2- 172.21.15.46:0/2183299734 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3c480d2910 0x7f3c480d31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:16.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.970+0000 7f3c4ef5a640 1 --2- 172.21.15.46:0/2183299734 >> v2:172.21.15.162:3300/0 conn(0x7f3c4815d330 0x7f3c480b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:16.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.970+0000 7f3c4ef5a640 1 -- 172.21.15.46:0/2183299734 >> 172.21.15.46:0/2183299734 conn(0x7f3c48072020 msgr2=0x7f3c48072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:16.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.970+0000 7f3c4ef5a640 1 -- 172.21.15.46:0/2183299734 shutdown_connections 2023-12-02T00:31:16.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.970+0000 7f3c4ef5a640 1 -- 172.21.15.46:0/2183299734 wait complete. 2023-12-02T00:31:16.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.972+0000 7f3c4ef5a640 1 Processor -- start 2023-12-02T00:31:16.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.972+0000 7f3c4ef5a640 1 -- start start 2023-12-02T00:31:16.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.972+0000 7f3c4ef5a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c480d2910 0x7f3c480c7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:16.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.972+0000 7f3c4ef5a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c480d3730 0x7f3c480ca730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:16.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.972+0000 7f3c4ef5a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3c4815d330 0x7f3c481f3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:16.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.972+0000 7f3c4ef5a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4815f740 con 0x7f3c480d3730 2023-12-02T00:31:16.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.972+0000 7f3c4ef5a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4815d710 con 0x7f3c480d2910 2023-12-02T00:31:16.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.972+0000 7f3c4ef5a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4815d9f0 con 0x7f3c4815d330 2023-12-02T00:31:16.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.973+0000 7f3c3ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c480d3730 0x7f3c480ca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:16.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.973+0000 7f3c3ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c480d3730 0x7f3c480ca730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39742/0 (socket says 172.21.15.46:39742) 2023-12-02T00:31:16.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.973+0000 7f3c3ffff640 1 -- 172.21.15.46:0/910035737 learned_addr learned my addr 172.21.15.46:0/910035737 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:16.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.973+0000 7f3c4ccf6640 1 --2- 172.21.15.46:0/910035737 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c480d2910 0x7f3c480c7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:16.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.973+0000 7f3c4d4f7640 1 --2- 172.21.15.46:0/910035737 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3c4815d330 0x7f3c481f3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:16.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.973+0000 7f3c3ffff640 1 -- 172.21.15.46:0/910035737 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3c4815d330 msgr2=0x7f3c481f3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:16.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.973+0000 7f3c3ffff640 1 --2- 172.21.15.46:0/910035737 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3c4815d330 0x7f3c481f3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:16.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.973+0000 7f3c3ffff640 1 -- 172.21.15.46:0/910035737 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c480d2910 msgr2=0x7f3c480c7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:16.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.973+0000 7f3c3ffff640 1 --2- 172.21.15.46:0/910035737 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c480d2910 0x7f3c480c7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:16.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.973+0000 7f3c3ffff640 1 -- 172.21.15.46:0/910035737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c40009b30 con 0x7f3c480d3730 2023-12-02T00:31:16.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.973+0000 7f3c4ccf6640 1 --2- 172.21.15.46:0/910035737 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c480d2910 0x7f3c480c7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:31:16.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.973+0000 7f3c4d4f7640 1 --2- 172.21.15.46:0/910035737 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3c4815d330 0x7f3c481f3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:31:16.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.973+0000 7f3c3ffff640 1 --2- 172.21.15.46:0/910035737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c480d3730 0x7f3c480ca730 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f3c3800e870 tx=0x7f3c3800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:16.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.973+0000 7f3c3d7fa640 1 -- 172.21.15.46:0/910035737 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3800ee60 con 0x7f3c480d3730 2023-12-02T00:31:16.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.973+0000 7f3c3d7fa640 1 -- 172.21.15.46:0/910035737 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c3800c940 con 0x7f3c480d3730 2023-12-02T00:31:16.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.973+0000 7f3c4ef5a640 1 -- 172.21.15.46:0/910035737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c4815dcd0 con 0x7f3c480d3730 2023-12-02T00:31:16.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.973+0000 7f3c3d7fa640 1 -- 172.21.15.46:0/910035737 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3800ccf0 con 0x7f3c480d3730 2023-12-02T00:31:16.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.973+0000 7f3c4ef5a640 1 -- 172.21.15.46:0/910035737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c4815f310 con 0x7f3c480d3730 2023-12-02T00:31:16.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.974+0000 7f3c4ef5a640 1 -- 172.21.15.46:0/910035737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c4815f560 con 0x7f3c480d3730 2023-12-02T00:31:16.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.975+0000 7f3c3d7fa640 1 -- 172.21.15.46:0/910035737 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3c38013470 con 0x7f3c480d3730 2023-12-02T00:31:16.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.975+0000 7f3c3d7fa640 1 --2- 172.21.15.46:0/910035737 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3c3406ae40 0x7f3c3406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:16.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.975+0000 7f3c3d7fa640 1 -- 172.21.15.46:0/910035737 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3c38089640 con 0x7f3c480d3730 2023-12-02T00:31:16.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.976+0000 7f3c4ccf6640 1 --2- 172.21.15.46:0/910035737 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3c3406ae40 0x7f3c3406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:16.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.976+0000 7f3c4ccf6640 1 --2- 172.21.15.46:0/910035737 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3c3406ae40 0x7f3c3406d300 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3c44009b90 tx=0x7f3c44008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:16.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:16.979+0000 7f3c3d7fa640 1 -- 172.21.15.46:0/910035737 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3c380546f0 con 0x7f3c480d3730 2023-12-02T00:31:17.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.167+0000 7f3c4ef5a640 1 -- 172.21.15.46:0/910035737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3c4815dec0 con 0x7f3c480d3730 2023-12-02T00:31:17.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.168+0000 7f3c3d7fa640 1 -- 172.21.15.46:0/910035737 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v51) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3c38057d10 con 0x7f3c480d3730 2023-12-02T00:31:17.169 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2023-12-02T00:31:17.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.170+0000 7f3c4ef5a640 1 -- 172.21.15.46:0/910035737 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3c3406ae40 msgr2=0x7f3c3406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:17.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.170+0000 7f3c4ef5a640 1 --2- 172.21.15.46:0/910035737 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3c3406ae40 0x7f3c3406d300 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3c44009b90 tx=0x7f3c44008040 comp rx=0 tx=0).stop 2023-12-02T00:31:17.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.170+0000 7f3c4ef5a640 1 -- 172.21.15.46:0/910035737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c480d3730 msgr2=0x7f3c480ca730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:17.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.170+0000 7f3c4ef5a640 1 --2- 172.21.15.46:0/910035737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c480d3730 0x7f3c480ca730 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f3c3800e870 tx=0x7f3c3800ed30 comp rx=0 tx=0).stop 2023-12-02T00:31:17.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.170+0000 7f3c4ef5a640 1 -- 172.21.15.46:0/910035737 shutdown_connections 2023-12-02T00:31:17.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.170+0000 7f3c4ef5a640 1 --2- 172.21.15.46:0/910035737 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3c3406ae40 0x7f3c3406d300 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:17.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.170+0000 7f3c4ef5a640 1 --2- 172.21.15.46:0/910035737 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3c4815d330 0x7f3c481f3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:17.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.170+0000 7f3c4ef5a640 1 --2- 172.21.15.46:0/910035737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c480d3730 0x7f3c480ca730 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:17.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.170+0000 7f3c4ef5a640 1 --2- 172.21.15.46:0/910035737 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c480d2910 0x7f3c480c7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:17.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.170+0000 7f3c4ef5a640 1 -- 172.21.15.46:0/910035737 >> 172.21.15.46:0/910035737 conn(0x7f3c48072020 msgr2=0x7f3c480b8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:17.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.170+0000 7f3c4ef5a640 1 -- 172.21.15.46:0/910035737 shutdown_connections 2023-12-02T00:31:17.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.170+0000 7f3c4ef5a640 1 -- 172.21.15.46:0/910035737 wait complete. 2023-12-02T00:31:17.181 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi046.front.sepia.ceph.com... 2023-12-02T00:31:17.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:31:17.182 DEBUG:teuthology.orchestra.run.smithi046:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-12-02T00:31:17.211 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[110046]: fuse finished with error 0 and tester_r 0 2023-12-02T00:31:17.213 INFO:teuthology.orchestra.run:waiting for 300 2023-12-02T00:31:17.255 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2023-12-02T00:31:17.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:31:17.255 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-02T00:31:17.313 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-02T00:31:17.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.459+0000 7f39933a9640 1 Processor -- start 2023-12-02T00:31:17.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.459+0000 7f39933a9640 1 -- start start 2023-12-02T00:31:17.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.460+0000 7f39933a9640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f398c0d2910 0x7f398c0d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:17.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.460+0000 7f39933a9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f398c0d3730 0x7f398c0b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:17.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.460+0000 7f39933a9640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f398c0b2b10 con 0x7f398c15d330 2023-12-02T00:31:17.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.460+0000 7f39933a9640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f398c0d1590 con 0x7f398c0d2910 2023-12-02T00:31:17.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.460+0000 7f39933a9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f398c0d1870 con 0x7f398c0d3730 2023-12-02T00:31:17.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.460+0000 7f3990944640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f398c0d2910 0x7f398c0d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:17.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.460+0000 7f3991145640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f398c15d330 0x7f398c0b8b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57178/0 (socket says 172.21.15.46:57178) 2023-12-02T00:31:17.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.460+0000 7f3991145640 1 -- 172.21.15.46:0/2859858000 learned_addr learned my addr 172.21.15.46:0/2859858000 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:31:17.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.461+0000 7f3991946640 1 --2- 172.21.15.46:0/2859858000 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f398c0d3730 0x7f398c0b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:17.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.461+0000 7f3990944640 1 -- 172.21.15.46:0/2859858000 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f398c0d3730 msgr2=0x7f398c0b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:17.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.461+0000 7f3990944640 1 --2- 172.21.15.46:0/2859858000 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f398c0d3730 0x7f398c0b6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:17.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.461+0000 7f3990944640 1 -- 172.21.15.46:0/2859858000 >> v1:172.21.15.46:6789/0 conn(0x7f398c15d330 legacy=0x7f398c0b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:17.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.461+0000 7f3990944640 1 -- 172.21.15.46:0/2859858000 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f398c0d3b90 con 0x7f398c0d2910 2023-12-02T00:31:17.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.461+0000 7f398bfff640 1 -- 172.21.15.46:0/2859858000 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 387532346 0 0) 0x7f398c0b2b10 con 0x7f398c15d330 2023-12-02T00:31:17.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.462+0000 7f3990944640 1 --2- 172.21.15.46:0/2859858000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f398c0d2910 0x7f398c0d31f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f39800097a0 tx=0x7f398002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f75cdc070572fee4 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:17.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.462+0000 7f398bfff640 1 -- 172.21.15.46:0/2859858000 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398002e070 con 0x7f398c0d2910 2023-12-02T00:31:17.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.462+0000 7f398bfff640 1 -- 172.21.15.46:0/2859858000 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39800315a0 con 0x7f398c0d2910 2023-12-02T00:31:17.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.462+0000 7f398bfff640 1 -- 172.21.15.46:0/2859858000 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3980031930 con 0x7f398c0d2910 2023-12-02T00:31:17.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.462+0000 7f39933a9640 1 -- 172.21.15.46:0/2859858000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f398c0d2910 msgr2=0x7f398c0d31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:17.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.462+0000 7f39933a9640 1 --2- 172.21.15.46:0/2859858000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f398c0d2910 0x7f398c0d31f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f39800097a0 tx=0x7f398002b840 comp rx=0 tx=0).stop 2023-12-02T00:31:17.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.462+0000 7f39933a9640 1 -- 172.21.15.46:0/2859858000 shutdown_connections 2023-12-02T00:31:17.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.462+0000 7f39933a9640 1 --2- 172.21.15.46:0/2859858000 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f398c0d3730 0x7f398c0b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:17.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.462+0000 7f39933a9640 1 --2- 172.21.15.46:0/2859858000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f398c0d2910 0x7f398c0d31f0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:17.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.462+0000 7f39933a9640 1 -- 172.21.15.46:0/2859858000 >> 172.21.15.46:0/2859858000 conn(0x7f398c072020 msgr2=0x7f398c072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:17.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.462+0000 7f39933a9640 1 -- 172.21.15.46:0/2859858000 shutdown_connections 2023-12-02T00:31:17.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.462+0000 7f39933a9640 1 -- 172.21.15.46:0/2859858000 wait complete. 2023-12-02T00:31:17.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.463+0000 7f39933a9640 1 Processor -- start 2023-12-02T00:31:17.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.464+0000 7f39933a9640 1 -- start start 2023-12-02T00:31:17.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.464+0000 7f39933a9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f398c0d2910 0x7f398c0c9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:17.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.464+0000 7f39933a9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f398c0d3730 0x7f398c1601f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:17.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.464+0000 7f39933a9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f398c15d330 0x7f398c0c7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:17.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.464+0000 7f39933a9640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c156050 con 0x7f398c0d3730 2023-12-02T00:31:17.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.464+0000 7f39933a9640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c0c9870 con 0x7f398c15d330 2023-12-02T00:31:17.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.464+0000 7f39933a9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f398c1579e0 con 0x7f398c0d2910 2023-12-02T00:31:17.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.464+0000 7f3990944640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f398c0d3730 0x7f398c1601f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:17.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.464+0000 7f3990944640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f398c0d3730 0x7f398c1601f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39772/0 (socket says 172.21.15.46:39772) 2023-12-02T00:31:17.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.464+0000 7f3990944640 1 -- 172.21.15.46:0/499947252 learned_addr learned my addr 172.21.15.46:0/499947252 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:17.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.465+0000 7f3991145640 1 --2- 172.21.15.46:0/499947252 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f398c0d2910 0x7f398c0c9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:17.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.465+0000 7f3991946640 1 --2- 172.21.15.46:0/499947252 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f398c15d330 0x7f398c0c7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:17.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.465+0000 7f3990944640 1 -- 172.21.15.46:0/499947252 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f398c0d2910 msgr2=0x7f398c0c9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:17.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.465+0000 7f3990944640 1 --2- 172.21.15.46:0/499947252 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f398c0d2910 0x7f398c0c9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:17.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.465+0000 7f3990944640 1 -- 172.21.15.46:0/499947252 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f398c15d330 msgr2=0x7f398c0c7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:17.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.465+0000 7f3990944640 1 --2- 172.21.15.46:0/499947252 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f398c15d330 0x7f398c0c7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:17.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.465+0000 7f3990944640 1 -- 172.21.15.46:0/499947252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3980009400 con 0x7f398c0d3730 2023-12-02T00:31:17.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.465+0000 7f3991946640 1 --2- 172.21.15.46:0/499947252 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f398c15d330 0x7f398c0c7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:31:17.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.465+0000 7f3990944640 1 --2- 172.21.15.46:0/499947252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f398c0d3730 0x7f398c1601f0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f3980034040 tx=0x7f39800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:17.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.465+0000 7f3989ffb640 1 -- 172.21.15.46:0/499947252 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3980004230 con 0x7f398c0d3730 2023-12-02T00:31:17.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.465+0000 7f3989ffb640 1 -- 172.21.15.46:0/499947252 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3980031f00 con 0x7f398c0d3730 2023-12-02T00:31:17.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.465+0000 7f3989ffb640 1 -- 172.21.15.46:0/499947252 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3980032640 con 0x7f398c0d3730 2023-12-02T00:31:17.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.465+0000 7f39933a9640 1 -- 172.21.15.46:0/499947252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f398c157cc0 con 0x7f398c0d3730 2023-12-02T00:31:17.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.465+0000 7f39933a9640 1 -- 172.21.15.46:0/499947252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f398c15d830 con 0x7f398c0d3730 2023-12-02T00:31:17.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.466+0000 7f39933a9640 1 -- 172.21.15.46:0/499947252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f398c0d3520 con 0x7f398c0d3730 2023-12-02T00:31:17.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.467+0000 7f3989ffb640 1 -- 172.21.15.46:0/499947252 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f39800327a0 con 0x7f398c0d3730 2023-12-02T00:31:17.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.467+0000 7f3989ffb640 1 --2- 172.21.15.46:0/499947252 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f397806ae40 0x7f397806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:17.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.467+0000 7f3989ffb640 1 -- 172.21.15.46:0/499947252 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f39800ab660 con 0x7f398c0d3730 2023-12-02T00:31:17.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.467+0000 7f3991145640 1 --2- 172.21.15.46:0/499947252 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f397806ae40 0x7f397806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:17.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.468+0000 7f3991145640 1 --2- 172.21.15.46:0/499947252 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f397806ae40 0x7f397806d300 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3984000f60 tx=0x7f3984007b50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:17.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.471+0000 7f3989ffb640 1 -- 172.21.15.46:0/499947252 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3980076710 con 0x7f398c0d3730 2023-12-02T00:31:17.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.687+0000 7f39933a9640 1 -- 172.21.15.46:0/499947252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f398c06bf00 con 0x7f398c0d3730 2023-12-02T00:31:17.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.688+0000 7f3989ffb640 1 -- 172.21.15.46:0/499947252 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3980079d30 con 0x7f398c0d3730 2023-12-02T00:31:17.689 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-02T00:31:17.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.690+0000 7f39933a9640 1 -- 172.21.15.46:0/499947252 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f397806ae40 msgr2=0x7f397806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:17.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.690+0000 7f39933a9640 1 --2- 172.21.15.46:0/499947252 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f397806ae40 0x7f397806d300 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3984000f60 tx=0x7f3984007b50 comp rx=0 tx=0).stop 2023-12-02T00:31:17.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.690+0000 7f39933a9640 1 -- 172.21.15.46:0/499947252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f398c0d3730 msgr2=0x7f398c1601f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:17.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.690+0000 7f39933a9640 1 --2- 172.21.15.46:0/499947252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f398c0d3730 0x7f398c1601f0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f3980034040 tx=0x7f39800046c0 comp rx=0 tx=0).stop 2023-12-02T00:31:17.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.690+0000 7f39933a9640 1 -- 172.21.15.46:0/499947252 shutdown_connections 2023-12-02T00:31:17.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.690+0000 7f39933a9640 1 --2- 172.21.15.46:0/499947252 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f397806ae40 0x7f397806d300 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:17.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.690+0000 7f39933a9640 1 --2- 172.21.15.46:0/499947252 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f398c15d330 0x7f398c0c7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:17.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.690+0000 7f39933a9640 1 --2- 172.21.15.46:0/499947252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f398c0d3730 0x7f398c1601f0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:17.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.690+0000 7f39933a9640 1 --2- 172.21.15.46:0/499947252 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f398c0d2910 0x7f398c0c9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:17.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.690+0000 7f39933a9640 1 -- 172.21.15.46:0/499947252 >> 172.21.15.46:0/499947252 conn(0x7f398c072020 msgr2=0x7f398c0b55e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:17.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.690+0000 7f39933a9640 1 -- 172.21.15.46:0/499947252 shutdown_connections 2023-12-02T00:31:17.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:17.690+0000 7f39933a9640 1 -- 172.21.15.46:0/499947252 wait complete. 2023-12-02T00:31:17.702 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-02T00:31:17.702 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-02T00:31:17.702 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2023-12-02T00:31:17.702 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-02T00:31:17.702 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-02T00:31:17.702 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-02T00:31:17.702 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-02T00:31:17.702 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-02T00:31:17.703 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-02T00:31:17.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:31:17.703 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2023-12-02T00:31:17.719 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-02T00:31:17.719 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-02T00:31:17.719 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2023-12-02T00:31:17.719 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:31:17.719 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2023-12-02T00:31:17.720 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:31:17.720 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-02T00:31:17.720 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2023-12-02T00:31:17.720 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2023-12-02T00:31:17.721 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-02T00:31:17.721 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2023-12-02T00:31:17.722 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2023-12-02T00:31:17.722 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-02T00:31:17.722 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2023-12-02T00:31:17.722 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2023-12-02T00:31:17.722 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-12-02T00:31:17.722 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42501sec preferred_lft 42501sec 2023-12-02T00:31:17.722 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link noprefixroute 2023-12-02T00:31:17.722 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:31:17.722 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-02T00:31:17.722 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2023-12-02T00:31:17.722 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2023-12-02T00:31:17.722 INFO:teuthology.orchestra.run.smithi046.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-02T00:31:17.722 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether b6:1a:8b:80:15:f1 brd ff:ff:ff:ff:ff:ff 2023-12-02T00:31:17.722 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-02T00:31:17.723 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:31:17.723 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::eca8:9ff:feb3:1726/64 scope link 2023-12-02T00:31:17.723 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:31:17.723 INFO:teuthology.orchestra.run.smithi046.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-02T00:31:17.723 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether b6:1a:8b:80:15:f1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-02T00:31:17.723 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::b41a:8bff:fe80:15f1/64 scope link 2023-12-02T00:31:17.723 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:31:17.723 INFO:teuthology.orchestra.run.smithi046.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-02T00:31:17.723 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether c6:93:a5:c0:e2:af brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-02T00:31:17.723 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c493:a5ff:fec0:e2af/64 scope link 2023-12-02T00:31:17.723 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:31:17.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:31:17.724 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-02T00:31:17.789 INFO:teuthology.orchestra.run.smithi046.stdout:1 2023-12-02T00:31:17.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:31:17.791 DEBUG:teuthology.orchestra.run.smithi046:> route 2023-12-02T00:31:17.848 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2023-12-02T00:31:17.848 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-02T00:31:17.848 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-12-02T00:31:17.848 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-12-02T00:31:17.848 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-02T00:31:17.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:31:17.849 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2023-12-02T00:31:17.849 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2023-12-02T00:31:17.849 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-02T00:31:17.849 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-02T00:31:17.850 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-02T00:31:17.850 DEBUG:teuthology.orchestra.run.smithi046:> ') 2023-12-02T00:31:17.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:31:17.955 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2023-12-02T00:31:17.970 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-02T00:31:17.970 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-02T00:31:17.970 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-02T00:31:17.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:31:17.970 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2023-12-02T00:31:18.040 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-12-02T00:31:18.041 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-02T00:31:18.041 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-02T00:31:18.096 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-02T00:31:18.096 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-02T00:31:18.097 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-02T00:31:18.154 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2023-12-02T00:31:18.226 DEBUG:teuthology.orchestra.run.smithi046:> cat /proc/self/mounts | awk '{print $2}' 2023-12-02T00:31:18.283 INFO:teuthology.orchestra.run.smithi046.stdout:/proc 2023-12-02T00:31:18.283 INFO:teuthology.orchestra.run.smithi046.stdout:/sys 2023-12-02T00:31:18.283 INFO:teuthology.orchestra.run.smithi046.stdout:/dev 2023-12-02T00:31:18.283 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/security 2023-12-02T00:31:18.283 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/shm 2023-12-02T00:31:18.283 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/pts 2023-12-02T00:31:18.284 INFO:teuthology.orchestra.run.smithi046.stdout:/run 2023-12-02T00:31:18.284 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/cgroup 2023-12-02T00:31:18.284 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/pstore 2023-12-02T00:31:18.284 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/bpf 2023-12-02T00:31:18.284 INFO:teuthology.orchestra.run.smithi046.stdout:/ 2023-12-02T00:31:18.284 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/selinux 2023-12-02T00:31:18.284 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2023-12-02T00:31:18.284 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/mqueue 2023-12-02T00:31:18.284 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/hugepages 2023-12-02T00:31:18.284 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/debug 2023-12-02T00:31:18.284 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/tracing 2023-12-02T00:31:18.284 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/fuse/connections 2023-12-02T00:31:18.284 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/config 2023-12-02T00:31:18.284 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-sysctl.service 2023-12-02T00:31:18.284 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-02T00:31:18.284 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-02T00:31:18.285 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/nfs/rpc_pipefs 2023-12-02T00:31:18.285 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/1000 2023-12-02T00:31:18.285 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph 2023-12-02T00:31:18.285 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-0 2023-12-02T00:31:18.285 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-1 2023-12-02T00:31:18.285 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-2 2023-12-02T00:31:18.285 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-3 2023-12-02T00:31:18.285 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns 2023-12-02T00:31:18.285 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-02T00:31:18.285 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-02T00:31:18.285 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-02T00:31:18.285 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-02T00:31:18.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:31:18.286 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2023-12-02T00:31:18.339 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-12-02T00:31:18.339 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-12-02T00:31:18.381 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2023-12-02T00:31:18.412 DEBUG:teuthology.orchestra.run.smithi046:> cat /proc/self/mounts | awk '{print $2}' 2023-12-02T00:31:18.468 INFO:teuthology.orchestra.run.smithi046.stdout:/proc 2023-12-02T00:31:18.468 INFO:teuthology.orchestra.run.smithi046.stdout:/sys 2023-12-02T00:31:18.468 INFO:teuthology.orchestra.run.smithi046.stdout:/dev 2023-12-02T00:31:18.468 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/security 2023-12-02T00:31:18.468 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/shm 2023-12-02T00:31:18.469 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/pts 2023-12-02T00:31:18.469 INFO:teuthology.orchestra.run.smithi046.stdout:/run 2023-12-02T00:31:18.469 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/cgroup 2023-12-02T00:31:18.469 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/pstore 2023-12-02T00:31:18.469 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/bpf 2023-12-02T00:31:18.469 INFO:teuthology.orchestra.run.smithi046.stdout:/ 2023-12-02T00:31:18.469 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/selinux 2023-12-02T00:31:18.469 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2023-12-02T00:31:18.469 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/mqueue 2023-12-02T00:31:18.469 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/hugepages 2023-12-02T00:31:18.469 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/debug 2023-12-02T00:31:18.469 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/tracing 2023-12-02T00:31:18.470 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/fuse/connections 2023-12-02T00:31:18.470 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/config 2023-12-02T00:31:18.470 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-sysctl.service 2023-12-02T00:31:18.470 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-02T00:31:18.470 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-02T00:31:18.470 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/nfs/rpc_pipefs 2023-12-02T00:31:18.470 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/1000 2023-12-02T00:31:18.470 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph 2023-12-02T00:31:18.470 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-0 2023-12-02T00:31:18.470 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-1 2023-12-02T00:31:18.470 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-2 2023-12-02T00:31:18.471 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-3 2023-12-02T00:31:18.471 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns 2023-12-02T00:31:18.471 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-02T00:31:18.471 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-02T00:31:18.471 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-02T00:31:18.471 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-02T00:31:18.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:31:18.471 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2023-12-02T00:31:18.474 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:2023-12-02T00:31:18.473+0000 7f4de7e4a480 -1 init, newargv = 0x55dd0a7cf740 newargc=15 2023-12-02T00:31:18.475 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[111010]: starting ceph client 2023-12-02T00:31:18.478 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[111010]: starting fuse 2023-12-02T00:31:18.490 INFO:teuthology.orchestra.run.smithi046.stdout:41 2023-12-02T00:31:18.491 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-12-02T00:31:18.491 DEBUG:teuthology.orchestra.run.smithi046:> sudo stdin-killer -- python3 -c ' 2023-12-02T00:31:18.491 DEBUG:teuthology.orchestra.run.smithi046:> import glob 2023-12-02T00:31:18.491 DEBUG:teuthology.orchestra.run.smithi046:> import re 2023-12-02T00:31:18.491 DEBUG:teuthology.orchestra.run.smithi046:> import os 2023-12-02T00:31:18.491 DEBUG:teuthology.orchestra.run.smithi046:> import subprocess 2023-12-02T00:31:18.491 DEBUG:teuthology.orchestra.run.smithi046:> 2023-12-02T00:31:18.491 DEBUG:teuthology.orchestra.run.smithi046:> def _find_admin_socket(client_name): 2023-12-02T00:31:18.491 DEBUG:teuthology.orchestra.run.smithi046:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-12-02T00:31:18.492 DEBUG:teuthology.orchestra.run.smithi046:> files = glob.glob(asok_path) 2023-12-02T00:31:18.492 DEBUG:teuthology.orchestra.run.smithi046:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-12-02T00:31:18.492 DEBUG:teuthology.orchestra.run.smithi046:> 2023-12-02T00:31:18.492 DEBUG:teuthology.orchestra.run.smithi046:> # Given a non-glob path, it better be there 2023-12-02T00:31:18.492 DEBUG:teuthology.orchestra.run.smithi046:> if "*" not in asok_path: 2023-12-02T00:31:18.492 DEBUG:teuthology.orchestra.run.smithi046:> assert(len(files) == 1) 2023-12-02T00:31:18.492 DEBUG:teuthology.orchestra.run.smithi046:> return files[0] 2023-12-02T00:31:18.492 DEBUG:teuthology.orchestra.run.smithi046:> 2023-12-02T00:31:18.492 DEBUG:teuthology.orchestra.run.smithi046:> for f in files: 2023-12-02T00:31:18.492 DEBUG:teuthology.orchestra.run.smithi046:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-02T00:31:18.492 DEBUG:teuthology.orchestra.run.smithi046:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-02T00:31:18.492 DEBUG:teuthology.orchestra.run.smithi046:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-02T00:31:18.492 DEBUG:teuthology.orchestra.run.smithi046:> contents = proc_f.read() 2023-12-02T00:31:18.492 DEBUG:teuthology.orchestra.run.smithi046:> if mountpoint in contents: 2023-12-02T00:31:18.492 DEBUG:teuthology.orchestra.run.smithi046:> return f 2023-12-02T00:31:18.493 DEBUG:teuthology.orchestra.run.smithi046:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-02T00:31:18.493 DEBUG:teuthology.orchestra.run.smithi046:> 2023-12-02T00:31:18.493 DEBUG:teuthology.orchestra.run.smithi046:> print(_find_admin_socket("client.0")) 2023-12-02T00:31:18.493 DEBUG:teuthology.orchestra.run.smithi046:> ' 2023-12-02T00:31:18.615 INFO:teuthology.orchestra.run.smithi046.stdout:/var/run/ceph/ceph-client.0.111010.asok 2023-12-02T00:31:18.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-12-02T00:31:18.626 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.111010.asok 2023-12-02T00:31:18.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:31:18.626 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.111010.asok status 2023-12-02T00:31:18.767 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-12-02T00:31:18.767 INFO:teuthology.orchestra.run.smithi046.stdout: "metadata": { 2023-12-02T00:31:18.767 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_sha1": "bed7681480ccb0192b91a35e691658867e1e96b7", 2023-12-02T00:31:18.767 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_version": "ceph version 18.0.0-7631-gbed76814 (bed7681480ccb0192b91a35e691658867e1e96b7) reef (dev)", 2023-12-02T00:31:18.767 INFO:teuthology.orchestra.run.smithi046.stdout: "entity_id": "0", 2023-12-02T00:31:18.767 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": "smithi046", 2023-12-02T00:31:18.768 INFO:teuthology.orchestra.run.smithi046.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-12-02T00:31:18.768 INFO:teuthology.orchestra.run.smithi046.stdout: "pid": "111010", 2023-12-02T00:31:18.768 INFO:teuthology.orchestra.run.smithi046.stdout: "root": "/" 2023-12-02T00:31:18.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:31:18.768 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_count": 0, 2023-12-02T00:31:18.768 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_pinned_count": 0, 2023-12-02T00:31:18.768 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 6261, 2023-12-02T00:31:18.768 INFO:teuthology.orchestra.run.smithi046.stdout: "inst": { 2023-12-02T00:31:18.768 INFO:teuthology.orchestra.run.smithi046.stdout: "name": { 2023-12-02T00:31:18.768 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "client", 2023-12-02T00:31:18.768 INFO:teuthology.orchestra.run.smithi046.stdout: "num": 6261 2023-12-02T00:31:18.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:31:18.768 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2023-12-02T00:31:18.768 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2023-12-02T00:31:18.768 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2023-12-02T00:31:18.769 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2557768689 2023-12-02T00:31:18.769 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:31:18.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:31:18.769 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2023-12-02T00:31:18.769 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2023-12-02T00:31:18.769 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2023-12-02T00:31:18.769 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2557768689 2023-12-02T00:31:18.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:31:18.769 INFO:teuthology.orchestra.run.smithi046.stdout: "inst_str": "client.6261 192.168.0.1:0/2557768689", 2023-12-02T00:31:18.769 INFO:teuthology.orchestra.run.smithi046.stdout: "addr_str": "192.168.0.1:0/2557768689", 2023-12-02T00:31:18.769 INFO:teuthology.orchestra.run.smithi046.stdout: "inode_count": 1, 2023-12-02T00:31:18.769 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_epoch": 24, 2023-12-02T00:31:18.769 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch": 51, 2023-12-02T00:31:18.769 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch_barrier": 0, 2023-12-02T00:31:18.769 INFO:teuthology.orchestra.run.smithi046.stdout: "blocklisted": false, 2023-12-02T00:31:18.770 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs" 2023-12-02T00:31:18.770 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-12-02T00:31:18.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:31:18.776 DEBUG:teuthology.orchestra.run.smithi046:> stat --file-system '--printf=%T 2023-12-02T00:31:18.776 DEBUG:teuthology.orchestra.run.smithi046:> ' -- /home/ubuntu/cephtest/mnt.0 2023-12-02T00:31:18.792 INFO:teuthology.orchestra.run.smithi046.stdout:fuseblk 2023-12-02T00:31:18.792 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-12-02T00:31:18.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:31:18.792 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-02T00:31:18.862 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2023-12-02T00:31:18.863 DEBUG:teuthology.orchestra.run.smithi046:> import os 2023-12-02T00:31:18.863 DEBUG:teuthology.orchestra.run.smithi046:> import errno 2023-12-02T00:31:18.863 DEBUG:teuthology.orchestra.run.smithi046:> 2023-12-02T00:31:18.863 DEBUG:teuthology.orchestra.run.smithi046:> fd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR) 2023-12-02T00:31:18.863 DEBUG:teuthology.orchestra.run.smithi046:> try: 2023-12-02T00:31:18.863 DEBUG:teuthology.orchestra.run.smithi046:> if False: 2023-12-02T00:31:18.863 DEBUG:teuthology.orchestra.run.smithi046:> ret = os.read(fd, 1024) 2023-12-02T00:31:18.863 DEBUG:teuthology.orchestra.run.smithi046:> else: 2023-12-02T00:31:18.863 DEBUG:teuthology.orchestra.run.smithi046:> os.write(fd, b'"'"'content'"'"') 2023-12-02T00:31:18.863 DEBUG:teuthology.orchestra.run.smithi046:> except OSError as e: 2023-12-02T00:31:18.863 DEBUG:teuthology.orchestra.run.smithi046:> if e.errno != errno.EPERM: 2023-12-02T00:31:18.863 DEBUG:teuthology.orchestra.run.smithi046:> raise 2023-12-02T00:31:18.863 DEBUG:teuthology.orchestra.run.smithi046:> else: 2023-12-02T00:31:18.863 DEBUG:teuthology.orchestra.run.smithi046:> raise RuntimeError("client does not check permission of data pool") 2023-12-02T00:31:18.863 DEBUG:teuthology.orchestra.run.smithi046:> ' 2023-12-02T00:31:18.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-12-02T00:31:18.981 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow w pool=cephfs_data' 2023-12-02T00:31:19.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.134+0000 7fdd3fba5640 1 Processor -- start 2023-12-02T00:31:19.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.134+0000 7fdd3fba5640 1 -- start start 2023-12-02T00:31:19.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.135+0000 7fdd3fba5640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdd380b3400 0x7fdd38168360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:19.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.135+0000 7fdd3fba5640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd380cf660 con 0x7fdd38146010 2023-12-02T00:31:19.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.135+0000 7fdd3fba5640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd38161170 con 0x7fdd38146d70 2023-12-02T00:31:19.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.135+0000 7fdd3fba5640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd38161450 con 0x7fdd380b3400 2023-12-02T00:31:19.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.135+0000 7fdd3e142640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdd380b3400 0x7fdd38168360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:19.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.135+0000 7fdd3d941640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fdd38146010 0x7fdd380cef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57182/0 (socket says 172.21.15.46:57182) 2023-12-02T00:31:19.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.135+0000 7fdd3d941640 1 -- 172.21.15.46:0/2240642045 learned_addr learned my addr 172.21.15.46:0/2240642045 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:31:19.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.135+0000 7fdd3e142640 1 -- 172.21.15.46:0/2240642045 >> v1:172.21.15.162:6789/0 conn(0x7fdd38146d70 legacy=0x7fdd380b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:19.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.135+0000 7fdd3e142640 1 -- 172.21.15.46:0/2240642045 >> v1:172.21.15.46:6789/0 conn(0x7fdd38146010 legacy=0x7fdd380cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:19.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.135+0000 7fdd3e142640 1 -- 172.21.15.46:0/2240642045 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd38161730 con 0x7fdd380b3400 2023-12-02T00:31:19.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.135+0000 7fdd3c93f640 1 -- 172.21.15.46:0/2240642045 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2574793168 0 0) 0x7fdd380cf660 con 0x7fdd38146010 2023-12-02T00:31:19.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.136+0000 7fdd3e142640 1 --2- 172.21.15.46:0/2240642045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd380b3400 0x7fdd38168360 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fdd30009d90 tx=0x7fdd3002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5c4dbd9676af313 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:19.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.136+0000 7fdd3c93f640 1 -- 172.21.15.46:0/2240642045 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3002cc60 con 0x7fdd380b3400 2023-12-02T00:31:19.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.136+0000 7fdd3c93f640 1 -- 172.21.15.46:0/2240642045 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd3002cdc0 con 0x7fdd380b3400 2023-12-02T00:31:19.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.136+0000 7fdd3c93f640 1 -- 172.21.15.46:0/2240642045 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd30034660 con 0x7fdd380b3400 2023-12-02T00:31:19.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.137+0000 7fdd3fba5640 1 -- 172.21.15.46:0/2240642045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd380b3400 msgr2=0x7fdd38168360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:19.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.137+0000 7fdd3fba5640 1 --2- 172.21.15.46:0/2240642045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd380b3400 0x7fdd38168360 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fdd30009d90 tx=0x7fdd3002beb0 comp rx=0 tx=0).stop 2023-12-02T00:31:19.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.137+0000 7fdd3fba5640 1 -- 172.21.15.46:0/2240642045 shutdown_connections 2023-12-02T00:31:19.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.137+0000 7fdd3fba5640 1 --2- 172.21.15.46:0/2240642045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd380b3400 0x7fdd38168360 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:19.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.137+0000 7fdd3fba5640 1 -- 172.21.15.46:0/2240642045 >> 172.21.15.46:0/2240642045 conn(0x7fdd3806a4d0 msgr2=0x7fdd3806a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:19.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.137+0000 7fdd3fba5640 1 -- 172.21.15.46:0/2240642045 shutdown_connections 2023-12-02T00:31:19.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.137+0000 7fdd3fba5640 1 -- 172.21.15.46:0/2240642045 wait complete. 2023-12-02T00:31:19.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.138+0000 7fdd3fba5640 1 Processor -- start 2023-12-02T00:31:19.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.138+0000 7fdd3fba5640 1 -- start start 2023-12-02T00:31:19.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.139+0000 7fdd3fba5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd380b3400 0x7fdd380b5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:19.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.139+0000 7fdd3fba5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd38146010 0x7fdd380ce260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:19.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.139+0000 7fdd3fba5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd38146d70 0x7fdd3815f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:19.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.139+0000 7fdd3fba5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3815ea70 con 0x7fdd380b3400 2023-12-02T00:31:19.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.139+0000 7fdd3fba5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd380cc600 con 0x7fdd38146d70 2023-12-02T00:31:19.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.139+0000 7fdd3fba5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd380cc8e0 con 0x7fdd38146010 2023-12-02T00:31:19.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.139+0000 7fdd3d941640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd380b3400 0x7fdd380b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:19.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.139+0000 7fdd3d941640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd380b3400 0x7fdd380b5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39794/0 (socket says 172.21.15.46:39794) 2023-12-02T00:31:19.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.139+0000 7fdd3d941640 1 -- 172.21.15.46:0/1722993155 learned_addr learned my addr 172.21.15.46:0/1722993155 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:19.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.139+0000 7fdd3d140640 1 --2- 172.21.15.46:0/1722993155 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd38146010 0x7fdd380ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:19.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.139+0000 7fdd3e142640 1 --2- 172.21.15.46:0/1722993155 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd38146d70 0x7fdd3815f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:19.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.139+0000 7fdd3d941640 1 -- 172.21.15.46:0/1722993155 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd38146010 msgr2=0x7fdd380ce260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:19.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.139+0000 7fdd3d941640 1 --2- 172.21.15.46:0/1722993155 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd38146010 0x7fdd380ce260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:19.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.139+0000 7fdd3d941640 1 -- 172.21.15.46:0/1722993155 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd38146d70 msgr2=0x7fdd3815f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:19.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.139+0000 7fdd3d941640 1 --2- 172.21.15.46:0/1722993155 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd38146d70 0x7fdd3815f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:19.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.139+0000 7fdd3d941640 1 -- 172.21.15.46:0/1722993155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd30009a70 con 0x7fdd380b3400 2023-12-02T00:31:19.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.139+0000 7fdd3d941640 1 --2- 172.21.15.46:0/1722993155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd380b3400 0x7fdd380b5450 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fdd3400c4d0 tx=0x7fdd3400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:19.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.139+0000 7fdd2e7fc640 1 -- 172.21.15.46:0/1722993155 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd340074e0 con 0x7fdd380b3400 2023-12-02T00:31:19.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.140+0000 7fdd2e7fc640 1 -- 172.21.15.46:0/1722993155 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd34007660 con 0x7fdd380b3400 2023-12-02T00:31:19.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.140+0000 7fdd3fba5640 1 -- 172.21.15.46:0/1722993155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd38146570 con 0x7fdd380b3400 2023-12-02T00:31:19.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.140+0000 7fdd2e7fc640 1 -- 172.21.15.46:0/1722993155 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd34004a20 con 0x7fdd380b3400 2023-12-02T00:31:19.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.140+0000 7fdd3fba5640 1 -- 172.21.15.46:0/1722993155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd380b5990 con 0x7fdd380b3400 2023-12-02T00:31:19.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.140+0000 7fdd3fba5640 1 -- 172.21.15.46:0/1722993155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd3815ef00 con 0x7fdd380b3400 2023-12-02T00:31:19.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.141+0000 7fdd2e7fc640 1 -- 172.21.15.46:0/1722993155 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdd34004b80 con 0x7fdd380b3400 2023-12-02T00:31:19.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.141+0000 7fdd2e7fc640 1 --2- 172.21.15.46:0/1722993155 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fdd1c06ae40 0x7fdd1c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:19.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.142+0000 7fdd2e7fc640 1 -- 172.21.15.46:0/1722993155 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdd34088b40 con 0x7fdd380b3400 2023-12-02T00:31:19.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.142+0000 7fdd3d140640 1 --2- 172.21.15.46:0/1722993155 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fdd1c06ae40 0x7fdd1c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:19.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.142+0000 7fdd3d140640 1 --2- 172.21.15.46:0/1722993155 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fdd1c06ae40 0x7fdd1c06d300 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdd28006fd0 tx=0x7fdd28008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:19.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.146+0000 7fdd2e7fc640 1 -- 172.21.15.46:0/1722993155 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdd3815ef00 con 0x7fdd380b3400 2023-12-02T00:31:19.393 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.392+0000 7fdd3fba5640 1 -- 172.21.15.46:0/1722993155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow w pool=cephfs_data"]} v 0) v1 -- 0x7fdd3815ef00 con 0x7fdd380b3400 2023-12-02T00:31:19.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.394+0000 7fdd2e7fc640 1 -- 172.21.15.46:0/1722993155 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow w pool=cephfs_data"]}]=0 updated caps for client.0 v19) v1 ==== 183+0+140 (secure 0 0 0) 0x7fdd3815ef00 con 0x7fdd380b3400 2023-12-02T00:31:19.396 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2023-12-02T00:31:19.396 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAJempl5Lx1DhAAjpK6ajoQ5jQiUr/EMpFeLg== 2023-12-02T00:31:19.396 INFO:teuthology.orchestra.run.smithi046.stdout: caps mds = "allow" 2023-12-02T00:31:19.396 INFO:teuthology.orchestra.run.smithi046.stdout: caps mon = "allow r" 2023-12-02T00:31:19.396 INFO:teuthology.orchestra.run.smithi046.stdout: caps osd = "allow w pool=cephfs_data" 2023-12-02T00:31:19.396 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2023-12-02T00:31:19.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.397+0000 7fdd3fba5640 1 -- 172.21.15.46:0/1722993155 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fdd1c06ae40 msgr2=0x7fdd1c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:19.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.397+0000 7fdd3fba5640 1 --2- 172.21.15.46:0/1722993155 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fdd1c06ae40 0x7fdd1c06d300 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdd28006fd0 tx=0x7fdd28008040 comp rx=0 tx=0).stop 2023-12-02T00:31:19.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.397+0000 7fdd3fba5640 1 -- 172.21.15.46:0/1722993155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd380b3400 msgr2=0x7fdd380b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:19.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.397+0000 7fdd3fba5640 1 --2- 172.21.15.46:0/1722993155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd380b3400 0x7fdd380b5450 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fdd3400c4d0 tx=0x7fdd3400c990 comp rx=0 tx=0).stop 2023-12-02T00:31:19.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.397+0000 7fdd3fba5640 1 -- 172.21.15.46:0/1722993155 shutdown_connections 2023-12-02T00:31:19.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.397+0000 7fdd3fba5640 1 --2- 172.21.15.46:0/1722993155 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fdd1c06ae40 0x7fdd1c06d300 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:19.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.397+0000 7fdd3fba5640 1 --2- 172.21.15.46:0/1722993155 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd38146d70 0x7fdd3815f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:19.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.397+0000 7fdd3fba5640 1 --2- 172.21.15.46:0/1722993155 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd38146010 0x7fdd380ce260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:19.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.397+0000 7fdd3fba5640 1 --2- 172.21.15.46:0/1722993155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd380b3400 0x7fdd380b5450 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:19.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.397+0000 7fdd3fba5640 1 -- 172.21.15.46:0/1722993155 >> 172.21.15.46:0/1722993155 conn(0x7fdd3806a4d0 msgr2=0x7fdd3806a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:19.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.397+0000 7fdd3fba5640 1 -- 172.21.15.46:0/1722993155 shutdown_connections 2023-12-02T00:31:19.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.397+0000 7fdd3fba5640 1 -- 172.21.15.46:0/1722993155 wait complete. 2023-12-02T00:31:19.408 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-02T00:31:19.409 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-02T00:31:19.424 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:31:19.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.569+0000 7f3c8a7d5640 1 Processor -- start 2023-12-02T00:31:19.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.569+0000 7f3c8a7d5640 1 -- start start 2023-12-02T00:31:19.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.570+0000 7f3c8a7d5640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3c84143490 0x7f3c840b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:19.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.570+0000 7f3c8a7d5640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c8415e2a0 con 0x7f3c8415abf0 2023-12-02T00:31:19.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.570+0000 7f3c8a7d5640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c8415b240 con 0x7f3c8415e730 2023-12-02T00:31:19.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.570+0000 7f3c8a7d5640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c8415b520 con 0x7f3c84143490 2023-12-02T00:31:19.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.570+0000 7f3c83fff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3c84143490 0x7f3c840b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:19.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.570+0000 7f3c88d72640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f3c8415abf0 0x7f3c840b2f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57198/0 (socket says 172.21.15.46:57198) 2023-12-02T00:31:19.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.570+0000 7f3c88d72640 1 -- 172.21.15.46:0/2007329779 learned_addr learned my addr 172.21.15.46:0/2007329779 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:31:19.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.570+0000 7f3c83fff640 1 -- 172.21.15.46:0/2007329779 >> v1:172.21.15.162:6789/0 conn(0x7f3c8415e730 legacy=0x7f3c8415a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:19.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.570+0000 7f3c83fff640 1 -- 172.21.15.46:0/2007329779 >> v1:172.21.15.46:6789/0 conn(0x7f3c8415abf0 legacy=0x7f3c840b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:19.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.570+0000 7f3c83fff640 1 -- 172.21.15.46:0/2007329779 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c8415dc50 con 0x7f3c84143490 2023-12-02T00:31:19.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.571+0000 7f3c83fff640 1 --2- 172.21.15.46:0/2007329779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c84143490 0x7f3c840b0cf0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f3c780097a0 tx=0x7f3c7802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f68cf2e0e50b2713 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:19.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.571+0000 7f3c82ffd640 1 -- 172.21.15.46:0/2007329779 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c78031830 con 0x7f3c84143490 2023-12-02T00:31:19.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.571+0000 7f3c82ffd640 1 -- 172.21.15.46:0/2007329779 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c78031990 con 0x7f3c84143490 2023-12-02T00:31:19.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.571+0000 7f3c82ffd640 1 -- 172.21.15.46:0/2007329779 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c78031d20 con 0x7f3c84143490 2023-12-02T00:31:19.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.572+0000 7f3c8a7d5640 1 -- 172.21.15.46:0/2007329779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c84143490 msgr2=0x7f3c840b0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:19.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.572+0000 7f3c8a7d5640 1 --2- 172.21.15.46:0/2007329779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c84143490 0x7f3c840b0cf0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f3c780097a0 tx=0x7f3c7802b8d0 comp rx=0 tx=0).stop 2023-12-02T00:31:19.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.572+0000 7f3c8a7d5640 1 -- 172.21.15.46:0/2007329779 shutdown_connections 2023-12-02T00:31:19.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.572+0000 7f3c8a7d5640 1 --2- 172.21.15.46:0/2007329779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c84143490 0x7f3c840b0cf0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:19.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.572+0000 7f3c8a7d5640 1 -- 172.21.15.46:0/2007329779 >> 172.21.15.46:0/2007329779 conn(0x7f3c8406a4d0 msgr2=0x7f3c8406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:19.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.572+0000 7f3c8a7d5640 1 -- 172.21.15.46:0/2007329779 shutdown_connections 2023-12-02T00:31:19.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.572+0000 7f3c8a7d5640 1 -- 172.21.15.46:0/2007329779 wait complete. 2023-12-02T00:31:19.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.573+0000 7f3c8a7d5640 1 Processor -- start 2023-12-02T00:31:19.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.573+0000 7f3c8a7d5640 1 -- start start 2023-12-02T00:31:19.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.574+0000 7f3c8a7d5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3c84143490 0x7f3c841e5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:19.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.574+0000 7f3c8a7d5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c8415abf0 0x7f3c84159c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:19.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.574+0000 7f3c8a7d5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c8415e730 0x7f3c841f66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:19.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.574+0000 7f3c8a7d5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c84131390 con 0x7f3c8415abf0 2023-12-02T00:31:19.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.574+0000 7f3c8a7d5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c841f6ed0 con 0x7f3c8415e730 2023-12-02T00:31:19.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.574+0000 7f3c8a7d5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c841f71b0 con 0x7f3c84143490 2023-12-02T00:31:19.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.574+0000 7f3c837fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c8415abf0 0x7f3c84159c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:19.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.574+0000 7f3c837fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c8415abf0 0x7f3c84159c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39814/0 (socket says 172.21.15.46:39814) 2023-12-02T00:31:19.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.574+0000 7f3c837fe640 1 -- 172.21.15.46:0/3987719562 learned_addr learned my addr 172.21.15.46:0/3987719562 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:19.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.574+0000 7f3c83fff640 1 --2- 172.21.15.46:0/3987719562 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3c84143490 0x7f3c841e5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:19.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.574+0000 7f3c88d72640 1 --2- 172.21.15.46:0/3987719562 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c8415e730 0x7f3c841f66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:19.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.574+0000 7f3c837fe640 1 -- 172.21.15.46:0/3987719562 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3c84143490 msgr2=0x7f3c841e5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:19.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.574+0000 7f3c837fe640 1 --2- 172.21.15.46:0/3987719562 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3c84143490 0x7f3c841e5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:19.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.574+0000 7f3c837fe640 1 -- 172.21.15.46:0/3987719562 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c8415e730 msgr2=0x7f3c841f66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:19.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.574+0000 7f3c837fe640 1 --2- 172.21.15.46:0/3987719562 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c8415e730 0x7f3c841f66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:19.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.574+0000 7f3c837fe640 1 -- 172.21.15.46:0/3987719562 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c78009400 con 0x7f3c8415abf0 2023-12-02T00:31:19.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.574+0000 7f3c837fe640 1 --2- 172.21.15.46:0/3987719562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c8415abf0 0x7f3c84159c10 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f3c7400b3f0 tx=0x7f3c7400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:19.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.575+0000 7f3c80ff9640 1 -- 172.21.15.46:0/3987719562 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c74011830 con 0x7f3c8415abf0 2023-12-02T00:31:19.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.575+0000 7f3c80ff9640 1 -- 172.21.15.46:0/3987719562 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c74011990 con 0x7f3c8415abf0 2023-12-02T00:31:19.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.575+0000 7f3c80ff9640 1 -- 172.21.15.46:0/3987719562 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c74011d40 con 0x7f3c8415abf0 2023-12-02T00:31:19.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.575+0000 7f3c8a7d5640 1 -- 172.21.15.46:0/3987719562 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c841f7490 con 0x7f3c8415abf0 2023-12-02T00:31:19.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.575+0000 7f3c8a7d5640 1 -- 172.21.15.46:0/3987719562 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c841f6be0 con 0x7f3c8415abf0 2023-12-02T00:31:19.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.576+0000 7f3c8a7d5640 1 -- 172.21.15.46:0/3987719562 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c840b4380 con 0x7f3c8415abf0 2023-12-02T00:31:19.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.576+0000 7f3c80ff9640 1 -- 172.21.15.46:0/3987719562 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3c7401a400 con 0x7f3c8415abf0 2023-12-02T00:31:19.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.576+0000 7f3c80ff9640 1 --2- 172.21.15.46:0/3987719562 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3c7006ae70 0x7f3c7006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:19.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.577+0000 7f3c80ff9640 1 -- 172.21.15.46:0/3987719562 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3c74089520 con 0x7f3c8415abf0 2023-12-02T00:31:19.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.577+0000 7f3c83fff640 1 --2- 172.21.15.46:0/3987719562 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3c7006ae70 0x7f3c7006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:19.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.577+0000 7f3c83fff640 1 --2- 172.21.15.46:0/3987719562 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3c7006ae70 0x7f3c7006d330 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3c78036000 tx=0x7f3c7803c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:19.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.581+0000 7f3c80ff9640 1 -- 172.21.15.46:0/3987719562 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3c740544b0 con 0x7f3c8415abf0 2023-12-02T00:31:19.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.759+0000 7f3c8a7d5640 1 -- 172.21.15.46:0/3987719562 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3c84069400 con 0x7f3c8415abf0 2023-12-02T00:31:19.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.760+0000 7f3c80ff9640 1 -- 172.21.15.46:0/3987719562 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+16545 (secure 0 0 0) 0x7f3c74057ad0 con 0x7f3c8415abf0 2023-12-02T00:31:19.761 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:31:19.761 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":51,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:31:02.837102+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-12-02T00:30:59.469070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-12-02T00:31:00.174610+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:31:19.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.762+0000 7f3c8a7d5640 1 -- 172.21.15.46:0/3987719562 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3c7006ae70 msgr2=0x7f3c7006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:19.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.762+0000 7f3c8a7d5640 1 --2- 172.21.15.46:0/3987719562 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3c7006ae70 0x7f3c7006d330 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3c78036000 tx=0x7f3c7803c040 comp rx=0 tx=0).stop 2023-12-02T00:31:19.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.762+0000 7f3c8a7d5640 1 -- 172.21.15.46:0/3987719562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c8415abf0 msgr2=0x7f3c84159c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:19.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.762+0000 7f3c8a7d5640 1 --2- 172.21.15.46:0/3987719562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c8415abf0 0x7f3c84159c10 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f3c7400b3f0 tx=0x7f3c7400b8b0 comp rx=0 tx=0).stop 2023-12-02T00:31:19.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.762+0000 7f3c8a7d5640 1 -- 172.21.15.46:0/3987719562 shutdown_connections 2023-12-02T00:31:19.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.762+0000 7f3c8a7d5640 1 --2- 172.21.15.46:0/3987719562 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3c7006ae70 0x7f3c7006d330 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:19.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.762+0000 7f3c8a7d5640 1 --2- 172.21.15.46:0/3987719562 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c8415e730 0x7f3c841f66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:19.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.762+0000 7f3c8a7d5640 1 --2- 172.21.15.46:0/3987719562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c8415abf0 0x7f3c84159c10 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:19.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.762+0000 7f3c8a7d5640 1 --2- 172.21.15.46:0/3987719562 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3c84143490 0x7f3c841e5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:19.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.762+0000 7f3c8a7d5640 1 -- 172.21.15.46:0/3987719562 >> 172.21.15.46:0/3987719562 conn(0x7f3c8406a4d0 msgr2=0x7f3c84070350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:19.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.762+0000 7f3c8a7d5640 1 -- 172.21.15.46:0/3987719562 shutdown_connections 2023-12-02T00:31:19.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.762+0000 7f3c8a7d5640 1 -- 172.21.15.46:0/3987719562 wait complete. 2023-12-02T00:31:19.774 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-02T00:28:37.213645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-02T00:30:59.469070+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-12-02T00:31:00.174610+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-02T00:31:19.775 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-02T00:31:19.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.883+0000 7f258c290640 1 Processor -- start 2023-12-02T00:31:19.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.883+0000 7f258c290640 1 -- start start 2023-12-02T00:31:19.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.884+0000 7f258c290640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2584143490 0x7f25840b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:19.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.884+0000 7f258c290640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f258415abf0 0x7f25840b1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:19.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.884+0000 7f258c290640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f258415e2a0 con 0x7f258415e730 2023-12-02T00:31:19.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.884+0000 7f258c290640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f258415b520 con 0x7f258415abf0 2023-12-02T00:31:19.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.884+0000 7f258c290640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2584143870 con 0x7f2584143490 2023-12-02T00:31:19.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.884+0000 7f258a82d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f258415abf0 0x7f25840b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:19.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.884+0000 7f258982b640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f258415e730 0x7f258415a4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57210/0 (socket says 172.21.15.46:57210) 2023-12-02T00:31:19.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.884+0000 7f258982b640 1 -- 172.21.15.46:0/1654277651 learned_addr learned my addr 172.21.15.46:0/1654277651 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:31:19.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.884+0000 7f258a02c640 1 --2- 172.21.15.46:0/1654277651 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2584143490 0x7f25840b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:19.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.884+0000 7f258a82d640 1 -- 172.21.15.46:0/1654277651 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2584143490 msgr2=0x7f25840b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:19.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.884+0000 7f258a82d640 1 --2- 172.21.15.46:0/1654277651 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2584143490 0x7f25840b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:19.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.884+0000 7f258a82d640 1 -- 172.21.15.46:0/1654277651 >> v1:172.21.15.46:6789/0 conn(0x7f258415e730 legacy=0x7f258415a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:19.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.884+0000 7f258a82d640 1 -- 172.21.15.46:0/1654277651 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f258415b240 con 0x7f258415abf0 2023-12-02T00:31:19.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.884+0000 7f258902a640 1 -- 172.21.15.46:0/1654277651 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2868361649 0 0) 0x7f258415e2a0 con 0x7f258415e730 2023-12-02T00:31:19.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.885+0000 7f258a82d640 1 --2- 172.21.15.46:0/1654277651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f258415abf0 0x7f25840b1230 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f2580009d90 tx=0x7f258002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d804b02ad7aa4f8 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:19.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.885+0000 7f258902a640 1 -- 172.21.15.46:0/1654277651 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258002cc50 con 0x7f258415abf0 2023-12-02T00:31:19.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.885+0000 7f258902a640 1 -- 172.21.15.46:0/1654277651 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f258002cdb0 con 0x7f258415abf0 2023-12-02T00:31:19.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.885+0000 7f258902a640 1 -- 172.21.15.46:0/1654277651 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2580034660 con 0x7f258415abf0 2023-12-02T00:31:19.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.886+0000 7f258c290640 1 -- 172.21.15.46:0/1654277651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f258415abf0 msgr2=0x7f25840b1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:19.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.886+0000 7f258c290640 1 --2- 172.21.15.46:0/1654277651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f258415abf0 0x7f25840b1230 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f2580009d90 tx=0x7f258002bea0 comp rx=0 tx=0).stop 2023-12-02T00:31:19.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.886+0000 7f258c290640 1 -- 172.21.15.46:0/1654277651 shutdown_connections 2023-12-02T00:31:19.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.886+0000 7f258c290640 1 --2- 172.21.15.46:0/1654277651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f258415abf0 0x7f25840b1230 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:19.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.886+0000 7f258c290640 1 --2- 172.21.15.46:0/1654277651 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2584143490 0x7f25840b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:19.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.886+0000 7f258c290640 1 -- 172.21.15.46:0/1654277651 >> 172.21.15.46:0/1654277651 conn(0x7f258406a4d0 msgr2=0x7f258406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:19.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.886+0000 7f258c290640 1 -- 172.21.15.46:0/1654277651 shutdown_connections 2023-12-02T00:31:19.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.886+0000 7f258c290640 1 -- 172.21.15.46:0/1654277651 wait complete. 2023-12-02T00:31:19.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.887+0000 7f258c290640 1 Processor -- start 2023-12-02T00:31:19.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.887+0000 7f258c290640 1 -- start start 2023-12-02T00:31:19.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.888+0000 7f258c290640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2584143490 0x7f2584159a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:19.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.888+0000 7f258c290640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f258415abf0 0x7f25840aa520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:19.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.888+0000 7f258c290640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f258415e730 0x7f25840aaa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:19.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.888+0000 7f258c290640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258413f680 con 0x7f258415e730 2023-12-02T00:31:19.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.888+0000 7f258c290640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2584159fc0 con 0x7f2584143490 2023-12-02T00:31:19.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.888+0000 7f258c290640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25840ad9c0 con 0x7f258415abf0 2023-12-02T00:31:19.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.888+0000 7f258a82d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f258415e730 0x7f25840aaa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:19.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.888+0000 7f258a82d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f258415e730 0x7f25840aaa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39828/0 (socket says 172.21.15.46:39828) 2023-12-02T00:31:19.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.888+0000 7f258a82d640 1 -- 172.21.15.46:0/4091865210 learned_addr learned my addr 172.21.15.46:0/4091865210 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:19.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.888+0000 7f258982b640 1 --2- 172.21.15.46:0/4091865210 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f258415abf0 0x7f25840aa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:19.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.888+0000 7f258a82d640 1 -- 172.21.15.46:0/4091865210 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f258415abf0 msgr2=0x7f25840aa520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:19.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.888+0000 7f258a82d640 1 --2- 172.21.15.46:0/4091865210 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f258415abf0 0x7f25840aa520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:19.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.888+0000 7f258a82d640 1 -- 172.21.15.46:0/4091865210 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2584143490 msgr2=0x7f2584159a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:19.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.888+0000 7f258a82d640 1 --2- 172.21.15.46:0/4091865210 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2584143490 0x7f2584159a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:19.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.888+0000 7f258a82d640 1 -- 172.21.15.46:0/4091865210 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2580009a70 con 0x7f258415e730 2023-12-02T00:31:19.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.888+0000 7f258a82d640 1 --2- 172.21.15.46:0/4091865210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f258415e730 0x7f25840aaa60 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f2580009d90 tx=0x7f2580004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:19.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.889+0000 7f257affd640 1 -- 172.21.15.46:0/4091865210 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258003e610 con 0x7f258415e730 2023-12-02T00:31:19.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.889+0000 7f257affd640 1 -- 172.21.15.46:0/4091865210 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f258003e790 con 0x7f258415e730 2023-12-02T00:31:19.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.889+0000 7f258c290640 1 -- 172.21.15.46:0/4091865210 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25840adca0 con 0x7f258415e730 2023-12-02T00:31:19.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.889+0000 7f257affd640 1 -- 172.21.15.46:0/4091865210 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258003eb40 con 0x7f258415e730 2023-12-02T00:31:19.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.889+0000 7f258c290640 1 -- 172.21.15.46:0/4091865210 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25840ae260 con 0x7f258415e730 2023-12-02T00:31:19.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.890+0000 7f258c290640 1 -- 172.21.15.46:0/4091865210 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25840b4380 con 0x7f258415e730 2023-12-02T00:31:19.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.890+0000 7f257affd640 1 -- 172.21.15.46:0/4091865210 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f258003b050 con 0x7f258415e730 2023-12-02T00:31:19.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.890+0000 7f257affd640 1 --2- 172.21.15.46:0/4091865210 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f256806ae40 0x7f256806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:19.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.891+0000 7f257affd640 1 -- 172.21.15.46:0/4091865210 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f25800abc20 con 0x7f258415e730 2023-12-02T00:31:19.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.891+0000 7f258a02c640 1 --2- 172.21.15.46:0/4091865210 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f256806ae40 0x7f256806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:19.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.891+0000 7f258a02c640 1 --2- 172.21.15.46:0/4091865210 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f256806ae40 0x7f256806d300 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f257c0098f0 tx=0x7f257c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:19.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:19.895+0000 7f257affd640 1 -- 172.21.15.46:0/4091865210 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2580076cd0 con 0x7f258415e730 2023-12-02T00:31:20.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.070+0000 7f258c290640 1 -- 172.21.15.46:0/4091865210 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2584069400 con 0x7f258415e730 2023-12-02T00:31:20.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.070+0000 7f257affd640 1 -- 172.21.15.46:0/4091865210 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) v1 ==== 93+0+10 (secure 0 0 0) 0x7f258007a2f0 con 0x7f258415e730 2023-12-02T00:31:20.071 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-12-02T00:31:20.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.072+0000 7f258c290640 1 -- 172.21.15.46:0/4091865210 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f256806ae40 msgr2=0x7f256806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:20.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.072+0000 7f258c290640 1 --2- 172.21.15.46:0/4091865210 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f256806ae40 0x7f256806d300 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f257c0098f0 tx=0x7f257c008040 comp rx=0 tx=0).stop 2023-12-02T00:31:20.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.072+0000 7f258c290640 1 -- 172.21.15.46:0/4091865210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f258415e730 msgr2=0x7f25840aaa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:20.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.072+0000 7f258c290640 1 --2- 172.21.15.46:0/4091865210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f258415e730 0x7f25840aaa60 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f2580009d90 tx=0x7f2580004a40 comp rx=0 tx=0).stop 2023-12-02T00:31:20.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.072+0000 7f258c290640 1 -- 172.21.15.46:0/4091865210 shutdown_connections 2023-12-02T00:31:20.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.072+0000 7f258c290640 1 --2- 172.21.15.46:0/4091865210 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f256806ae40 0x7f256806d300 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:20.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.072+0000 7f258c290640 1 --2- 172.21.15.46:0/4091865210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f258415e730 0x7f25840aaa60 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:20.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.072+0000 7f258c290640 1 --2- 172.21.15.46:0/4091865210 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f258415abf0 0x7f25840aa520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:20.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.072+0000 7f258c290640 1 --2- 172.21.15.46:0/4091865210 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2584143490 0x7f2584159a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:20.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.072+0000 7f258c290640 1 -- 172.21.15.46:0/4091865210 >> 172.21.15.46:0/4091865210 conn(0x7f258406a4d0 msgr2=0x7f2584071d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:20.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.072+0000 7f258c290640 1 -- 172.21.15.46:0/4091865210 shutdown_connections 2023-12-02T00:31:20.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.072+0000 7f258c290640 1 -- 172.21.15.46:0/4091865210 wait complete. 2023-12-02T00:31:20.084 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-02T00:31:20.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.193+0000 7f0092fa5640 1 Processor -- start 2023-12-02T00:31:20.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.193+0000 7f0092fa5640 1 -- start start 2023-12-02T00:31:20.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.194+0000 7f0092fa5640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f008c06ea50 0x7f008c06ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:20.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.194+0000 7f0092fa5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f008c160fb0 0x7f008c161440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:20.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.194+0000 7f0092fa5640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f008c0b3ff0 0x7f008c0b4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:20.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.194+0000 7f0092fa5640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f008c161d20 con 0x7f008c06ea50 2023-12-02T00:31:20.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.194+0000 7f0092fa5640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f008c06f370 con 0x7f008c0b3ff0 2023-12-02T00:31:20.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.194+0000 7f0092fa5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f008c15e760 con 0x7f008c160fb0 2023-12-02T00:31:20.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.194+0000 7f0090d41640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f008c06ea50 0x7f008c06ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:20.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.194+0000 7f0090d41640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f008c06ea50 0x7f008c06ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39838/0 (socket says 172.21.15.46:39838) 2023-12-02T00:31:20.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.194+0000 7f0090d41640 1 -- 172.21.15.46:0/3420368807 learned_addr learned my addr 172.21.15.46:0/3420368807 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:20.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.194+0000 7f008bfff640 1 --2- 172.21.15.46:0/3420368807 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f008c160fb0 0x7f008c161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:20.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.194+0000 7f0091542640 1 --2- 172.21.15.46:0/3420368807 >> v2:172.21.15.162:3300/0 conn(0x7f008c0b3ff0 0x7f008c0b4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:20.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.194+0000 7f0090d41640 1 -- 172.21.15.46:0/3420368807 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f008c160fb0 msgr2=0x7f008c161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:20.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.194+0000 7f0090d41640 1 --2- 172.21.15.46:0/3420368807 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f008c160fb0 0x7f008c161440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:20.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.194+0000 7f0090d41640 1 -- 172.21.15.46:0/3420368807 >> v2:172.21.15.162:3300/0 conn(0x7f008c0b3ff0 msgr2=0x7f008c0b4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:20.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.194+0000 7f0090d41640 1 --2- 172.21.15.46:0/3420368807 >> v2:172.21.15.162:3300/0 conn(0x7f008c0b3ff0 0x7f008c0b4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:20.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.194+0000 7f0090d41640 1 -- 172.21.15.46:0/3420368807 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f008c15d9a0 con 0x7f008c06ea50 2023-12-02T00:31:20.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.195+0000 7f0090d41640 1 --2- 172.21.15.46:0/3420368807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f008c06ea50 0x7f008c06ee30 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f008c0b6690 tx=0x7f008002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deda5b4727d76e40 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:20.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.195+0000 7f008b7fe640 1 -- 172.21.15.46:0/3420368807 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0080031870 con 0x7f008c06ea50 2023-12-02T00:31:20.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.195+0000 7f008b7fe640 1 -- 172.21.15.46:0/3420368807 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00800319d0 con 0x7f008c06ea50 2023-12-02T00:31:20.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.195+0000 7f008b7fe640 1 -- 172.21.15.46:0/3420368807 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0080031d60 con 0x7f008c06ea50 2023-12-02T00:31:20.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.196+0000 7f0092fa5640 1 -- 172.21.15.46:0/3420368807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f008c06ea50 msgr2=0x7f008c06ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:20.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.196+0000 7f0092fa5640 1 --2- 172.21.15.46:0/3420368807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f008c06ea50 0x7f008c06ee30 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f008c0b6690 tx=0x7f008002b910 comp rx=0 tx=0).stop 2023-12-02T00:31:20.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.196+0000 7f0092fa5640 1 -- 172.21.15.46:0/3420368807 shutdown_connections 2023-12-02T00:31:20.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.196+0000 7f0092fa5640 1 --2- 172.21.15.46:0/3420368807 >> v2:172.21.15.162:3300/0 conn(0x7f008c0b3ff0 0x7f008c0b4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:20.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.196+0000 7f0092fa5640 1 --2- 172.21.15.46:0/3420368807 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f008c160fb0 0x7f008c161440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:20.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.196+0000 7f0092fa5640 1 --2- 172.21.15.46:0/3420368807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f008c06ea50 0x7f008c06ee30 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:20.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.196+0000 7f0092fa5640 1 -- 172.21.15.46:0/3420368807 >> 172.21.15.46:0/3420368807 conn(0x7f008c06a6b0 msgr2=0x7f008c06aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:20.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.196+0000 7f0092fa5640 1 -- 172.21.15.46:0/3420368807 shutdown_connections 2023-12-02T00:31:20.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.196+0000 7f0092fa5640 1 -- 172.21.15.46:0/3420368807 wait complete. 2023-12-02T00:31:20.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.197+0000 7f0092fa5640 1 Processor -- start 2023-12-02T00:31:20.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.197+0000 7f0092fa5640 1 -- start start 2023-12-02T00:31:20.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.198+0000 7f0092fa5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f008c06ea50 0x7f008c156cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:20.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.198+0000 7f0092fa5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f008c0b3ff0 0x7f008c1584d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:20.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.198+0000 7f0092fa5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f008c160fb0 0x7f008c15f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:20.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.198+0000 7f0092fa5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008c0d07c0 con 0x7f008c06ea50 2023-12-02T00:31:20.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.198+0000 7f0092fa5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008c157230 con 0x7f008c0b3ff0 2023-12-02T00:31:20.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.198+0000 7f0092fa5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f008c158a10 con 0x7f008c160fb0 2023-12-02T00:31:20.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.198+0000 7f0090d41640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f008c06ea50 0x7f008c156cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:20.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.198+0000 7f0090d41640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f008c06ea50 0x7f008c156cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39840/0 (socket says 172.21.15.46:39840) 2023-12-02T00:31:20.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.198+0000 7f0090d41640 1 -- 172.21.15.46:0/2421111455 learned_addr learned my addr 172.21.15.46:0/2421111455 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:20.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.198+0000 7f008bfff640 1 --2- 172.21.15.46:0/2421111455 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f008c0b3ff0 0x7f008c1584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:20.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.198+0000 7f0091542640 1 --2- 172.21.15.46:0/2421111455 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f008c160fb0 0x7f008c15f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:20.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.198+0000 7f0090d41640 1 -- 172.21.15.46:0/2421111455 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f008c160fb0 msgr2=0x7f008c15f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:20.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.198+0000 7f0090d41640 1 --2- 172.21.15.46:0/2421111455 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f008c160fb0 0x7f008c15f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:20.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.198+0000 7f0090d41640 1 -- 172.21.15.46:0/2421111455 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f008c0b3ff0 msgr2=0x7f008c1584d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:20.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.198+0000 7f0090d41640 1 --2- 172.21.15.46:0/2421111455 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f008c0b3ff0 0x7f008c1584d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:20.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.198+0000 7f0090d41640 1 -- 172.21.15.46:0/2421111455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00800094d0 con 0x7f008c06ea50 2023-12-02T00:31:20.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.198+0000 7f0091542640 1 --2- 172.21.15.46:0/2421111455 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f008c160fb0 0x7f008c15f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:31:20.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.199+0000 7f0090d41640 1 --2- 172.21.15.46:0/2421111455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f008c06ea50 0x7f008c156cf0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f008002b950 tx=0x7f00800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:20.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.199+0000 7f00897fa640 1 -- 172.21.15.46:0/2421111455 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0080031860 con 0x7f008c06ea50 2023-12-02T00:31:20.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.199+0000 7f00897fa640 1 -- 172.21.15.46:0/2421111455 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00800443f0 con 0x7f008c06ea50 2023-12-02T00:31:20.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.199+0000 7f0092fa5640 1 -- 172.21.15.46:0/2421111455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f008c158040 con 0x7f008c06ea50 2023-12-02T00:31:20.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.199+0000 7f00897fa640 1 -- 172.21.15.46:0/2421111455 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00800447a0 con 0x7f008c06ea50 2023-12-02T00:31:20.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.199+0000 7f0092fa5640 1 -- 172.21.15.46:0/2421111455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f008c1f3f40 con 0x7f008c06ea50 2023-12-02T00:31:20.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.200+0000 7f0092fa5640 1 -- 172.21.15.46:0/2421111455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f008c069400 con 0x7f008c06ea50 2023-12-02T00:31:20.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.200+0000 7f00897fa640 1 -- 172.21.15.46:0/2421111455 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0080044900 con 0x7f008c06ea50 2023-12-02T00:31:20.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.200+0000 7f00897fa640 1 --2- 172.21.15.46:0/2421111455 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f007006ae40 0x7f007006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:20.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.201+0000 7f00897fa640 1 -- 172.21.15.46:0/2421111455 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f008002d040 con 0x7f008c06ea50 2023-12-02T00:31:20.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.201+0000 7f008bfff640 1 --2- 172.21.15.46:0/2421111455 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f007006ae40 0x7f007006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:20.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.201+0000 7f008bfff640 1 --2- 172.21.15.46:0/2421111455 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f007006ae40 0x7f007006d300 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f008c159010 tx=0x7f0078008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:20.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.205+0000 7f00897fa640 1 -- 172.21.15.46:0/2421111455 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0080076f00 con 0x7f008c06ea50 2023-12-02T00:31:20.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.380+0000 7f0092fa5640 1 -- 172.21.15.46:0/2421111455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f008c06ab90 con 0x7f008c06ea50 2023-12-02T00:31:20.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.381+0000 7f00897fa640 1 -- 172.21.15.46:0/2421111455 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v51) v1 ==== 104+0+11 (secure 0 0 0) 0x7f00800abe60 con 0x7f008c06ea50 2023-12-02T00:31:20.382 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-12-02T00:31:20.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.383+0000 7f0092fa5640 1 -- 172.21.15.46:0/2421111455 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f007006ae40 msgr2=0x7f007006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:20.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.383+0000 7f0092fa5640 1 --2- 172.21.15.46:0/2421111455 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f007006ae40 0x7f007006d300 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f008c159010 tx=0x7f0078008040 comp rx=0 tx=0).stop 2023-12-02T00:31:20.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.383+0000 7f0092fa5640 1 -- 172.21.15.46:0/2421111455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f008c06ea50 msgr2=0x7f008c156cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:20.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.383+0000 7f0092fa5640 1 --2- 172.21.15.46:0/2421111455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f008c06ea50 0x7f008c156cf0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f008002b950 tx=0x7f00800046c0 comp rx=0 tx=0).stop 2023-12-02T00:31:20.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.383+0000 7f0092fa5640 1 -- 172.21.15.46:0/2421111455 shutdown_connections 2023-12-02T00:31:20.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.383+0000 7f0092fa5640 1 --2- 172.21.15.46:0/2421111455 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f007006ae40 0x7f007006d300 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:20.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.383+0000 7f0092fa5640 1 --2- 172.21.15.46:0/2421111455 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f008c160fb0 0x7f008c15f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:20.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.383+0000 7f0092fa5640 1 --2- 172.21.15.46:0/2421111455 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f008c0b3ff0 0x7f008c1584d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:20.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.383+0000 7f0092fa5640 1 --2- 172.21.15.46:0/2421111455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f008c06ea50 0x7f008c156cf0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:20.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.383+0000 7f0092fa5640 1 -- 172.21.15.46:0/2421111455 >> 172.21.15.46:0/2421111455 conn(0x7f008c06a6b0 msgr2=0x7f008c140840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:20.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.383+0000 7f0092fa5640 1 -- 172.21.15.46:0/2421111455 shutdown_connections 2023-12-02T00:31:20.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.383+0000 7f0092fa5640 1 -- 172.21.15.46:0/2421111455 wait complete. 2023-12-02T00:31:20.398 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-02T00:31:20.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.506+0000 7f34dfbf5640 1 Processor -- start 2023-12-02T00:31:20.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.506+0000 7f34dfbf5640 1 -- start start 2023-12-02T00:31:20.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.507+0000 7f34dfbf5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34d815c2f0 0x7f34d806eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:20.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.507+0000 7f34dfbf5640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f34d806f040 0x7f34d8140800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:20.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.507+0000 7f34dfbf5640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34d815ba40 con 0x7f34d8140d40 2023-12-02T00:31:20.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.507+0000 7f34dfbf5640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34d815c6d0 con 0x7f34d806f040 2023-12-02T00:31:20.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.507+0000 7f34dfbf5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34d813e530 con 0x7f34d815c2f0 2023-12-02T00:31:20.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.507+0000 7f34de192640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f34d8140d40 0x7f34d815ec70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57222/0 (socket says 172.21.15.46:57222) 2023-12-02T00:31:20.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.507+0000 7f34de192640 1 -- 172.21.15.46:0/481997772 learned_addr learned my addr 172.21.15.46:0/481997772 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:31:20.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.507+0000 7f34dd190640 1 --2- 172.21.15.46:0/481997772 >> v2:172.21.15.162:3300/0 conn(0x7f34d806f040 0x7f34d8140800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:20.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.507+0000 7f34dd991640 1 --2- 172.21.15.46:0/481997772 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34d815c2f0 0x7f34d806eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:20.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.507+0000 7f34dc98f640 1 -- 172.21.15.46:0/481997772 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1480197786 0 0) 0x7f34d815ba40 con 0x7f34d8140d40 2023-12-02T00:31:20.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.507+0000 7f34dc98f640 1 -- 172.21.15.46:0/481997772 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34c4003520 con 0x7f34d8140d40 2023-12-02T00:31:20.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.508+0000 7f34dc98f640 1 -- 172.21.15.46:0/481997772 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1762234283 0 0) 0x7f34c4003520 con 0x7f34d8140d40 2023-12-02T00:31:20.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.508+0000 7f34dc98f640 1 -- 172.21.15.46:0/481997772 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34d815c2f0 msgr2=0x7f34d806eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:20.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.508+0000 7f34dc98f640 1 --2- 172.21.15.46:0/481997772 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34d815c2f0 0x7f34d806eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:20.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.508+0000 7f34dc98f640 1 -- 172.21.15.46:0/481997772 >> v2:172.21.15.162:3300/0 conn(0x7f34d806f040 msgr2=0x7f34d8140800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:20.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.508+0000 7f34dc98f640 1 --2- 172.21.15.46:0/481997772 >> v2:172.21.15.162:3300/0 conn(0x7f34d806f040 0x7f34d8140800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:20.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.508+0000 7f34dc98f640 1 -- 172.21.15.46:0/481997772 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d81411d0 con 0x7f34d8140d40 2023-12-02T00:31:20.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.508+0000 7f34dc98f640 1 -- 172.21.15.46:0/481997772 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7f34d00019f0 con 0x7f34d8140d40 2023-12-02T00:31:20.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.508+0000 7f34dc98f640 1 -- 172.21.15.46:0/481997772 >> v1:172.21.15.46:6789/0 conn(0x7f34d8140d40 legacy=0x7f34d815ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:20.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.508+0000 7f34dc98f640 1 --2- 172.21.15.46:0/481997772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34c40039a0 0x7f34c4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:20.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.508+0000 7f34dc98f640 1 --2- 172.21.15.46:0/481997772 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34c40047c0 0x7f34c4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:20.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.508+0000 7f34dc98f640 1 --2- 172.21.15.46:0/481997772 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34d815c2f0 0x7f34c4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:20.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.508+0000 7f34dc98f640 1 -- 172.21.15.46:0/481997772 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d813e530 con 0x7f34c40039a0 2023-12-02T00:31:20.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.508+0000 7f34dc98f640 1 -- 172.21.15.46:0/481997772 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c400b980 con 0x7f34c40047c0 2023-12-02T00:31:20.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.508+0000 7f34dc98f640 1 -- 172.21.15.46:0/481997772 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34c400bac0 con 0x7f34d815c2f0 2023-12-02T00:31:20.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.508+0000 7f34dd991640 1 --2- 172.21.15.46:0/481997772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34c40039a0 0x7f34c4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:20.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.508+0000 7f34dd190640 1 --2- 172.21.15.46:0/481997772 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34c40047c0 0x7f34c4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:20.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.508+0000 7f34de192640 1 --2- 172.21.15.46:0/481997772 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34d815c2f0 0x7f34c4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:20.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.508+0000 7f34dd991640 1 -- 172.21.15.46:0/481997772 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34d815c2f0 msgr2=0x7f34c4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:20.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.508+0000 7f34dd991640 1 --2- 172.21.15.46:0/481997772 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34d815c2f0 0x7f34c4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:20.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.508+0000 7f34dd991640 1 -- 172.21.15.46:0/481997772 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34c40047c0 msgr2=0x7f34c4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:20.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.508+0000 7f34dd991640 1 --2- 172.21.15.46:0/481997772 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34c40047c0 0x7f34c4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:20.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.508+0000 7f34dd991640 1 -- 172.21.15.46:0/481997772 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34c4003520 con 0x7f34c40039a0 2023-12-02T00:31:20.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.509+0000 7f34dd991640 1 --2- 172.21.15.46:0/481997772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34c40039a0 0x7f34c4003dc0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f34d4009ac0 tx=0x7f34c40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4210881e0e6ffd09 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:20.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.509+0000 7f34dc98f640 1 -- 172.21.15.46:0/481997772 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d402c6b0 con 0x7f34c40039a0 2023-12-02T00:31:20.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.509+0000 7f34dc98f640 1 -- 172.21.15.46:0/481997772 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34d402c810 con 0x7f34c40039a0 2023-12-02T00:31:20.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.509+0000 7f34dc98f640 1 -- 172.21.15.46:0/481997772 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d402cba0 con 0x7f34c40039a0 2023-12-02T00:31:20.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.509+0000 7f34dfbf5640 1 -- 172.21.15.46:0/481997772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34c40039a0 msgr2=0x7f34c4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:20.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.509+0000 7f34dfbf5640 1 --2- 172.21.15.46:0/481997772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34c40039a0 0x7f34c4003dc0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f34d4009ac0 tx=0x7f34c40057f0 comp rx=0 tx=0).stop 2023-12-02T00:31:20.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.509+0000 7f34de192640 1 -- 172.21.15.46:0/481997772 reap_dead start 2023-12-02T00:31:20.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.509+0000 7f34dfbf5640 1 -- 172.21.15.46:0/481997772 shutdown_connections 2023-12-02T00:31:20.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.509+0000 7f34dfbf5640 1 -- 172.21.15.46:0/481997772 >> 172.21.15.46:0/481997772 conn(0x7f34d806a6b0 msgr2=0x7f34d806aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:20.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.510+0000 7f34dfbf5640 1 -- 172.21.15.46:0/481997772 shutdown_connections 2023-12-02T00:31:20.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.510+0000 7f34dfbf5640 1 -- 172.21.15.46:0/481997772 wait complete. 2023-12-02T00:31:20.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.511+0000 7f34dfbf5640 1 Processor -- start 2023-12-02T00:31:20.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.511+0000 7f34dfbf5640 1 -- start start 2023-12-02T00:31:20.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.511+0000 7f34dfbf5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34c40039a0 0x7f34d81535c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:20.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.511+0000 7f34dfbf5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34d8152a30 0x7f34d81f4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:20.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.511+0000 7f34dfbf5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34d81f4bb0 0x7f34d81f5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:20.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.511+0000 7f34dfbf5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d813ba10 con 0x7f34c40039a0 2023-12-02T00:31:20.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.511+0000 7f34dfbf5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d813bea0 con 0x7f34d81f4bb0 2023-12-02T00:31:20.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.511+0000 7f34dfbf5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34d813ae00 con 0x7f34d8152a30 2023-12-02T00:31:20.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.512+0000 7f34dd991640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34c40039a0 0x7f34d81535c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:20.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.512+0000 7f34dd991640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34c40039a0 0x7f34d81535c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39872/0 (socket says 172.21.15.46:39872) 2023-12-02T00:31:20.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.512+0000 7f34dd991640 1 -- 172.21.15.46:0/328420630 learned_addr learned my addr 172.21.15.46:0/328420630 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:20.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.512+0000 7f34dd190640 1 --2- 172.21.15.46:0/328420630 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34d8152a30 0x7f34d81f4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:20.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.512+0000 7f34de192640 1 --2- 172.21.15.46:0/328420630 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34d81f4bb0 0x7f34d81f5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:20.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.512+0000 7f34dd991640 1 -- 172.21.15.46:0/328420630 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34d8152a30 msgr2=0x7f34d81f4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:20.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.512+0000 7f34dd991640 1 --2- 172.21.15.46:0/328420630 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34d8152a30 0x7f34d81f4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:20.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.512+0000 7f34dd991640 1 -- 172.21.15.46:0/328420630 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34d81f4bb0 msgr2=0x7f34d81f5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:20.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.512+0000 7f34dd991640 1 --2- 172.21.15.46:0/328420630 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34d81f4bb0 0x7f34d81f5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:20.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.512+0000 7f34dd991640 1 -- 172.21.15.46:0/328420630 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d806f4a0 con 0x7f34c40039a0 2023-12-02T00:31:20.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.512+0000 7f34dd190640 1 --2- 172.21.15.46:0/328420630 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34d8152a30 0x7f34d81f4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:31:20.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.512+0000 7f34de192640 1 --2- 172.21.15.46:0/328420630 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34d81f4bb0 0x7f34d81f5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:31:20.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.512+0000 7f34dd991640 1 --2- 172.21.15.46:0/328420630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34c40039a0 0x7f34d81535c0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f34d402c250 tx=0x7f34d4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:20.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.512+0000 7f34ce7fc640 1 -- 172.21.15.46:0/328420630 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d4003810 con 0x7f34c40039a0 2023-12-02T00:31:20.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.512+0000 7f34ce7fc640 1 -- 172.21.15.46:0/328420630 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34d4003970 con 0x7f34c40039a0 2023-12-02T00:31:20.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.512+0000 7f34dfbf5640 1 -- 172.21.15.46:0/328420630 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34d813b0e0 con 0x7f34c40039a0 2023-12-02T00:31:20.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.512+0000 7f34ce7fc640 1 -- 172.21.15.46:0/328420630 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d4003ce0 con 0x7f34c40039a0 2023-12-02T00:31:20.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.512+0000 7f34dfbf5640 1 -- 172.21.15.46:0/328420630 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34d813b6a0 con 0x7f34c40039a0 2023-12-02T00:31:20.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.513+0000 7f34dfbf5640 1 -- 172.21.15.46:0/328420630 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34d8069400 con 0x7f34c40039a0 2023-12-02T00:31:20.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.514+0000 7f34ce7fc640 1 -- 172.21.15.46:0/328420630 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f34d4047450 con 0x7f34c40039a0 2023-12-02T00:31:20.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.514+0000 7f34ce7fc640 1 --2- 172.21.15.46:0/328420630 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f34bc06ae70 0x7f34bc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:20.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.514+0000 7f34ce7fc640 1 -- 172.21.15.46:0/328420630 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f34d40ab890 con 0x7f34c40039a0 2023-12-02T00:31:20.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.514+0000 7f34dd190640 1 --2- 172.21.15.46:0/328420630 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f34bc06ae70 0x7f34bc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:20.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.515+0000 7f34dd190640 1 --2- 172.21.15.46:0/328420630 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f34bc06ae70 0x7f34bc06d330 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f34d8155b60 tx=0x7f34c8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:20.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.518+0000 7f34ce7fc640 1 -- 172.21.15.46:0/328420630 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f34d40769f0 con 0x7f34c40039a0 2023-12-02T00:31:20.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.697+0000 7f34dfbf5640 1 -- 172.21.15.46:0/328420630 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f34d8070d30 con 0x7f34c40039a0 2023-12-02T00:31:20.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.698+0000 7f34ce7fc640 1 -- 172.21.15.46:0/328420630 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v51) v1 ==== 100+0+11 (secure 0 0 0) 0x7f34d40044e0 con 0x7f34c40039a0 2023-12-02T00:31:20.699 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-12-02T00:31:20.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.700+0000 7f34dfbf5640 1 -- 172.21.15.46:0/328420630 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f34bc06ae70 msgr2=0x7f34bc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:20.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.700+0000 7f34dfbf5640 1 --2- 172.21.15.46:0/328420630 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f34bc06ae70 0x7f34bc06d330 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f34d8155b60 tx=0x7f34c8008040 comp rx=0 tx=0).stop 2023-12-02T00:31:20.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.700+0000 7f34dfbf5640 1 -- 172.21.15.46:0/328420630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34c40039a0 msgr2=0x7f34d81535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:20.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.700+0000 7f34dfbf5640 1 --2- 172.21.15.46:0/328420630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34c40039a0 0x7f34d81535c0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f34d402c250 tx=0x7f34d4004730 comp rx=0 tx=0).stop 2023-12-02T00:31:20.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.700+0000 7f34dfbf5640 1 -- 172.21.15.46:0/328420630 shutdown_connections 2023-12-02T00:31:20.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.700+0000 7f34dfbf5640 1 --2- 172.21.15.46:0/328420630 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f34bc06ae70 0x7f34bc06d330 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:20.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.700+0000 7f34dfbf5640 1 --2- 172.21.15.46:0/328420630 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34d81f4bb0 0x7f34d81f5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:20.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.700+0000 7f34dfbf5640 1 --2- 172.21.15.46:0/328420630 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34d8152a30 0x7f34d81f4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:20.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.700+0000 7f34dfbf5640 1 --2- 172.21.15.46:0/328420630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34c40039a0 0x7f34d81535c0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:20.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.700+0000 7f34dfbf5640 1 -- 172.21.15.46:0/328420630 >> 172.21.15.46:0/328420630 conn(0x7f34d806a6b0 msgr2=0x7f34d813fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:20.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.700+0000 7f34dfbf5640 1 -- 172.21.15.46:0/328420630 shutdown_connections 2023-12-02T00:31:20.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.700+0000 7f34dfbf5640 1 -- 172.21.15.46:0/328420630 wait complete. 2023-12-02T00:31:20.712 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-02T00:31:20.756 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-02T00:31:20.759 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-02T00:31:20.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.820+0000 7f28bcbc9640 1 Processor -- start 2023-12-02T00:31:20.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.820+0000 7f28bcbc9640 1 -- start start 2023-12-02T00:31:20.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.821+0000 7f28bcbc9640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f28b815d330 0x7f28b80b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:20.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.821+0000 7f28bcbc9640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f28b80d2910 0x7f28b80d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:20.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.821+0000 7f28bcbc9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f28b80d3730 0x7f28b80b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:20.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.821+0000 7f28bcbc9640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28b80b2b10 con 0x7f28b80d2910 2023-12-02T00:31:20.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.821+0000 7f28bcbc9640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28b80d3b90 con 0x7f28b815d330 2023-12-02T00:31:20.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.821+0000 7f28bcbc9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28b80d1590 con 0x7f28b80d3730 2023-12-02T00:31:20.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.821+0000 7f28b5d9b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f28b80d2910 0x7f28b80d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:20.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.821+0000 7f28b5d9b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f28b80d2910 0x7f28b80d31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39880/0 (socket says 172.21.15.46:39880) 2023-12-02T00:31:20.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.821+0000 7f28b5d9b640 1 -- 172.21.15.46:0/817049652 learned_addr learned my addr 172.21.15.46:0/817049652 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:20.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.821+0000 7f28b659c640 1 --2- 172.21.15.46:0/817049652 >> v2:172.21.15.162:3300/0 conn(0x7f28b815d330 0x7f28b80b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:20.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.821+0000 7f28b6d9d640 1 --2- 172.21.15.46:0/817049652 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f28b80d3730 0x7f28b80b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:20.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.821+0000 7f28b5d9b640 1 -- 172.21.15.46:0/817049652 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f28b80d3730 msgr2=0x7f28b80b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:20.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.821+0000 7f28b5d9b640 1 --2- 172.21.15.46:0/817049652 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f28b80d3730 0x7f28b80b6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:20.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.821+0000 7f28b5d9b640 1 -- 172.21.15.46:0/817049652 >> v2:172.21.15.162:3300/0 conn(0x7f28b815d330 msgr2=0x7f28b80b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:20.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.821+0000 7f28b5d9b640 1 --2- 172.21.15.46:0/817049652 >> v2:172.21.15.162:3300/0 conn(0x7f28b815d330 0x7f28b80b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:20.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.821+0000 7f28b5d9b640 1 -- 172.21.15.46:0/817049652 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28b80d1870 con 0x7f28b80d2910 2023-12-02T00:31:20.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.822+0000 7f28b5d9b640 1 --2- 172.21.15.46:0/817049652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28b80d2910 0x7f28b80d31f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f28b80d0d10 tx=0x7f28ac02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d318ceecb7128088 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:20.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.822+0000 7f28b559a640 1 -- 172.21.15.46:0/817049652 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ac0317e0 con 0x7f28b80d2910 2023-12-02T00:31:20.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.822+0000 7f28b559a640 1 -- 172.21.15.46:0/817049652 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28ac031940 con 0x7f28b80d2910 2023-12-02T00:31:20.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.822+0000 7f28b559a640 1 -- 172.21.15.46:0/817049652 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ac031cd0 con 0x7f28b80d2910 2023-12-02T00:31:20.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.823+0000 7f28bcbc9640 1 -- 172.21.15.46:0/817049652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28b80d2910 msgr2=0x7f28b80d31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:20.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.823+0000 7f28bcbc9640 1 --2- 172.21.15.46:0/817049652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28b80d2910 0x7f28b80d31f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f28b80d0d10 tx=0x7f28ac02b880 comp rx=0 tx=0).stop 2023-12-02T00:31:20.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.823+0000 7f28bcbc9640 1 -- 172.21.15.46:0/817049652 shutdown_connections 2023-12-02T00:31:20.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.823+0000 7f28bcbc9640 1 --2- 172.21.15.46:0/817049652 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f28b80d3730 0x7f28b80b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:20.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.823+0000 7f28bcbc9640 1 --2- 172.21.15.46:0/817049652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28b80d2910 0x7f28b80d31f0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:20.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.823+0000 7f28bcbc9640 1 --2- 172.21.15.46:0/817049652 >> v2:172.21.15.162:3300/0 conn(0x7f28b815d330 0x7f28b80b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:20.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.823+0000 7f28bcbc9640 1 -- 172.21.15.46:0/817049652 >> 172.21.15.46:0/817049652 conn(0x7f28b8072020 msgr2=0x7f28b8072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:20.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.823+0000 7f28bcbc9640 1 -- 172.21.15.46:0/817049652 shutdown_connections 2023-12-02T00:31:20.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.823+0000 7f28bcbc9640 1 -- 172.21.15.46:0/817049652 wait complete. 2023-12-02T00:31:20.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.824+0000 7f28bcbc9640 1 Processor -- start 2023-12-02T00:31:20.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.824+0000 7f28bcbc9640 1 -- start start 2023-12-02T00:31:20.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.825+0000 7f28bcbc9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28b80d2910 0x7f28b81f6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:20.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.825+0000 7f28bcbc9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f28b80d3730 0x7f28b80c94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:20.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.825+0000 7f28bcbc9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28b815d330 0x7f28b815d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:20.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.825+0000 7f28bcbc9640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b81f7120 con 0x7f28b80d2910 2023-12-02T00:31:20.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.825+0000 7f28bcbc9640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b815fac0 con 0x7f28b815d330 2023-12-02T00:31:20.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.825+0000 7f28bcbc9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28b815fda0 con 0x7f28b80d3730 2023-12-02T00:31:20.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.825+0000 7f28b659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28b80d2910 0x7f28b81f6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:20.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.825+0000 7f28b659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28b80d2910 0x7f28b81f6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39896/0 (socket says 172.21.15.46:39896) 2023-12-02T00:31:20.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.825+0000 7f28b659c640 1 -- 172.21.15.46:0/3429406549 learned_addr learned my addr 172.21.15.46:0/3429406549 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:20.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.825+0000 7f28b6d9d640 1 --2- 172.21.15.46:0/3429406549 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28b815d330 0x7f28b815d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:20.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.825+0000 7f28b5d9b640 1 --2- 172.21.15.46:0/3429406549 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f28b80d3730 0x7f28b80c94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:20.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.826+0000 7f28b659c640 1 -- 172.21.15.46:0/3429406549 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f28b80d3730 msgr2=0x7f28b80c94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:20.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.826+0000 7f28b659c640 1 --2- 172.21.15.46:0/3429406549 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f28b80d3730 0x7f28b80c94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:20.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.826+0000 7f28b659c640 1 -- 172.21.15.46:0/3429406549 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28b815d330 msgr2=0x7f28b815d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:20.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.826+0000 7f28b659c640 1 --2- 172.21.15.46:0/3429406549 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28b815d330 0x7f28b815d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:20.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.826+0000 7f28b659c640 1 -- 172.21.15.46:0/3429406549 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28ac0094d0 con 0x7f28b80d2910 2023-12-02T00:31:20.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.826+0000 7f28b659c640 1 --2- 172.21.15.46:0/3429406549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28b80d2910 0x7f28b81f6be0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f28a800d5f0 tx=0x7f28a800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:20.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.826+0000 7f28a6ffd640 1 -- 172.21.15.46:0/3429406549 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a80125f0 con 0x7f28b80d2910 2023-12-02T00:31:20.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.826+0000 7f28a6ffd640 1 -- 172.21.15.46:0/3429406549 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28a8012750 con 0x7f28b80d2910 2023-12-02T00:31:20.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.826+0000 7f28bcbc9640 1 -- 172.21.15.46:0/3429406549 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28b8160080 con 0x7f28b80d2910 2023-12-02T00:31:20.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.826+0000 7f28a6ffd640 1 -- 172.21.15.46:0/3429406549 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a8012b00 con 0x7f28b80d2910 2023-12-02T00:31:20.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.826+0000 7f28bcbc9640 1 -- 172.21.15.46:0/3429406549 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28b8160640 con 0x7f28b80d2910 2023-12-02T00:31:20.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.827+0000 7f28bcbc9640 1 -- 172.21.15.46:0/3429406549 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28b8069400 con 0x7f28b80d2910 2023-12-02T00:31:20.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.827+0000 7f28a6ffd640 1 -- 172.21.15.46:0/3429406549 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f28a8012c60 con 0x7f28b80d2910 2023-12-02T00:31:20.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.828+0000 7f28a6ffd640 1 --2- 172.21.15.46:0/3429406549 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f289806ae40 0x7f289806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:20.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.828+0000 7f28a6ffd640 1 -- 172.21.15.46:0/3429406549 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f28a8089a00 con 0x7f28b80d2910 2023-12-02T00:31:20.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.828+0000 7f28b5d9b640 1 --2- 172.21.15.46:0/3429406549 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f289806ae40 0x7f289806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:20.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.829+0000 7f28b5d9b640 1 --2- 172.21.15.46:0/3429406549 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f289806ae40 0x7f289806d300 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f28ac036000 tx=0x7f28ac03d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:20.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:20.832+0000 7f28a6ffd640 1 -- 172.21.15.46:0/3429406549 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f28a8054ae0 con 0x7f28b80d2910 2023-12-02T00:31:21.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.022+0000 7f28bcbc9640 1 -- 172.21.15.46:0/3429406549 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f28b80b8b30 con 0x7f28b80d2910 2023-12-02T00:31:21.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.023+0000 7f28a6ffd640 1 -- 172.21.15.46:0/3429406549 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v51) v1 ==== 80+0+0 (secure 0 0 0) 0x7f28a8058100 con 0x7f28b80d2910 2023-12-02T00:31:21.024 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2023-12-02T00:31:21.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.025+0000 7f28bcbc9640 1 -- 172.21.15.46:0/3429406549 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f289806ae40 msgr2=0x7f289806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:21.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.025+0000 7f28bcbc9640 1 --2- 172.21.15.46:0/3429406549 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f289806ae40 0x7f289806d300 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f28ac036000 tx=0x7f28ac03d040 comp rx=0 tx=0).stop 2023-12-02T00:31:21.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.025+0000 7f28bcbc9640 1 -- 172.21.15.46:0/3429406549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28b80d2910 msgr2=0x7f28b81f6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:21.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.025+0000 7f28bcbc9640 1 --2- 172.21.15.46:0/3429406549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28b80d2910 0x7f28b81f6be0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f28a800d5f0 tx=0x7f28a800dab0 comp rx=0 tx=0).stop 2023-12-02T00:31:21.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.025+0000 7f28bcbc9640 1 -- 172.21.15.46:0/3429406549 shutdown_connections 2023-12-02T00:31:21.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.025+0000 7f28bcbc9640 1 --2- 172.21.15.46:0/3429406549 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f289806ae40 0x7f289806d300 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:21.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.025+0000 7f28bcbc9640 1 --2- 172.21.15.46:0/3429406549 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28b815d330 0x7f28b815d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:21.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.025+0000 7f28bcbc9640 1 --2- 172.21.15.46:0/3429406549 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f28b80d3730 0x7f28b80c94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:21.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.025+0000 7f28bcbc9640 1 --2- 172.21.15.46:0/3429406549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28b80d2910 0x7f28b81f6be0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:21.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.025+0000 7f28bcbc9640 1 -- 172.21.15.46:0/3429406549 >> 172.21.15.46:0/3429406549 conn(0x7f28b8072020 msgr2=0x7f28b80d0050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:21.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.025+0000 7f28bcbc9640 1 -- 172.21.15.46:0/3429406549 shutdown_connections 2023-12-02T00:31:21.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.025+0000 7f28bcbc9640 1 -- 172.21.15.46:0/3429406549 wait complete. 2023-12-02T00:31:21.037 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-02T00:31:21.037 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-02T00:31:21.093 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:31:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.238+0000 7ff839e64640 1 Processor -- start 2023-12-02T00:31:21.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.238+0000 7ff839e64640 1 -- start start 2023-12-02T00:31:21.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.239+0000 7ff839e64640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff83415d330 0x7ff8340b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:21.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.239+0000 7ff839e64640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff8340d2910 0x7ff8340d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:21.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.239+0000 7ff839e64640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff8340d2d90 0x7ff8340d3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:21.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.239+0000 7ff839e64640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8340b2b10 con 0x7ff8340d2910 2023-12-02T00:31:21.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.239+0000 7ff839e64640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8340d1590 con 0x7ff8340d2d90 2023-12-02T00:31:21.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.239+0000 7ff839e64640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8340d1870 con 0x7ff83415d330 2023-12-02T00:31:21.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.239+0000 7ff832ffd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff8340d2910 0x7ff8340d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:21.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.239+0000 7ff832ffd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff8340d2910 0x7ff8340d31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39900/0 (socket says 172.21.15.46:39900) 2023-12-02T00:31:21.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.239+0000 7ff832ffd640 1 -- 172.21.15.46:0/1816326745 learned_addr learned my addr 172.21.15.46:0/1816326745 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:21.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.240+0000 7ff8337fe640 1 --2- 172.21.15.46:0/1816326745 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff83415d330 0x7ff8340b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:21.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.240+0000 7ff832ffd640 1 -- 172.21.15.46:0/1816326745 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff83415d330 msgr2=0x7ff8340b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:21.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.240+0000 7ff833fff640 1 --2- 172.21.15.46:0/1816326745 >> v2:172.21.15.162:3300/0 conn(0x7ff8340d2d90 0x7ff8340d3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:21.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.240+0000 7ff832ffd640 1 --2- 172.21.15.46:0/1816326745 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff83415d330 0x7ff8340b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:21.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.240+0000 7ff832ffd640 1 -- 172.21.15.46:0/1816326745 >> v2:172.21.15.162:3300/0 conn(0x7ff8340d2d90 msgr2=0x7ff8340d3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:21.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.240+0000 7ff832ffd640 1 --2- 172.21.15.46:0/1816326745 >> v2:172.21.15.162:3300/0 conn(0x7ff8340d2d90 0x7ff8340d3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:21.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.240+0000 7ff832ffd640 1 -- 172.21.15.46:0/1816326745 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8340d1b50 con 0x7ff8340d2910 2023-12-02T00:31:21.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.240+0000 7ff833fff640 1 --2- 172.21.15.46:0/1816326745 >> v2:172.21.15.162:3300/0 conn(0x7ff8340d2d90 0x7ff8340d3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:31:21.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.240+0000 7ff8337fe640 1 --2- 172.21.15.46:0/1816326745 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff83415d330 0x7ff8340b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:31:21.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.240+0000 7ff832ffd640 1 --2- 172.21.15.46:0/1816326745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8340d2910 0x7ff8340d31f0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7ff8340d1540 tx=0x7ff82802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6e5ebdfe62d8dbb server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:21.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.241+0000 7ff8327fc640 1 -- 172.21.15.46:0/1816326745 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8280317e0 con 0x7ff8340d2910 2023-12-02T00:31:21.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.241+0000 7ff8327fc640 1 -- 172.21.15.46:0/1816326745 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff828031940 con 0x7ff8340d2910 2023-12-02T00:31:21.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.241+0000 7ff8327fc640 1 -- 172.21.15.46:0/1816326745 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff828031cd0 con 0x7ff8340d2910 2023-12-02T00:31:21.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.241+0000 7ff839e64640 1 -- 172.21.15.46:0/1816326745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8340d2910 msgr2=0x7ff8340d31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:21.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.241+0000 7ff839e64640 1 --2- 172.21.15.46:0/1816326745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8340d2910 0x7ff8340d31f0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7ff8340d1540 tx=0x7ff82802b880 comp rx=0 tx=0).stop 2023-12-02T00:31:21.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.241+0000 7ff839e64640 1 -- 172.21.15.46:0/1816326745 shutdown_connections 2023-12-02T00:31:21.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.241+0000 7ff839e64640 1 --2- 172.21.15.46:0/1816326745 >> v2:172.21.15.162:3300/0 conn(0x7ff8340d2d90 0x7ff8340d3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:21.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.241+0000 7ff839e64640 1 --2- 172.21.15.46:0/1816326745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8340d2910 0x7ff8340d31f0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:21.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.241+0000 7ff839e64640 1 --2- 172.21.15.46:0/1816326745 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff83415d330 0x7ff8340b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:21.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.241+0000 7ff839e64640 1 -- 172.21.15.46:0/1816326745 >> 172.21.15.46:0/1816326745 conn(0x7ff834072020 msgr2=0x7ff834072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:21.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.241+0000 7ff839e64640 1 -- 172.21.15.46:0/1816326745 shutdown_connections 2023-12-02T00:31:21.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.241+0000 7ff839e64640 1 -- 172.21.15.46:0/1816326745 wait complete. 2023-12-02T00:31:21.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.242+0000 7ff839e64640 1 Processor -- start 2023-12-02T00:31:21.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.243+0000 7ff839e64640 1 -- start start 2023-12-02T00:31:21.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.243+0000 7ff839e64640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8340d2910 0x7ff8341f2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:21.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.243+0000 7ff839e64640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8340d2d90 0x7ff8340c75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:21.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.243+0000 7ff839e64640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff83415d330 0x7ff83415f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:21.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.243+0000 7ff839e64640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8340cad10 con 0x7ff8340d2d90 2023-12-02T00:31:21.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.243+0000 7ff839e64640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8340c87c0 con 0x7ff8340d2910 2023-12-02T00:31:21.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.243+0000 7ff839e64640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8340c7f20 con 0x7ff83415d330 2023-12-02T00:31:21.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.243+0000 7ff832ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8340d2d90 0x7ff8340c75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:21.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.243+0000 7ff832ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8340d2d90 0x7ff8340c75b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39902/0 (socket says 172.21.15.46:39902) 2023-12-02T00:31:21.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.243+0000 7ff832ffd640 1 -- 172.21.15.46:0/71084127 learned_addr learned my addr 172.21.15.46:0/71084127 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:21.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.244+0000 7ff833fff640 1 --2- 172.21.15.46:0/71084127 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff83415d330 0x7ff83415f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:21.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.244+0000 7ff8337fe640 1 --2- 172.21.15.46:0/71084127 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8340d2910 0x7ff8341f2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:21.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.244+0000 7ff832ffd640 1 -- 172.21.15.46:0/71084127 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff83415d330 msgr2=0x7ff83415f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:21.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.244+0000 7ff832ffd640 1 --2- 172.21.15.46:0/71084127 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff83415d330 0x7ff83415f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:21.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.244+0000 7ff832ffd640 1 -- 172.21.15.46:0/71084127 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8340d2910 msgr2=0x7ff8341f2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:21.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.244+0000 7ff832ffd640 1 --2- 172.21.15.46:0/71084127 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8340d2910 0x7ff8341f2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:21.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.244+0000 7ff832ffd640 1 -- 172.21.15.46:0/71084127 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8280094d0 con 0x7ff8340d2d90 2023-12-02T00:31:21.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.244+0000 7ff8337fe640 1 --2- 172.21.15.46:0/71084127 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8340d2910 0x7ff8341f2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:31:21.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.244+0000 7ff833fff640 1 --2- 172.21.15.46:0/71084127 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff83415d330 0x7ff83415f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:31:21.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.244+0000 7ff832ffd640 1 --2- 172.21.15.46:0/71084127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8340d2d90 0x7ff8340c75b0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7ff828000c00 tx=0x7ff8280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:21.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.244+0000 7ff81bfff640 1 -- 172.21.15.46:0/71084127 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff828032900 con 0x7ff8340d2d90 2023-12-02T00:31:21.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.244+0000 7ff81bfff640 1 -- 172.21.15.46:0/71084127 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff828032a60 con 0x7ff8340d2d90 2023-12-02T00:31:21.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.244+0000 7ff839e64640 1 -- 172.21.15.46:0/71084127 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8340c8200 con 0x7ff8340d2d90 2023-12-02T00:31:21.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.244+0000 7ff81bfff640 1 -- 172.21.15.46:0/71084127 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff828032e10 con 0x7ff8340d2d90 2023-12-02T00:31:21.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.244+0000 7ff839e64640 1 -- 172.21.15.46:0/71084127 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8340caf40 con 0x7ff8340d2d90 2023-12-02T00:31:21.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.245+0000 7ff839e64640 1 -- 172.21.15.46:0/71084127 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff834069400 con 0x7ff8340d2d90 2023-12-02T00:31:21.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.245+0000 7ff81bfff640 1 -- 172.21.15.46:0/71084127 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff82804e450 con 0x7ff8340d2d90 2023-12-02T00:31:21.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.246+0000 7ff81bfff640 1 --2- 172.21.15.46:0/71084127 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff82006ae40 0x7ff82006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:21.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.246+0000 7ff81bfff640 1 -- 172.21.15.46:0/71084127 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff8280ab910 con 0x7ff8340d2d90 2023-12-02T00:31:21.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.246+0000 7ff8337fe640 1 --2- 172.21.15.46:0/71084127 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff82006ae40 0x7ff82006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:21.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.247+0000 7ff8337fe640 1 --2- 172.21.15.46:0/71084127 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff82006ae40 0x7ff82006d300 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff82400aa20 tx=0x7ff824008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:21.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.250+0000 7ff81bfff640 1 -- 172.21.15.46:0/71084127 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff828076a70 con 0x7ff8340d2d90 2023-12-02T00:31:21.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.427+0000 7ff839e64640 1 -- 172.21.15.46:0/71084127 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff83415f2d0 con 0x7ff8340d2d90 2023-12-02T00:31:21.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.428+0000 7ff81bfff640 1 -- 172.21.15.46:0/71084127 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+16545 (secure 0 0 0) 0x7ff82807a090 con 0x7ff8340d2d90 2023-12-02T00:31:21.430 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:31:21.431 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":51,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:31:02.837102+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-12-02T00:30:59.469070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-12-02T00:31:00.174610+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:31:21.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.430+0000 7ff839e64640 1 -- 172.21.15.46:0/71084127 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff82006ae40 msgr2=0x7ff82006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:21.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.430+0000 7ff839e64640 1 --2- 172.21.15.46:0/71084127 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff82006ae40 0x7ff82006d300 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff82400aa20 tx=0x7ff824008040 comp rx=0 tx=0).stop 2023-12-02T00:31:21.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.430+0000 7ff839e64640 1 -- 172.21.15.46:0/71084127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8340d2d90 msgr2=0x7ff8340c75b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:21.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.430+0000 7ff839e64640 1 --2- 172.21.15.46:0/71084127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8340d2d90 0x7ff8340c75b0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7ff828000c00 tx=0x7ff8280046c0 comp rx=0 tx=0).stop 2023-12-02T00:31:21.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.430+0000 7ff839e64640 1 -- 172.21.15.46:0/71084127 shutdown_connections 2023-12-02T00:31:21.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.430+0000 7ff839e64640 1 --2- 172.21.15.46:0/71084127 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff82006ae40 0x7ff82006d300 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:21.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.430+0000 7ff839e64640 1 --2- 172.21.15.46:0/71084127 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff83415d330 0x7ff83415f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:21.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.430+0000 7ff839e64640 1 --2- 172.21.15.46:0/71084127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8340d2d90 0x7ff8340c75b0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:21.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.430+0000 7ff839e64640 1 --2- 172.21.15.46:0/71084127 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8340d2910 0x7ff8341f2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:21.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.430+0000 7ff839e64640 1 -- 172.21.15.46:0/71084127 >> 172.21.15.46:0/71084127 conn(0x7ff834072020 msgr2=0x7ff8340b4410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:21.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.430+0000 7ff839e64640 1 -- 172.21.15.46:0/71084127 shutdown_connections 2023-12-02T00:31:21.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.430+0000 7ff839e64640 1 -- 172.21.15.46:0/71084127 wait complete. 2023-12-02T00:31:21.443 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-02T00:28:37.213645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-02T00:30:59.469070+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-12-02T00:31:00.174610+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-02T00:31:21.443 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-02T00:31:21.552 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.551+0000 7f81b346e640 1 Processor -- start 2023-12-02T00:31:21.552 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.551+0000 7f81b346e640 1 -- start start 2023-12-02T00:31:21.556 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.552+0000 7f81b346e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f81ac06abe0 0x7f81ac1618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:21.556 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.552+0000 7f81b346e640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81ac1406f0 con 0x7f81ac140310 2023-12-02T00:31:21.556 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.552+0000 7f81b346e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81ac143960 con 0x7f81ac1409d0 2023-12-02T00:31:21.556 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.552+0000 7f81b346e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81ac161e20 con 0x7f81ac06abe0 2023-12-02T00:31:21.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.552+0000 7f81b120a640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f81ac140310 0x7f81ac06a4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57226/0 (socket says 172.21.15.46:57226) 2023-12-02T00:31:21.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.552+0000 7f81b120a640 1 -- 172.21.15.46:0/1432281144 learned_addr learned my addr 172.21.15.46:0/1432281144 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:31:21.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.552+0000 7f81b0a09640 1 --2- 172.21.15.46:0/1432281144 >> v2:172.21.15.162:3300/0 conn(0x7f81ac06abe0 0x7f81ac1618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:21.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.552+0000 7f81abfff640 1 -- 172.21.15.46:0/1432281144 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 800619140 0 0) 0x7f81ac1406f0 con 0x7f81ac140310 2023-12-02T00:31:21.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.552+0000 7f81abfff640 1 -- 172.21.15.46:0/1432281144 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8194003520 con 0x7f81ac140310 2023-12-02T00:31:21.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.553+0000 7f81abfff640 1 -- 172.21.15.46:0/1432281144 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3987225521 0 0) 0x7f8194003520 con 0x7f81ac140310 2023-12-02T00:31:21.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.553+0000 7f81abfff640 1 -- 172.21.15.46:0/1432281144 >> v2:172.21.15.162:3300/0 conn(0x7f81ac06abe0 msgr2=0x7f81ac1618e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:21.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.553+0000 7f81abfff640 1 --2- 172.21.15.46:0/1432281144 >> v2:172.21.15.162:3300/0 conn(0x7f81ac06abe0 0x7f81ac1618e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:21.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.553+0000 7f81abfff640 1 -- 172.21.15.46:0/1432281144 >> v1:172.21.15.162:6789/0 conn(0x7f81ac1409d0 legacy=0x7f81ac06efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:21.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.553+0000 7f81abfff640 1 -- 172.21.15.46:0/1432281144 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81ac142de0 con 0x7f81ac140310 2023-12-02T00:31:21.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.553+0000 7f81abfff640 1 -- 172.21.15.46:0/1432281144 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7f81a4001670 con 0x7f81ac140310 2023-12-02T00:31:21.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.553+0000 7f81abfff640 1 -- 172.21.15.46:0/1432281144 >> v1:172.21.15.46:6789/0 conn(0x7f81ac140310 legacy=0x7f81ac06a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:21.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.553+0000 7f81abfff640 1 --2- 172.21.15.46:0/1432281144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8194003970 0x7f8194003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:21.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.553+0000 7f81abfff640 1 --2- 172.21.15.46:0/1432281144 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81940047c0 0x7f8194004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:21.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.553+0000 7f81abfff640 1 --2- 172.21.15.46:0/1432281144 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8194005180 0x7f819400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:21.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.553+0000 7f81abfff640 1 -- 172.21.15.46:0/1432281144 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac161e20 con 0x7f8194003970 2023-12-02T00:31:21.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.553+0000 7f81abfff640 1 -- 172.21.15.46:0/1432281144 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819400c120 con 0x7f8194005180 2023-12-02T00:31:21.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.553+0000 7f81abfff640 1 -- 172.21.15.46:0/1432281144 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f819400c2e0 con 0x7f81940047c0 2023-12-02T00:31:21.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.553+0000 7f81b120a640 1 --2- 172.21.15.46:0/1432281144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8194003970 0x7f8194003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:21.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.553+0000 7f81b0a09640 1 --2- 172.21.15.46:0/1432281144 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81940047c0 0x7f8194004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:21.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.553+0000 7f81b1a0b640 1 --2- 172.21.15.46:0/1432281144 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8194005180 0x7f819400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:21.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.553+0000 7f81b120a640 1 -- 172.21.15.46:0/1432281144 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81940047c0 msgr2=0x7f8194004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:21.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.553+0000 7f81b120a640 1 --2- 172.21.15.46:0/1432281144 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81940047c0 0x7f8194004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:21.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.553+0000 7f81b120a640 1 -- 172.21.15.46:0/1432281144 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8194005180 msgr2=0x7f819400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:21.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.553+0000 7f81b120a640 1 --2- 172.21.15.46:0/1432281144 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8194005180 0x7f819400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:21.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.553+0000 7f81b120a640 1 -- 172.21.15.46:0/1432281144 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8194003520 con 0x7f8194003970 2023-12-02T00:31:21.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.553+0000 7f81b1a0b640 1 --2- 172.21.15.46:0/1432281144 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8194005180 0x7f819400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:31:21.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.553+0000 7f81b0a09640 1 --2- 172.21.15.46:0/1432281144 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81940047c0 0x7f8194004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:31:21.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.553+0000 7f81b1a0b640 1 -- 172.21.15.46:0/1432281144 reap_dead start 2023-12-02T00:31:21.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.554+0000 7f81b120a640 1 --2- 172.21.15.46:0/1432281144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8194003970 0x7f8194003d90 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f81a400c240 tx=0x7f81940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf77a5fe328a5ab8 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:21.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.554+0000 7f81abfff640 1 -- 172.21.15.46:0/1432281144 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a4005ad0 con 0x7f8194003970 2023-12-02T00:31:21.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.554+0000 7f81abfff640 1 -- 172.21.15.46:0/1432281144 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a4005c30 con 0x7f8194003970 2023-12-02T00:31:21.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.554+0000 7f81abfff640 1 -- 172.21.15.46:0/1432281144 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a402c8b0 con 0x7f8194003970 2023-12-02T00:31:21.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.555+0000 7f81b346e640 1 -- 172.21.15.46:0/1432281144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8194003970 msgr2=0x7f8194003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:21.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.555+0000 7f81b346e640 1 --2- 172.21.15.46:0/1432281144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8194003970 0x7f8194003d90 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f81a400c240 tx=0x7f81940057f0 comp rx=0 tx=0).stop 2023-12-02T00:31:21.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.555+0000 7f81b346e640 1 -- 172.21.15.46:0/1432281144 shutdown_connections 2023-12-02T00:31:21.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.555+0000 7f81b346e640 1 --2- 172.21.15.46:0/1432281144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8194003970 0x7f8194003d90 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:21.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.555+0000 7f81b346e640 1 -- 172.21.15.46:0/1432281144 >> 172.21.15.46:0/1432281144 conn(0x7f81ac0b1e50 msgr2=0x7f81ac0b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:21.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.555+0000 7f81b346e640 1 -- 172.21.15.46:0/1432281144 shutdown_connections 2023-12-02T00:31:21.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.555+0000 7f81b346e640 1 -- 172.21.15.46:0/1432281144 wait complete. 2023-12-02T00:31:21.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.556+0000 7f81b346e640 1 Processor -- start 2023-12-02T00:31:21.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.556+0000 7f81b346e640 1 -- start start 2023-12-02T00:31:21.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.557+0000 7f81b346e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8194003970 0x7f81ac154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:21.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.557+0000 7f81b346e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81ac13d740 0x7f81ac1462f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:21.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.557+0000 7f81b346e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81ac0b4200 0x7f81ac1538b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:21.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.557+0000 7f81b346e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac146bd0 con 0x7f81ac0b4200 2023-12-02T00:31:21.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.557+0000 7f81b346e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac144f00 con 0x7f81ac13d740 2023-12-02T00:31:21.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.557+0000 7f81b346e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac15f410 con 0x7f8194003970 2023-12-02T00:31:21.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.557+0000 7f81b1a0b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81ac0b4200 0x7f81ac1538b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:21.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.557+0000 7f81b1a0b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81ac0b4200 0x7f81ac1538b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39920/0 (socket says 172.21.15.46:39920) 2023-12-02T00:31:21.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.557+0000 7f81b1a0b640 1 -- 172.21.15.46:0/1090844759 learned_addr learned my addr 172.21.15.46:0/1090844759 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:21.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.557+0000 7f81b0a09640 1 --2- 172.21.15.46:0/1090844759 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81ac13d740 0x7f81ac1462f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:21.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.557+0000 7f81b120a640 1 --2- 172.21.15.46:0/1090844759 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8194003970 0x7f81ac154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:21.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.557+0000 7f81b1a0b640 1 -- 172.21.15.46:0/1090844759 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8194003970 msgr2=0x7f81ac154960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:21.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.557+0000 7f81b1a0b640 1 --2- 172.21.15.46:0/1090844759 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8194003970 0x7f81ac154960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:21.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.557+0000 7f81b1a0b640 1 -- 172.21.15.46:0/1090844759 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81ac13d740 msgr2=0x7f81ac1462f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:21.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.557+0000 7f81b1a0b640 1 --2- 172.21.15.46:0/1090844759 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81ac13d740 0x7f81ac1462f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:21.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.557+0000 7f81b1a0b640 1 -- 172.21.15.46:0/1090844759 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81ac15eae0 con 0x7f81ac0b4200 2023-12-02T00:31:21.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.557+0000 7f81b1a0b640 1 --2- 172.21.15.46:0/1090844759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81ac0b4200 0x7f81ac1538b0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f81ac147040 tx=0x7f819c00eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:21.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.557+0000 7f81a9ffb640 1 -- 172.21.15.46:0/1090844759 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819c00eed0 con 0x7f81ac0b4200 2023-12-02T00:31:21.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.557+0000 7f81a9ffb640 1 -- 172.21.15.46:0/1090844759 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f819c003970 con 0x7f81ac0b4200 2023-12-02T00:31:21.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.557+0000 7f81b346e640 1 -- 172.21.15.46:0/1090844759 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81ac15edc0 con 0x7f81ac0b4200 2023-12-02T00:31:21.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.557+0000 7f81a9ffb640 1 -- 172.21.15.46:0/1090844759 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819c003d20 con 0x7f81ac0b4200 2023-12-02T00:31:21.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.558+0000 7f81b346e640 1 -- 172.21.15.46:0/1090844759 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81ac146830 con 0x7f81ac0b4200 2023-12-02T00:31:21.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.559+0000 7f81b346e640 1 -- 172.21.15.46:0/1090844759 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81ac161030 con 0x7f81ac0b4200 2023-12-02T00:31:21.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.559+0000 7f81a9ffb640 1 -- 172.21.15.46:0/1090844759 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f819c0050d0 con 0x7f81ac0b4200 2023-12-02T00:31:21.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.559+0000 7f81a9ffb640 1 --2- 172.21.15.46:0/1090844759 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f819806ae40 0x7f819806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:21.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.559+0000 7f81a9ffb640 1 -- 172.21.15.46:0/1090844759 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f819c0892b0 con 0x7f81ac0b4200 2023-12-02T00:31:21.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.559+0000 7f81b120a640 1 --2- 172.21.15.46:0/1090844759 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f819806ae40 0x7f819806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:21.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.560+0000 7f81b120a640 1 --2- 172.21.15.46:0/1090844759 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f819806ae40 0x7f819806d300 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f81a4034000 tx=0x7f81a4009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:21.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.564+0000 7f81a9ffb640 1 -- 172.21.15.46:0/1090844759 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f819c0544b0 con 0x7f81ac0b4200 2023-12-02T00:31:21.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.740+0000 7f81b346e640 1 -- 172.21.15.46:0/1090844759 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f81ac069400 con 0x7f81ac0b4200 2023-12-02T00:31:21.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.741+0000 7f81a9ffb640 1 -- 172.21.15.46:0/1090844759 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) v1 ==== 93+0+10 (secure 0 0 0) 0x7f819c057ad0 con 0x7f81ac0b4200 2023-12-02T00:31:21.742 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-12-02T00:31:21.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.743+0000 7f81b346e640 1 -- 172.21.15.46:0/1090844759 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f819806ae40 msgr2=0x7f819806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:21.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.743+0000 7f81b346e640 1 --2- 172.21.15.46:0/1090844759 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f819806ae40 0x7f819806d300 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f81a4034000 tx=0x7f81a4009030 comp rx=0 tx=0).stop 2023-12-02T00:31:21.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.743+0000 7f81b346e640 1 -- 172.21.15.46:0/1090844759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81ac0b4200 msgr2=0x7f81ac1538b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:21.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.743+0000 7f81b346e640 1 --2- 172.21.15.46:0/1090844759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81ac0b4200 0x7f81ac1538b0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f81ac147040 tx=0x7f819c00eb90 comp rx=0 tx=0).stop 2023-12-02T00:31:21.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.743+0000 7f81b346e640 1 -- 172.21.15.46:0/1090844759 shutdown_connections 2023-12-02T00:31:21.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.743+0000 7f81b346e640 1 --2- 172.21.15.46:0/1090844759 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f819806ae40 0x7f819806d300 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:21.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.743+0000 7f81b346e640 1 --2- 172.21.15.46:0/1090844759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81ac0b4200 0x7f81ac1538b0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:21.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.743+0000 7f81b346e640 1 --2- 172.21.15.46:0/1090844759 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81ac13d740 0x7f81ac1462f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:21.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.743+0000 7f81b346e640 1 --2- 172.21.15.46:0/1090844759 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8194003970 0x7f81ac154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:21.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.743+0000 7f81b346e640 1 -- 172.21.15.46:0/1090844759 >> 172.21.15.46:0/1090844759 conn(0x7f81ac0b1e50 msgr2=0x7f81ac0714a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:21.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.743+0000 7f81b346e640 1 -- 172.21.15.46:0/1090844759 shutdown_connections 2023-12-02T00:31:21.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.743+0000 7f81b346e640 1 -- 172.21.15.46:0/1090844759 wait complete. 2023-12-02T00:31:21.755 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-02T00:31:21.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.863+0000 7feeb1925640 1 Processor -- start 2023-12-02T00:31:21.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.863+0000 7feeb1925640 1 -- start start 2023-12-02T00:31:21.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.864+0000 7feeb1925640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7feeac1401f0 0x7feeac0c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:21.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.864+0000 7feeb1925640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7feeac06fe30 0x7feeac0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:21.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.864+0000 7feeb1925640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeac1438c0 con 0x7feeac160f20 2023-12-02T00:31:21.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.864+0000 7feeb1925640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeac070800 con 0x7feeac06fe30 2023-12-02T00:31:21.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.864+0000 7feeb1925640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeac1429e0 con 0x7feeac1401f0 2023-12-02T00:31:21.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.864+0000 7feeaa7fc640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7feeac06fe30 0x7feeac0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:21.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.864+0000 7feeaa7fc640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7feeac06fe30 0x7feeac0702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39930/0 (socket says 172.21.15.46:39930) 2023-12-02T00:31:21.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.864+0000 7feeaa7fc640 1 -- 172.21.15.46:0/118955174 learned_addr learned my addr 172.21.15.46:0/118955174 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:21.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.865+0000 7feeaaffd640 1 --2- 172.21.15.46:0/118955174 >> v2:172.21.15.162:3300/0 conn(0x7feeac1401f0 0x7feeac0c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:21.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.865+0000 7feeaa7fc640 1 -- 172.21.15.46:0/118955174 >> v2:172.21.15.162:3300/0 conn(0x7feeac1401f0 msgr2=0x7feeac0c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:21.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.865+0000 7feeaa7fc640 1 --2- 172.21.15.46:0/118955174 >> v2:172.21.15.162:3300/0 conn(0x7feeac1401f0 0x7feeac0c6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:21.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.865+0000 7feeaa7fc640 1 -- 172.21.15.46:0/118955174 >> v1:172.21.15.162:6789/0 conn(0x7feeac160f20 legacy=0x7feeac161390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:21.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.865+0000 7feeaa7fc640 1 -- 172.21.15.46:0/118955174 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeac142cb0 con 0x7feeac06fe30 2023-12-02T00:31:21.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.865+0000 7feeaa7fc640 1 --2- 172.21.15.46:0/118955174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feeac06fe30 0x7feeac0702c0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7feeac06f450 tx=0x7fee9402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec654b8a2af33882 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:21.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.866+0000 7feea9ffb640 1 -- 172.21.15.46:0/118955174 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee94031870 con 0x7feeac06fe30 2023-12-02T00:31:21.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.866+0000 7feea9ffb640 1 -- 172.21.15.46:0/118955174 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee940319d0 con 0x7feeac06fe30 2023-12-02T00:31:21.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.866+0000 7feea9ffb640 1 -- 172.21.15.46:0/118955174 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee94031d60 con 0x7feeac06fe30 2023-12-02T00:31:21.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.866+0000 7feeb1925640 1 -- 172.21.15.46:0/118955174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feeac06fe30 msgr2=0x7feeac0702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:21.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.866+0000 7feeb1925640 1 --2- 172.21.15.46:0/118955174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feeac06fe30 0x7feeac0702c0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7feeac06f450 tx=0x7fee9402b910 comp rx=0 tx=0).stop 2023-12-02T00:31:21.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.866+0000 7feeb1925640 1 -- 172.21.15.46:0/118955174 shutdown_connections 2023-12-02T00:31:21.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.866+0000 7feeb1925640 1 --2- 172.21.15.46:0/118955174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feeac06fe30 0x7feeac0702c0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:21.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.866+0000 7feeb1925640 1 --2- 172.21.15.46:0/118955174 >> v2:172.21.15.162:3300/0 conn(0x7feeac1401f0 0x7feeac0c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:21.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.866+0000 7feeb1925640 1 -- 172.21.15.46:0/118955174 >> 172.21.15.46:0/118955174 conn(0x7feeac06e7d0 msgr2=0x7feeac06ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:21.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.866+0000 7feeb1925640 1 -- 172.21.15.46:0/118955174 shutdown_connections 2023-12-02T00:31:21.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.866+0000 7feeb1925640 1 -- 172.21.15.46:0/118955174 wait complete. 2023-12-02T00:31:21.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.867+0000 7feeb1925640 1 Processor -- start 2023-12-02T00:31:21.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.868+0000 7feeb1925640 1 -- start start 2023-12-02T00:31:21.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.868+0000 7feeb1925640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feeac06fe30 0x7feeac13f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:21.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.868+0000 7feeb1925640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feeac1401f0 0x7feeac0b51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:21.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.868+0000 7feeb1925640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feeac160f20 0x7feeac15fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:21.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.868+0000 7feeb1925640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeac154f30 con 0x7feeac160f20 2023-12-02T00:31:21.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.868+0000 7feeb1925640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeac155210 con 0x7feeac1401f0 2023-12-02T00:31:21.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.868+0000 7feeb1925640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeac1554f0 con 0x7feeac06fe30 2023-12-02T00:31:21.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.868+0000 7feeab7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feeac160f20 0x7feeac15fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:21.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.868+0000 7feeab7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feeac160f20 0x7feeac15fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39932/0 (socket says 172.21.15.46:39932) 2023-12-02T00:31:21.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.868+0000 7feeab7fe640 1 -- 172.21.15.46:0/45060688 learned_addr learned my addr 172.21.15.46:0/45060688 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:21.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.869+0000 7feeaaffd640 1 --2- 172.21.15.46:0/45060688 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feeac06fe30 0x7feeac13f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:21.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.869+0000 7feeaa7fc640 1 --2- 172.21.15.46:0/45060688 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feeac1401f0 0x7feeac0b51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:21.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.869+0000 7feeab7fe640 1 -- 172.21.15.46:0/45060688 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feeac06fe30 msgr2=0x7feeac13f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:21.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.869+0000 7feeab7fe640 1 --2- 172.21.15.46:0/45060688 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feeac06fe30 0x7feeac13f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:21.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.869+0000 7feeab7fe640 1 -- 172.21.15.46:0/45060688 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feeac1401f0 msgr2=0x7feeac0b51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:21.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.869+0000 7feeab7fe640 1 --2- 172.21.15.46:0/45060688 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feeac1401f0 0x7feeac0b51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:21.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.869+0000 7feeab7fe640 1 -- 172.21.15.46:0/45060688 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee940094d0 con 0x7feeac160f20 2023-12-02T00:31:21.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.869+0000 7feeab7fe640 1 --2- 172.21.15.46:0/45060688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feeac160f20 0x7feeac15fab0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fee9c00b990 tx=0x7fee9c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:21.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.869+0000 7feea3fff640 1 -- 172.21.15.46:0/45060688 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9c00cba0 con 0x7feeac160f20 2023-12-02T00:31:21.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.869+0000 7feea3fff640 1 -- 172.21.15.46:0/45060688 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee9c00cd00 con 0x7feeac160f20 2023-12-02T00:31:21.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.869+0000 7feeb1925640 1 -- 172.21.15.46:0/45060688 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeac1557d0 con 0x7feeac160f20 2023-12-02T00:31:21.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.869+0000 7feea3fff640 1 -- 172.21.15.46:0/45060688 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9c014680 con 0x7feeac160f20 2023-12-02T00:31:21.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.869+0000 7feeb1925640 1 -- 172.21.15.46:0/45060688 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeac1f3520 con 0x7feeac160f20 2023-12-02T00:31:21.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.870+0000 7feeb1925640 1 -- 172.21.15.46:0/45060688 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feeac155cf0 con 0x7feeac160f20 2023-12-02T00:31:21.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.871+0000 7feea3fff640 1 -- 172.21.15.46:0/45060688 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fee9c0147e0 con 0x7feeac160f20 2023-12-02T00:31:21.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.871+0000 7feea3fff640 1 --2- 172.21.15.46:0/45060688 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fee8c06ae40 0x7fee8c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:21.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.871+0000 7feea3fff640 1 -- 172.21.15.46:0/45060688 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fee9c089410 con 0x7feeac160f20 2023-12-02T00:31:21.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.871+0000 7feeaaffd640 1 --2- 172.21.15.46:0/45060688 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fee8c06ae40 0x7fee8c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:21.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.872+0000 7feeaaffd640 1 --2- 172.21.15.46:0/45060688 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fee8c06ae40 0x7fee8c06d300 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7feeac06d290 tx=0x7feea4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:21.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:21.875+0000 7feea3fff640 1 -- 172.21.15.46:0/45060688 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7feeac155cf0 con 0x7feeac160f20 2023-12-02T00:31:22.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.050+0000 7feeb1925640 1 -- 172.21.15.46:0/45060688 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7feeac155cf0 con 0x7feeac160f20 2023-12-02T00:31:22.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.051+0000 7feea3fff640 1 -- 172.21.15.46:0/45060688 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v51) v1 ==== 104+0+11 (secure 0 0 0) 0x7feeac155cf0 con 0x7feeac160f20 2023-12-02T00:31:22.052 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-12-02T00:31:22.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.053+0000 7feeb1925640 1 -- 172.21.15.46:0/45060688 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fee8c06ae40 msgr2=0x7fee8c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:22.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.053+0000 7feeb1925640 1 --2- 172.21.15.46:0/45060688 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fee8c06ae40 0x7fee8c06d300 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7feeac06d290 tx=0x7feea4008040 comp rx=0 tx=0).stop 2023-12-02T00:31:22.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.053+0000 7feeb1925640 1 -- 172.21.15.46:0/45060688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feeac160f20 msgr2=0x7feeac15fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:22.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.053+0000 7feeb1925640 1 --2- 172.21.15.46:0/45060688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feeac160f20 0x7feeac15fab0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fee9c00b990 tx=0x7fee9c00be50 comp rx=0 tx=0).stop 2023-12-02T00:31:22.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.053+0000 7feeb1925640 1 -- 172.21.15.46:0/45060688 shutdown_connections 2023-12-02T00:31:22.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.053+0000 7feeb1925640 1 --2- 172.21.15.46:0/45060688 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fee8c06ae40 0x7fee8c06d300 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:22.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.053+0000 7feeb1925640 1 --2- 172.21.15.46:0/45060688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feeac160f20 0x7feeac15fab0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:22.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.053+0000 7feeb1925640 1 --2- 172.21.15.46:0/45060688 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feeac1401f0 0x7feeac0b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:22.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.053+0000 7feeb1925640 1 --2- 172.21.15.46:0/45060688 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feeac06fe30 0x7feeac13f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:22.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.053+0000 7feeb1925640 1 -- 172.21.15.46:0/45060688 >> 172.21.15.46:0/45060688 conn(0x7feeac06e7d0 msgr2=0x7feeac0b18e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:22.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.053+0000 7feeb1925640 1 -- 172.21.15.46:0/45060688 shutdown_connections 2023-12-02T00:31:22.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.053+0000 7feeb1925640 1 -- 172.21.15.46:0/45060688 wait complete. 2023-12-02T00:31:22.065 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-02T00:31:22.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.174+0000 7fb590a87640 1 Processor -- start 2023-12-02T00:31:22.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.174+0000 7fb590a87640 1 -- start start 2023-12-02T00:31:22.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.175+0000 7fb590a87640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb58c143490 0x7fb58c0b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:22.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.175+0000 7fb590a87640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb58c15e2a0 con 0x7fb58c15e730 2023-12-02T00:31:22.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.175+0000 7fb590a87640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb58c15b240 con 0x7fb58c15abf0 2023-12-02T00:31:22.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.175+0000 7fb590a87640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb58c15b520 con 0x7fb58c143490 2023-12-02T00:31:22.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.175+0000 7fb589d9b640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb58c15e730 0x7fb58c15a4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57232/0 (socket says 172.21.15.46:57232) 2023-12-02T00:31:22.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.175+0000 7fb589d9b640 1 -- 172.21.15.46:0/560392737 learned_addr learned my addr 172.21.15.46:0/560392737 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:31:22.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.175+0000 7fb58a59c640 1 --2- 172.21.15.46:0/560392737 >> v2:172.21.15.162:3300/0 conn(0x7fb58c143490 0x7fb58c0b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:22.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.175+0000 7fb58959a640 1 -- 172.21.15.46:0/560392737 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2211330022 0 0) 0x7fb58c15e2a0 con 0x7fb58c15e730 2023-12-02T00:31:22.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.175+0000 7fb58959a640 1 -- 172.21.15.46:0/560392737 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb574003520 con 0x7fb58c15e730 2023-12-02T00:31:22.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.175+0000 7fb58959a640 1 -- 172.21.15.46:0/560392737 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3421835013 0 0) 0x7fb574003520 con 0x7fb58c15e730 2023-12-02T00:31:22.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.175+0000 7fb58959a640 1 -- 172.21.15.46:0/560392737 >> v2:172.21.15.162:3300/0 conn(0x7fb58c143490 msgr2=0x7fb58c0b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:22.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.175+0000 7fb58959a640 1 --2- 172.21.15.46:0/560392737 >> v2:172.21.15.162:3300/0 conn(0x7fb58c143490 0x7fb58c0b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:22.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.175+0000 7fb58959a640 1 -- 172.21.15.46:0/560392737 >> v1:172.21.15.162:6789/0 conn(0x7fb58c15abf0 legacy=0x7fb58c0b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:22.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.175+0000 7fb58959a640 1 -- 172.21.15.46:0/560392737 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb58c15dc50 con 0x7fb58c15e730 2023-12-02T00:31:22.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.175+0000 7fb58959a640 1 -- 172.21.15.46:0/560392737 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7fb580001670 con 0x7fb58c15e730 2023-12-02T00:31:22.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.175+0000 7fb58959a640 1 -- 172.21.15.46:0/560392737 >> v1:172.21.15.46:6789/0 conn(0x7fb58c15e730 legacy=0x7fb58c15a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:22.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.175+0000 7fb58959a640 1 --2- 172.21.15.46:0/560392737 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb574003970 0x7fb574003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:22.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.175+0000 7fb58959a640 1 --2- 172.21.15.46:0/560392737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5740047c0 0x7fb574004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:22.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.175+0000 7fb58959a640 1 --2- 172.21.15.46:0/560392737 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb574005180 0x7fb57400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:22.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.175+0000 7fb58959a640 1 -- 172.21.15.46:0/560392737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58c15b520 con 0x7fb5740047c0 2023-12-02T00:31:22.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.175+0000 7fb58959a640 1 -- 172.21.15.46:0/560392737 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57400c020 con 0x7fb574003970 2023-12-02T00:31:22.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.175+0000 7fb58959a640 1 -- 172.21.15.46:0/560392737 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb57400c1e0 con 0x7fb574005180 2023-12-02T00:31:22.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.176+0000 7fb589d9b640 1 --2- 172.21.15.46:0/560392737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5740047c0 0x7fb574004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:22.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.176+0000 7fb58a59c640 1 --2- 172.21.15.46:0/560392737 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb574003970 0x7fb574003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:22.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.176+0000 7fb58ad9d640 1 --2- 172.21.15.46:0/560392737 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb574005180 0x7fb57400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:22.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.176+0000 7fb589d9b640 1 -- 172.21.15.46:0/560392737 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb574005180 msgr2=0x7fb57400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:22.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.176+0000 7fb589d9b640 1 --2- 172.21.15.46:0/560392737 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb574005180 0x7fb57400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:22.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.176+0000 7fb589d9b640 1 -- 172.21.15.46:0/560392737 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb574003970 msgr2=0x7fb574003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:22.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.176+0000 7fb589d9b640 1 --2- 172.21.15.46:0/560392737 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb574003970 0x7fb574003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:22.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.176+0000 7fb589d9b640 1 -- 172.21.15.46:0/560392737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb574003520 con 0x7fb5740047c0 2023-12-02T00:31:22.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.176+0000 7fb58ad9d640 1 --2- 172.21.15.46:0/560392737 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb574005180 0x7fb57400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:31:22.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.176+0000 7fb58ad9d640 1 -- 172.21.15.46:0/560392737 reap_dead start 2023-12-02T00:31:22.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.177+0000 7fb589d9b640 1 --2- 172.21.15.46:0/560392737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5740047c0 0x7fb574004c40 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fb58000c240 tx=0x7fb5740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d03d73cba418295 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:22.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.177+0000 7fb58959a640 1 -- 172.21.15.46:0/560392737 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb580005ad0 con 0x7fb5740047c0 2023-12-02T00:31:22.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.177+0000 7fb58959a640 1 -- 172.21.15.46:0/560392737 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb580005c30 con 0x7fb5740047c0 2023-12-02T00:31:22.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.177+0000 7fb58959a640 1 -- 172.21.15.46:0/560392737 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58002c8b0 con 0x7fb5740047c0 2023-12-02T00:31:22.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.177+0000 7fb590a87640 1 -- 172.21.15.46:0/560392737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5740047c0 msgr2=0x7fb574004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:22.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.177+0000 7fb590a87640 1 --2- 172.21.15.46:0/560392737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5740047c0 0x7fb574004c40 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fb58000c240 tx=0x7fb5740057f0 comp rx=0 tx=0).stop 2023-12-02T00:31:22.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.177+0000 7fb590a87640 1 -- 172.21.15.46:0/560392737 shutdown_connections 2023-12-02T00:31:22.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.177+0000 7fb590a87640 1 --2- 172.21.15.46:0/560392737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5740047c0 0x7fb574004c40 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:22.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.177+0000 7fb590a87640 1 -- 172.21.15.46:0/560392737 >> 172.21.15.46:0/560392737 conn(0x7fb58c06a4d0 msgr2=0x7fb58c06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:22.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.177+0000 7fb590a87640 1 -- 172.21.15.46:0/560392737 shutdown_connections 2023-12-02T00:31:22.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.177+0000 7fb590a87640 1 -- 172.21.15.46:0/560392737 wait complete. 2023-12-02T00:31:22.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.179+0000 7fb590a87640 1 Processor -- start 2023-12-02T00:31:22.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.179+0000 7fb590a87640 1 -- start start 2023-12-02T00:31:22.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.179+0000 7fb590a87640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5740047c0 0x7fb58c159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:22.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.179+0000 7fb590a87640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb58c0c8d20 0x7fb58c137f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:22.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.179+0000 7fb590a87640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb58c138460 0x7fb58c0cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:22.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.179+0000 7fb590a87640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58c13a940 con 0x7fb58c0c8d20 2023-12-02T00:31:22.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.179+0000 7fb590a87640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58c1595d0 con 0x7fb5740047c0 2023-12-02T00:31:22.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.179+0000 7fb590a87640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb58c139dc0 con 0x7fb58c138460 2023-12-02T00:31:22.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.180+0000 7fb589d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb58c0c8d20 0x7fb58c137f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:22.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.180+0000 7fb589d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb58c0c8d20 0x7fb58c137f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39942/0 (socket says 172.21.15.46:39942) 2023-12-02T00:31:22.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.180+0000 7fb589d9b640 1 -- 172.21.15.46:0/1661663437 learned_addr learned my addr 172.21.15.46:0/1661663437 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:22.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.180+0000 7fb58ad9d640 1 --2- 172.21.15.46:0/1661663437 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb58c138460 0x7fb58c0cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:22.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.180+0000 7fb58a59c640 1 --2- 172.21.15.46:0/1661663437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5740047c0 0x7fb58c159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:22.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.180+0000 7fb589d9b640 1 -- 172.21.15.46:0/1661663437 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb58c138460 msgr2=0x7fb58c0cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:22.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.180+0000 7fb589d9b640 1 --2- 172.21.15.46:0/1661663437 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb58c138460 0x7fb58c0cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:22.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.180+0000 7fb589d9b640 1 -- 172.21.15.46:0/1661663437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5740047c0 msgr2=0x7fb58c159090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:22.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.180+0000 7fb589d9b640 1 --2- 172.21.15.46:0/1661663437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5740047c0 0x7fb58c159090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:22.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.180+0000 7fb589d9b640 1 -- 172.21.15.46:0/1661663437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb58c13a0a0 con 0x7fb58c0c8d20 2023-12-02T00:31:22.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.180+0000 7fb589d9b640 1 --2- 172.21.15.46:0/1661663437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb58c0c8d20 0x7fb58c137f20 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fb580001930 tx=0x7fb58002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:22.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.180+0000 7fb57affd640 1 -- 172.21.15.46:0/1661663437 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58003c040 con 0x7fb58c0c8d20 2023-12-02T00:31:22.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.180+0000 7fb57affd640 1 -- 172.21.15.46:0/1661663437 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb58002ccc0 con 0x7fb58c0c8d20 2023-12-02T00:31:22.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.180+0000 7fb590a87640 1 -- 172.21.15.46:0/1661663437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb58c13a380 con 0x7fb58c0c8d20 2023-12-02T00:31:22.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.180+0000 7fb57affd640 1 -- 172.21.15.46:0/1661663437 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb580007290 con 0x7fb58c0c8d20 2023-12-02T00:31:22.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.180+0000 7fb590a87640 1 -- 172.21.15.46:0/1661663437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb58c159a90 con 0x7fb58c0c8d20 2023-12-02T00:31:22.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.181+0000 7fb590a87640 1 -- 172.21.15.46:0/1661663437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb58c069400 con 0x7fb58c0c8d20 2023-12-02T00:31:22.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.182+0000 7fb57affd640 1 -- 172.21.15.46:0/1661663437 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb580032070 con 0x7fb58c0c8d20 2023-12-02T00:31:22.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.182+0000 7fb57affd640 1 --2- 172.21.15.46:0/1661663437 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb56c06ae40 0x7fb56c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:22.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.182+0000 7fb57affd640 1 -- 172.21.15.46:0/1661663437 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb5800aaa80 con 0x7fb58c0c8d20 2023-12-02T00:31:22.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.183+0000 7fb58a59c640 1 --2- 172.21.15.46:0/1661663437 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb56c06ae40 0x7fb56c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:22.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.183+0000 7fb58a59c640 1 --2- 172.21.15.46:0/1661663437 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb56c06ae40 0x7fb56c06d300 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb57c009b80 tx=0x7fb57c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:22.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.186+0000 7fb57affd640 1 -- 172.21.15.46:0/1661663437 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb580075bc0 con 0x7fb58c0c8d20 2023-12-02T00:31:22.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.364+0000 7fb590a87640 1 -- 172.21.15.46:0/1661663437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb58c143b30 con 0x7fb58c0c8d20 2023-12-02T00:31:22.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.364+0000 7fb57affd640 1 -- 172.21.15.46:0/1661663437 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v51) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb5800791e0 con 0x7fb58c0c8d20 2023-12-02T00:31:22.365 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-12-02T00:31:22.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.366+0000 7fb590a87640 1 -- 172.21.15.46:0/1661663437 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb56c06ae40 msgr2=0x7fb56c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:22.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.366+0000 7fb590a87640 1 --2- 172.21.15.46:0/1661663437 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb56c06ae40 0x7fb56c06d300 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb57c009b80 tx=0x7fb57c008040 comp rx=0 tx=0).stop 2023-12-02T00:31:22.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.366+0000 7fb590a87640 1 -- 172.21.15.46:0/1661663437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb58c0c8d20 msgr2=0x7fb58c137f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:22.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.366+0000 7fb590a87640 1 --2- 172.21.15.46:0/1661663437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb58c0c8d20 0x7fb58c137f20 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fb580001930 tx=0x7fb58002e070 comp rx=0 tx=0).stop 2023-12-02T00:31:22.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.366+0000 7fb590a87640 1 -- 172.21.15.46:0/1661663437 shutdown_connections 2023-12-02T00:31:22.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.366+0000 7fb590a87640 1 --2- 172.21.15.46:0/1661663437 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb56c06ae40 0x7fb56c06d300 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:22.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.366+0000 7fb590a87640 1 --2- 172.21.15.46:0/1661663437 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb58c138460 0x7fb58c0cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:22.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.366+0000 7fb590a87640 1 --2- 172.21.15.46:0/1661663437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb58c0c8d20 0x7fb58c137f20 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:22.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.366+0000 7fb590a87640 1 --2- 172.21.15.46:0/1661663437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5740047c0 0x7fb58c159090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:22.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.366+0000 7fb590a87640 1 -- 172.21.15.46:0/1661663437 >> 172.21.15.46:0/1661663437 conn(0x7fb58c06a4d0 msgr2=0x7fb58c0b3670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:22.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.366+0000 7fb590a87640 1 -- 172.21.15.46:0/1661663437 shutdown_connections 2023-12-02T00:31:22.369 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.366+0000 7fb590a87640 1 -- 172.21.15.46:0/1661663437 wait complete. 2023-12-02T00:31:22.379 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-02T00:31:22.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.486+0000 7f8d6cd68640 1 Processor -- start 2023-12-02T00:31:22.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.486+0000 7f8d6cd68640 1 -- start start 2023-12-02T00:31:22.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.487+0000 7f8d6cd68640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8d6806ea50 0x7f8d6806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:22.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.487+0000 7f8d6cd68640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8d68160fb0 0x7f8d68161440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:22.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.487+0000 7f8d6cd68640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d680b3ff0 0x7f8d680b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:22.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.487+0000 7f8d6cd68640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d68161d20 con 0x7f8d6806ea50 2023-12-02T00:31:22.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.487+0000 7f8d6cd68640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d6806f370 con 0x7f8d68160fb0 2023-12-02T00:31:22.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.487+0000 7f8d6cd68640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6815e760 con 0x7f8d680b3ff0 2023-12-02T00:31:22.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.487+0000 7f8d6659c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8d6806ea50 0x7f8d6806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:22.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.487+0000 7f8d6659c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8d6806ea50 0x7f8d6806ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39946/0 (socket says 172.21.15.46:39946) 2023-12-02T00:31:22.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.487+0000 7f8d6659c640 1 -- 172.21.15.46:0/34542249 learned_addr learned my addr 172.21.15.46:0/34542249 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:22.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.487+0000 7f8d66d9d640 1 --2- 172.21.15.46:0/34542249 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d680b3ff0 0x7f8d680b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:22.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.487+0000 7f8d65d9b640 1 --2- 172.21.15.46:0/34542249 >> v2:172.21.15.162:3300/0 conn(0x7f8d68160fb0 0x7f8d68161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:22.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.487+0000 7f8d6659c640 1 -- 172.21.15.46:0/34542249 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d680b3ff0 msgr2=0x7f8d680b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:22.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.487+0000 7f8d6659c640 1 --2- 172.21.15.46:0/34542249 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d680b3ff0 0x7f8d680b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:22.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.487+0000 7f8d6659c640 1 -- 172.21.15.46:0/34542249 >> v2:172.21.15.162:3300/0 conn(0x7f8d68160fb0 msgr2=0x7f8d68161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:22.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.487+0000 7f8d6659c640 1 --2- 172.21.15.46:0/34542249 >> v2:172.21.15.162:3300/0 conn(0x7f8d68160fb0 0x7f8d68161440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:22.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.487+0000 7f8d6659c640 1 -- 172.21.15.46:0/34542249 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d6815d9a0 con 0x7f8d6806ea50 2023-12-02T00:31:22.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.488+0000 7f8d6659c640 1 --2- 172.21.15.46:0/34542249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d6806ea50 0x7f8d6806ee30 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f8d680b5b30 tx=0x7f8d5802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b08b28d73e22d713 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:22.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.488+0000 7f8d6559a640 1 -- 172.21.15.46:0/34542249 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d580317e0 con 0x7f8d6806ea50 2023-12-02T00:31:22.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.488+0000 7f8d6559a640 1 -- 172.21.15.46:0/34542249 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d58031940 con 0x7f8d6806ea50 2023-12-02T00:31:22.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.488+0000 7f8d6559a640 1 -- 172.21.15.46:0/34542249 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d58031cd0 con 0x7f8d6806ea50 2023-12-02T00:31:22.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.488+0000 7f8d6cd68640 1 -- 172.21.15.46:0/34542249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d6806ea50 msgr2=0x7f8d6806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:22.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.488+0000 7f8d6cd68640 1 --2- 172.21.15.46:0/34542249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d6806ea50 0x7f8d6806ee30 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f8d680b5b30 tx=0x7f8d5802b880 comp rx=0 tx=0).stop 2023-12-02T00:31:22.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.489+0000 7f8d6cd68640 1 -- 172.21.15.46:0/34542249 shutdown_connections 2023-12-02T00:31:22.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.489+0000 7f8d6cd68640 1 --2- 172.21.15.46:0/34542249 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d680b3ff0 0x7f8d680b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:22.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.489+0000 7f8d6cd68640 1 --2- 172.21.15.46:0/34542249 >> v2:172.21.15.162:3300/0 conn(0x7f8d68160fb0 0x7f8d68161440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:22.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.489+0000 7f8d6cd68640 1 --2- 172.21.15.46:0/34542249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d6806ea50 0x7f8d6806ee30 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:22.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.489+0000 7f8d6cd68640 1 -- 172.21.15.46:0/34542249 >> 172.21.15.46:0/34542249 conn(0x7f8d6806a6b0 msgr2=0x7f8d6806aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:22.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.489+0000 7f8d6cd68640 1 -- 172.21.15.46:0/34542249 shutdown_connections 2023-12-02T00:31:22.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.489+0000 7f8d6cd68640 1 -- 172.21.15.46:0/34542249 wait complete. 2023-12-02T00:31:22.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.490+0000 7f8d6cd68640 1 Processor -- start 2023-12-02T00:31:22.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.490+0000 7f8d6cd68640 1 -- start start 2023-12-02T00:31:22.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.491+0000 7f8d6cd68640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d6806ea50 0x7f8d6815aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:22.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.491+0000 7f8d6cd68640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8d680b3ff0 0x7f8d68156230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:22.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.491+0000 7f8d6cd68640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d68160fb0 0x7f8d681317f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:22.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.491+0000 7f8d6cd68640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d68155da0 con 0x7f8d68160fb0 2023-12-02T00:31:22.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.491+0000 7f8d6cd68640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d681521e0 con 0x7f8d680b3ff0 2023-12-02T00:31:22.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.491+0000 7f8d6cd68640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d68155a50 con 0x7f8d6806ea50 2023-12-02T00:31:22.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.491+0000 7f8d66d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d68160fb0 0x7f8d681317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:22.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.491+0000 7f8d66d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d68160fb0 0x7f8d681317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39956/0 (socket says 172.21.15.46:39956) 2023-12-02T00:31:22.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.491+0000 7f8d66d9d640 1 -- 172.21.15.46:0/825789433 learned_addr learned my addr 172.21.15.46:0/825789433 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:22.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.491+0000 7f8d6659c640 1 --2- 172.21.15.46:0/825789433 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d6806ea50 0x7f8d6815aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:22.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.491+0000 7f8d66d9d640 1 -- 172.21.15.46:0/825789433 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d6806ea50 msgr2=0x7f8d6815aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:22.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.491+0000 7f8d66d9d640 1 --2- 172.21.15.46:0/825789433 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d6806ea50 0x7f8d6815aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:22.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.491+0000 7f8d66d9d640 1 -- 172.21.15.46:0/825789433 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8d680b3ff0 msgr2=0x7f8d68156230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:22.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.491+0000 7f8d65d9b640 1 --2- 172.21.15.46:0/825789433 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8d680b3ff0 0x7f8d68156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:22.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.491+0000 7f8d66d9d640 1 --2- 172.21.15.46:0/825789433 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8d680b3ff0 0x7f8d68156230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:22.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.491+0000 7f8d66d9d640 1 -- 172.21.15.46:0/825789433 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d580094d0 con 0x7f8d68160fb0 2023-12-02T00:31:22.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.491+0000 7f8d6659c640 1 --2- 172.21.15.46:0/825789433 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d6806ea50 0x7f8d6815aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:31:22.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.491+0000 7f8d66d9d640 1 --2- 172.21.15.46:0/825789433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d68160fb0 0x7f8d681317f0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f8d6000efc0 tx=0x7f8d6000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:22.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.491+0000 7f8d56ffd640 1 -- 172.21.15.46:0/825789433 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d60009b10 con 0x7f8d68160fb0 2023-12-02T00:31:22.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.491+0000 7f8d56ffd640 1 -- 172.21.15.46:0/825789433 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d6000f040 con 0x7f8d68160fb0 2023-12-02T00:31:22.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.492+0000 7f8d6cd68640 1 -- 172.21.15.46:0/825789433 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d68151940 con 0x7f8d68160fb0 2023-12-02T00:31:22.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.492+0000 7f8d56ffd640 1 -- 172.21.15.46:0/825789433 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d600146c0 con 0x7f8d68160fb0 2023-12-02T00:31:22.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.492+0000 7f8d6cd68640 1 -- 172.21.15.46:0/825789433 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d68151f00 con 0x7f8d68160fb0 2023-12-02T00:31:22.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.492+0000 7f8d6cd68640 1 -- 172.21.15.46:0/825789433 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d68155f50 con 0x7f8d68160fb0 2023-12-02T00:31:22.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.493+0000 7f8d56ffd640 1 -- 172.21.15.46:0/825789433 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8d60014820 con 0x7f8d68160fb0 2023-12-02T00:31:22.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.493+0000 7f8d56ffd640 1 --2- 172.21.15.46:0/825789433 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8d4806ae40 0x7f8d4806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:22.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.494+0000 7f8d56ffd640 1 -- 172.21.15.46:0/825789433 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8d6008a350 con 0x7f8d68160fb0 2023-12-02T00:31:22.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.494+0000 7f8d6659c640 1 --2- 172.21.15.46:0/825789433 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8d4806ae40 0x7f8d4806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:22.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.494+0000 7f8d6659c640 1 --2- 172.21.15.46:0/825789433 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8d4806ae40 0x7f8d4806d300 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8d58036000 tx=0x7f8d5803d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:22.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.498+0000 7f8d56ffd640 1 -- 172.21.15.46:0/825789433 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8d68155f50 con 0x7f8d68160fb0 2023-12-02T00:31:22.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.688+0000 7f8d6cd68640 1 -- 172.21.15.46:0/825789433 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8d6806ee30 con 0x7f8d68160fb0 2023-12-02T00:31:22.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.688+0000 7f8d56ffd640 1 -- 172.21.15.46:0/825789433 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v51) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8d60055490 con 0x7f8d68160fb0 2023-12-02T00:31:22.689 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2023-12-02T00:31:22.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.690+0000 7f8d6cd68640 1 -- 172.21.15.46:0/825789433 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8d4806ae40 msgr2=0x7f8d4806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:22.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.690+0000 7f8d6cd68640 1 --2- 172.21.15.46:0/825789433 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8d4806ae40 0x7f8d4806d300 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8d58036000 tx=0x7f8d5803d040 comp rx=0 tx=0).stop 2023-12-02T00:31:22.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.690+0000 7f8d6cd68640 1 -- 172.21.15.46:0/825789433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d68160fb0 msgr2=0x7f8d681317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:22.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.690+0000 7f8d6cd68640 1 --2- 172.21.15.46:0/825789433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d68160fb0 0x7f8d681317f0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f8d6000efc0 tx=0x7f8d6000c490 comp rx=0 tx=0).stop 2023-12-02T00:31:22.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.690+0000 7f8d6cd68640 1 -- 172.21.15.46:0/825789433 shutdown_connections 2023-12-02T00:31:22.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.690+0000 7f8d6cd68640 1 --2- 172.21.15.46:0/825789433 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8d4806ae40 0x7f8d4806d300 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:22.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.690+0000 7f8d6cd68640 1 --2- 172.21.15.46:0/825789433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d68160fb0 0x7f8d681317f0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:22.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.690+0000 7f8d6cd68640 1 --2- 172.21.15.46:0/825789433 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8d680b3ff0 0x7f8d68156230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:22.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.690+0000 7f8d6cd68640 1 --2- 172.21.15.46:0/825789433 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d6806ea50 0x7f8d6815aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:22.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.690+0000 7f8d6cd68640 1 -- 172.21.15.46:0/825789433 >> 172.21.15.46:0/825789433 conn(0x7f8d6806a6b0 msgr2=0x7f8d680b0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:22.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.690+0000 7f8d6cd68640 1 -- 172.21.15.46:0/825789433 shutdown_connections 2023-12-02T00:31:22.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:22.690+0000 7f8d6cd68640 1 -- 172.21.15.46:0/825789433 wait complete. 2023-12-02T00:31:22.702 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi046.front.sepia.ceph.com... 2023-12-02T00:31:22.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:31:22.703 DEBUG:teuthology.orchestra.run.smithi046:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-12-02T00:31:22.733 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[111010]: fuse finished with error 0 and tester_r 0 2023-12-02T00:31:22.735 INFO:teuthology.orchestra.run:waiting for 300 2023-12-02T00:31:28.837 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2023-12-02T00:31:28.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:31:28.837 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-02T00:31:28.857 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-02T00:31:29.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.014+0000 7f6d8e6c2640 1 Processor -- start 2023-12-02T00:31:29.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.014+0000 7f6d8e6c2640 1 -- start start 2023-12-02T00:31:29.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.015+0000 7f6d8e6c2640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6d880ae480 0x7f6d8806d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:29.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.015+0000 7f6d8e6c2640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6d88072390 0x7f6d8806d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:29.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.015+0000 7f6d8e6c2640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d880ae910 con 0x7f6d8815d2b0 2023-12-02T00:31:29.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.015+0000 7f6d8e6c2640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d880b2200 con 0x7f6d88072390 2023-12-02T00:31:29.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.015+0000 7f6d8e6c2640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d880b24e0 con 0x7f6d880ae480 2023-12-02T00:31:29.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.015+0000 7f6d8cc5f640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6d88072390 0x7f6d8806d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:29.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.015+0000 7f6d8cc5f640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6d88072390 0x7f6d8806d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60308/0 (socket says 172.21.15.46:60308) 2023-12-02T00:31:29.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.015+0000 7f6d8cc5f640 1 -- 172.21.15.46:0/501731424 learned_addr learned my addr 172.21.15.46:0/501731424 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:29.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.015+0000 7f6d877fe640 1 --2- 172.21.15.46:0/501731424 >> v2:172.21.15.162:3300/0 conn(0x7f6d880ae480 0x7f6d8806d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:29.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.015+0000 7f6d8cc5f640 1 -- 172.21.15.46:0/501731424 >> v2:172.21.15.162:3300/0 conn(0x7f6d880ae480 msgr2=0x7f6d8806d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:29.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.015+0000 7f6d8cc5f640 1 --2- 172.21.15.46:0/501731424 >> v2:172.21.15.162:3300/0 conn(0x7f6d880ae480 0x7f6d8806d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:29.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.015+0000 7f6d8cc5f640 1 -- 172.21.15.46:0/501731424 >> v1:172.21.15.162:6789/0 conn(0x7f6d8815d2b0 legacy=0x7f6d88071c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:29.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.015+0000 7f6d8cc5f640 1 -- 172.21.15.46:0/501731424 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d880b40a0 con 0x7f6d88072390 2023-12-02T00:31:29.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.015+0000 7f6d877fe640 1 --2- 172.21.15.46:0/501731424 >> v2:172.21.15.162:3300/0 conn(0x7f6d880ae480 0x7f6d8806d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:31:29.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.016+0000 7f6d8cc5f640 1 --2- 172.21.15.46:0/501731424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d88072390 0x7f6d8806d880 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f6d88160e00 tx=0x7f6d8002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43b9d9e7ae3bbf7c server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:29.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.016+0000 7f6d86ffd640 1 -- 172.21.15.46:0/501731424 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8002c9f0 con 0x7f6d88072390 2023-12-02T00:31:29.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.016+0000 7f6d86ffd640 1 -- 172.21.15.46:0/501731424 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d8002cb50 con 0x7f6d88072390 2023-12-02T00:31:29.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.016+0000 7f6d86ffd640 1 -- 172.21.15.46:0/501731424 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d800343f0 con 0x7f6d88072390 2023-12-02T00:31:29.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.017+0000 7f6d8e6c2640 1 -- 172.21.15.46:0/501731424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d88072390 msgr2=0x7f6d8806d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:29.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.017+0000 7f6d8e6c2640 1 --2- 172.21.15.46:0/501731424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d88072390 0x7f6d8806d880 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f6d88160e00 tx=0x7f6d8002be50 comp rx=0 tx=0).stop 2023-12-02T00:31:29.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.017+0000 7f6d8e6c2640 1 -- 172.21.15.46:0/501731424 shutdown_connections 2023-12-02T00:31:29.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.017+0000 7f6d8e6c2640 1 --2- 172.21.15.46:0/501731424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d88072390 0x7f6d8806d880 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:29.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.017+0000 7f6d8e6c2640 1 --2- 172.21.15.46:0/501731424 >> v2:172.21.15.162:3300/0 conn(0x7f6d880ae480 0x7f6d8806d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:29.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.017+0000 7f6d8e6c2640 1 -- 172.21.15.46:0/501731424 >> 172.21.15.46:0/501731424 conn(0x7f6d880b4720 msgr2=0x7f6d880b4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:29.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.017+0000 7f6d8e6c2640 1 -- 172.21.15.46:0/501731424 shutdown_connections 2023-12-02T00:31:29.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.017+0000 7f6d8e6c2640 1 -- 172.21.15.46:0/501731424 wait complete. 2023-12-02T00:31:29.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.018+0000 7f6d8e6c2640 1 Processor -- start 2023-12-02T00:31:29.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.019+0000 7f6d8e6c2640 1 -- start start 2023-12-02T00:31:29.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.019+0000 7f6d8e6c2640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d88072390 0x7f6d88154cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:29.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.019+0000 7f6d8e6c2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d8815d2b0 0x7f6d881f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:29.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.019+0000 7f6d8e6c2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d881f2620 0x7f6d88143210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:29.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.019+0000 7f6d8e6c2640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8815b520 con 0x7f6d8815d2b0 2023-12-02T00:31:29.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.019+0000 7f6d8e6c2640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d881f2ad0 con 0x7f6d88072390 2023-12-02T00:31:29.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.019+0000 7f6d8e6c2640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8815ab80 con 0x7f6d881f2620 2023-12-02T00:31:29.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.019+0000 7f6d877fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d8815d2b0 0x7f6d881f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:29.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.019+0000 7f6d877fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d8815d2b0 0x7f6d881f20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60322/0 (socket says 172.21.15.46:60322) 2023-12-02T00:31:29.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.019+0000 7f6d877fe640 1 -- 172.21.15.46:0/1665764812 learned_addr learned my addr 172.21.15.46:0/1665764812 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:29.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.020+0000 7f6d87fff640 1 --2- 172.21.15.46:0/1665764812 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d88072390 0x7f6d88154cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:29.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.020+0000 7f6d8cc5f640 1 --2- 172.21.15.46:0/1665764812 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d881f2620 0x7f6d88143210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:29.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.020+0000 7f6d877fe640 1 -- 172.21.15.46:0/1665764812 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d881f2620 msgr2=0x7f6d88143210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:29.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.020+0000 7f6d877fe640 1 --2- 172.21.15.46:0/1665764812 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d881f2620 0x7f6d88143210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:29.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.020+0000 7f6d877fe640 1 -- 172.21.15.46:0/1665764812 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d88072390 msgr2=0x7f6d88154cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:29.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.020+0000 7f6d877fe640 1 --2- 172.21.15.46:0/1665764812 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d88072390 0x7f6d88154cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:29.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.020+0000 7f6d877fe640 1 -- 172.21.15.46:0/1665764812 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d80009b30 con 0x7f6d8815d2b0 2023-12-02T00:31:29.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.020+0000 7f6d877fe640 1 --2- 172.21.15.46:0/1665764812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d8815d2b0 0x7f6d881f20e0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f6d7800c7f0 tx=0x7f6d7800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:29.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.020+0000 7f6d84ff9640 1 -- 172.21.15.46:0/1665764812 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d78007a30 con 0x7f6d8815d2b0 2023-12-02T00:31:29.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.020+0000 7f6d84ff9640 1 -- 172.21.15.46:0/1665764812 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d78007b90 con 0x7f6d8815d2b0 2023-12-02T00:31:29.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.020+0000 7f6d8e6c2640 1 -- 172.21.15.46:0/1665764812 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d880b2af0 con 0x7f6d8815d2b0 2023-12-02T00:31:29.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.020+0000 7f6d84ff9640 1 -- 172.21.15.46:0/1665764812 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7800ce40 con 0x7f6d8815d2b0 2023-12-02T00:31:29.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.020+0000 7f6d8e6c2640 1 -- 172.21.15.46:0/1665764812 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d8815b140 con 0x7f6d8815d2b0 2023-12-02T00:31:29.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.021+0000 7f6d8e6c2640 1 -- 172.21.15.46:0/1665764812 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d880b2950 con 0x7f6d8815d2b0 2023-12-02T00:31:29.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.022+0000 7f6d84ff9640 1 -- 172.21.15.46:0/1665764812 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6d780113f0 con 0x7f6d8815d2b0 2023-12-02T00:31:29.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.022+0000 7f6d84ff9640 1 --2- 172.21.15.46:0/1665764812 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f6d7406ae40 0x7f6d7406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:29.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.022+0000 7f6d84ff9640 1 -- 172.21.15.46:0/1665764812 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6d780897e0 con 0x7f6d8815d2b0 2023-12-02T00:31:29.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.022+0000 7f6d87fff640 1 --2- 172.21.15.46:0/1665764812 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f6d7406ae40 0x7f6d7406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:29.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.023+0000 7f6d87fff640 1 --2- 172.21.15.46:0/1665764812 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f6d7406ae40 0x7f6d7406d300 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c006fd0 tx=0x7f6d7c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:29.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.026+0000 7f6d84ff9640 1 -- 172.21.15.46:0/1665764812 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6d880b2950 con 0x7f6d8815d2b0 2023-12-02T00:31:29.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.240+0000 7f6d8e6c2640 1 -- 172.21.15.46:0/1665764812 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6d8815b660 con 0x7f6d8815d2b0 2023-12-02T00:31:29.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.240+0000 7f6d84ff9640 1 -- 172.21.15.46:0/1665764812 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6d8815b660 con 0x7f6d8815d2b0 2023-12-02T00:31:29.242 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-02T00:31:29.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.242+0000 7f6d8e6c2640 1 -- 172.21.15.46:0/1665764812 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f6d7406ae40 msgr2=0x7f6d7406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:29.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.242+0000 7f6d8e6c2640 1 --2- 172.21.15.46:0/1665764812 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f6d7406ae40 0x7f6d7406d300 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c006fd0 tx=0x7f6d7c008040 comp rx=0 tx=0).stop 2023-12-02T00:31:29.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.242+0000 7f6d8e6c2640 1 -- 172.21.15.46:0/1665764812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d8815d2b0 msgr2=0x7f6d881f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:29.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.242+0000 7f6d8e6c2640 1 --2- 172.21.15.46:0/1665764812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d8815d2b0 0x7f6d881f20e0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f6d7800c7f0 tx=0x7f6d7800cbf0 comp rx=0 tx=0).stop 2023-12-02T00:31:29.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.242+0000 7f6d8e6c2640 1 -- 172.21.15.46:0/1665764812 shutdown_connections 2023-12-02T00:31:29.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.242+0000 7f6d8e6c2640 1 --2- 172.21.15.46:0/1665764812 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f6d7406ae40 0x7f6d7406d300 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:29.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.242+0000 7f6d8e6c2640 1 --2- 172.21.15.46:0/1665764812 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d881f2620 0x7f6d88143210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:29.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.242+0000 7f6d8e6c2640 1 --2- 172.21.15.46:0/1665764812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d8815d2b0 0x7f6d881f20e0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:29.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.242+0000 7f6d8e6c2640 1 --2- 172.21.15.46:0/1665764812 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d88072390 0x7f6d88154cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:29.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.242+0000 7f6d8e6c2640 1 -- 172.21.15.46:0/1665764812 >> 172.21.15.46:0/1665764812 conn(0x7f6d880b4720 msgr2=0x7f6d8815e170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:29.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.242+0000 7f6d8e6c2640 1 -- 172.21.15.46:0/1665764812 shutdown_connections 2023-12-02T00:31:29.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:29.242+0000 7f6d8e6c2640 1 -- 172.21.15.46:0/1665764812 wait complete. 2023-12-02T00:31:29.254 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-02T00:31:29.254 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-02T00:31:29.254 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2023-12-02T00:31:29.255 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-02T00:31:29.255 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-02T00:31:29.255 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-02T00:31:29.255 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-02T00:31:29.255 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-02T00:31:29.255 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-02T00:31:29.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:31:29.255 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2023-12-02T00:31:29.271 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-02T00:31:29.271 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-02T00:31:29.272 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2023-12-02T00:31:29.272 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:31:29.272 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2023-12-02T00:31:29.272 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:31:29.272 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-02T00:31:29.272 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2023-12-02T00:31:29.272 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2023-12-02T00:31:29.272 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-02T00:31:29.273 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2023-12-02T00:31:29.273 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2023-12-02T00:31:29.273 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-02T00:31:29.273 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2023-12-02T00:31:29.273 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2023-12-02T00:31:29.273 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-12-02T00:31:29.273 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42490sec preferred_lft 42490sec 2023-12-02T00:31:29.273 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link noprefixroute 2023-12-02T00:31:29.273 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:31:29.273 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-02T00:31:29.274 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2023-12-02T00:31:29.274 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2023-12-02T00:31:29.274 INFO:teuthology.orchestra.run.smithi046.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-02T00:31:29.274 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether b6:1a:8b:80:15:f1 brd ff:ff:ff:ff:ff:ff 2023-12-02T00:31:29.274 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-02T00:31:29.274 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:31:29.274 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::eca8:9ff:feb3:1726/64 scope link 2023-12-02T00:31:29.274 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:31:29.274 INFO:teuthology.orchestra.run.smithi046.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-02T00:31:29.275 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether b6:1a:8b:80:15:f1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-02T00:31:29.275 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::b41a:8bff:fe80:15f1/64 scope link 2023-12-02T00:31:29.275 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:31:29.275 INFO:teuthology.orchestra.run.smithi046.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-02T00:31:29.275 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether c6:93:a5:c0:e2:af brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-02T00:31:29.275 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c493:a5ff:fec0:e2af/64 scope link 2023-12-02T00:31:29.275 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-02T00:31:29.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:31:29.276 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-02T00:31:29.341 INFO:teuthology.orchestra.run.smithi046.stdout:1 2023-12-02T00:31:29.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:31:29.342 DEBUG:teuthology.orchestra.run.smithi046:> route 2023-12-02T00:31:29.399 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2023-12-02T00:31:29.399 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-02T00:31:29.399 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-12-02T00:31:29.399 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-12-02T00:31:29.399 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-02T00:31:29.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:31:29.399 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2023-12-02T00:31:29.400 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2023-12-02T00:31:29.400 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-02T00:31:29.400 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-02T00:31:29.400 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-02T00:31:29.400 DEBUG:teuthology.orchestra.run.smithi046:> ') 2023-12-02T00:31:29.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:31:29.504 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2023-12-02T00:31:29.520 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-02T00:31:29.520 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-02T00:31:29.520 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-02T00:31:29.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:31:29.521 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2023-12-02T00:31:29.590 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-12-02T00:31:29.591 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-02T00:31:29.591 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-02T00:31:29.646 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-02T00:31:29.647 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-02T00:31:29.647 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-02T00:31:29.702 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2023-12-02T00:31:29.773 DEBUG:teuthology.orchestra.run.smithi046:> cat /proc/self/mounts | awk '{print $2}' 2023-12-02T00:31:29.829 INFO:teuthology.orchestra.run.smithi046.stdout:/proc 2023-12-02T00:31:29.829 INFO:teuthology.orchestra.run.smithi046.stdout:/sys 2023-12-02T00:31:29.829 INFO:teuthology.orchestra.run.smithi046.stdout:/dev 2023-12-02T00:31:29.830 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/security 2023-12-02T00:31:29.830 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/shm 2023-12-02T00:31:29.830 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/pts 2023-12-02T00:31:29.830 INFO:teuthology.orchestra.run.smithi046.stdout:/run 2023-12-02T00:31:29.830 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/cgroup 2023-12-02T00:31:29.830 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/pstore 2023-12-02T00:31:29.830 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/bpf 2023-12-02T00:31:29.830 INFO:teuthology.orchestra.run.smithi046.stdout:/ 2023-12-02T00:31:29.830 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/selinux 2023-12-02T00:31:29.830 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2023-12-02T00:31:29.830 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/mqueue 2023-12-02T00:31:29.830 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/hugepages 2023-12-02T00:31:29.830 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/debug 2023-12-02T00:31:29.830 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/tracing 2023-12-02T00:31:29.830 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/fuse/connections 2023-12-02T00:31:29.831 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/config 2023-12-02T00:31:29.831 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-sysctl.service 2023-12-02T00:31:29.831 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-02T00:31:29.831 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-02T00:31:29.831 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/nfs/rpc_pipefs 2023-12-02T00:31:29.831 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/1000 2023-12-02T00:31:29.831 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph 2023-12-02T00:31:29.831 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-0 2023-12-02T00:31:29.831 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-1 2023-12-02T00:31:29.831 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-2 2023-12-02T00:31:29.831 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-3 2023-12-02T00:31:29.831 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns 2023-12-02T00:31:29.831 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-02T00:31:29.831 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-02T00:31:29.832 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-02T00:31:29.832 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-02T00:31:29.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:31:29.832 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2023-12-02T00:31:29.885 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-12-02T00:31:29.886 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-12-02T00:31:29.928 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2023-12-02T00:31:29.959 DEBUG:teuthology.orchestra.run.smithi046:> cat /proc/self/mounts | awk '{print $2}' 2023-12-02T00:31:30.016 INFO:teuthology.orchestra.run.smithi046.stdout:/proc 2023-12-02T00:31:30.016 INFO:teuthology.orchestra.run.smithi046.stdout:/sys 2023-12-02T00:31:30.017 INFO:teuthology.orchestra.run.smithi046.stdout:/dev 2023-12-02T00:31:30.017 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/security 2023-12-02T00:31:30.017 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/shm 2023-12-02T00:31:30.017 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/pts 2023-12-02T00:31:30.017 INFO:teuthology.orchestra.run.smithi046.stdout:/run 2023-12-02T00:31:30.017 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/cgroup 2023-12-02T00:31:30.017 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/pstore 2023-12-02T00:31:30.017 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/bpf 2023-12-02T00:31:30.017 INFO:teuthology.orchestra.run.smithi046.stdout:/ 2023-12-02T00:31:30.018 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/selinux 2023-12-02T00:31:30.018 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2023-12-02T00:31:30.018 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/mqueue 2023-12-02T00:31:30.018 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/hugepages 2023-12-02T00:31:30.018 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/debug 2023-12-02T00:31:30.018 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/tracing 2023-12-02T00:31:30.018 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/fuse/connections 2023-12-02T00:31:30.018 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/config 2023-12-02T00:31:30.019 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-sysctl.service 2023-12-02T00:31:30.019 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-02T00:31:30.019 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-02T00:31:30.019 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/nfs/rpc_pipefs 2023-12-02T00:31:30.019 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/1000 2023-12-02T00:31:30.019 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph 2023-12-02T00:31:30.019 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-0 2023-12-02T00:31:30.019 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-1 2023-12-02T00:31:30.019 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-2 2023-12-02T00:31:30.020 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-3 2023-12-02T00:31:30.020 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns 2023-12-02T00:31:30.020 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-02T00:31:30.020 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-02T00:31:30.020 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-02T00:31:30.020 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-02T00:31:30.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:31:30.021 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2023-12-02T00:31:30.022 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:2023-12-02T00:31:30.020+0000 7fb5a5bd7480 -1 init, newargv = 0x55feb50ae610 newargc=15 2023-12-02T00:31:30.022 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[111990]: starting ceph client 2023-12-02T00:31:30.025 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[111990]: starting fuse 2023-12-02T00:31:30.037 INFO:teuthology.orchestra.run.smithi046.stdout:41 2023-12-02T00:31:30.038 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-12-02T00:31:30.038 DEBUG:teuthology.orchestra.run.smithi046:> sudo stdin-killer -- python3 -c ' 2023-12-02T00:31:30.038 DEBUG:teuthology.orchestra.run.smithi046:> import glob 2023-12-02T00:31:30.038 DEBUG:teuthology.orchestra.run.smithi046:> import re 2023-12-02T00:31:30.038 DEBUG:teuthology.orchestra.run.smithi046:> import os 2023-12-02T00:31:30.038 DEBUG:teuthology.orchestra.run.smithi046:> import subprocess 2023-12-02T00:31:30.038 DEBUG:teuthology.orchestra.run.smithi046:> 2023-12-02T00:31:30.038 DEBUG:teuthology.orchestra.run.smithi046:> def _find_admin_socket(client_name): 2023-12-02T00:31:30.038 DEBUG:teuthology.orchestra.run.smithi046:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-12-02T00:31:30.039 DEBUG:teuthology.orchestra.run.smithi046:> files = glob.glob(asok_path) 2023-12-02T00:31:30.039 DEBUG:teuthology.orchestra.run.smithi046:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-12-02T00:31:30.039 DEBUG:teuthology.orchestra.run.smithi046:> 2023-12-02T00:31:30.039 DEBUG:teuthology.orchestra.run.smithi046:> # Given a non-glob path, it better be there 2023-12-02T00:31:30.039 DEBUG:teuthology.orchestra.run.smithi046:> if "*" not in asok_path: 2023-12-02T00:31:30.039 DEBUG:teuthology.orchestra.run.smithi046:> assert(len(files) == 1) 2023-12-02T00:31:30.039 DEBUG:teuthology.orchestra.run.smithi046:> return files[0] 2023-12-02T00:31:30.039 DEBUG:teuthology.orchestra.run.smithi046:> 2023-12-02T00:31:30.039 DEBUG:teuthology.orchestra.run.smithi046:> for f in files: 2023-12-02T00:31:30.040 DEBUG:teuthology.orchestra.run.smithi046:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-02T00:31:30.040 DEBUG:teuthology.orchestra.run.smithi046:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-02T00:31:30.040 DEBUG:teuthology.orchestra.run.smithi046:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-02T00:31:30.040 DEBUG:teuthology.orchestra.run.smithi046:> contents = proc_f.read() 2023-12-02T00:31:30.040 DEBUG:teuthology.orchestra.run.smithi046:> if mountpoint in contents: 2023-12-02T00:31:30.040 DEBUG:teuthology.orchestra.run.smithi046:> return f 2023-12-02T00:31:30.040 DEBUG:teuthology.orchestra.run.smithi046:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-02T00:31:30.040 DEBUG:teuthology.orchestra.run.smithi046:> 2023-12-02T00:31:30.040 DEBUG:teuthology.orchestra.run.smithi046:> print(_find_admin_socket("client.0")) 2023-12-02T00:31:30.041 DEBUG:teuthology.orchestra.run.smithi046:> ' 2023-12-02T00:31:30.163 INFO:teuthology.orchestra.run.smithi046.stdout:/var/run/ceph/ceph-client.0.111990.asok 2023-12-02T00:31:30.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-12-02T00:31:30.174 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.111990.asok 2023-12-02T00:31:30.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:31:30.174 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.111990.asok status 2023-12-02T00:31:30.317 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-12-02T00:31:30.317 INFO:teuthology.orchestra.run.smithi046.stdout: "metadata": { 2023-12-02T00:31:30.317 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_sha1": "bed7681480ccb0192b91a35e691658867e1e96b7", 2023-12-02T00:31:30.317 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_version": "ceph version 18.0.0-7631-gbed76814 (bed7681480ccb0192b91a35e691658867e1e96b7) reef (dev)", 2023-12-02T00:31:30.317 INFO:teuthology.orchestra.run.smithi046.stdout: "entity_id": "0", 2023-12-02T00:31:30.317 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": "smithi046", 2023-12-02T00:31:30.317 INFO:teuthology.orchestra.run.smithi046.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-12-02T00:31:30.317 INFO:teuthology.orchestra.run.smithi046.stdout: "pid": "111990", 2023-12-02T00:31:30.318 INFO:teuthology.orchestra.run.smithi046.stdout: "root": "/" 2023-12-02T00:31:30.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:31:30.318 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_count": 0, 2023-12-02T00:31:30.318 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_pinned_count": 0, 2023-12-02T00:31:30.318 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 6348, 2023-12-02T00:31:30.318 INFO:teuthology.orchestra.run.smithi046.stdout: "inst": { 2023-12-02T00:31:30.318 INFO:teuthology.orchestra.run.smithi046.stdout: "name": { 2023-12-02T00:31:30.318 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "client", 2023-12-02T00:31:30.318 INFO:teuthology.orchestra.run.smithi046.stdout: "num": 6348 2023-12-02T00:31:30.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:31:30.318 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2023-12-02T00:31:30.318 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2023-12-02T00:31:30.318 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2023-12-02T00:31:30.318 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3706589357 2023-12-02T00:31:30.318 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-02T00:31:30.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:31:30.319 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2023-12-02T00:31:30.319 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2023-12-02T00:31:30.319 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2023-12-02T00:31:30.319 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3706589357 2023-12-02T00:31:30.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-02T00:31:30.319 INFO:teuthology.orchestra.run.smithi046.stdout: "inst_str": "client.6348 192.168.0.1:0/3706589357", 2023-12-02T00:31:30.319 INFO:teuthology.orchestra.run.smithi046.stdout: "addr_str": "192.168.0.1:0/3706589357", 2023-12-02T00:31:30.319 INFO:teuthology.orchestra.run.smithi046.stdout: "inode_count": 1, 2023-12-02T00:31:30.319 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_epoch": 24, 2023-12-02T00:31:30.319 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch": 51, 2023-12-02T00:31:30.319 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch_barrier": 0, 2023-12-02T00:31:30.319 INFO:teuthology.orchestra.run.smithi046.stdout: "blocklisted": false, 2023-12-02T00:31:30.319 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs" 2023-12-02T00:31:30.319 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-12-02T00:31:30.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:31:30.326 DEBUG:teuthology.orchestra.run.smithi046:> stat --file-system '--printf=%T 2023-12-02T00:31:30.326 DEBUG:teuthology.orchestra.run.smithi046:> ' -- /home/ubuntu/cephtest/mnt.0 2023-12-02T00:31:30.341 INFO:teuthology.orchestra.run.smithi046.stdout:fuseblk 2023-12-02T00:31:30.341 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-12-02T00:31:30.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:31:30.341 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-02T00:31:30.412 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2023-12-02T00:31:30.412 DEBUG:teuthology.orchestra.run.smithi046:> import os 2023-12-02T00:31:30.412 DEBUG:teuthology.orchestra.run.smithi046:> import errno 2023-12-02T00:31:30.412 DEBUG:teuthology.orchestra.run.smithi046:> 2023-12-02T00:31:30.412 DEBUG:teuthology.orchestra.run.smithi046:> fd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR) 2023-12-02T00:31:30.412 DEBUG:teuthology.orchestra.run.smithi046:> try: 2023-12-02T00:31:30.412 DEBUG:teuthology.orchestra.run.smithi046:> if True: 2023-12-02T00:31:30.412 DEBUG:teuthology.orchestra.run.smithi046:> ret = os.read(fd, 1024) 2023-12-02T00:31:30.413 DEBUG:teuthology.orchestra.run.smithi046:> else: 2023-12-02T00:31:30.413 DEBUG:teuthology.orchestra.run.smithi046:> os.write(fd, b'"'"'content'"'"') 2023-12-02T00:31:30.413 DEBUG:teuthology.orchestra.run.smithi046:> except OSError as e: 2023-12-02T00:31:30.413 DEBUG:teuthology.orchestra.run.smithi046:> if e.errno != errno.EPERM: 2023-12-02T00:31:30.413 DEBUG:teuthology.orchestra.run.smithi046:> raise 2023-12-02T00:31:30.413 DEBUG:teuthology.orchestra.run.smithi046:> else: 2023-12-02T00:31:30.413 DEBUG:teuthology.orchestra.run.smithi046:> raise RuntimeError("client does not check permission of data pool") 2023-12-02T00:31:30.413 DEBUG:teuthology.orchestra.run.smithi046:> ' 2023-12-02T00:31:30.520 INFO:teuthology.orchestra.run.smithi046.stderr:Traceback (most recent call last): 2023-12-02T00:31:30.520 INFO:teuthology.orchestra.run.smithi046.stderr: File "", line 15, in 2023-12-02T00:31:30.520 INFO:teuthology.orchestra.run.smithi046.stderr:RuntimeError: client does not check permission of data pool 2023-12-02T00:31:30.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30 stdin-killer INFO: command exited with status 1: exiting normally with same code! 2023-12-02T00:31:30.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-02T00:31:30.530 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2023-12-02T00:31:30.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.683+0000 7f8b0a7d0640 1 Processor -- start 2023-12-02T00:31:30.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.683+0000 7f8b0a7d0640 1 -- start start 2023-12-02T00:31:30.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.684+0000 7f8b0a7d0640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8b041464d0 0x7f8b040b2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:30.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.684+0000 7f8b0a7d0640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b04146d50 con 0x7f8b04140480 2023-12-02T00:31:30.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.684+0000 7f8b0a7d0640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b040ad700 con 0x7f8b0406bbf0 2023-12-02T00:31:30.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.684+0000 7f8b0a7d0640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b040ada30 con 0x7f8b041464d0 2023-12-02T00:31:30.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.684+0000 7f8b08d6d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8b041464d0 0x7f8b040b2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:30.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.684+0000 7f8b03fff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8b04140480 0x7f8b0406b4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49742/0 (socket says 172.21.15.46:49742) 2023-12-02T00:31:30.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.684+0000 7f8b03fff640 1 -- 172.21.15.46:0/2365579340 learned_addr learned my addr 172.21.15.46:0/2365579340 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:31:30.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.684+0000 7f8b08d6d640 1 -- 172.21.15.46:0/2365579340 >> v1:172.21.15.162:6789/0 conn(0x7f8b0406bbf0 legacy=0x7f8b040ade50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:30.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.684+0000 7f8b08d6d640 1 -- 172.21.15.46:0/2365579340 >> v1:172.21.15.46:6789/0 conn(0x7f8b04140480 legacy=0x7f8b0406b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:30.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.684+0000 7f8b08d6d640 1 -- 172.21.15.46:0/2365579340 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b040b37c0 con 0x7f8b041464d0 2023-12-02T00:31:30.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.685+0000 7f8b08d6d640 1 --2- 172.21.15.46:0/2365579340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b041464d0 0x7f8b040b2820 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f8afc009d90 tx=0x7f8afc02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcac7e31ce73e5c5 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:30.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.685+0000 7f8b02ffd640 1 -- 172.21.15.46:0/2365579340 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8afc02cbd0 con 0x7f8b041464d0 2023-12-02T00:31:30.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.685+0000 7f8b02ffd640 1 -- 172.21.15.46:0/2365579340 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8afc02cd30 con 0x7f8b041464d0 2023-12-02T00:31:30.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.685+0000 7f8b02ffd640 1 -- 172.21.15.46:0/2365579340 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8afc02cea0 con 0x7f8b041464d0 2023-12-02T00:31:30.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.685+0000 7f8b0a7d0640 1 -- 172.21.15.46:0/2365579340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b041464d0 msgr2=0x7f8b040b2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:30.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.685+0000 7f8b0a7d0640 1 --2- 172.21.15.46:0/2365579340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b041464d0 0x7f8b040b2820 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f8afc009d90 tx=0x7f8afc02be20 comp rx=0 tx=0).stop 2023-12-02T00:31:30.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.685+0000 7f8b0a7d0640 1 -- 172.21.15.46:0/2365579340 shutdown_connections 2023-12-02T00:31:30.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.685+0000 7f8b0a7d0640 1 --2- 172.21.15.46:0/2365579340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b041464d0 0x7f8b040b2820 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:30.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.686+0000 7f8b0a7d0640 1 -- 172.21.15.46:0/2365579340 >> 172.21.15.46:0/2365579340 conn(0x7f8b04070c70 msgr2=0x7f8b04071050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:30.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.686+0000 7f8b0a7d0640 1 -- 172.21.15.46:0/2365579340 shutdown_connections 2023-12-02T00:31:30.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.686+0000 7f8b0a7d0640 1 -- 172.21.15.46:0/2365579340 wait complete. 2023-12-02T00:31:30.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.687+0000 7f8b0a7d0640 1 Processor -- start 2023-12-02T00:31:30.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.687+0000 7f8b0a7d0640 1 -- start start 2023-12-02T00:31:30.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.688+0000 7f8b0a7d0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b0406bbf0 0x7f8b0415cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:30.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.688+0000 7f8b0a7d0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b04140480 0x7f8b0415e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:30.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.688+0000 7f8b0a7d0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b041464d0 0x7f8b041f40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:30.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.688+0000 7f8b0a7d0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b04139560 con 0x7f8b0406bbf0 2023-12-02T00:31:30.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.688+0000 7f8b0a7d0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b04160410 con 0x7f8b04140480 2023-12-02T00:31:30.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.688+0000 7f8b0a7d0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0415d000 con 0x7f8b041464d0 2023-12-02T00:31:30.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.688+0000 7f8b03fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b0406bbf0 0x7f8b0415cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:30.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.688+0000 7f8b03fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b0406bbf0 0x7f8b0415cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60332/0 (socket says 172.21.15.46:60332) 2023-12-02T00:31:30.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.688+0000 7f8b03fff640 1 -- 172.21.15.46:0/2990073015 learned_addr learned my addr 172.21.15.46:0/2990073015 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:30.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.688+0000 7f8b037fe640 1 --2- 172.21.15.46:0/2990073015 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b04140480 0x7f8b0415e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:30.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.688+0000 7f8b08d6d640 1 --2- 172.21.15.46:0/2990073015 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b041464d0 0x7f8b041f40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:30.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.688+0000 7f8b03fff640 1 -- 172.21.15.46:0/2990073015 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b041464d0 msgr2=0x7f8b041f40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:30.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.688+0000 7f8b03fff640 1 --2- 172.21.15.46:0/2990073015 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b041464d0 0x7f8b041f40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:30.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.688+0000 7f8b03fff640 1 -- 172.21.15.46:0/2990073015 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b04140480 msgr2=0x7f8b0415e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:30.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.688+0000 7f8b03fff640 1 --2- 172.21.15.46:0/2990073015 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b04140480 0x7f8b0415e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:30.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.688+0000 7f8b03fff640 1 -- 172.21.15.46:0/2990073015 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8afc009a70 con 0x7f8b0406bbf0 2023-12-02T00:31:30.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.688+0000 7f8b037fe640 1 --2- 172.21.15.46:0/2990073015 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b04140480 0x7f8b0415e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:31:30.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.688+0000 7f8b03fff640 1 --2- 172.21.15.46:0/2990073015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b0406bbf0 0x7f8b0415cac0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f8af400c450 tx=0x7f8af400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:30.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.688+0000 7f8b00ff9640 1 -- 172.21.15.46:0/2990073015 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af4005ce0 con 0x7f8b0406bbf0 2023-12-02T00:31:30.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.688+0000 7f8b00ff9640 1 -- 172.21.15.46:0/2990073015 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8af4005e40 con 0x7f8b0406bbf0 2023-12-02T00:31:30.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.688+0000 7f8b00ff9640 1 -- 172.21.15.46:0/2990073015 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af40072b0 con 0x7f8b0406bbf0 2023-12-02T00:31:30.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.688+0000 7f8b0a7d0640 1 -- 172.21.15.46:0/2990073015 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b04138bc0 con 0x7f8b0406bbf0 2023-12-02T00:31:30.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.689+0000 7f8b0a7d0640 1 -- 172.21.15.46:0/2990073015 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b04139180 con 0x7f8b0406bbf0 2023-12-02T00:31:30.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.689+0000 7f8b0a7d0640 1 -- 172.21.15.46:0/2990073015 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b04069400 con 0x7f8b0406bbf0 2023-12-02T00:31:30.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.690+0000 7f8b00ff9640 1 -- 172.21.15.46:0/2990073015 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8af4007410 con 0x7f8b0406bbf0 2023-12-02T00:31:30.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.690+0000 7f8b00ff9640 1 --2- 172.21.15.46:0/2990073015 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8af006ae40 0x7f8af006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:30.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.690+0000 7f8b00ff9640 1 -- 172.21.15.46:0/2990073015 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8af4088aa0 con 0x7f8b0406bbf0 2023-12-02T00:31:30.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.691+0000 7f8b037fe640 1 --2- 172.21.15.46:0/2990073015 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8af006ae40 0x7f8af006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:30.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.691+0000 7f8b037fe640 1 --2- 172.21.15.46:0/2990073015 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8af006ae40 0x7f8af006d300 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8b0415f230 tx=0x7f8af8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:30.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.695+0000 7f8b00ff9640 1 -- 172.21.15.46:0/2990073015 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8af4053be0 con 0x7f8b0406bbf0 2023-12-02T00:31:30.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.938+0000 7f8b0a7d0640 1 -- 172.21.15.46:0/2990073015 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f8b04071a70 con 0x7f8b0406bbf0 2023-12-02T00:31:30.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.940+0000 7f8b00ff9640 1 -- 172.21.15.46:0/2990073015 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v20) v1 ==== 184+0+141 (secure 0 0 0) 0x7f8af4017020 con 0x7f8b0406bbf0 2023-12-02T00:31:30.942 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2023-12-02T00:31:30.942 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAJempl5Lx1DhAAjpK6ajoQ5jQiUr/EMpFeLg== 2023-12-02T00:31:30.942 INFO:teuthology.orchestra.run.smithi046.stdout: caps mds = "allow" 2023-12-02T00:31:30.942 INFO:teuthology.orchestra.run.smithi046.stdout: caps mon = "allow r" 2023-12-02T00:31:30.942 INFO:teuthology.orchestra.run.smithi046.stdout: caps osd = "allow rw pool=cephfs_data" 2023-12-02T00:31:30.942 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2023-12-02T00:31:30.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.942+0000 7f8b0a7d0640 1 -- 172.21.15.46:0/2990073015 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8af006ae40 msgr2=0x7f8af006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:30.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.942+0000 7f8b0a7d0640 1 --2- 172.21.15.46:0/2990073015 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8af006ae40 0x7f8af006d300 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8b0415f230 tx=0x7f8af8009040 comp rx=0 tx=0).stop 2023-12-02T00:31:30.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.942+0000 7f8b0a7d0640 1 -- 172.21.15.46:0/2990073015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b0406bbf0 msgr2=0x7f8b0415cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:30.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.942+0000 7f8b0a7d0640 1 --2- 172.21.15.46:0/2990073015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b0406bbf0 0x7f8b0415cac0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f8af400c450 tx=0x7f8af400c910 comp rx=0 tx=0).stop 2023-12-02T00:31:30.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.942+0000 7f8b0a7d0640 1 -- 172.21.15.46:0/2990073015 shutdown_connections 2023-12-02T00:31:30.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.942+0000 7f8b0a7d0640 1 --2- 172.21.15.46:0/2990073015 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8af006ae40 0x7f8af006d300 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:30.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.942+0000 7f8b0a7d0640 1 --2- 172.21.15.46:0/2990073015 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b041464d0 0x7f8b041f40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:30.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.942+0000 7f8b0a7d0640 1 --2- 172.21.15.46:0/2990073015 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b04140480 0x7f8b0415e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:30.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.942+0000 7f8b0a7d0640 1 --2- 172.21.15.46:0/2990073015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b0406bbf0 0x7f8b0415cac0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:30.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.942+0000 7f8b0a7d0640 1 -- 172.21.15.46:0/2990073015 >> 172.21.15.46:0/2990073015 conn(0x7f8b04070c70 msgr2=0x7f8b0406dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:30.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.943+0000 7f8b0a7d0640 1 -- 172.21.15.46:0/2990073015 shutdown_connections 2023-12-02T00:31:30.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:30.943+0000 7f8b0a7d0640 1 -- 172.21.15.46:0/2990073015 wait complete. 2023-12-02T00:31:30.954 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-02T00:31:30.954 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-02T00:31:30.998 DEBUG:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-02T00:31:31.041 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-02T00:31:31.042 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-02T00:31:31.042 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-02T00:31:31.057 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:31:31.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.204+0000 7fc960e17640 1 Processor -- start 2023-12-02T00:31:31.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.204+0000 7fc960e17640 1 -- start start 2023-12-02T00:31:31.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.205+0000 7fc960e17640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc95c15e730 0x7fc95c15abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:31.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.205+0000 7fc960e17640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc95c15e2a0 con 0x7fc95c0b1400 2023-12-02T00:31:31.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.205+0000 7fc960e17640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc95c15b240 con 0x7fc95c143490 2023-12-02T00:31:31.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.205+0000 7fc960e17640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc95c15b520 con 0x7fc95c15e730 2023-12-02T00:31:31.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.205+0000 7fc959d9b640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc95c0b1400 0x7fc95c15a4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49758/0 (socket says 172.21.15.46:49758) 2023-12-02T00:31:31.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.205+0000 7fc959d9b640 1 -- 172.21.15.46:0/2538130281 learned_addr learned my addr 172.21.15.46:0/2538130281 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:31:31.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.206+0000 7fc95959a640 1 -- 172.21.15.46:0/2538130281 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2192795421 0 0) 0x7fc95c15e2a0 con 0x7fc95c0b1400 2023-12-02T00:31:31.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.206+0000 7fc95959a640 1 -- 172.21.15.46:0/2538130281 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc948005730 con 0x7fc95c0b1400 2023-12-02T00:31:31.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.206+0000 7fc95ad9d640 1 --2- 172.21.15.46:0/2538130281 >> v2:172.21.15.162:3300/0 conn(0x7fc95c15e730 0x7fc95c15abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:31.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.206+0000 7fc95959a640 1 -- 172.21.15.46:0/2538130281 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1735717263 0 0) 0x7fc948005730 con 0x7fc95c0b1400 2023-12-02T00:31:31.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.206+0000 7fc95959a640 1 -- 172.21.15.46:0/2538130281 >> v2:172.21.15.162:3300/0 conn(0x7fc95c15e730 msgr2=0x7fc95c15abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:31.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.206+0000 7fc95959a640 1 --2- 172.21.15.46:0/2538130281 >> v2:172.21.15.162:3300/0 conn(0x7fc95c15e730 0x7fc95c15abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:31.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.206+0000 7fc95959a640 1 -- 172.21.15.46:0/2538130281 >> v1:172.21.15.162:6789/0 conn(0x7fc95c143490 legacy=0x7fc95c0b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:31.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.206+0000 7fc95959a640 1 -- 172.21.15.46:0/2538130281 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc95c15dc50 con 0x7fc95c0b1400 2023-12-02T00:31:31.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.206+0000 7fc95959a640 1 -- 172.21.15.46:0/2538130281 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7fc944001670 con 0x7fc95c0b1400 2023-12-02T00:31:31.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.206+0000 7fc95ad9d640 1 --2- 172.21.15.46:0/2538130281 >> v2:172.21.15.162:3300/0 conn(0x7fc95c15e730 0x7fc95c15abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:31:31.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.206+0000 7fc95959a640 1 -- 172.21.15.46:0/2538130281 >> v1:172.21.15.46:6789/0 conn(0x7fc95c0b1400 legacy=0x7fc95c15a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:31.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.206+0000 7fc95959a640 1 --2- 172.21.15.46:0/2538130281 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc948005b80 0x7fc948005f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:31.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.206+0000 7fc95959a640 1 --2- 172.21.15.46:0/2538130281 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc9480069d0 0x7fc948006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:31.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.206+0000 7fc95959a640 1 --2- 172.21.15.46:0/2538130281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc948007390 0x7fc94800db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:31.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.206+0000 7fc95959a640 1 -- 172.21.15.46:0/2538130281 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95c15b520 con 0x7fc948007390 2023-12-02T00:31:31.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.206+0000 7fc95959a640 1 -- 172.21.15.46:0/2538130281 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94800e2c0 con 0x7fc948005b80 2023-12-02T00:31:31.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.206+0000 7fc95959a640 1 -- 172.21.15.46:0/2538130281 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc94800e480 con 0x7fc9480069d0 2023-12-02T00:31:31.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.206+0000 7fc95ad9d640 1 --2- 172.21.15.46:0/2538130281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc948007390 0x7fc94800db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:31.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.206+0000 7fc95a59c640 1 --2- 172.21.15.46:0/2538130281 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc948005b80 0x7fc948005f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:31.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.206+0000 7fc959d9b640 1 --2- 172.21.15.46:0/2538130281 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc9480069d0 0x7fc948006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:31.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.206+0000 7fc95ad9d640 1 -- 172.21.15.46:0/2538130281 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc9480069d0 msgr2=0x7fc948006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:31.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.206+0000 7fc95ad9d640 1 --2- 172.21.15.46:0/2538130281 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc9480069d0 0x7fc948006e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:31.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.206+0000 7fc95ad9d640 1 -- 172.21.15.46:0/2538130281 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc948005b80 msgr2=0x7fc948005f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:31.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.206+0000 7fc95ad9d640 1 --2- 172.21.15.46:0/2538130281 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc948005b80 0x7fc948005f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:31.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.206+0000 7fc95ad9d640 1 -- 172.21.15.46:0/2538130281 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc948005730 con 0x7fc948007390 2023-12-02T00:31:31.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.206+0000 7fc95a59c640 1 --2- 172.21.15.46:0/2538130281 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc948005b80 0x7fc948005f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:31:31.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.207+0000 7fc95ad9d640 1 -- 172.21.15.46:0/2538130281 reap_dead start 2023-12-02T00:31:31.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.207+0000 7fc95ad9d640 1 --2- 172.21.15.46:0/2538130281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc948007390 0x7fc94800db90 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fc94c0073b0 tx=0x7fc948007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aa2899b1f681f7d server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:31.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.207+0000 7fc95959a640 1 -- 172.21.15.46:0/2538130281 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94c02cd90 con 0x7fc948007390 2023-12-02T00:31:31.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.207+0000 7fc95959a640 1 -- 172.21.15.46:0/2538130281 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc94c034440 con 0x7fc948007390 2023-12-02T00:31:31.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.207+0000 7fc95959a640 1 -- 172.21.15.46:0/2538130281 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94c0347d0 con 0x7fc948007390 2023-12-02T00:31:31.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.208+0000 7fc960e17640 1 -- 172.21.15.46:0/2538130281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc948007390 msgr2=0x7fc94800db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:31.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.208+0000 7fc960e17640 1 --2- 172.21.15.46:0/2538130281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc948007390 0x7fc94800db90 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fc94c0073b0 tx=0x7fc948007a00 comp rx=0 tx=0).stop 2023-12-02T00:31:31.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.208+0000 7fc960e17640 1 -- 172.21.15.46:0/2538130281 shutdown_connections 2023-12-02T00:31:31.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.208+0000 7fc960e17640 1 --2- 172.21.15.46:0/2538130281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc948007390 0x7fc94800db90 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:31.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.208+0000 7fc960e17640 1 -- 172.21.15.46:0/2538130281 >> 172.21.15.46:0/2538130281 conn(0x7fc95c06a4d0 msgr2=0x7fc95c06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:31.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.208+0000 7fc960e17640 1 -- 172.21.15.46:0/2538130281 shutdown_connections 2023-12-02T00:31:31.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.208+0000 7fc960e17640 1 -- 172.21.15.46:0/2538130281 wait complete. 2023-12-02T00:31:31.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.209+0000 7fc960e17640 1 Processor -- start 2023-12-02T00:31:31.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.209+0000 7fc960e17640 1 -- start start 2023-12-02T00:31:31.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.210+0000 7fc960e17640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc948007390 0x7fc95c1598b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:31.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.210+0000 7fc960e17640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc95c15a4e0 0x7fc95c15a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:31.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.210+0000 7fc960e17640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc95c1302b0 0x7fc95c130730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:31.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.210+0000 7fc960e17640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95c15aea0 con 0x7fc95c1302b0 2023-12-02T00:31:31.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.210+0000 7fc960e17640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95c12e950 con 0x7fc948007390 2023-12-02T00:31:31.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.210+0000 7fc960e17640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc95c130c70 con 0x7fc95c15a4e0 2023-12-02T00:31:31.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.210+0000 7fc95ad9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc95c1302b0 0x7fc95c130730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:31.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.210+0000 7fc95ad9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc95c1302b0 0x7fc95c130730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60354/0 (socket says 172.21.15.46:60354) 2023-12-02T00:31:31.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.210+0000 7fc95ad9d640 1 -- 172.21.15.46:0/1413667162 learned_addr learned my addr 172.21.15.46:0/1413667162 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:31.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.210+0000 7fc959d9b640 1 --2- 172.21.15.46:0/1413667162 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc95c15a4e0 0x7fc95c15a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:31.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.210+0000 7fc95a59c640 1 --2- 172.21.15.46:0/1413667162 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc948007390 0x7fc95c1598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:31.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.210+0000 7fc95ad9d640 1 -- 172.21.15.46:0/1413667162 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc95c15a4e0 msgr2=0x7fc95c15a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:31.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.210+0000 7fc95ad9d640 1 --2- 172.21.15.46:0/1413667162 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc95c15a4e0 0x7fc95c15a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:31.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.210+0000 7fc95ad9d640 1 -- 172.21.15.46:0/1413667162 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc948007390 msgr2=0x7fc95c1598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:31.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.210+0000 7fc95ad9d640 1 --2- 172.21.15.46:0/1413667162 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc948007390 0x7fc95c1598b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:31.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.210+0000 7fc95ad9d640 1 -- 172.21.15.46:0/1413667162 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc95c131200 con 0x7fc95c1302b0 2023-12-02T00:31:31.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.210+0000 7fc95a59c640 1 --2- 172.21.15.46:0/1413667162 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc948007390 0x7fc95c1598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:31:31.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.210+0000 7fc95ad9d640 1 --2- 172.21.15.46:0/1413667162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc95c1302b0 0x7fc95c130730 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fc94c02c800 tx=0x7fc94c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:31.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.211+0000 7fc952ffd640 1 -- 172.21.15.46:0/1413667162 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94c03f440 con 0x7fc95c1302b0 2023-12-02T00:31:31.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.211+0000 7fc952ffd640 1 -- 172.21.15.46:0/1413667162 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc94c03f5a0 con 0x7fc95c1302b0 2023-12-02T00:31:31.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.211+0000 7fc952ffd640 1 -- 172.21.15.46:0/1413667162 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94c03f950 con 0x7fc95c1302b0 2023-12-02T00:31:31.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.211+0000 7fc960e17640 1 -- 172.21.15.46:0/1413667162 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc95c130f20 con 0x7fc95c1302b0 2023-12-02T00:31:31.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.211+0000 7fc960e17640 1 -- 172.21.15.46:0/1413667162 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc95c12e5e0 con 0x7fc95c1302b0 2023-12-02T00:31:31.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.212+0000 7fc960e17640 1 -- 172.21.15.46:0/1413667162 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc95c13fbb0 con 0x7fc95c1302b0 2023-12-02T00:31:31.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.212+0000 7fc952ffd640 1 -- 172.21.15.46:0/1413667162 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc94c03b040 con 0x7fc95c1302b0 2023-12-02T00:31:31.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.212+0000 7fc952ffd640 1 --2- 172.21.15.46:0/1413667162 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc93c06ae40 0x7fc93c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:31.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.213+0000 7fc952ffd640 1 -- 172.21.15.46:0/1413667162 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fc94c0ab670 con 0x7fc95c1302b0 2023-12-02T00:31:31.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.213+0000 7fc95a59c640 1 --2- 172.21.15.46:0/1413667162 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc93c06ae40 0x7fc93c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:31.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.213+0000 7fc95a59c640 1 --2- 172.21.15.46:0/1413667162 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc93c06ae40 0x7fc93c06d300 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc954007fd0 tx=0x7fc954009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:31.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.217+0000 7fc952ffd640 1 -- 172.21.15.46:0/1413667162 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc94c0766f0 con 0x7fc95c1302b0 2023-12-02T00:31:31.393 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.392+0000 7fc960e17640 1 -- 172.21.15.46:0/1413667162 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc95c069400 con 0x7fc95c1302b0 2023-12-02T00:31:31.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.393+0000 7fc952ffd640 1 -- 172.21.15.46:0/1413667162 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+16585 (secure 0 0 0) 0x7fc94c079d10 con 0x7fc95c1302b0 2023-12-02T00:31:31.395 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:31:31.395 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":52,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:31:30.728357+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-12-02T00:30:59.469070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-12-02T00:31:00.174610+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.28","mappings":[{"from":0,"to":6}]},{"pgid":"8.38","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:31:31.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.395+0000 7fc960e17640 1 -- 172.21.15.46:0/1413667162 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc93c06ae40 msgr2=0x7fc93c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:31.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.395+0000 7fc960e17640 1 --2- 172.21.15.46:0/1413667162 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc93c06ae40 0x7fc93c06d300 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc954007fd0 tx=0x7fc954009040 comp rx=0 tx=0).stop 2023-12-02T00:31:31.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.395+0000 7fc960e17640 1 -- 172.21.15.46:0/1413667162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc95c1302b0 msgr2=0x7fc95c130730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:31.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.395+0000 7fc960e17640 1 --2- 172.21.15.46:0/1413667162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc95c1302b0 0x7fc95c130730 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fc94c02c800 tx=0x7fc94c004ab0 comp rx=0 tx=0).stop 2023-12-02T00:31:31.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.395+0000 7fc960e17640 1 -- 172.21.15.46:0/1413667162 shutdown_connections 2023-12-02T00:31:31.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.395+0000 7fc960e17640 1 --2- 172.21.15.46:0/1413667162 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc93c06ae40 0x7fc93c06d300 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:31.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.395+0000 7fc960e17640 1 --2- 172.21.15.46:0/1413667162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc95c1302b0 0x7fc95c130730 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:31.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.395+0000 7fc960e17640 1 --2- 172.21.15.46:0/1413667162 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc95c15a4e0 0x7fc95c15a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:31.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.395+0000 7fc960e17640 1 --2- 172.21.15.46:0/1413667162 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc948007390 0x7fc95c1598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:31.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.395+0000 7fc960e17640 1 -- 172.21.15.46:0/1413667162 >> 172.21.15.46:0/1413667162 conn(0x7fc95c06a4d0 msgr2=0x7fc95c0716b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:31.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.395+0000 7fc960e17640 1 -- 172.21.15.46:0/1413667162 shutdown_connections 2023-12-02T00:31:31.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.395+0000 7fc960e17640 1 -- 172.21.15.46:0/1413667162 wait complete. 2023-12-02T00:31:31.407 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-02T00:28:37.213645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-02T00:30:59.469070+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-12-02T00:31:00.174610+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-02T00:31:31.408 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-02T00:31:31.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.517+0000 7fd9e2a26640 1 Processor -- start 2023-12-02T00:31:31.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.517+0000 7fd9e2a26640 1 -- start start 2023-12-02T00:31:31.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.518+0000 7fd9e2a26640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd9dc160fb0 0x7fd9dc161440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:31.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.518+0000 7fd9e2a26640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd9dc0d0ed0 0x7fd9dc0d1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:31.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.518+0000 7fd9e2a26640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9dc161d20 con 0x7fd9dc06ea50 2023-12-02T00:31:31.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.518+0000 7fd9e2a26640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9dc15e760 con 0x7fd9dc0d0ed0 2023-12-02T00:31:31.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.518+0000 7fd9e2a26640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc15d9a0 con 0x7fd9dc160fb0 2023-12-02T00:31:31.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.518+0000 7fd9db7fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd9dc160fb0 0x7fd9dc161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:31.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.518+0000 7fd9e0fc3640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd9dc0d0ed0 0x7fd9dc0d1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:31.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.518+0000 7fd9db7fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd9dc160fb0 0x7fd9dc161440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.46:52178/0 (socket says 172.21.15.46:52178) 2023-12-02T00:31:31.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.518+0000 7fd9db7fe640 1 -- 172.21.15.46:0/3070481802 learned_addr learned my addr 172.21.15.46:0/3070481802 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:31.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.518+0000 7fd9e0fc3640 1 -- 172.21.15.46:0/3070481802 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd9dc160fb0 msgr2=0x7fd9dc161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:31.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.518+0000 7fd9e0fc3640 1 --2- 172.21.15.46:0/3070481802 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd9dc160fb0 0x7fd9dc161440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:31.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.518+0000 7fd9e0fc3640 1 -- 172.21.15.46:0/3070481802 >> v1:172.21.15.162:6789/0 conn(0x7fd9dc06ea50 legacy=0x7fd9dc06ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:31.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.518+0000 7fd9e0fc3640 1 -- 172.21.15.46:0/3070481802 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9dc15dc80 con 0x7fd9dc0d0ed0 2023-12-02T00:31:31.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.518+0000 7fd9daffd640 1 -- 172.21.15.46:0/3070481802 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1570088480 0 0) 0x7fd9dc161d20 con 0x7fd9dc06ea50 2023-12-02T00:31:31.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.519+0000 7fd9e0fc3640 1 --2- 172.21.15.46:0/3070481802 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9dc0d0ed0 0x7fd9dc0d1360 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4009d90 tx=0x7fd9d402bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c91b45df16bb33 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:31.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.519+0000 7fd9daffd640 1 -- 172.21.15.46:0/3070481802 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d402cc50 con 0x7fd9dc0d0ed0 2023-12-02T00:31:31.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.520+0000 7fd9daffd640 1 -- 172.21.15.46:0/3070481802 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d402cdb0 con 0x7fd9dc0d0ed0 2023-12-02T00:31:31.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.520+0000 7fd9daffd640 1 -- 172.21.15.46:0/3070481802 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d4034660 con 0x7fd9dc0d0ed0 2023-12-02T00:31:31.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.520+0000 7fd9e2a26640 1 -- 172.21.15.46:0/3070481802 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9dc0d0ed0 msgr2=0x7fd9dc0d1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:31.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.520+0000 7fd9e2a26640 1 --2- 172.21.15.46:0/3070481802 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9dc0d0ed0 0x7fd9dc0d1360 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4009d90 tx=0x7fd9d402bea0 comp rx=0 tx=0).stop 2023-12-02T00:31:31.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.520+0000 7fd9e2a26640 1 -- 172.21.15.46:0/3070481802 shutdown_connections 2023-12-02T00:31:31.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.520+0000 7fd9e2a26640 1 --2- 172.21.15.46:0/3070481802 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd9dc160fb0 0x7fd9dc161440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:31.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.520+0000 7fd9e2a26640 1 --2- 172.21.15.46:0/3070481802 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9dc0d0ed0 0x7fd9dc0d1360 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:31.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.520+0000 7fd9e2a26640 1 -- 172.21.15.46:0/3070481802 >> 172.21.15.46:0/3070481802 conn(0x7fd9dc06a6b0 msgr2=0x7fd9dc06aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:31.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.520+0000 7fd9e2a26640 1 -- 172.21.15.46:0/3070481802 shutdown_connections 2023-12-02T00:31:31.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.520+0000 7fd9e2a26640 1 -- 172.21.15.46:0/3070481802 wait complete. 2023-12-02T00:31:31.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.521+0000 7fd9e2a26640 1 Processor -- start 2023-12-02T00:31:31.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.521+0000 7fd9e2a26640 1 -- start start 2023-12-02T00:31:31.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.522+0000 7fd9e2a26640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9dc06ea50 0x7fd9dc13d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:31.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.522+0000 7fd9e2a26640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd9dc0d0ed0 0x7fd9dc13b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:31.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.522+0000 7fd9e2a26640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9dc160fb0 0x7fd9dc1f6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:31.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.522+0000 7fd9e2a26640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc157e30 con 0x7fd9dc160fb0 2023-12-02T00:31:31.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.522+0000 7fd9e2a26640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc1583f0 con 0x7fd9dc06ea50 2023-12-02T00:31:31.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.522+0000 7fd9e2a26640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc13d5a0 con 0x7fd9dc0d0ed0 2023-12-02T00:31:31.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.522+0000 7fd9e0fc3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9dc160fb0 0x7fd9dc1f6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:31.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.522+0000 7fd9e0fc3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9dc160fb0 0x7fd9dc1f6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60364/0 (socket says 172.21.15.46:60364) 2023-12-02T00:31:31.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.522+0000 7fd9e0fc3640 1 -- 172.21.15.46:0/4257467247 learned_addr learned my addr 172.21.15.46:0/4257467247 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:31.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.522+0000 7fd9db7fe640 1 --2- 172.21.15.46:0/4257467247 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd9dc0d0ed0 0x7fd9dc13b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:31.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.522+0000 7fd9dbfff640 1 --2- 172.21.15.46:0/4257467247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9dc06ea50 0x7fd9dc13d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:31.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.523+0000 7fd9e0fc3640 1 -- 172.21.15.46:0/4257467247 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd9dc0d0ed0 msgr2=0x7fd9dc13b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:31.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.523+0000 7fd9e0fc3640 1 --2- 172.21.15.46:0/4257467247 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd9dc0d0ed0 0x7fd9dc13b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:31.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.523+0000 7fd9e0fc3640 1 -- 172.21.15.46:0/4257467247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9dc06ea50 msgr2=0x7fd9dc13d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:31.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.523+0000 7fd9e0fc3640 1 --2- 172.21.15.46:0/4257467247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9dc06ea50 0x7fd9dc13d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:31.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.523+0000 7fd9e0fc3640 1 -- 172.21.15.46:0/4257467247 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d4009a70 con 0x7fd9dc160fb0 2023-12-02T00:31:31.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.523+0000 7fd9dbfff640 1 --2- 172.21.15.46:0/4257467247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9dc06ea50 0x7fd9dc13d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:31:31.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.523+0000 7fd9e0fc3640 1 --2- 172.21.15.46:0/4257467247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9dc160fb0 0x7fd9dc1f6390 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4009d90 tx=0x7fd9d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:31.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.523+0000 7fd9d8ff9640 1 -- 172.21.15.46:0/4257467247 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d4003b90 con 0x7fd9dc160fb0 2023-12-02T00:31:31.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.523+0000 7fd9d8ff9640 1 -- 172.21.15.46:0/4257467247 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d4003d10 con 0x7fd9dc160fb0 2023-12-02T00:31:31.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.523+0000 7fd9e2a26640 1 -- 172.21.15.46:0/4257467247 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9dc1f6e90 con 0x7fd9dc160fb0 2023-12-02T00:31:31.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.523+0000 7fd9d8ff9640 1 -- 172.21.15.46:0/4257467247 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d4004270 con 0x7fd9dc160fb0 2023-12-02T00:31:31.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.523+0000 7fd9e2a26640 1 -- 172.21.15.46:0/4257467247 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9dc158080 con 0x7fd9dc160fb0 2023-12-02T00:31:31.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.524+0000 7fd9e2a26640 1 -- 172.21.15.46:0/4257467247 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9dc158560 con 0x7fd9dc160fb0 2023-12-02T00:31:31.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.525+0000 7fd9d8ff9640 1 -- 172.21.15.46:0/4257467247 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd9d403b050 con 0x7fd9dc160fb0 2023-12-02T00:31:31.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.525+0000 7fd9d8ff9640 1 --2- 172.21.15.46:0/4257467247 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd9c806ae70 0x7fd9c806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:31.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.525+0000 7fd9d8ff9640 1 -- 172.21.15.46:0/4257467247 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd9d40ab9e0 con 0x7fd9dc160fb0 2023-12-02T00:31:31.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.525+0000 7fd9dbfff640 1 --2- 172.21.15.46:0/4257467247 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd9c806ae70 0x7fd9c806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:31.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.526+0000 7fd9dbfff640 1 --2- 172.21.15.46:0/4257467247 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd9c806ae70 0x7fd9c806d330 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0000f60 tx=0x7fd9d0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:31.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.529+0000 7fd9d8ff9640 1 -- 172.21.15.46:0/4257467247 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd9d4076a50 con 0x7fd9dc160fb0 2023-12-02T00:31:31.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.704+0000 7fd9e2a26640 1 -- 172.21.15.46:0/4257467247 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd9dc069400 con 0x7fd9dc160fb0 2023-12-02T00:31:31.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.705+0000 7fd9d8ff9640 1 -- 172.21.15.46:0/4257467247 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v52) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd9d40390a0 con 0x7fd9dc160fb0 2023-12-02T00:31:31.706 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-12-02T00:31:31.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.707+0000 7fd9e2a26640 1 -- 172.21.15.46:0/4257467247 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd9c806ae70 msgr2=0x7fd9c806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:31.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.707+0000 7fd9e2a26640 1 --2- 172.21.15.46:0/4257467247 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd9c806ae70 0x7fd9c806d330 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0000f60 tx=0x7fd9d0009040 comp rx=0 tx=0).stop 2023-12-02T00:31:31.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.707+0000 7fd9e2a26640 1 -- 172.21.15.46:0/4257467247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9dc160fb0 msgr2=0x7fd9dc1f6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:31.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.707+0000 7fd9e2a26640 1 --2- 172.21.15.46:0/4257467247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9dc160fb0 0x7fd9dc1f6390 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4009d90 tx=0x7fd9d4004a40 comp rx=0 tx=0).stop 2023-12-02T00:31:31.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.707+0000 7fd9e2a26640 1 -- 172.21.15.46:0/4257467247 shutdown_connections 2023-12-02T00:31:31.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.707+0000 7fd9e2a26640 1 --2- 172.21.15.46:0/4257467247 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd9c806ae70 0x7fd9c806d330 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:31.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.707+0000 7fd9e2a26640 1 --2- 172.21.15.46:0/4257467247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9dc160fb0 0x7fd9dc1f6390 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:31.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.707+0000 7fd9e2a26640 1 --2- 172.21.15.46:0/4257467247 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd9dc0d0ed0 0x7fd9dc13b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:31.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.707+0000 7fd9e2a26640 1 --2- 172.21.15.46:0/4257467247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9dc06ea50 0x7fd9dc13d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:31.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.707+0000 7fd9e2a26640 1 -- 172.21.15.46:0/4257467247 >> 172.21.15.46:0/4257467247 conn(0x7fd9dc06a6b0 msgr2=0x7fd9dc06ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:31.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.707+0000 7fd9e2a26640 1 -- 172.21.15.46:0/4257467247 shutdown_connections 2023-12-02T00:31:31.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.707+0000 7fd9e2a26640 1 -- 172.21.15.46:0/4257467247 wait complete. 2023-12-02T00:31:31.719 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-02T00:31:31.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.828+0000 7f6a47bfc640 1 Processor -- start 2023-12-02T00:31:31.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.828+0000 7f6a47bfc640 1 -- start start 2023-12-02T00:31:31.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.829+0000 7f6a47bfc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a4006abe0 0x7f6a401618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:31.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.829+0000 7f6a47bfc640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6a401409d0 0x7f6a4006efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:31.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.829+0000 7f6a47bfc640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a401406f0 con 0x7f6a40140310 2023-12-02T00:31:31.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.829+0000 7f6a47bfc640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a40161e20 con 0x7f6a401409d0 2023-12-02T00:31:31.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.829+0000 7f6a47bfc640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a40143960 con 0x7f6a4006abe0 2023-12-02T00:31:31.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.829+0000 7f6a46199640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6a401409d0 0x7f6a4006efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:31.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.829+0000 7f6a45998640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6a40140310 0x7f6a4006a4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49774/0 (socket says 172.21.15.46:49774) 2023-12-02T00:31:31.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.829+0000 7f6a45998640 1 -- 172.21.15.46:0/3111613917 learned_addr learned my addr 172.21.15.46:0/3111613917 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:31:31.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.829+0000 7f6a45197640 1 --2- 172.21.15.46:0/3111613917 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a4006abe0 0x7f6a401618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:31.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.829+0000 7f6a46199640 1 -- 172.21.15.46:0/3111613917 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a4006abe0 msgr2=0x7f6a401618e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:31.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.829+0000 7f6a46199640 1 --2- 172.21.15.46:0/3111613917 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a4006abe0 0x7f6a401618e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:31.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.829+0000 7f6a46199640 1 -- 172.21.15.46:0/3111613917 >> v1:172.21.15.46:6789/0 conn(0x7f6a40140310 legacy=0x7f6a4006a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:31.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.829+0000 7f6a46199640 1 -- 172.21.15.46:0/3111613917 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a40142de0 con 0x7f6a401409d0 2023-12-02T00:31:31.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.829+0000 7f6a44996640 1 -- 172.21.15.46:0/3111613917 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2905976718 0 0) 0x7f6a401406f0 con 0x7f6a40140310 2023-12-02T00:31:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.830+0000 7f6a46199640 1 --2- 172.21.15.46:0/3111613917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a401409d0 0x7f6a4006efb0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f6a38009d90 tx=0x7f6a3802be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=994ff4813b3d5abd server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.830+0000 7f6a44996640 1 -- 172.21.15.46:0/3111613917 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3802cbc0 con 0x7f6a401409d0 2023-12-02T00:31:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.830+0000 7f6a44996640 1 -- 172.21.15.46:0/3111613917 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a3802cd20 con 0x7f6a401409d0 2023-12-02T00:31:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.830+0000 7f6a44996640 1 -- 172.21.15.46:0/3111613917 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a38034660 con 0x7f6a401409d0 2023-12-02T00:31:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.831+0000 7f6a47bfc640 1 -- 172.21.15.46:0/3111613917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a401409d0 msgr2=0x7f6a4006efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.831+0000 7f6a47bfc640 1 --2- 172.21.15.46:0/3111613917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a401409d0 0x7f6a4006efb0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f6a38009d90 tx=0x7f6a3802be10 comp rx=0 tx=0).stop 2023-12-02T00:31:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.831+0000 7f6a47bfc640 1 -- 172.21.15.46:0/3111613917 shutdown_connections 2023-12-02T00:31:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.831+0000 7f6a47bfc640 1 --2- 172.21.15.46:0/3111613917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a401409d0 0x7f6a4006efb0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.831+0000 7f6a47bfc640 1 --2- 172.21.15.46:0/3111613917 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a4006abe0 0x7f6a401618e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.831+0000 7f6a47bfc640 1 -- 172.21.15.46:0/3111613917 >> 172.21.15.46:0/3111613917 conn(0x7f6a400b1e50 msgr2=0x7f6a400b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.831+0000 7f6a47bfc640 1 -- 172.21.15.46:0/3111613917 shutdown_connections 2023-12-02T00:31:31.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.831+0000 7f6a47bfc640 1 -- 172.21.15.46:0/3111613917 wait complete. 2023-12-02T00:31:31.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.832+0000 7f6a47bfc640 1 Processor -- start 2023-12-02T00:31:31.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.832+0000 7f6a47bfc640 1 -- start start 2023-12-02T00:31:31.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.833+0000 7f6a47bfc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a4006abe0 0x7f6a401f2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:31.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.833+0000 7f6a47bfc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a40140310 0x7f6a40156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:31.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.833+0000 7f6a47bfc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a401409d0 0x7f6a400b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:31.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.833+0000 7f6a47bfc640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a400ae2f0 con 0x7f6a40140310 2023-12-02T00:31:31.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.833+0000 7f6a47bfc640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a401383e0 con 0x7f6a4006abe0 2023-12-02T00:31:31.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.833+0000 7f6a47bfc640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a400ad670 con 0x7f6a401409d0 2023-12-02T00:31:31.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.833+0000 7f6a45197640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a40140310 0x7f6a40156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:31.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.833+0000 7f6a45197640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a40140310 0x7f6a40156be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60378/0 (socket says 172.21.15.46:60378) 2023-12-02T00:31:31.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.833+0000 7f6a45197640 1 -- 172.21.15.46:0/2125312676 learned_addr learned my addr 172.21.15.46:0/2125312676 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:31.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.833+0000 7f6a46199640 1 --2- 172.21.15.46:0/2125312676 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a401409d0 0x7f6a400b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:31.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.833+0000 7f6a45998640 1 --2- 172.21.15.46:0/2125312676 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a4006abe0 0x7f6a401f2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:31.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.833+0000 7f6a45197640 1 -- 172.21.15.46:0/2125312676 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a401409d0 msgr2=0x7f6a400b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:31.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.833+0000 7f6a45197640 1 --2- 172.21.15.46:0/2125312676 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a401409d0 0x7f6a400b04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:31.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.833+0000 7f6a45197640 1 -- 172.21.15.46:0/2125312676 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a4006abe0 msgr2=0x7f6a401f2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:31.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.833+0000 7f6a45197640 1 --2- 172.21.15.46:0/2125312676 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a4006abe0 0x7f6a401f2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:31.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.833+0000 7f6a45197640 1 -- 172.21.15.46:0/2125312676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a38009a70 con 0x7f6a40140310 2023-12-02T00:31:31.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.833+0000 7f6a46199640 1 --2- 172.21.15.46:0/2125312676 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a401409d0 0x7f6a400b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:31:31.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.833+0000 7f6a45197640 1 --2- 172.21.15.46:0/2125312676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a40140310 0x7f6a40156be0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f6a3000c7f0 tx=0x7f6a3000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:31.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.833+0000 7f6a367fc640 1 -- 172.21.15.46:0/2125312676 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a30004220 con 0x7f6a40140310 2023-12-02T00:31:31.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.834+0000 7f6a367fc640 1 -- 172.21.15.46:0/2125312676 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a30004380 con 0x7f6a40140310 2023-12-02T00:31:31.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.834+0000 7f6a47bfc640 1 -- 172.21.15.46:0/2125312676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a400ad950 con 0x7f6a40140310 2023-12-02T00:31:31.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.834+0000 7f6a367fc640 1 -- 172.21.15.46:0/2125312676 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a30007840 con 0x7f6a40140310 2023-12-02T00:31:31.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.834+0000 7f6a47bfc640 1 -- 172.21.15.46:0/2125312676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a400adf10 con 0x7f6a40140310 2023-12-02T00:31:31.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.834+0000 7f6a47bfc640 1 -- 172.21.15.46:0/2125312676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a40069400 con 0x7f6a40140310 2023-12-02T00:31:31.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.835+0000 7f6a367fc640 1 -- 172.21.15.46:0/2125312676 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6a30007bb0 con 0x7f6a40140310 2023-12-02T00:31:31.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.835+0000 7f6a367fc640 1 --2- 172.21.15.46:0/2125312676 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f6a2406ae40 0x7f6a2406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:31.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.835+0000 7f6a367fc640 1 -- 172.21.15.46:0/2125312676 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f6a30089410 con 0x7f6a40140310 2023-12-02T00:31:31.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.835+0000 7f6a45998640 1 --2- 172.21.15.46:0/2125312676 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f6a2406ae40 0x7f6a2406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:31.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.836+0000 7f6a45998640 1 --2- 172.21.15.46:0/2125312676 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f6a2406ae40 0x7f6a2406d300 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c000f60 tx=0x7f6a3c007d50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:31.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:31.840+0000 7f6a367fc640 1 -- 172.21.15.46:0/2125312676 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6a300545c0 con 0x7f6a40140310 2023-12-02T00:31:32.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.018+0000 7f6a47bfc640 1 -- 172.21.15.46:0/2125312676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6a4006ea60 con 0x7f6a40140310 2023-12-02T00:31:32.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.018+0000 7f6a367fc640 1 -- 172.21.15.46:0/2125312676 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v53) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6a30007e60 con 0x7f6a40140310 2023-12-02T00:31:32.020 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-12-02T00:31:32.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.020+0000 7f6a47bfc640 1 -- 172.21.15.46:0/2125312676 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f6a2406ae40 msgr2=0x7f6a2406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:32.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.020+0000 7f6a47bfc640 1 --2- 172.21.15.46:0/2125312676 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f6a2406ae40 0x7f6a2406d300 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c000f60 tx=0x7f6a3c007d50 comp rx=0 tx=0).stop 2023-12-02T00:31:32.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.020+0000 7f6a47bfc640 1 -- 172.21.15.46:0/2125312676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a40140310 msgr2=0x7f6a40156be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:32.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.020+0000 7f6a47bfc640 1 --2- 172.21.15.46:0/2125312676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a40140310 0x7f6a40156be0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f6a3000c7f0 tx=0x7f6a3000ccb0 comp rx=0 tx=0).stop 2023-12-02T00:31:32.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.020+0000 7f6a47bfc640 1 -- 172.21.15.46:0/2125312676 shutdown_connections 2023-12-02T00:31:32.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.020+0000 7f6a47bfc640 1 --2- 172.21.15.46:0/2125312676 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f6a2406ae40 0x7f6a2406d300 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:32.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.020+0000 7f6a47bfc640 1 --2- 172.21.15.46:0/2125312676 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a401409d0 0x7f6a400b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:32.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.020+0000 7f6a47bfc640 1 --2- 172.21.15.46:0/2125312676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a40140310 0x7f6a40156be0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:32.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.020+0000 7f6a47bfc640 1 --2- 172.21.15.46:0/2125312676 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a4006abe0 0x7f6a401f2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:32.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.020+0000 7f6a47bfc640 1 -- 172.21.15.46:0/2125312676 >> 172.21.15.46:0/2125312676 conn(0x7f6a400b1e50 msgr2=0x7f6a4015d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:32.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.020+0000 7f6a47bfc640 1 -- 172.21.15.46:0/2125312676 shutdown_connections 2023-12-02T00:31:32.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.020+0000 7f6a47bfc640 1 -- 172.21.15.46:0/2125312676 wait complete. 2023-12-02T00:31:32.032 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-02T00:31:32.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.142+0000 7ff6bdea2640 1 Processor -- start 2023-12-02T00:31:32.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.142+0000 7ff6bdea2640 1 -- start start 2023-12-02T00:31:32.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.143+0000 7ff6bdea2640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff6b80ceb70 0x7ff6b80cc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:32.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.143+0000 7ff6bdea2640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff6b80b4370 0x7ff6b80b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:32.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.143+0000 7ff6bdea2640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6b8071770 con 0x7ff6b8144980 2023-12-02T00:31:32.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.143+0000 7ff6bdea2640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6b815ced0 con 0x7ff6b80ceb70 2023-12-02T00:31:32.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.143+0000 7ff6bdea2640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6b80b4ce0 con 0x7ff6b80b4370 2023-12-02T00:31:32.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.143+0000 7ff6b6ffd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff6b80ceb70 0x7ff6b80cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:32.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.143+0000 7ff6b77fe640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff6b8144980 0x7ff6b815c7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49782/0 (socket says 172.21.15.46:49782) 2023-12-02T00:31:32.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.143+0000 7ff6b77fe640 1 -- 172.21.15.46:0/3332386181 learned_addr learned my addr 172.21.15.46:0/3332386181 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:31:32.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.143+0000 7ff6b7fff640 1 --2- 172.21.15.46:0/3332386181 >> v2:172.21.15.162:3300/0 conn(0x7ff6b80b4370 0x7ff6b80b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:32.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.143+0000 7ff6b6ffd640 1 -- 172.21.15.46:0/3332386181 >> v2:172.21.15.162:3300/0 conn(0x7ff6b80b4370 msgr2=0x7ff6b80b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:32.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.143+0000 7ff6b6ffd640 1 --2- 172.21.15.46:0/3332386181 >> v2:172.21.15.162:3300/0 conn(0x7ff6b80b4370 0x7ff6b80b47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:32.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.143+0000 7ff6b6ffd640 1 -- 172.21.15.46:0/3332386181 >> v1:172.21.15.46:6789/0 conn(0x7ff6b8144980 legacy=0x7ff6b815c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:32.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.143+0000 7ff6b6ffd640 1 -- 172.21.15.46:0/3332386181 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b80ccdc0 con 0x7ff6b80ceb70 2023-12-02T00:31:32.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.144+0000 7ff6b6ffd640 1 --2- 172.21.15.46:0/3332386181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6b80ceb70 0x7ff6b80cc450 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ff6b80716e0 tx=0x7ff6ac02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=722d6cccbfd0179f server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:32.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.144+0000 7ff6b67fc640 1 -- 172.21.15.46:0/3332386181 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac031830 con 0x7ff6b80ceb70 2023-12-02T00:31:32.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.144+0000 7ff6b67fc640 1 -- 172.21.15.46:0/3332386181 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6ac031990 con 0x7ff6b80ceb70 2023-12-02T00:31:32.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.144+0000 7ff6b67fc640 1 -- 172.21.15.46:0/3332386181 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac031d20 con 0x7ff6b80ceb70 2023-12-02T00:31:32.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.145+0000 7ff6bdea2640 1 -- 172.21.15.46:0/3332386181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6b80ceb70 msgr2=0x7ff6b80cc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:32.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.145+0000 7ff6bdea2640 1 --2- 172.21.15.46:0/3332386181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6b80ceb70 0x7ff6b80cc450 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ff6b80716e0 tx=0x7ff6ac02b8d0 comp rx=0 tx=0).stop 2023-12-02T00:31:32.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.145+0000 7ff6bdea2640 1 -- 172.21.15.46:0/3332386181 shutdown_connections 2023-12-02T00:31:32.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.145+0000 7ff6bdea2640 1 --2- 172.21.15.46:0/3332386181 >> v2:172.21.15.162:3300/0 conn(0x7ff6b80b4370 0x7ff6b80b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:32.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.145+0000 7ff6bdea2640 1 --2- 172.21.15.46:0/3332386181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6b80ceb70 0x7ff6b80cc450 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:32.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.145+0000 7ff6bdea2640 1 -- 172.21.15.46:0/3332386181 >> 172.21.15.46:0/3332386181 conn(0x7ff6b806a4d0 msgr2=0x7ff6b806a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:32.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.145+0000 7ff6bdea2640 1 -- 172.21.15.46:0/3332386181 shutdown_connections 2023-12-02T00:31:32.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.145+0000 7ff6bdea2640 1 -- 172.21.15.46:0/3332386181 wait complete. 2023-12-02T00:31:32.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.146+0000 7ff6bdea2640 1 Processor -- start 2023-12-02T00:31:32.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.146+0000 7ff6bdea2640 1 -- start start 2023-12-02T00:31:32.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.147+0000 7ff6bdea2640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6b80b4370 0x7ff6b8140cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:32.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.147+0000 7ff6bdea2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6b80ceb70 0x7ff6b80b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:32.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.147+0000 7ff6bdea2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6b8144980 0x7ff6b80b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:32.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.147+0000 7ff6bdea2640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b80c88b0 con 0x7ff6b80ceb70 2023-12-02T00:31:32.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.147+0000 7ff6bdea2640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b80b2990 con 0x7ff6b80b4370 2023-12-02T00:31:32.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.147+0000 7ff6bdea2640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b80b2c70 con 0x7ff6b8144980 2023-12-02T00:31:32.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.147+0000 7ff6b6ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6b80ceb70 0x7ff6b80b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:32.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.147+0000 7ff6b6ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6b80ceb70 0x7ff6b80b3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60408/0 (socket says 172.21.15.46:60408) 2023-12-02T00:31:32.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.147+0000 7ff6b6ffd640 1 -- 172.21.15.46:0/3567233993 learned_addr learned my addr 172.21.15.46:0/3567233993 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:32.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.147+0000 7ff6b77fe640 1 --2- 172.21.15.46:0/3567233993 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6b80b4370 0x7ff6b8140cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:32.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.147+0000 7ff6b7fff640 1 --2- 172.21.15.46:0/3567233993 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6b8144980 0x7ff6b80b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:32.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.147+0000 7ff6b6ffd640 1 -- 172.21.15.46:0/3567233993 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6b8144980 msgr2=0x7ff6b80b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:32.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.147+0000 7ff6b6ffd640 1 --2- 172.21.15.46:0/3567233993 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6b8144980 0x7ff6b80b2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:32.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.147+0000 7ff6b6ffd640 1 -- 172.21.15.46:0/3567233993 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6b80b4370 msgr2=0x7ff6b8140cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:32.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.147+0000 7ff6b6ffd640 1 --2- 172.21.15.46:0/3567233993 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6b80b4370 0x7ff6b8140cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:32.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.147+0000 7ff6b6ffd640 1 -- 172.21.15.46:0/3567233993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6ac009400 con 0x7ff6b80ceb70 2023-12-02T00:31:32.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.147+0000 7ff6b6ffd640 1 --2- 172.21.15.46:0/3567233993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6b80ceb70 0x7ff6b80b3d90 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac009890 tx=0x7ff6ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:32.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.148+0000 7ff69ffff640 1 -- 172.21.15.46:0/3567233993 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac03ba70 con 0x7ff6b80ceb70 2023-12-02T00:31:32.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.148+0000 7ff69ffff640 1 -- 172.21.15.46:0/3567233993 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6ac03bbf0 con 0x7ff6b80ceb70 2023-12-02T00:31:32.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.148+0000 7ff69ffff640 1 -- 172.21.15.46:0/3567233993 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac045400 con 0x7ff6b80ceb70 2023-12-02T00:31:32.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.148+0000 7ff6bdea2640 1 -- 172.21.15.46:0/3567233993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6b80b2f50 con 0x7ff6b80ceb70 2023-12-02T00:31:32.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.148+0000 7ff6bdea2640 1 -- 172.21.15.46:0/3567233993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6b80b26a0 con 0x7ff6b80ceb70 2023-12-02T00:31:32.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.149+0000 7ff6bdea2640 1 -- 172.21.15.46:0/3567233993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6b80c8d40 con 0x7ff6b80ceb70 2023-12-02T00:31:32.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.149+0000 7ff69ffff640 1 -- 172.21.15.46:0/3567233993 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff6ac04e440 con 0x7ff6b80ceb70 2023-12-02T00:31:32.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.149+0000 7ff69ffff640 1 --2- 172.21.15.46:0/3567233993 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff6a406ae40 0x7ff6a406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:32.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.150+0000 7ff69ffff640 1 -- 172.21.15.46:0/3567233993 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff6ac0ab870 con 0x7ff6b80ceb70 2023-12-02T00:31:32.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.150+0000 7ff6b77fe640 1 --2- 172.21.15.46:0/3567233993 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff6a406ae40 0x7ff6a406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:32.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.150+0000 7ff6b77fe640 1 --2- 172.21.15.46:0/3567233993 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff6a406ae40 0x7ff6a406d300 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8005880 tx=0x7ff6a8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:32.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.154+0000 7ff69ffff640 1 -- 172.21.15.46:0/3567233993 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff6b80c8d40 con 0x7ff6b80ceb70 2023-12-02T00:31:32.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.328+0000 7ff6bdea2640 1 -- 172.21.15.46:0/3567233993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff6b80c8d40 con 0x7ff6b80ceb70 2023-12-02T00:31:32.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.329+0000 7ff69ffff640 1 -- 172.21.15.46:0/3567233993 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v53) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff6ac035020 con 0x7ff6b80ceb70 2023-12-02T00:31:32.330 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-12-02T00:31:32.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.331+0000 7ff6bdea2640 1 -- 172.21.15.46:0/3567233993 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff6a406ae40 msgr2=0x7ff6a406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:32.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.331+0000 7ff6bdea2640 1 --2- 172.21.15.46:0/3567233993 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff6a406ae40 0x7ff6a406d300 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8005880 tx=0x7ff6a8009040 comp rx=0 tx=0).stop 2023-12-02T00:31:32.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.331+0000 7ff6bdea2640 1 -- 172.21.15.46:0/3567233993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6b80ceb70 msgr2=0x7ff6b80b3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:32.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.331+0000 7ff6bdea2640 1 --2- 172.21.15.46:0/3567233993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6b80ceb70 0x7ff6b80b3d90 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac009890 tx=0x7ff6ac0046c0 comp rx=0 tx=0).stop 2023-12-02T00:31:32.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.331+0000 7ff6bdea2640 1 -- 172.21.15.46:0/3567233993 shutdown_connections 2023-12-02T00:31:32.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.331+0000 7ff6bdea2640 1 --2- 172.21.15.46:0/3567233993 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7ff6a406ae40 0x7ff6a406d300 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:32.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.331+0000 7ff6bdea2640 1 --2- 172.21.15.46:0/3567233993 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6b8144980 0x7ff6b80b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:32.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.331+0000 7ff6bdea2640 1 --2- 172.21.15.46:0/3567233993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6b80ceb70 0x7ff6b80b3d90 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:32.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.331+0000 7ff6bdea2640 1 --2- 172.21.15.46:0/3567233993 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6b80b4370 0x7ff6b8140cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:32.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.331+0000 7ff6bdea2640 1 -- 172.21.15.46:0/3567233993 >> 172.21.15.46:0/3567233993 conn(0x7ff6b806a4d0 msgr2=0x7ff6b81590b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:32.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.331+0000 7ff6bdea2640 1 -- 172.21.15.46:0/3567233993 shutdown_connections 2023-12-02T00:31:32.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.331+0000 7ff6bdea2640 1 -- 172.21.15.46:0/3567233993 wait complete. 2023-12-02T00:31:32.343 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-02T00:31:32.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.450+0000 7fdc77e26640 1 Processor -- start 2023-12-02T00:31:32.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.450+0000 7fdc77e26640 1 -- start start 2023-12-02T00:31:32.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.451+0000 7fdc77e26640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fdc7006ea50 0x7fdc7006ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:32.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.451+0000 7fdc77e26640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdc70160fb0 0x7fdc70161440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:32.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.451+0000 7fdc77e26640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc70161d20 con 0x7fdc700b3ff0 2023-12-02T00:31:32.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.451+0000 7fdc77e26640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc700d1ee0 con 0x7fdc70160fb0 2023-12-02T00:31:32.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.451+0000 7fdc77e26640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc7006f370 con 0x7fdc7006ea50 2023-12-02T00:31:32.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.451+0000 7fdc753c1640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdc70160fb0 0x7fdc70161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:32.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.451+0000 7fdc763c3640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fdc700b3ff0 0x7fdc700b4460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49798/0 (socket says 172.21.15.46:49798) 2023-12-02T00:31:32.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.451+0000 7fdc763c3640 1 -- 172.21.15.46:0/577153149 learned_addr learned my addr 172.21.15.46:0/577153149 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:31:32.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.451+0000 7fdc753c1640 1 -- 172.21.15.46:0/577153149 >> v2:172.21.15.162:3300/0 conn(0x7fdc7006ea50 msgr2=0x7fdc7006ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:32.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.451+0000 7fdc75bc2640 1 --2- 172.21.15.46:0/577153149 >> v2:172.21.15.162:3300/0 conn(0x7fdc7006ea50 0x7fdc7006ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:32.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.451+0000 7fdc753c1640 1 --2- 172.21.15.46:0/577153149 >> v2:172.21.15.162:3300/0 conn(0x7fdc7006ea50 0x7fdc7006ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:32.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.451+0000 7fdc753c1640 1 -- 172.21.15.46:0/577153149 >> v1:172.21.15.46:6789/0 conn(0x7fdc700b3ff0 legacy=0x7fdc700b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:32.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.451+0000 7fdc753c1640 1 -- 172.21.15.46:0/577153149 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc7015e760 con 0x7fdc70160fb0 2023-12-02T00:31:32.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.451+0000 7fdc74bc0640 1 -- 172.21.15.46:0/577153149 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 508413555 0 0) 0x7fdc70161d20 con 0x7fdc700b3ff0 2023-12-02T00:31:32.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.451+0000 7fdc75bc2640 1 --2- 172.21.15.46:0/577153149 >> v2:172.21.15.162:3300/0 conn(0x7fdc7006ea50 0x7fdc7006ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:31:32.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.452+0000 7fdc753c1640 1 --2- 172.21.15.46:0/577153149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc70160fb0 0x7fdc70161440 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fdc7006f680 tx=0x7fdc6002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df95fe69628efcd4 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:32.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.452+0000 7fdc74bc0640 1 -- 172.21.15.46:0/577153149 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6002e070 con 0x7fdc70160fb0 2023-12-02T00:31:32.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.452+0000 7fdc74bc0640 1 -- 172.21.15.46:0/577153149 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc600315a0 con 0x7fdc70160fb0 2023-12-02T00:31:32.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.452+0000 7fdc74bc0640 1 -- 172.21.15.46:0/577153149 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc60031930 con 0x7fdc70160fb0 2023-12-02T00:31:32.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.453+0000 7fdc77e26640 1 -- 172.21.15.46:0/577153149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc70160fb0 msgr2=0x7fdc70161440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:32.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.453+0000 7fdc77e26640 1 --2- 172.21.15.46:0/577153149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc70160fb0 0x7fdc70161440 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fdc7006f680 tx=0x7fdc6002b840 comp rx=0 tx=0).stop 2023-12-02T00:31:32.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.453+0000 7fdc77e26640 1 -- 172.21.15.46:0/577153149 shutdown_connections 2023-12-02T00:31:32.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.453+0000 7fdc77e26640 1 --2- 172.21.15.46:0/577153149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc70160fb0 0x7fdc70161440 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:32.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.453+0000 7fdc77e26640 1 --2- 172.21.15.46:0/577153149 >> v2:172.21.15.162:3300/0 conn(0x7fdc7006ea50 0x7fdc7006ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:32.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.453+0000 7fdc77e26640 1 -- 172.21.15.46:0/577153149 >> 172.21.15.46:0/577153149 conn(0x7fdc7006a6b0 msgr2=0x7fdc7006aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:32.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.453+0000 7fdc77e26640 1 -- 172.21.15.46:0/577153149 shutdown_connections 2023-12-02T00:31:32.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.453+0000 7fdc77e26640 1 -- 172.21.15.46:0/577153149 wait complete. 2023-12-02T00:31:32.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.454+0000 7fdc77e26640 1 Processor -- start 2023-12-02T00:31:32.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.454+0000 7fdc77e26640 1 -- start start 2023-12-02T00:31:32.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.455+0000 7fdc77e26640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc7006ea50 0x7fdc7006cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:32.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.455+0000 7fdc77e26640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdc700b3ff0 0x7fdc7006c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:32.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.455+0000 7fdc77e26640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdc70160fb0 0x7fdc701e8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:32.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.455+0000 7fdc77e26640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc701e7440 con 0x7fdc7006ea50 2023-12-02T00:31:32.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.455+0000 7fdc77e26640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7006d250 con 0x7fdc70160fb0 2023-12-02T00:31:32.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.455+0000 7fdc77e26640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc701e7690 con 0x7fdc700b3ff0 2023-12-02T00:31:32.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.455+0000 7fdc75bc2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc7006ea50 0x7fdc7006cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:32.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.455+0000 7fdc75bc2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc7006ea50 0x7fdc7006cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60426/0 (socket says 172.21.15.46:60426) 2023-12-02T00:31:32.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.455+0000 7fdc75bc2640 1 -- 172.21.15.46:0/1675267282 learned_addr learned my addr 172.21.15.46:0/1675267282 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:32.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.455+0000 7fdc763c3640 1 --2- 172.21.15.46:0/1675267282 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdc70160fb0 0x7fdc701e8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:32.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.455+0000 7fdc753c1640 1 --2- 172.21.15.46:0/1675267282 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdc700b3ff0 0x7fdc7006c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:32.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.455+0000 7fdc75bc2640 1 -- 172.21.15.46:0/1675267282 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdc700b3ff0 msgr2=0x7fdc7006c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:32.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.455+0000 7fdc75bc2640 1 --2- 172.21.15.46:0/1675267282 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdc700b3ff0 0x7fdc7006c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:32.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.455+0000 7fdc75bc2640 1 -- 172.21.15.46:0/1675267282 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdc70160fb0 msgr2=0x7fdc701e8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:32.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.455+0000 7fdc75bc2640 1 --2- 172.21.15.46:0/1675267282 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdc70160fb0 0x7fdc701e8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:32.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.455+0000 7fdc75bc2640 1 -- 172.21.15.46:0/1675267282 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc60009400 con 0x7fdc7006ea50 2023-12-02T00:31:32.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.455+0000 7fdc753c1640 1 --2- 172.21.15.46:0/1675267282 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdc700b3ff0 0x7fdc7006c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:31:32.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.455+0000 7fdc75bc2640 1 --2- 172.21.15.46:0/1675267282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc7006ea50 0x7fdc7006cc00 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c00c7f0 tx=0x7fdc6c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:32.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.455+0000 7fdc667fc640 1 -- 172.21.15.46:0/1675267282 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c007820 con 0x7fdc7006ea50 2023-12-02T00:31:32.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.455+0000 7fdc667fc640 1 -- 172.21.15.46:0/1675267282 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc6c007980 con 0x7fdc7006ea50 2023-12-02T00:31:32.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.455+0000 7fdc667fc640 1 -- 172.21.15.46:0/1675267282 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c007cf0 con 0x7fdc7006ea50 2023-12-02T00:31:32.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.456+0000 7fdc77e26640 1 -- 172.21.15.46:0/1675267282 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc701e7970 con 0x7fdc7006ea50 2023-12-02T00:31:32.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.456+0000 7fdc77e26640 1 -- 172.21.15.46:0/1675267282 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc70156520 con 0x7fdc7006ea50 2023-12-02T00:31:32.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.457+0000 7fdc77e26640 1 -- 172.21.15.46:0/1675267282 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc700b5310 con 0x7fdc7006ea50 2023-12-02T00:31:32.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.457+0000 7fdc667fc640 1 -- 172.21.15.46:0/1675267282 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdc6c0113f0 con 0x7fdc7006ea50 2023-12-02T00:31:32.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.457+0000 7fdc667fc640 1 --2- 172.21.15.46:0/1675267282 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fdc5406ae70 0x7fdc5406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:32.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.457+0000 7fdc667fc640 1 -- 172.21.15.46:0/1675267282 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fdc6c089970 con 0x7fdc7006ea50 2023-12-02T00:31:32.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.458+0000 7fdc753c1640 1 --2- 172.21.15.46:0/1675267282 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fdc5406ae70 0x7fdc5406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:32.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.458+0000 7fdc753c1640 1 --2- 172.21.15.46:0/1675267282 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fdc5406ae70 0x7fdc5406d330 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fdc60036000 tx=0x7fdc6003d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:32.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.462+0000 7fdc667fc640 1 -- 172.21.15.46:0/1675267282 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdc6c0549e0 con 0x7fdc7006ea50 2023-12-02T00:31:32.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.649+0000 7fdc77e26640 1 -- 172.21.15.46:0/1675267282 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fdc70069400 con 0x7fdc7006ea50 2023-12-02T00:31:32.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.650+0000 7fdc667fc640 1 -- 172.21.15.46:0/1675267282 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v53) v1 ==== 80+0+0 (secure 0 0 0) 0x7fdc6c01f080 con 0x7fdc7006ea50 2023-12-02T00:31:32.651 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2023-12-02T00:31:32.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.651+0000 7fdc77e26640 1 -- 172.21.15.46:0/1675267282 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fdc5406ae70 msgr2=0x7fdc5406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:32.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.651+0000 7fdc77e26640 1 --2- 172.21.15.46:0/1675267282 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fdc5406ae70 0x7fdc5406d330 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fdc60036000 tx=0x7fdc6003d040 comp rx=0 tx=0).stop 2023-12-02T00:31:32.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.651+0000 7fdc77e26640 1 -- 172.21.15.46:0/1675267282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc7006ea50 msgr2=0x7fdc7006cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:32.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.651+0000 7fdc77e26640 1 --2- 172.21.15.46:0/1675267282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc7006ea50 0x7fdc7006cc00 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c00c7f0 tx=0x7fdc6c00ccb0 comp rx=0 tx=0).stop 2023-12-02T00:31:32.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.651+0000 7fdc77e26640 1 -- 172.21.15.46:0/1675267282 shutdown_connections 2023-12-02T00:31:32.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.651+0000 7fdc77e26640 1 --2- 172.21.15.46:0/1675267282 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fdc5406ae70 0x7fdc5406d330 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:32.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.651+0000 7fdc77e26640 1 --2- 172.21.15.46:0/1675267282 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdc70160fb0 0x7fdc701e8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:32.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.651+0000 7fdc77e26640 1 --2- 172.21.15.46:0/1675267282 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdc700b3ff0 0x7fdc7006c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:32.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.651+0000 7fdc77e26640 1 --2- 172.21.15.46:0/1675267282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc7006ea50 0x7fdc7006cc00 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:32.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.651+0000 7fdc77e26640 1 -- 172.21.15.46:0/1675267282 >> 172.21.15.46:0/1675267282 conn(0x7fdc7006a6b0 msgr2=0x7fdc7013e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:32.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.651+0000 7fdc77e26640 1 -- 172.21.15.46:0/1675267282 shutdown_connections 2023-12-02T00:31:32.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:32.652+0000 7fdc77e26640 1 -- 172.21.15.46:0/1675267282 wait complete. 2023-12-02T00:31:32.663 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi046.front.sepia.ceph.com... 2023-12-02T00:31:32.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:31:32.663 DEBUG:teuthology.orchestra.run.smithi046:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-12-02T00:31:32.692 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[111990]: fuse finished with error 0 and tester_r 0 2023-12-02T00:31:32.694 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2023-12-02T00:31:32.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:31:32.694 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-02T00:31:32.813 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-12-02T00:31:32.814 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-02T00:31:32.833 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056379,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497728k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15716 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248860k,nr_inodes=812215,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497728k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-02T00:31:32.833 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi162.front.sepia.ceph.com 2023-12-02T00:31:32.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:31:32.833 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-12-02T00:31:32.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-02T00:31:32.892 INFO:teuthology.orchestra.run.smithi162.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-12-02T00:31:32.892 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-02T00:31:32.892 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-02T00:31:32.909 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057150,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498968k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19546 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249480k,nr_inodes=812370,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498968k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-02T00:31:32.909 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2023-12-02T00:31:32.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:31:32.909 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-12-02T00:31:32.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-02T00:31:32.964 INFO:teuthology.orchestra.run.smithi046.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-12-02T00:31:32.965 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-12-02T00:31:32.965 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-02T00:31:32.983 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056379,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497728k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15716 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248860k,nr_inodes=812215,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497728k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-02T00:31:32.984 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi162.front.sepia.ceph.com 2023-12-02T00:31:32.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:31:32.984 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-12-02T00:31:33.041 INFO:teuthology.orchestra.run.smithi162.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-12-02T00:31:33.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-02T00:31:33.042 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-02T00:31:33.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.147+0000 7f44b7d68640 1 Processor -- start 2023-12-02T00:31:33.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.147+0000 7f44b7d68640 1 -- start start 2023-12-02T00:31:33.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.148+0000 7f44b7d68640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f44b00ad640 0x7f44b006d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:33.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.148+0000 7f44b7d68640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f44b006bbf0 0x7f44b006d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:33.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.148+0000 7f44b7d68640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44b015d720 con 0x7f44b015d340 2023-12-02T00:31:33.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.148+0000 7f44b7d68640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44b00adae0 con 0x7f44b006bbf0 2023-12-02T00:31:33.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.148+0000 7f44b7d68640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44b00addc0 con 0x7f44b00ad640 2023-12-02T00:31:33.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.148+0000 7f44b6305640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f44b006bbf0 0x7f44b006d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:33.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.148+0000 7f44b6305640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f44b006bbf0 0x7f44b006d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44958/0 (socket says 172.21.15.46:44958) 2023-12-02T00:31:33.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.148+0000 7f44b6305640 1 -- 172.21.15.46:0/555393791 learned_addr learned my addr 172.21.15.46:0/555393791 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:33.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.149+0000 7f44b5303640 1 --2- 172.21.15.46:0/555393791 >> v2:172.21.15.162:3300/0 conn(0x7f44b00ad640 0x7f44b006d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:33.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.149+0000 7f44b6305640 1 -- 172.21.15.46:0/555393791 >> v2:172.21.15.162:3300/0 conn(0x7f44b00ad640 msgr2=0x7f44b006d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:33.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.149+0000 7f44b6305640 1 --2- 172.21.15.46:0/555393791 >> v2:172.21.15.162:3300/0 conn(0x7f44b00ad640 0x7f44b006d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:33.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.149+0000 7f44b6305640 1 -- 172.21.15.46:0/555393791 >> v1:172.21.15.162:6789/0 conn(0x7f44b015d340 legacy=0x7f44b006b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:33.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.149+0000 7f44b6305640 1 -- 172.21.15.46:0/555393791 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b00ae0a0 con 0x7f44b006bbf0 2023-12-02T00:31:33.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.149+0000 7f44b5303640 1 --2- 172.21.15.46:0/555393791 >> v2:172.21.15.162:3300/0 conn(0x7f44b00ad640 0x7f44b006d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:31:33.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.149+0000 7f44b6305640 1 --2- 172.21.15.46:0/555393791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44b006bbf0 0x7f44b006d5f0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f44b0072310 tx=0x7f44a802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dcf197a17e97a54 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:33.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.150+0000 7f44b4b02640 1 -- 172.21.15.46:0/555393791 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a8031870 con 0x7f44b006bbf0 2023-12-02T00:31:33.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.150+0000 7f44b4b02640 1 -- 172.21.15.46:0/555393791 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44a80319d0 con 0x7f44b006bbf0 2023-12-02T00:31:33.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.150+0000 7f44b4b02640 1 -- 172.21.15.46:0/555393791 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a8031d60 con 0x7f44b006bbf0 2023-12-02T00:31:33.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.150+0000 7f44b7d68640 1 -- 172.21.15.46:0/555393791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44b006bbf0 msgr2=0x7f44b006d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:33.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.150+0000 7f44b7d68640 1 --2- 172.21.15.46:0/555393791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44b006bbf0 0x7f44b006d5f0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f44b0072310 tx=0x7f44a802b910 comp rx=0 tx=0).stop 2023-12-02T00:31:33.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.150+0000 7f44b7d68640 1 -- 172.21.15.46:0/555393791 shutdown_connections 2023-12-02T00:31:33.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.150+0000 7f44b7d68640 1 --2- 172.21.15.46:0/555393791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44b006bbf0 0x7f44b006d5f0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:33.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.150+0000 7f44b7d68640 1 --2- 172.21.15.46:0/555393791 >> v2:172.21.15.162:3300/0 conn(0x7f44b00ad640 0x7f44b006d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:33.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.150+0000 7f44b7d68640 1 -- 172.21.15.46:0/555393791 >> 172.21.15.46:0/555393791 conn(0x7f44b0070e70 msgr2=0x7f44b0071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:33.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.150+0000 7f44b7d68640 1 -- 172.21.15.46:0/555393791 shutdown_connections 2023-12-02T00:31:33.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.150+0000 7f44b7d68640 1 -- 172.21.15.46:0/555393791 wait complete. 2023-12-02T00:31:33.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.151+0000 7f44b7d68640 1 Processor -- start 2023-12-02T00:31:33.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.151+0000 7f44b7d68640 1 -- start start 2023-12-02T00:31:33.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.152+0000 7f44b7d68640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44b006bbf0 0x7f44b0160130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:33.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.152+0000 7f44b7d68640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f44b00ad640 0x7f44b01565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:33.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.152+0000 7f44b7d68640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44b015d340 0x7f44b01f6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:33.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.152+0000 7f44b7d68640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b0157450 con 0x7f44b015d340 2023-12-02T00:31:33.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.152+0000 7f44b7d68640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b01541f0 con 0x7f44b006bbf0 2023-12-02T00:31:33.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.152+0000 7f44b7d68640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44b01576c0 con 0x7f44b00ad640 2023-12-02T00:31:33.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.152+0000 7f44b6305640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44b015d340 0x7f44b01f6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:33.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.152+0000 7f44b6305640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44b015d340 0x7f44b01f6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44974/0 (socket says 172.21.15.46:44974) 2023-12-02T00:31:33.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.152+0000 7f44b6305640 1 -- 172.21.15.46:0/1895079813 learned_addr learned my addr 172.21.15.46:0/1895079813 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:33.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.152+0000 7f44b5303640 1 --2- 172.21.15.46:0/1895079813 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f44b00ad640 0x7f44b01565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:33.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.152+0000 7f44b5b04640 1 --2- 172.21.15.46:0/1895079813 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44b006bbf0 0x7f44b0160130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:33.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.153+0000 7f44b6305640 1 -- 172.21.15.46:0/1895079813 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f44b00ad640 msgr2=0x7f44b01565b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:33.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.153+0000 7f44b6305640 1 --2- 172.21.15.46:0/1895079813 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f44b00ad640 0x7f44b01565b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:33.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.153+0000 7f44b6305640 1 -- 172.21.15.46:0/1895079813 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44b006bbf0 msgr2=0x7f44b0160130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:33.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.153+0000 7f44b6305640 1 --2- 172.21.15.46:0/1895079813 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44b006bbf0 0x7f44b0160130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:33.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.153+0000 7f44b6305640 1 -- 172.21.15.46:0/1895079813 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a80094d0 con 0x7f44b015d340 2023-12-02T00:31:33.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.153+0000 7f44b5b04640 1 --2- 172.21.15.46:0/1895079813 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44b006bbf0 0x7f44b0160130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:31:33.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.153+0000 7f44b5303640 1 --2- 172.21.15.46:0/1895079813 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f44b00ad640 0x7f44b01565b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:31:33.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.153+0000 7f44b6305640 1 --2- 172.21.15.46:0/1895079813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44b015d340 0x7f44b01f6800 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f44a802b950 tx=0x7f44a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:33.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.153+0000 7f44ae7fc640 1 -- 172.21.15.46:0/1895079813 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a80443f0 con 0x7f44b015d340 2023-12-02T00:31:33.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.153+0000 7f44ae7fc640 1 -- 172.21.15.46:0/1895079813 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44a8044570 con 0x7f44b015d340 2023-12-02T00:31:33.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.153+0000 7f44b7d68640 1 -- 172.21.15.46:0/1895079813 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44b01579a0 con 0x7f44b015d340 2023-12-02T00:31:33.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.153+0000 7f44ae7fc640 1 -- 172.21.15.46:0/1895079813 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a8044920 con 0x7f44b015d340 2023-12-02T00:31:33.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.153+0000 7f44b7d68640 1 -- 172.21.15.46:0/1895079813 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44b0153e30 con 0x7f44b015d340 2023-12-02T00:31:33.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.154+0000 7f44b7d68640 1 -- 172.21.15.46:0/1895079813 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44b0157bb0 con 0x7f44b015d340 2023-12-02T00:31:33.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.154+0000 7f44ae7fc640 1 -- 172.21.15.46:0/1895079813 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f44a8044a80 con 0x7f44b015d340 2023-12-02T00:31:33.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.155+0000 7f44ae7fc640 1 --2- 172.21.15.46:0/1895079813 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f449c06ae40 0x7f449c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:33.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.155+0000 7f44ae7fc640 1 -- 172.21.15.46:0/1895079813 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f44a802d040 con 0x7f44b015d340 2023-12-02T00:31:33.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.155+0000 7f44b5b04640 1 --2- 172.21.15.46:0/1895079813 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f449c06ae40 0x7f449c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:33.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.156+0000 7f44b5b04640 1 --2- 172.21.15.46:0/1895079813 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f449c06ae40 0x7f449c06d300 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f44a0009fc0 tx=0x7f44a0009550 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:33.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.159+0000 7f44ae7fc640 1 -- 172.21.15.46:0/1895079813 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f44a8076ef0 con 0x7f44b015d340 2023-12-02T00:31:33.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.371+0000 7f44b7d68640 1 -- 172.21.15.46:0/1895079813 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f44b0069400 con 0x7f44b015d340 2023-12-02T00:31:33.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.371+0000 7f44ae7fc640 1 -- 172.21.15.46:0/1895079813 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f44a807a510 con 0x7f44b015d340 2023-12-02T00:31:33.372 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:31:33.373 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5189,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/4047806587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":4047806587},{"type":"v1","addr":"172.21.15.162:6837","nonce":4047806587}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5620,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/1418783715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1418783715},{"type":"v1","addr":"172.21.15.162:6836","nonce":1418783715}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5979,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/452915985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":452915985},{"type":"v1","addr":"172.21.15.46:6833","nonce":452915985}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-02T00:31:01.187535+0000","modified":"2023-12-02T00:31:04.211234+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5994":{"gid":5994,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/903694874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":903694874},{"type":"v1","addr":"172.21.15.46:6835","nonce":903694874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-12-02T00:31:33.373 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 24 2023-12-02T00:31:33.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.373+0000 7f44b7d68640 1 -- 172.21.15.46:0/1895079813 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f449c06ae40 msgr2=0x7f449c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:33.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.373+0000 7f44b7d68640 1 --2- 172.21.15.46:0/1895079813 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f449c06ae40 0x7f449c06d300 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f44a0009fc0 tx=0x7f44a0009550 comp rx=0 tx=0).stop 2023-12-02T00:31:33.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.373+0000 7f44b7d68640 1 -- 172.21.15.46:0/1895079813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44b015d340 msgr2=0x7f44b01f6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:33.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.373+0000 7f44b7d68640 1 --2- 172.21.15.46:0/1895079813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44b015d340 0x7f44b01f6800 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f44a802b950 tx=0x7f44a80046c0 comp rx=0 tx=0).stop 2023-12-02T00:31:33.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.373+0000 7f44b7d68640 1 -- 172.21.15.46:0/1895079813 shutdown_connections 2023-12-02T00:31:33.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.373+0000 7f44b7d68640 1 --2- 172.21.15.46:0/1895079813 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f449c06ae40 0x7f449c06d300 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:33.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.373+0000 7f44b7d68640 1 --2- 172.21.15.46:0/1895079813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44b015d340 0x7f44b01f6800 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:33.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.373+0000 7f44b7d68640 1 --2- 172.21.15.46:0/1895079813 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f44b00ad640 0x7f44b01565b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:33.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.373+0000 7f44b7d68640 1 --2- 172.21.15.46:0/1895079813 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44b006bbf0 0x7f44b0160130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:33.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.373+0000 7f44b7d68640 1 -- 172.21.15.46:0/1895079813 >> 172.21.15.46:0/1895079813 conn(0x7f44b0070e70 msgr2=0x7f44b00b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:33.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.373+0000 7f44b7d68640 1 -- 172.21.15.46:0/1895079813 shutdown_connections 2023-12-02T00:31:33.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.373+0000 7f44b7d68640 1 -- 172.21.15.46:0/1895079813 wait complete. 2023-12-02T00:31:33.386 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:31:33.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.492+0000 7f1826487640 1 Processor -- start 2023-12-02T00:31:33.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.492+0000 7f1826487640 1 -- start start 2023-12-02T00:31:33.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.493+0000 7f1826487640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f182006abe0 0x7f18201618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:33.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.493+0000 7f1826487640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f18201409d0 0x7f182006efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:33.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.493+0000 7f1826487640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18201406f0 con 0x7f1820140310 2023-12-02T00:31:33.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.493+0000 7f1826487640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1820161e20 con 0x7f182006abe0 2023-12-02T00:31:33.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.493+0000 7f1826487640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1820143960 con 0x7f18201409d0 2023-12-02T00:31:33.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.493+0000 7f181f7fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f182006abe0 0x7f18201618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:33.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.493+0000 7f181ffff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1820140310 0x7f182006a4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50800/0 (socket says 172.21.15.46:50800) 2023-12-02T00:31:33.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.493+0000 7f181f7fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f182006abe0 0x7f18201618e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44986/0 (socket says 172.21.15.46:44986) 2023-12-02T00:31:33.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.493+0000 7f181ffff640 1 -- 172.21.15.46:0/3695543422 learned_addr learned my addr 172.21.15.46:0/3695543422 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:31:33.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.493+0000 7f1824a24640 1 --2- 172.21.15.46:0/3695543422 >> v2:172.21.15.162:3300/0 conn(0x7f18201409d0 0x7f182006efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:33.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.493+0000 7f181f7fe640 1 -- 172.21.15.46:0/3695543422 >> v2:172.21.15.162:3300/0 conn(0x7f18201409d0 msgr2=0x7f182006efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:33.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.493+0000 7f181f7fe640 1 --2- 172.21.15.46:0/3695543422 >> v2:172.21.15.162:3300/0 conn(0x7f18201409d0 0x7f182006efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:33.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.493+0000 7f181effd640 1 -- 172.21.15.46:0/3695543422 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1967580153 0 0) 0x7f18201406f0 con 0x7f1820140310 2023-12-02T00:31:33.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.493+0000 7f181f7fe640 1 -- 172.21.15.46:0/3695543422 >> v1:172.21.15.46:6789/0 conn(0x7f1820140310 legacy=0x7f182006a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:33.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.493+0000 7f181f7fe640 1 -- 172.21.15.46:0/3695543422 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1820142de0 con 0x7f182006abe0 2023-12-02T00:31:33.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.494+0000 7f181f7fe640 1 --2- 172.21.15.46:0/3695543422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f182006abe0 0x7f18201618e0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f1820154960 tx=0x7f181002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0d2306c4b8c997e server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:33.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.494+0000 7f181effd640 1 -- 172.21.15.46:0/3695543422 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181002e070 con 0x7f182006abe0 2023-12-02T00:31:33.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.494+0000 7f181effd640 1 -- 172.21.15.46:0/3695543422 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18100315a0 con 0x7f182006abe0 2023-12-02T00:31:33.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.494+0000 7f181effd640 1 -- 172.21.15.46:0/3695543422 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1810031930 con 0x7f182006abe0 2023-12-02T00:31:33.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.495+0000 7f1826487640 1 -- 172.21.15.46:0/3695543422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f182006abe0 msgr2=0x7f18201618e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:33.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.495+0000 7f1826487640 1 --2- 172.21.15.46:0/3695543422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f182006abe0 0x7f18201618e0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f1820154960 tx=0x7f181002b840 comp rx=0 tx=0).stop 2023-12-02T00:31:33.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.495+0000 7f1826487640 1 -- 172.21.15.46:0/3695543422 shutdown_connections 2023-12-02T00:31:33.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.495+0000 7f1826487640 1 --2- 172.21.15.46:0/3695543422 >> v2:172.21.15.162:3300/0 conn(0x7f18201409d0 0x7f182006efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:33.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.495+0000 7f1826487640 1 --2- 172.21.15.46:0/3695543422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f182006abe0 0x7f18201618e0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:33.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.495+0000 7f1826487640 1 -- 172.21.15.46:0/3695543422 >> 172.21.15.46:0/3695543422 conn(0x7f18200b1e50 msgr2=0x7f18200b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:33.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.495+0000 7f1826487640 1 -- 172.21.15.46:0/3695543422 shutdown_connections 2023-12-02T00:31:33.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.495+0000 7f1826487640 1 -- 172.21.15.46:0/3695543422 wait complete. 2023-12-02T00:31:33.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.496+0000 7f1826487640 1 Processor -- start 2023-12-02T00:31:33.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.496+0000 7f1826487640 1 -- start start 2023-12-02T00:31:33.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.497+0000 7f1826487640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f182006abe0 0x7f18201f7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:33.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.497+0000 7f1826487640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1820140310 0x7f18200b4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:33.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.497+0000 7f1826487640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18201409d0 0x7f18200b2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:33.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.497+0000 7f1826487640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18200b4410 con 0x7f18201409d0 2023-12-02T00:31:33.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.497+0000 7f1826487640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18200b3790 con 0x7f1820140310 2023-12-02T00:31:33.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.497+0000 7f1826487640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18200b3a70 con 0x7f182006abe0 2023-12-02T00:31:33.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.497+0000 7f1824a24640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18201409d0 0x7f18200b2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:33.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.497+0000 7f1824a24640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18201409d0 0x7f18200b2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45000/0 (socket says 172.21.15.46:45000) 2023-12-02T00:31:33.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.497+0000 7f1824a24640 1 -- 172.21.15.46:0/1620033970 learned_addr learned my addr 172.21.15.46:0/1620033970 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:33.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.497+0000 7f181ffff640 1 --2- 172.21.15.46:0/1620033970 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f182006abe0 0x7f18201f7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:33.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.497+0000 7f181f7fe640 1 --2- 172.21.15.46:0/1620033970 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1820140310 0x7f18200b4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:33.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.497+0000 7f1824a24640 1 -- 172.21.15.46:0/1620033970 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f182006abe0 msgr2=0x7f18201f7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:33.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.497+0000 7f1824a24640 1 --2- 172.21.15.46:0/1620033970 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f182006abe0 0x7f18201f7000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:33.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.497+0000 7f1824a24640 1 -- 172.21.15.46:0/1620033970 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1820140310 msgr2=0x7f18200b4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:33.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.497+0000 7f1824a24640 1 --2- 172.21.15.46:0/1620033970 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1820140310 0x7f18200b4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:33.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.497+0000 7f1824a24640 1 -- 172.21.15.46:0/1620033970 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1810009400 con 0x7f18201409d0 2023-12-02T00:31:33.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.497+0000 7f1824a24640 1 --2- 172.21.15.46:0/1620033970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18201409d0 0x7f18200b2f40 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f181800cd70 tx=0x7f181800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:33.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.497+0000 7f181cff9640 1 -- 172.21.15.46:0/1620033970 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1818003b90 con 0x7f18201409d0 2023-12-02T00:31:33.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.498+0000 7f181cff9640 1 -- 172.21.15.46:0/1620033970 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1818003cf0 con 0x7f18201409d0 2023-12-02T00:31:33.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.498+0000 7f1826487640 1 -- 172.21.15.46:0/1620033970 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18200b3d50 con 0x7f18201409d0 2023-12-02T00:31:33.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.498+0000 7f181cff9640 1 -- 172.21.15.46:0/1620033970 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1818004240 con 0x7f18201409d0 2023-12-02T00:31:33.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.498+0000 7f1826487640 1 -- 172.21.15.46:0/1620033970 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18200b3480 con 0x7f18201409d0 2023-12-02T00:31:33.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.499+0000 7f1826487640 1 -- 172.21.15.46:0/1620033970 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18200b4550 con 0x7f18201409d0 2023-12-02T00:31:33.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.499+0000 7f181cff9640 1 -- 172.21.15.46:0/1620033970 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f18180043a0 con 0x7f18201409d0 2023-12-02T00:31:33.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.499+0000 7f181cff9640 1 --2- 172.21.15.46:0/1620033970 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f180c06ae40 0x7f180c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:33.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.499+0000 7f181cff9640 1 -- 172.21.15.46:0/1620033970 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f1818012070 con 0x7f18201409d0 2023-12-02T00:31:33.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.500+0000 7f181ffff640 1 --2- 172.21.15.46:0/1620033970 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f180c06ae40 0x7f180c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:33.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.500+0000 7f181ffff640 1 --2- 172.21.15.46:0/1620033970 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f180c06ae40 0x7f180c06d300 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1814007fd0 tx=0x7f1814009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:33.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.503+0000 7f181cff9640 1 -- 172.21.15.46:0/1620033970 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f18200b4550 con 0x7f18201409d0 2023-12-02T00:31:33.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.681+0000 7f1826487640 1 -- 172.21.15.46:0/1620033970 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f18200b4550 con 0x7f18201409d0 2023-12-02T00:31:33.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.682+0000 7f181cff9640 1 -- 172.21.15.46:0/1620033970 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+16585 (secure 0 0 0) 0x7f18200b4550 con 0x7f18201409d0 2023-12-02T00:31:33.684 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:31:33.684 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":53,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:31:31.941814+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-12-02T00:30:59.469070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-12-02T00:31:00.174610+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.28","mappings":[{"from":0,"to":6}]},{"pgid":"8.38","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:31:33.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.684+0000 7f1826487640 1 -- 172.21.15.46:0/1620033970 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f180c06ae40 msgr2=0x7f180c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:33.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.684+0000 7f1826487640 1 --2- 172.21.15.46:0/1620033970 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f180c06ae40 0x7f180c06d300 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1814007fd0 tx=0x7f1814009040 comp rx=0 tx=0).stop 2023-12-02T00:31:33.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.684+0000 7f1826487640 1 -- 172.21.15.46:0/1620033970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18201409d0 msgr2=0x7f18200b2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:33.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.684+0000 7f1826487640 1 --2- 172.21.15.46:0/1620033970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18201409d0 0x7f18200b2f40 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f181800cd70 tx=0x7f181800a660 comp rx=0 tx=0).stop 2023-12-02T00:31:33.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.684+0000 7f1826487640 1 -- 172.21.15.46:0/1620033970 shutdown_connections 2023-12-02T00:31:33.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.684+0000 7f1826487640 1 --2- 172.21.15.46:0/1620033970 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f180c06ae40 0x7f180c06d300 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:33.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.684+0000 7f1826487640 1 --2- 172.21.15.46:0/1620033970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18201409d0 0x7f18200b2f40 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:33.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.684+0000 7f1826487640 1 --2- 172.21.15.46:0/1620033970 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1820140310 0x7f18200b4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:33.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.684+0000 7f1826487640 1 --2- 172.21.15.46:0/1620033970 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f182006abe0 0x7f18201f7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:33.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.684+0000 7f1826487640 1 -- 172.21.15.46:0/1620033970 >> 172.21.15.46:0/1620033970 conn(0x7f18200b1e50 msgr2=0x7f182006d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:33.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.684+0000 7f1826487640 1 -- 172.21.15.46:0/1620033970 shutdown_connections 2023-12-02T00:31:33.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.684+0000 7f1826487640 1 -- 172.21.15.46:0/1620033970 wait complete. 2023-12-02T00:31:33.697 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-02T00:28:37.213645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-02T00:30:59.469070+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-12-02T00:31:00.174610+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-02T00:31:33.697 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-02T00:31:33.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.806+0000 7f672ba15640 1 Processor -- start 2023-12-02T00:31:33.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.806+0000 7f672ba15640 1 -- start start 2023-12-02T00:31:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.806+0000 7f672ba15640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f67240ad640 0x7f672406d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.807+0000 7f672ba15640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f672406bbf0 0x7f672406d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.807+0000 7f672ba15640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67240adae0 con 0x7f672415d1d0 2023-12-02T00:31:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.807+0000 7f672ba15640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67240addc0 con 0x7f67240ad640 2023-12-02T00:31:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.807+0000 7f672ba15640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67240ae0a0 con 0x7f672406bbf0 2023-12-02T00:31:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.807+0000 7f6728fb0640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f67240ad640 0x7f672406d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.807+0000 7f67297b1640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f672415d1d0 0x7f672406b4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50804/0 (socket says 172.21.15.46:50804) 2023-12-02T00:31:33.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.807+0000 7f6728fb0640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f67240ad640 0x7f672406d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45016/0 (socket says 172.21.15.46:45016) 2023-12-02T00:31:33.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.807+0000 7f67297b1640 1 -- 172.21.15.46:0/4252878873 learned_addr learned my addr 172.21.15.46:0/4252878873 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:31:33.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.807+0000 7f6729fb2640 1 --2- 172.21.15.46:0/4252878873 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f672406bbf0 0x7f672406d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:33.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.807+0000 7f6728fb0640 1 -- 172.21.15.46:0/4252878873 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f672406bbf0 msgr2=0x7f672406d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:33.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.807+0000 7f6728fb0640 1 --2- 172.21.15.46:0/4252878873 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f672406bbf0 0x7f672406d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:33.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.807+0000 7f6728fb0640 1 -- 172.21.15.46:0/4252878873 >> v1:172.21.15.46:6789/0 conn(0x7f672415d1d0 legacy=0x7f672406b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:33.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.807+0000 7f6728fb0640 1 -- 172.21.15.46:0/4252878873 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67240ae380 con 0x7f67240ad640 2023-12-02T00:31:33.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.807+0000 7f671bfff640 1 -- 172.21.15.46:0/4252878873 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3746164953 0 0) 0x7f67240adae0 con 0x7f672415d1d0 2023-12-02T00:31:33.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.807+0000 7f6729fb2640 1 --2- 172.21.15.46:0/4252878873 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f672406bbf0 0x7f672406d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:31:33.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.808+0000 7f6728fb0640 1 --2- 172.21.15.46:0/4252878873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67240ad640 0x7f672406d0b0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f67140097a0 tx=0x7f671402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f55fb9efe9d55d34 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:33.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.808+0000 7f671bfff640 1 -- 172.21.15.46:0/4252878873 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6714031830 con 0x7f67240ad640 2023-12-02T00:31:33.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.808+0000 7f671bfff640 1 -- 172.21.15.46:0/4252878873 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6714031990 con 0x7f67240ad640 2023-12-02T00:31:33.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.808+0000 7f671bfff640 1 -- 172.21.15.46:0/4252878873 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6714031d20 con 0x7f67240ad640 2023-12-02T00:31:33.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.808+0000 7f672ba15640 1 -- 172.21.15.46:0/4252878873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67240ad640 msgr2=0x7f672406d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:33.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.808+0000 7f672ba15640 1 --2- 172.21.15.46:0/4252878873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67240ad640 0x7f672406d0b0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f67140097a0 tx=0x7f671402b8d0 comp rx=0 tx=0).stop 2023-12-02T00:31:33.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.808+0000 7f672ba15640 1 -- 172.21.15.46:0/4252878873 shutdown_connections 2023-12-02T00:31:33.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.808+0000 7f672ba15640 1 --2- 172.21.15.46:0/4252878873 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f672406bbf0 0x7f672406d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:33.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.808+0000 7f672ba15640 1 --2- 172.21.15.46:0/4252878873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67240ad640 0x7f672406d0b0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:33.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.809+0000 7f672ba15640 1 -- 172.21.15.46:0/4252878873 >> 172.21.15.46:0/4252878873 conn(0x7f6724070e70 msgr2=0x7f6724071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:33.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.809+0000 7f672ba15640 1 -- 172.21.15.46:0/4252878873 shutdown_connections 2023-12-02T00:31:33.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.809+0000 7f672ba15640 1 -- 172.21.15.46:0/4252878873 wait complete. 2023-12-02T00:31:33.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.810+0000 7f672ba15640 1 Processor -- start 2023-12-02T00:31:33.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.810+0000 7f672ba15640 1 -- start start 2023-12-02T00:31:33.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.811+0000 7f672ba15640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f672406bbf0 0x7f67241584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:33.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.811+0000 7f672ba15640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67240ad640 0x7f672415fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:33.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.811+0000 7f672ba15640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f672415d1d0 0x7f672415ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:33.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.811+0000 7f672ba15640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67240cde20 con 0x7f672406bbf0 2023-12-02T00:31:33.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.811+0000 7f672ba15640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67240cd210 con 0x7f67240ad640 2023-12-02T00:31:33.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.811+0000 7f672ba15640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67240cd4f0 con 0x7f672415d1d0 2023-12-02T00:31:33.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.811+0000 7f67297b1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f672406bbf0 0x7f67241584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:33.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.811+0000 7f67297b1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f672406bbf0 0x7f67241584a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45026/0 (socket says 172.21.15.46:45026) 2023-12-02T00:31:33.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.811+0000 7f67297b1640 1 -- 172.21.15.46:0/3866281333 learned_addr learned my addr 172.21.15.46:0/3866281333 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:33.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.811+0000 7f67297b1640 1 -- 172.21.15.46:0/3866281333 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f672415d1d0 msgr2=0x7f672415ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:33.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.811+0000 7f6729fb2640 1 --2- 172.21.15.46:0/3866281333 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f672415d1d0 0x7f672415ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:33.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.811+0000 7f6728fb0640 1 --2- 172.21.15.46:0/3866281333 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67240ad640 0x7f672415fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:33.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.811+0000 7f67297b1640 1 --2- 172.21.15.46:0/3866281333 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f672415d1d0 0x7f672415ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:33.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.811+0000 7f67297b1640 1 -- 172.21.15.46:0/3866281333 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67240ad640 msgr2=0x7f672415fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:33.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.811+0000 7f67297b1640 1 --2- 172.21.15.46:0/3866281333 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67240ad640 0x7f672415fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:33.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.811+0000 7f67297b1640 1 -- 172.21.15.46:0/3866281333 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6714009400 con 0x7f672406bbf0 2023-12-02T00:31:33.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.811+0000 7f67297b1640 1 --2- 172.21.15.46:0/3866281333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f672406bbf0 0x7f67241584a0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f671c00c430 tx=0x7f671c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:33.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.811+0000 7f6719ffb640 1 -- 172.21.15.46:0/3866281333 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671c0074e0 con 0x7f672406bbf0 2023-12-02T00:31:33.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.811+0000 7f6719ffb640 1 -- 172.21.15.46:0/3866281333 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f671c007660 con 0x7f672406bbf0 2023-12-02T00:31:33.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.811+0000 7f6719ffb640 1 -- 172.21.15.46:0/3866281333 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671c004a20 con 0x7f672406bbf0 2023-12-02T00:31:33.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.811+0000 7f672ba15640 1 -- 172.21.15.46:0/3866281333 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67240cd7d0 con 0x7f672406bbf0 2023-12-02T00:31:33.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.812+0000 7f672ba15640 1 -- 172.21.15.46:0/3866281333 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67240ccec0 con 0x7f672406bbf0 2023-12-02T00:31:33.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.813+0000 7f672ba15640 1 -- 172.21.15.46:0/3866281333 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6724069400 con 0x7f672406bbf0 2023-12-02T00:31:33.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.813+0000 7f6719ffb640 1 -- 172.21.15.46:0/3866281333 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f671c004b80 con 0x7f672406bbf0 2023-12-02T00:31:33.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.813+0000 7f6719ffb640 1 --2- 172.21.15.46:0/3866281333 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f670806ae40 0x7f670806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:33.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.813+0000 7f6719ffb640 1 -- 172.21.15.46:0/3866281333 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f671c0894e0 con 0x7f672406bbf0 2023-12-02T00:31:33.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.814+0000 7f6728fb0640 1 --2- 172.21.15.46:0/3866281333 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f670806ae40 0x7f670806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:33.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.814+0000 7f6728fb0640 1 --2- 172.21.15.46:0/3866281333 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f670806ae40 0x7f670806d300 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6714036000 tx=0x7f6714008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:33.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:33.818+0000 7f6719ffb640 1 -- 172.21.15.46:0/3866281333 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f671c054550 con 0x7f672406bbf0 2023-12-02T00:31:34.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.000+0000 7f672ba15640 1 -- 172.21.15.46:0/3866281333 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f67240acf40 con 0x7f672406bbf0 2023-12-02T00:31:34.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.000+0000 7f6719ffb640 1 -- 172.21.15.46:0/3866281333 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v53) v1 ==== 93+0+10 (secure 0 0 0) 0x7f671c057b70 con 0x7f672406bbf0 2023-12-02T00:31:34.002 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-12-02T00:31:34.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.002+0000 7f672ba15640 1 -- 172.21.15.46:0/3866281333 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f670806ae40 msgr2=0x7f670806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:34.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.002+0000 7f672ba15640 1 --2- 172.21.15.46:0/3866281333 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f670806ae40 0x7f670806d300 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6714036000 tx=0x7f6714008030 comp rx=0 tx=0).stop 2023-12-02T00:31:34.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.002+0000 7f672ba15640 1 -- 172.21.15.46:0/3866281333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f672406bbf0 msgr2=0x7f67241584a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:34.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.002+0000 7f672ba15640 1 --2- 172.21.15.46:0/3866281333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f672406bbf0 0x7f67241584a0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f671c00c430 tx=0x7f671c00c8f0 comp rx=0 tx=0).stop 2023-12-02T00:31:34.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.003+0000 7f672ba15640 1 -- 172.21.15.46:0/3866281333 shutdown_connections 2023-12-02T00:31:34.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.003+0000 7f672ba15640 1 --2- 172.21.15.46:0/3866281333 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f670806ae40 0x7f670806d300 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:34.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.003+0000 7f672ba15640 1 --2- 172.21.15.46:0/3866281333 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f672415d1d0 0x7f672415ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:34.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.003+0000 7f672ba15640 1 --2- 172.21.15.46:0/3866281333 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67240ad640 0x7f672415fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:34.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.003+0000 7f672ba15640 1 --2- 172.21.15.46:0/3866281333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f672406bbf0 0x7f67241584a0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:34.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.003+0000 7f672ba15640 1 -- 172.21.15.46:0/3866281333 >> 172.21.15.46:0/3866281333 conn(0x7f6724070e70 msgr2=0x7f67240cbd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:34.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.003+0000 7f672ba15640 1 -- 172.21.15.46:0/3866281333 shutdown_connections 2023-12-02T00:31:34.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.003+0000 7f672ba15640 1 -- 172.21.15.46:0/3866281333 wait complete. 2023-12-02T00:31:34.015 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-02T00:31:34.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.123+0000 7f81ffd1d640 1 Processor -- start 2023-12-02T00:31:34.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.124+0000 7f81ffd1d640 1 -- start start 2023-12-02T00:31:34.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.124+0000 7f81ffd1d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81f80ad640 0x7f81f806d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:34.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.124+0000 7f81ffd1d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f81f806bbf0 0x7f81f806d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:34.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.124+0000 7f81ffd1d640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81f80adae0 con 0x7f81f815d1d0 2023-12-02T00:31:34.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.124+0000 7f81ffd1d640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81f80addc0 con 0x7f81f806bbf0 2023-12-02T00:31:34.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.124+0000 7f81ffd1d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81f80ae0a0 con 0x7f81f80ad640 2023-12-02T00:31:34.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.124+0000 7f81fdab9640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f81f815d1d0 0x7f81f806b4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50818/0 (socket says 172.21.15.46:50818) 2023-12-02T00:31:34.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.125+0000 7f81fdab9640 1 -- 172.21.15.46:0/1253909543 learned_addr learned my addr 172.21.15.46:0/1253909543 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:31:34.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.125+0000 7f81fd2b8640 1 --2- 172.21.15.46:0/1253909543 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81f80ad640 0x7f81f806d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:34.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.125+0000 7f81fe2ba640 1 --2- 172.21.15.46:0/1253909543 >> v2:172.21.15.162:3300/0 conn(0x7f81f806bbf0 0x7f81f806d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:34.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.125+0000 7f81fcab7640 1 -- 172.21.15.46:0/1253909543 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3833867451 0 0) 0x7f81f80adae0 con 0x7f81f815d1d0 2023-12-02T00:31:34.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.125+0000 7f81fcab7640 1 -- 172.21.15.46:0/1253909543 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81e4003520 con 0x7f81f815d1d0 2023-12-02T00:31:34.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.125+0000 7f81fcab7640 1 -- 172.21.15.46:0/1253909543 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3485235363 0 0) 0x7f81e4003520 con 0x7f81f815d1d0 2023-12-02T00:31:34.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.125+0000 7f81fcab7640 1 -- 172.21.15.46:0/1253909543 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81f80ad640 msgr2=0x7f81f806d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:34.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.125+0000 7f81fcab7640 1 --2- 172.21.15.46:0/1253909543 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81f80ad640 0x7f81f806d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:34.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.125+0000 7f81fcab7640 1 -- 172.21.15.46:0/1253909543 >> v2:172.21.15.162:3300/0 conn(0x7f81f806bbf0 msgr2=0x7f81f806d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:34.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.125+0000 7f81fcab7640 1 --2- 172.21.15.46:0/1253909543 >> v2:172.21.15.162:3300/0 conn(0x7f81f806bbf0 0x7f81f806d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:34.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.125+0000 7f81fcab7640 1 -- 172.21.15.46:0/1253909543 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f80ae380 con 0x7f81f815d1d0 2023-12-02T00:31:34.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.125+0000 7f81fcab7640 1 -- 172.21.15.46:0/1253909543 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7f81f0001670 con 0x7f81f815d1d0 2023-12-02T00:31:34.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.125+0000 7f81fcab7640 1 -- 172.21.15.46:0/1253909543 >> v1:172.21.15.46:6789/0 conn(0x7f81f815d1d0 legacy=0x7f81f806b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:34.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.125+0000 7f81fcab7640 1 --2- 172.21.15.46:0/1253909543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81e4003970 0x7f81e4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:34.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.125+0000 7f81fcab7640 1 --2- 172.21.15.46:0/1253909543 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81e40047c0 0x7f81e4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:34.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.125+0000 7f81fcab7640 1 --2- 172.21.15.46:0/1253909543 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81f80ad640 0x7f81e4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:34.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.125+0000 7f81fcab7640 1 -- 172.21.15.46:0/1253909543 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f80addc0 con 0x7f81e4003970 2023-12-02T00:31:34.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.125+0000 7f81fcab7640 1 -- 172.21.15.46:0/1253909543 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f80ae0a0 con 0x7f81e40047c0 2023-12-02T00:31:34.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.125+0000 7f81fcab7640 1 -- 172.21.15.46:0/1253909543 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81e400b980 con 0x7f81f80ad640 2023-12-02T00:31:34.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.125+0000 7f81fdab9640 1 --2- 172.21.15.46:0/1253909543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81e4003970 0x7f81e4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:34.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.125+0000 7f81fd2b8640 1 --2- 172.21.15.46:0/1253909543 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81e40047c0 0x7f81e4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:34.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.125+0000 7f81fe2ba640 1 --2- 172.21.15.46:0/1253909543 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81f80ad640 0x7f81e4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:34.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.126+0000 7f81fdab9640 1 -- 172.21.15.46:0/1253909543 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81f80ad640 msgr2=0x7f81e4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:34.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.126+0000 7f81fdab9640 1 --2- 172.21.15.46:0/1253909543 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81f80ad640 0x7f81e4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:34.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.126+0000 7f81fdab9640 1 -- 172.21.15.46:0/1253909543 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81e40047c0 msgr2=0x7f81e4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:34.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.126+0000 7f81fdab9640 1 --2- 172.21.15.46:0/1253909543 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81e40047c0 0x7f81e4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:34.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.126+0000 7f81fdab9640 1 -- 172.21.15.46:0/1253909543 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e4003520 con 0x7f81e4003970 2023-12-02T00:31:34.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.126+0000 7f81fd2b8640 1 --2- 172.21.15.46:0/1253909543 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81e40047c0 0x7f81e4004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:31:34.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.126+0000 7f81fdab9640 1 --2- 172.21.15.46:0/1253909543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81e4003970 0x7f81e4003d90 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f81f000c240 tx=0x7f81e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4af7f5020e0e2c13 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:34.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.126+0000 7f81fcab7640 1 -- 172.21.15.46:0/1253909543 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f0005ad0 con 0x7f81e4003970 2023-12-02T00:31:34.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.127+0000 7f81fcab7640 1 -- 172.21.15.46:0/1253909543 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81f0005c30 con 0x7f81e4003970 2023-12-02T00:31:34.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.127+0000 7f81fcab7640 1 -- 172.21.15.46:0/1253909543 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f002c8b0 con 0x7f81e4003970 2023-12-02T00:31:34.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.127+0000 7f81ffd1d640 1 -- 172.21.15.46:0/1253909543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81e4003970 msgr2=0x7f81e4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:34.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.127+0000 7f81ffd1d640 1 --2- 172.21.15.46:0/1253909543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81e4003970 0x7f81e4003d90 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f81f000c240 tx=0x7f81e40057f0 comp rx=0 tx=0).stop 2023-12-02T00:31:34.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.127+0000 7f81fe2ba640 1 -- 172.21.15.46:0/1253909543 reap_dead start 2023-12-02T00:31:34.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.127+0000 7f81ffd1d640 1 -- 172.21.15.46:0/1253909543 shutdown_connections 2023-12-02T00:31:34.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.127+0000 7f81ffd1d640 1 -- 172.21.15.46:0/1253909543 >> 172.21.15.46:0/1253909543 conn(0x7f81f8070e70 msgr2=0x7f81f8071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:34.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.127+0000 7f81ffd1d640 1 -- 172.21.15.46:0/1253909543 shutdown_connections 2023-12-02T00:31:34.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.127+0000 7f81ffd1d640 1 -- 172.21.15.46:0/1253909543 wait complete. 2023-12-02T00:31:34.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.128+0000 7f81ffd1d640 1 Processor -- start 2023-12-02T00:31:34.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.128+0000 7f81ffd1d640 1 -- start start 2023-12-02T00:31:34.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.129+0000 7f81ffd1d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81e4003970 0x7f81f8141f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:34.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.129+0000 7f81ffd1d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81f81446d0 0x7f81f8144b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:34.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.129+0000 7f81ffd1d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81f815ada0 0x7f81f815b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:34.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.129+0000 7f81ffd1d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f81f2aa0 con 0x7f81e4003970 2023-12-02T00:31:34.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.129+0000 7f81ffd1d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f8145090 con 0x7f81f81446d0 2023-12-02T00:31:34.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.129+0000 7f81ffd1d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81f815b790 con 0x7f81f815ada0 2023-12-02T00:31:34.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.129+0000 7f81fdab9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81e4003970 0x7f81f8141f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:34.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.129+0000 7f81fdab9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81e4003970 0x7f81f8141f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45038/0 (socket says 172.21.15.46:45038) 2023-12-02T00:31:34.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.129+0000 7f81fdab9640 1 -- 172.21.15.46:0/3716717020 learned_addr learned my addr 172.21.15.46:0/3716717020 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:34.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.129+0000 7f81fd2b8640 1 --2- 172.21.15.46:0/3716717020 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81f81446d0 0x7f81f8144b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:34.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.129+0000 7f81fe2ba640 1 --2- 172.21.15.46:0/3716717020 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81f815ada0 0x7f81f815b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:34.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.129+0000 7f81fdab9640 1 -- 172.21.15.46:0/3716717020 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81f815ada0 msgr2=0x7f81f815b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:34.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.129+0000 7f81fdab9640 1 --2- 172.21.15.46:0/3716717020 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81f815ada0 0x7f81f815b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:34.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.129+0000 7f81fdab9640 1 -- 172.21.15.46:0/3716717020 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81f81446d0 msgr2=0x7f81f8144b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:34.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.129+0000 7f81fdab9640 1 --2- 172.21.15.46:0/3716717020 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81f81446d0 0x7f81f8144b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:34.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.130+0000 7f81fdab9640 1 -- 172.21.15.46:0/3716717020 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f8145550 con 0x7f81e4003970 2023-12-02T00:31:34.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.130+0000 7f81fdab9640 1 --2- 172.21.15.46:0/3716717020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81e4003970 0x7f81f8141f20 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f81f000c360 tx=0x7f81f0001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:34.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.130+0000 7f81ee7fc640 1 -- 172.21.15.46:0/3716717020 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f003c040 con 0x7f81e4003970 2023-12-02T00:31:34.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.130+0000 7f81ee7fc640 1 -- 172.21.15.46:0/3716717020 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81f002ccc0 con 0x7f81e4003970 2023-12-02T00:31:34.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.130+0000 7f81ee7fc640 1 -- 172.21.15.46:0/3716717020 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f00072b0 con 0x7f81e4003970 2023-12-02T00:31:34.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.130+0000 7f81ffd1d640 1 -- 172.21.15.46:0/3716717020 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81f815b9b0 con 0x7f81e4003970 2023-12-02T00:31:34.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.130+0000 7f81ffd1d640 1 -- 172.21.15.46:0/3716717020 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81f815bf70 con 0x7f81e4003970 2023-12-02T00:31:34.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.131+0000 7f81ffd1d640 1 -- 172.21.15.46:0/3716717020 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81f8069400 con 0x7f81e4003970 2023-12-02T00:31:34.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.131+0000 7f81ee7fc640 1 -- 172.21.15.46:0/3716717020 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f81f0032070 con 0x7f81e4003970 2023-12-02T00:31:34.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.132+0000 7f81ee7fc640 1 --2- 172.21.15.46:0/3716717020 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f81dc06ae40 0x7f81dc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:34.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.132+0000 7f81ee7fc640 1 -- 172.21.15.46:0/3716717020 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f81f00aac50 con 0x7f81e4003970 2023-12-02T00:31:34.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.132+0000 7f81fd2b8640 1 --2- 172.21.15.46:0/3716717020 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f81dc06ae40 0x7f81dc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:34.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.132+0000 7f81fd2b8640 1 --2- 172.21.15.46:0/3716717020 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f81dc06ae40 0x7f81dc06d300 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f81f8153de0 tx=0x7f81e8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:34.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.136+0000 7f81ee7fc640 1 -- 172.21.15.46:0/3716717020 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f81f0075cd0 con 0x7f81e4003970 2023-12-02T00:31:34.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.311+0000 7f81ffd1d640 1 -- 172.21.15.46:0/3716717020 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f81f81537d0 con 0x7f81e4003970 2023-12-02T00:31:34.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.311+0000 7f81ee7fc640 1 -- 172.21.15.46:0/3716717020 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v53) v1 ==== 104+0+11 (secure 0 0 0) 0x7f81f0046020 con 0x7f81e4003970 2023-12-02T00:31:34.313 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-12-02T00:31:34.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.313+0000 7f81ffd1d640 1 -- 172.21.15.46:0/3716717020 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f81dc06ae40 msgr2=0x7f81dc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:34.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.313+0000 7f81ffd1d640 1 --2- 172.21.15.46:0/3716717020 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f81dc06ae40 0x7f81dc06d300 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f81f8153de0 tx=0x7f81e8008040 comp rx=0 tx=0).stop 2023-12-02T00:31:34.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.313+0000 7f81ffd1d640 1 -- 172.21.15.46:0/3716717020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81e4003970 msgr2=0x7f81f8141f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:34.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.313+0000 7f81ffd1d640 1 --2- 172.21.15.46:0/3716717020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81e4003970 0x7f81f8141f20 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f81f000c360 tx=0x7f81f0001a00 comp rx=0 tx=0).stop 2023-12-02T00:31:34.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.313+0000 7f81ffd1d640 1 -- 172.21.15.46:0/3716717020 shutdown_connections 2023-12-02T00:31:34.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.313+0000 7f81ffd1d640 1 --2- 172.21.15.46:0/3716717020 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f81dc06ae40 0x7f81dc06d300 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:34.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.313+0000 7f81ffd1d640 1 --2- 172.21.15.46:0/3716717020 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81f815ada0 0x7f81f815b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:34.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.313+0000 7f81ffd1d640 1 --2- 172.21.15.46:0/3716717020 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81f81446d0 0x7f81f8144b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:34.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.313+0000 7f81ffd1d640 1 --2- 172.21.15.46:0/3716717020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81e4003970 0x7f81f8141f20 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:34.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.313+0000 7f81ffd1d640 1 -- 172.21.15.46:0/3716717020 >> 172.21.15.46:0/3716717020 conn(0x7f81f8070e70 msgr2=0x7f81f806a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:34.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.313+0000 7f81ffd1d640 1 -- 172.21.15.46:0/3716717020 shutdown_connections 2023-12-02T00:31:34.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.313+0000 7f81ffd1d640 1 -- 172.21.15.46:0/3716717020 wait complete. 2023-12-02T00:31:34.325 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-02T00:31:34.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.434+0000 7f3d19417640 1 Processor -- start 2023-12-02T00:31:34.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.434+0000 7f3d19417640 1 -- start start 2023-12-02T00:31:34.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.435+0000 7f3d19417640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3d1406ea50 0x7f3d1406ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:34.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.435+0000 7f3d19417640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d14161d20 con 0x7f3d140d0ed0 2023-12-02T00:31:34.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.435+0000 7f3d19417640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d1406f370 con 0x7f3d14160fb0 2023-12-02T00:31:34.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.435+0000 7f3d19417640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d1415e760 con 0x7f3d1406ea50 2023-12-02T00:31:34.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.435+0000 7f3d137fe640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f3d140d0ed0 0x7f3d140d1320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50832/0 (socket says 172.21.15.46:50832) 2023-12-02T00:31:34.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.435+0000 7f3d137fe640 1 -- 172.21.15.46:0/2872155912 learned_addr learned my addr 172.21.15.46:0/2872155912 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:31:34.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.435+0000 7f3d12ffd640 1 --2- 172.21.15.46:0/2872155912 >> v2:172.21.15.162:3300/0 conn(0x7f3d1406ea50 0x7f3d1406ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:34.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.435+0000 7f3d11ffb640 1 -- 172.21.15.46:0/2872155912 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 658358258 0 0) 0x7f3d14161d20 con 0x7f3d140d0ed0 2023-12-02T00:31:34.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.435+0000 7f3d11ffb640 1 -- 172.21.15.46:0/2872155912 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d00003520 con 0x7f3d140d0ed0 2023-12-02T00:31:34.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.435+0000 7f3d11ffb640 1 -- 172.21.15.46:0/2872155912 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3908804884 0 0) 0x7f3d00003520 con 0x7f3d140d0ed0 2023-12-02T00:31:34.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.435+0000 7f3d11ffb640 1 -- 172.21.15.46:0/2872155912 >> v2:172.21.15.162:3300/0 conn(0x7f3d1406ea50 msgr2=0x7f3d1406ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:34.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.435+0000 7f3d11ffb640 1 --2- 172.21.15.46:0/2872155912 >> v2:172.21.15.162:3300/0 conn(0x7f3d1406ea50 0x7f3d1406ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:34.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.435+0000 7f3d11ffb640 1 -- 172.21.15.46:0/2872155912 >> v1:172.21.15.162:6789/0 conn(0x7f3d14160fb0 legacy=0x7f3d14161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:34.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.435+0000 7f3d11ffb640 1 -- 172.21.15.46:0/2872155912 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d1415d9a0 con 0x7f3d140d0ed0 2023-12-02T00:31:34.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.436+0000 7f3d11ffb640 1 -- 172.21.15.46:0/2872155912 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7f3d040019f0 con 0x7f3d140d0ed0 2023-12-02T00:31:34.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.436+0000 7f3d11ffb640 1 -- 172.21.15.46:0/2872155912 >> v1:172.21.15.46:6789/0 conn(0x7f3d140d0ed0 legacy=0x7f3d140d1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:34.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.436+0000 7f3d11ffb640 1 --2- 172.21.15.46:0/2872155912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d00003970 0x7f3d00003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:34.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.436+0000 7f3d11ffb640 1 --2- 172.21.15.46:0/2872155912 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d000047c0 0x7f3d00004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:34.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.436+0000 7f3d11ffb640 1 --2- 172.21.15.46:0/2872155912 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3d00005180 0x7f3d0000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:34.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.436+0000 7f3d11ffb640 1 -- 172.21.15.46:0/2872155912 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1415e760 con 0x7f3d00003970 2023-12-02T00:31:34.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.436+0000 7f3d11ffb640 1 -- 172.21.15.46:0/2872155912 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0000c120 con 0x7f3d000047c0 2023-12-02T00:31:34.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.436+0000 7f3d11ffb640 1 -- 172.21.15.46:0/2872155912 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0000c2e0 con 0x7f3d00005180 2023-12-02T00:31:34.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.436+0000 7f3d12ffd640 1 --2- 172.21.15.46:0/2872155912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d00003970 0x7f3d00003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:34.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.436+0000 7f3d127fc640 1 --2- 172.21.15.46:0/2872155912 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d000047c0 0x7f3d00004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:34.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.436+0000 7f3d137fe640 1 --2- 172.21.15.46:0/2872155912 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3d00005180 0x7f3d0000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:34.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.436+0000 7f3d12ffd640 1 -- 172.21.15.46:0/2872155912 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3d00005180 msgr2=0x7f3d0000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:34.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.436+0000 7f3d12ffd640 1 --2- 172.21.15.46:0/2872155912 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3d00005180 0x7f3d0000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:34.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.436+0000 7f3d12ffd640 1 -- 172.21.15.46:0/2872155912 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d000047c0 msgr2=0x7f3d00004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:34.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.436+0000 7f3d12ffd640 1 --2- 172.21.15.46:0/2872155912 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d000047c0 0x7f3d00004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:34.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.436+0000 7f3d12ffd640 1 -- 172.21.15.46:0/2872155912 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d00003520 con 0x7f3d00003970 2023-12-02T00:31:34.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.436+0000 7f3d137fe640 1 -- 172.21.15.46:0/2872155912 reap_dead start 2023-12-02T00:31:34.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.437+0000 7f3d12ffd640 1 --2- 172.21.15.46:0/2872155912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d00003970 0x7f3d00003d90 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c009ac0 tx=0x7f3d000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66fd29cb5192a790 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:34.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.437+0000 7f3d11ffb640 1 -- 172.21.15.46:0/2872155912 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c02c6b0 con 0x7f3d00003970 2023-12-02T00:31:34.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.437+0000 7f3d11ffb640 1 -- 172.21.15.46:0/2872155912 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d0c02c810 con 0x7f3d00003970 2023-12-02T00:31:34.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.437+0000 7f3d11ffb640 1 -- 172.21.15.46:0/2872155912 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c02cba0 con 0x7f3d00003970 2023-12-02T00:31:34.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.437+0000 7f3d19417640 1 -- 172.21.15.46:0/2872155912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d00003970 msgr2=0x7f3d00003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:34.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.437+0000 7f3d19417640 1 --2- 172.21.15.46:0/2872155912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d00003970 0x7f3d00003d90 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c009ac0 tx=0x7f3d000057f0 comp rx=0 tx=0).stop 2023-12-02T00:31:34.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.437+0000 7f3d19417640 1 -- 172.21.15.46:0/2872155912 shutdown_connections 2023-12-02T00:31:34.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.437+0000 7f3d19417640 1 --2- 172.21.15.46:0/2872155912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d00003970 0x7f3d00003d90 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:34.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.437+0000 7f3d19417640 1 -- 172.21.15.46:0/2872155912 >> 172.21.15.46:0/2872155912 conn(0x7f3d1406a6b0 msgr2=0x7f3d1406aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:34.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.438+0000 7f3d19417640 1 -- 172.21.15.46:0/2872155912 shutdown_connections 2023-12-02T00:31:34.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.438+0000 7f3d19417640 1 -- 172.21.15.46:0/2872155912 wait complete. 2023-12-02T00:31:34.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.439+0000 7f3d19417640 1 Processor -- start 2023-12-02T00:31:34.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.439+0000 7f3d19417640 1 -- start start 2023-12-02T00:31:34.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.440+0000 7f3d19417640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d00003970 0x7f3d1415a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:34.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.440+0000 7f3d19417640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3d1413f7e0 0x7f3d14132770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:34.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.440+0000 7f3d19417640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d141e8660 0x7f3d141e8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:34.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.440+0000 7f3d19417640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d141e9050 con 0x7f3d00003970 2023-12-02T00:31:34.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.440+0000 7f3d19417640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1415a730 con 0x7f3d141e8660 2023-12-02T00:31:34.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.440+0000 7f3d19417640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d14159830 con 0x7f3d1413f7e0 2023-12-02T00:31:34.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.440+0000 7f3d12ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d00003970 0x7f3d1415a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:34.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.440+0000 7f3d12ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d00003970 0x7f3d1415a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45056/0 (socket says 172.21.15.46:45056) 2023-12-02T00:31:34.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.440+0000 7f3d12ffd640 1 -- 172.21.15.46:0/1702589036 learned_addr learned my addr 172.21.15.46:0/1702589036 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:34.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.440+0000 7f3d127fc640 1 --2- 172.21.15.46:0/1702589036 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3d1413f7e0 0x7f3d14132770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:34.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.440+0000 7f3d137fe640 1 --2- 172.21.15.46:0/1702589036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d141e8660 0x7f3d141e8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:34.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.440+0000 7f3d12ffd640 1 -- 172.21.15.46:0/1702589036 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3d1413f7e0 msgr2=0x7f3d14132770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:34.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.440+0000 7f3d12ffd640 1 --2- 172.21.15.46:0/1702589036 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3d1413f7e0 0x7f3d14132770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:34.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.440+0000 7f3d12ffd640 1 -- 172.21.15.46:0/1702589036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d141e8660 msgr2=0x7f3d141e8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:34.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.440+0000 7f3d12ffd640 1 --2- 172.21.15.46:0/1702589036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d141e8660 0x7f3d141e8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:34.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.440+0000 7f3d12ffd640 1 -- 172.21.15.46:0/1702589036 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d14158a70 con 0x7f3d00003970 2023-12-02T00:31:34.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.440+0000 7f3d137fe640 1 --2- 172.21.15.46:0/1702589036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d141e8660 0x7f3d141e8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:31:34.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.440+0000 7f3d12ffd640 1 --2- 172.21.15.46:0/1702589036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d00003970 0x7f3d1415a1f0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c02c250 tx=0x7f3d0c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:34.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.440+0000 7f3d0b7fe640 1 -- 172.21.15.46:0/1702589036 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c003810 con 0x7f3d00003970 2023-12-02T00:31:34.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.440+0000 7f3d0b7fe640 1 -- 172.21.15.46:0/1702589036 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d0c003970 con 0x7f3d00003970 2023-12-02T00:31:34.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.440+0000 7f3d19417640 1 -- 172.21.15.46:0/1702589036 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d14158d50 con 0x7f3d00003970 2023-12-02T00:31:34.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.440+0000 7f3d0b7fe640 1 -- 172.21.15.46:0/1702589036 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c003d00 con 0x7f3d00003970 2023-12-02T00:31:34.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.440+0000 7f3d19417640 1 -- 172.21.15.46:0/1702589036 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d14159310 con 0x7f3d00003970 2023-12-02T00:31:34.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.441+0000 7f3d19417640 1 -- 172.21.15.46:0/1702589036 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d141594a0 con 0x7f3d00003970 2023-12-02T00:31:34.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.442+0000 7f3d0b7fe640 1 -- 172.21.15.46:0/1702589036 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3d0c034440 con 0x7f3d00003970 2023-12-02T00:31:34.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.442+0000 7f3d0b7fe640 1 --2- 172.21.15.46:0/1702589036 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3cf406ae40 0x7f3cf406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:34.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.442+0000 7f3d0b7fe640 1 -- 172.21.15.46:0/1702589036 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f3d0c0ab910 con 0x7f3d00003970 2023-12-02T00:31:34.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.442+0000 7f3d127fc640 1 --2- 172.21.15.46:0/1702589036 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3cf406ae40 0x7f3cf406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:34.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.443+0000 7f3d127fc640 1 --2- 172.21.15.46:0/1702589036 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3cf406ae40 0x7f3cf406d300 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc000c00 tx=0x7f3cfc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:34.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.447+0000 7f3d0b7fe640 1 -- 172.21.15.46:0/1702589036 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3d0c0769a0 con 0x7f3d00003970 2023-12-02T00:31:34.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.622+0000 7f3d19417640 1 -- 172.21.15.46:0/1702589036 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3d140b4e20 con 0x7f3d00003970 2023-12-02T00:31:34.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.623+0000 7f3d0b7fe640 1 -- 172.21.15.46:0/1702589036 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v53) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3d0c0390a0 con 0x7f3d00003970 2023-12-02T00:31:34.624 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-12-02T00:31:34.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.625+0000 7f3d19417640 1 -- 172.21.15.46:0/1702589036 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3cf406ae40 msgr2=0x7f3cf406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:34.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.625+0000 7f3d19417640 1 --2- 172.21.15.46:0/1702589036 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3cf406ae40 0x7f3cf406d300 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc000c00 tx=0x7f3cfc009040 comp rx=0 tx=0).stop 2023-12-02T00:31:34.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.625+0000 7f3d19417640 1 -- 172.21.15.46:0/1702589036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d00003970 msgr2=0x7f3d1415a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:34.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.625+0000 7f3d19417640 1 --2- 172.21.15.46:0/1702589036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d00003970 0x7f3d1415a1f0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c02c250 tx=0x7f3d0c004730 comp rx=0 tx=0).stop 2023-12-02T00:31:34.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.625+0000 7f3d19417640 1 -- 172.21.15.46:0/1702589036 shutdown_connections 2023-12-02T00:31:34.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.625+0000 7f3d19417640 1 --2- 172.21.15.46:0/1702589036 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f3cf406ae40 0x7f3cf406d300 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:34.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.625+0000 7f3d19417640 1 --2- 172.21.15.46:0/1702589036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d141e8660 0x7f3d141e8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:34.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.625+0000 7f3d19417640 1 --2- 172.21.15.46:0/1702589036 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3d1413f7e0 0x7f3d14132770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:34.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.625+0000 7f3d19417640 1 --2- 172.21.15.46:0/1702589036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d00003970 0x7f3d1415a1f0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:34.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.625+0000 7f3d19417640 1 -- 172.21.15.46:0/1702589036 >> 172.21.15.46:0/1702589036 conn(0x7f3d1406a6b0 msgr2=0x7f3d1415c650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:34.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.625+0000 7f3d19417640 1 -- 172.21.15.46:0/1702589036 shutdown_connections 2023-12-02T00:31:34.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.625+0000 7f3d19417640 1 -- 172.21.15.46:0/1702589036 wait complete. 2023-12-02T00:31:34.637 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-02T00:31:34.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.743+0000 7fb165100640 1 Processor -- start 2023-12-02T00:31:34.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.743+0000 7fb165100640 1 -- start start 2023-12-02T00:31:34.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.744+0000 7fb165100640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb16015d330 0x7fb1600b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:34.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.744+0000 7fb165100640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb1600d3900 0x7fb1600b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:34.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.744+0000 7fb165100640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1600b2b10 con 0x7fb1600d2910 2023-12-02T00:31:34.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.744+0000 7fb165100640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1600d1590 con 0x7fb16015d330 2023-12-02T00:31:34.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.744+0000 7fb165100640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1600d1870 con 0x7fb1600d3900 2023-12-02T00:31:34.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.744+0000 7fb15e59c640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb1600d2910 0x7fb1600d31f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50836/0 (socket says 172.21.15.46:50836) 2023-12-02T00:31:34.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.744+0000 7fb15e59c640 1 -- 172.21.15.46:0/378626030 learned_addr learned my addr 172.21.15.46:0/378626030 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:31:34.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.744+0000 7fb15ed9d640 1 --2- 172.21.15.46:0/378626030 >> v2:172.21.15.162:3300/0 conn(0x7fb16015d330 0x7fb1600b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:34.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.744+0000 7fb15f59e640 1 --2- 172.21.15.46:0/378626030 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb1600d3900 0x7fb1600b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:34.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.745+0000 7fb15dd9b640 1 -- 172.21.15.46:0/378626030 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3009393186 0 0) 0x7fb1600b2b10 con 0x7fb1600d2910 2023-12-02T00:31:34.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.745+0000 7fb15dd9b640 1 -- 172.21.15.46:0/378626030 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb148003520 con 0x7fb1600d2910 2023-12-02T00:31:34.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.745+0000 7fb15dd9b640 1 -- 172.21.15.46:0/378626030 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1180734282 0 0) 0x7fb148003520 con 0x7fb1600d2910 2023-12-02T00:31:34.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.745+0000 7fb15dd9b640 1 -- 172.21.15.46:0/378626030 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb1600d3900 msgr2=0x7fb1600b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:34.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.745+0000 7fb15dd9b640 1 --2- 172.21.15.46:0/378626030 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb1600d3900 0x7fb1600b6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:34.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.745+0000 7fb15dd9b640 1 -- 172.21.15.46:0/378626030 >> v2:172.21.15.162:3300/0 conn(0x7fb16015d330 msgr2=0x7fb1600b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:34.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.745+0000 7fb15dd9b640 1 --2- 172.21.15.46:0/378626030 >> v2:172.21.15.162:3300/0 conn(0x7fb16015d330 0x7fb1600b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:34.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.745+0000 7fb15dd9b640 1 -- 172.21.15.46:0/378626030 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1600d1b50 con 0x7fb1600d2910 2023-12-02T00:31:34.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.745+0000 7fb15dd9b640 1 -- 172.21.15.46:0/378626030 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7fb154001670 con 0x7fb1600d2910 2023-12-02T00:31:34.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.745+0000 7fb15ed9d640 1 --2- 172.21.15.46:0/378626030 >> v2:172.21.15.162:3300/0 conn(0x7fb16015d330 0x7fb1600b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:31:34.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.745+0000 7fb15dd9b640 1 -- 172.21.15.46:0/378626030 >> v1:172.21.15.46:6789/0 conn(0x7fb1600d2910 legacy=0x7fb1600d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:34.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.745+0000 7fb15dd9b640 1 --2- 172.21.15.46:0/378626030 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb148003970 0x7fb148003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:34.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.745+0000 7fb15dd9b640 1 --2- 172.21.15.46:0/378626030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1480047c0 0x7fb148004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:34.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.745+0000 7fb15dd9b640 1 --2- 172.21.15.46:0/378626030 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb1600d3900 0x7fb148005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:34.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.745+0000 7fb15dd9b640 1 -- 172.21.15.46:0/378626030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1600d1590 con 0x7fb1480047c0 2023-12-02T00:31:34.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.745+0000 7fb15dd9b640 1 -- 172.21.15.46:0/378626030 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1600d1870 con 0x7fb148003970 2023-12-02T00:31:34.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.745+0000 7fb15dd9b640 1 -- 172.21.15.46:0/378626030 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb14800b980 con 0x7fb1600d3900 2023-12-02T00:31:34.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.745+0000 7fb15e59c640 1 --2- 172.21.15.46:0/378626030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1480047c0 0x7fb148004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:34.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.745+0000 7fb15ed9d640 1 --2- 172.21.15.46:0/378626030 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb148003970 0x7fb148003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:34.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.745+0000 7fb15f59e640 1 --2- 172.21.15.46:0/378626030 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb1600d3900 0x7fb148005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:34.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.745+0000 7fb15e59c640 1 -- 172.21.15.46:0/378626030 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb1600d3900 msgr2=0x7fb148005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:34.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.745+0000 7fb15e59c640 1 --2- 172.21.15.46:0/378626030 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb1600d3900 0x7fb148005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:34.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.745+0000 7fb15e59c640 1 -- 172.21.15.46:0/378626030 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb148003970 msgr2=0x7fb148003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:34.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.745+0000 7fb15e59c640 1 --2- 172.21.15.46:0/378626030 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb148003970 0x7fb148003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:34.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.745+0000 7fb15e59c640 1 -- 172.21.15.46:0/378626030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb148003520 con 0x7fb1480047c0 2023-12-02T00:31:34.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.746+0000 7fb15e59c640 1 --2- 172.21.15.46:0/378626030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1480047c0 0x7fb148004c40 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fb15400c2c0 tx=0x7fb1480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc8db3a943e91e38 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:34.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.746+0000 7fb15dd9b640 1 -- 172.21.15.46:0/378626030 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb154005ad0 con 0x7fb1480047c0 2023-12-02T00:31:34.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.746+0000 7fb15dd9b640 1 -- 172.21.15.46:0/378626030 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb154005c30 con 0x7fb1480047c0 2023-12-02T00:31:34.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.746+0000 7fb15dd9b640 1 -- 172.21.15.46:0/378626030 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb15402c8b0 con 0x7fb1480047c0 2023-12-02T00:31:34.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.747+0000 7fb165100640 1 -- 172.21.15.46:0/378626030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1480047c0 msgr2=0x7fb148004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:34.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.747+0000 7fb165100640 1 --2- 172.21.15.46:0/378626030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1480047c0 0x7fb148004c40 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fb15400c2c0 tx=0x7fb1480057f0 comp rx=0 tx=0).stop 2023-12-02T00:31:34.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.747+0000 7fb15f59e640 1 -- 172.21.15.46:0/378626030 reap_dead start 2023-12-02T00:31:34.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.747+0000 7fb165100640 1 -- 172.21.15.46:0/378626030 shutdown_connections 2023-12-02T00:31:34.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.747+0000 7fb165100640 1 -- 172.21.15.46:0/378626030 >> 172.21.15.46:0/378626030 conn(0x7fb160072020 msgr2=0x7fb160072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:34.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.747+0000 7fb165100640 1 -- 172.21.15.46:0/378626030 shutdown_connections 2023-12-02T00:31:34.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.747+0000 7fb165100640 1 -- 172.21.15.46:0/378626030 wait complete. 2023-12-02T00:31:34.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.748+0000 7fb165100640 1 Processor -- start 2023-12-02T00:31:34.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.748+0000 7fb165100640 1 -- start start 2023-12-02T00:31:34.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.749+0000 7fb165100640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb1480047c0 0x7fb1600c7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:34.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.749+0000 7fb165100640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb160156d50 0x7fb160143520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:34.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.749+0000 7fb165100640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1601571d0 0x7fb160143a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:34.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.749+0000 7fb165100640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb160152ed0 con 0x7fb1601571d0 2023-12-02T00:31:34.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.749+0000 7fb165100640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1600c8fe0 con 0x7fb1480047c0 2023-12-02T00:31:34.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.749+0000 7fb165100640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1600c8130 con 0x7fb160156d50 2023-12-02T00:31:34.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.749+0000 7fb15f59e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1601571d0 0x7fb160143a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:34.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.749+0000 7fb15f59e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1601571d0 0x7fb160143a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45068/0 (socket says 172.21.15.46:45068) 2023-12-02T00:31:34.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.749+0000 7fb15f59e640 1 -- 172.21.15.46:0/4033759575 learned_addr learned my addr 172.21.15.46:0/4033759575 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:34.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.749+0000 7fb15ed9d640 1 --2- 172.21.15.46:0/4033759575 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb1480047c0 0x7fb1600c7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:34.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.749+0000 7fb15e59c640 1 --2- 172.21.15.46:0/4033759575 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb160156d50 0x7fb160143520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:34.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.749+0000 7fb15f59e640 1 -- 172.21.15.46:0/4033759575 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb160156d50 msgr2=0x7fb160143520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:34.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.749+0000 7fb15f59e640 1 --2- 172.21.15.46:0/4033759575 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb160156d50 0x7fb160143520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:34.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.749+0000 7fb15f59e640 1 -- 172.21.15.46:0/4033759575 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb1480047c0 msgr2=0x7fb1600c7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:34.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.749+0000 7fb15f59e640 1 --2- 172.21.15.46:0/4033759575 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb1480047c0 0x7fb1600c7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:34.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.749+0000 7fb15f59e640 1 -- 172.21.15.46:0/4033759575 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1600c83d0 con 0x7fb1601571d0 2023-12-02T00:31:34.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.750+0000 7fb15f59e640 1 --2- 172.21.15.46:0/4033759575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1601571d0 0x7fb160143a60 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fb16008c510 tx=0x7fb158011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:34.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.750+0000 7fb1477fe640 1 -- 172.21.15.46:0/4033759575 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb158011ee0 con 0x7fb1601571d0 2023-12-02T00:31:34.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.750+0000 7fb1477fe640 1 -- 172.21.15.46:0/4033759575 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb158004240 con 0x7fb1601571d0 2023-12-02T00:31:34.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.750+0000 7fb165100640 1 -- 172.21.15.46:0/4033759575 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1600c86b0 con 0x7fb1601571d0 2023-12-02T00:31:34.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.750+0000 7fb1477fe640 1 -- 172.21.15.46:0/4033759575 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb15800f770 con 0x7fb1601571d0 2023-12-02T00:31:34.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.750+0000 7fb165100640 1 -- 172.21.15.46:0/4033759575 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1600c8c70 con 0x7fb1601571d0 2023-12-02T00:31:34.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.751+0000 7fb165100640 1 -- 172.21.15.46:0/4033759575 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb16015cee0 con 0x7fb1601571d0 2023-12-02T00:31:34.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.751+0000 7fb1477fe640 1 -- 172.21.15.46:0/4033759575 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb15800f8d0 con 0x7fb1601571d0 2023-12-02T00:31:34.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.752+0000 7fb1477fe640 1 --2- 172.21.15.46:0/4033759575 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb14c06ae40 0x7fb14c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:34.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.752+0000 7fb1477fe640 1 -- 172.21.15.46:0/4033759575 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fb1580895f0 con 0x7fb1601571d0 2023-12-02T00:31:34.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.752+0000 7fb15ed9d640 1 --2- 172.21.15.46:0/4033759575 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb14c06ae40 0x7fb14c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:34.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.752+0000 7fb15ed9d640 1 --2- 172.21.15.46:0/4033759575 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb14c06ae40 0x7fb14c06d300 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb150009a40 tx=0x7fb15000b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:34.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.756+0000 7fb1477fe640 1 -- 172.21.15.46:0/4033759575 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb158054710 con 0x7fb1601571d0 2023-12-02T00:31:34.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.970+0000 7fb165100640 1 -- 172.21.15.46:0/4033759575 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb1600c9150 con 0x7fb1601571d0 2023-12-02T00:31:34.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.970+0000 7fb1477fe640 1 -- 172.21.15.46:0/4033759575 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fb158057d30 con 0x7fb1601571d0 2023-12-02T00:31:34.972 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:31:34.972 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5189,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/4047806587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":4047806587},{"type":"v1","addr":"172.21.15.162:6837","nonce":4047806587}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5620,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/1418783715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1418783715},{"type":"v1","addr":"172.21.15.162:6836","nonce":1418783715}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5979,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/452915985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":452915985},{"type":"v1","addr":"172.21.15.46:6833","nonce":452915985}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-02T00:31:01.187535+0000","modified":"2023-12-02T00:31:04.211234+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5994":{"gid":5994,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/903694874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":903694874},{"type":"v1","addr":"172.21.15.46:6835","nonce":903694874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-12-02T00:31:34.972 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 24 2023-12-02T00:31:34.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.972+0000 7fb165100640 1 -- 172.21.15.46:0/4033759575 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb14c06ae40 msgr2=0x7fb14c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:34.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.972+0000 7fb165100640 1 --2- 172.21.15.46:0/4033759575 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb14c06ae40 0x7fb14c06d300 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb150009a40 tx=0x7fb15000b040 comp rx=0 tx=0).stop 2023-12-02T00:31:34.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.972+0000 7fb165100640 1 -- 172.21.15.46:0/4033759575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1601571d0 msgr2=0x7fb160143a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:34.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.972+0000 7fb165100640 1 --2- 172.21.15.46:0/4033759575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1601571d0 0x7fb160143a60 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fb16008c510 tx=0x7fb158011cf0 comp rx=0 tx=0).stop 2023-12-02T00:31:34.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.973+0000 7fb165100640 1 -- 172.21.15.46:0/4033759575 shutdown_connections 2023-12-02T00:31:34.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.973+0000 7fb165100640 1 --2- 172.21.15.46:0/4033759575 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb14c06ae40 0x7fb14c06d300 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:34.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.973+0000 7fb165100640 1 --2- 172.21.15.46:0/4033759575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1601571d0 0x7fb160143a60 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:34.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.973+0000 7fb165100640 1 --2- 172.21.15.46:0/4033759575 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb160156d50 0x7fb160143520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:34.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.973+0000 7fb165100640 1 --2- 172.21.15.46:0/4033759575 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb1480047c0 0x7fb1600c7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:34.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.973+0000 7fb165100640 1 -- 172.21.15.46:0/4033759575 >> 172.21.15.46:0/4033759575 conn(0x7fb160072020 msgr2=0x7fb1600b4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:34.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.973+0000 7fb165100640 1 -- 172.21.15.46:0/4033759575 shutdown_connections 2023-12-02T00:31:34.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:34.973+0000 7fb165100640 1 -- 172.21.15.46:0/4033759575 wait complete. 2023-12-02T00:31:34.985 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:31:35.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.090+0000 7f3545296640 1 Processor -- start 2023-12-02T00:31:35.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.090+0000 7f3545296640 1 -- start start 2023-12-02T00:31:35.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.091+0000 7f3545296640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f354006f210 0x7f3540140800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:35.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.091+0000 7f3545296640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3540140d40 0x7f354013f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:35.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.091+0000 7f3545296640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f354015ba40 con 0x7f354015c2f0 2023-12-02T00:31:35.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.091+0000 7f3545296640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f354015c6d0 con 0x7f3540140d40 2023-12-02T00:31:35.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.091+0000 7f3545296640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f354013e530 con 0x7f354006f210 2023-12-02T00:31:35.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.091+0000 7f353f7fe640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3540140d40 0x7f354013f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:35.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.091+0000 7f353e7fc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f354006f210 0x7f3540140800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:35.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.091+0000 7f353f7fe640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3540140d40 0x7f354013f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.46:35338/0 (socket says 172.21.15.46:35338) 2023-12-02T00:31:35.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.091+0000 7f353effd640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f354015c2f0 0x7f354006eb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.46:36590/0 (socket says 172.21.15.46:36590) 2023-12-02T00:31:35.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.091+0000 7f353effd640 1 -- 172.21.15.46:0/3248367199 learned_addr learned my addr 172.21.15.46:0/3248367199 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:31:35.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.092+0000 7f353f7fe640 1 -- 172.21.15.46:0/3248367199 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f354006f210 msgr2=0x7f3540140800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:35.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.092+0000 7f353f7fe640 1 --2- 172.21.15.46:0/3248367199 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f354006f210 0x7f3540140800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:35.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.092+0000 7f353f7fe640 1 -- 172.21.15.46:0/3248367199 >> v1:172.21.15.162:6789/0 conn(0x7f354015c2f0 legacy=0x7f354006eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:35.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.092+0000 7f353f7fe640 1 -- 172.21.15.46:0/3248367199 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35401411d0 con 0x7f3540140d40 2023-12-02T00:31:35.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.092+0000 7f353e7fc640 1 --2- 172.21.15.46:0/3248367199 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f354006f210 0x7f3540140800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:31:35.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.092+0000 7f353dffb640 1 -- 172.21.15.46:0/3248367199 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4226836606 0 0) 0x7f354015ba40 con 0x7f354015c2f0 2023-12-02T00:31:35.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.093+0000 7f353f7fe640 1 --2- 172.21.15.46:0/3248367199 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3540140d40 0x7f354013f330 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3530009df0 tx=0x7f353002be70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b580937342bd74f server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:35.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.093+0000 7f353dffb640 1 -- 172.21.15.46:0/3248367199 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353002cc20 con 0x7f3540140d40 2023-12-02T00:31:35.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.093+0000 7f353dffb640 1 -- 172.21.15.46:0/3248367199 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f353002cd80 con 0x7f3540140d40 2023-12-02T00:31:35.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.093+0000 7f353dffb640 1 -- 172.21.15.46:0/3248367199 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3530034660 con 0x7f3540140d40 2023-12-02T00:31:35.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.093+0000 7f3545296640 1 -- 172.21.15.46:0/3248367199 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3540140d40 msgr2=0x7f354013f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:35.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.093+0000 7f3545296640 1 --2- 172.21.15.46:0/3248367199 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3540140d40 0x7f354013f330 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3530009df0 tx=0x7f353002be70 comp rx=0 tx=0).stop 2023-12-02T00:31:35.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.093+0000 7f3545296640 1 -- 172.21.15.46:0/3248367199 shutdown_connections 2023-12-02T00:31:35.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.093+0000 7f3545296640 1 --2- 172.21.15.46:0/3248367199 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f354006f210 0x7f3540140800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:35.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.093+0000 7f3545296640 1 --2- 172.21.15.46:0/3248367199 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3540140d40 0x7f354013f330 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:35.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.093+0000 7f3545296640 1 -- 172.21.15.46:0/3248367199 >> 172.21.15.46:0/3248367199 conn(0x7f354006a6b0 msgr2=0x7f354006aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:35.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.093+0000 7f3545296640 1 -- 172.21.15.46:0/3248367199 shutdown_connections 2023-12-02T00:31:35.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.093+0000 7f3545296640 1 -- 172.21.15.46:0/3248367199 wait complete. 2023-12-02T00:31:35.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.095+0000 7f3545296640 1 Processor -- start 2023-12-02T00:31:35.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.095+0000 7f3545296640 1 -- start start 2023-12-02T00:31:35.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.095+0000 7f3545296640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f354006f210 0x7f3540132810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:35.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.095+0000 7f3545296640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3540140d40 0x7f354006b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:35.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.095+0000 7f3545296640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f354015c2f0 0x7f354006bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:35.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.095+0000 7f3545296640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354013b350 con 0x7f3540140d40 2023-12-02T00:31:35.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.095+0000 7f3545296640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354006d5f0 con 0x7f354015c2f0 2023-12-02T00:31:35.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.095+0000 7f3545296640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f354006c550 con 0x7f354006f210 2023-12-02T00:31:35.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.096+0000 7f353e7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3540140d40 0x7f354006b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:35.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.096+0000 7f353e7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3540140d40 0x7f354006b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45078/0 (socket says 172.21.15.46:45078) 2023-12-02T00:31:35.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.096+0000 7f353e7fc640 1 -- 172.21.15.46:0/1495752590 learned_addr learned my addr 172.21.15.46:0/1495752590 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:35.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.096+0000 7f353effd640 1 --2- 172.21.15.46:0/1495752590 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f354006f210 0x7f3540132810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:35.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.096+0000 7f353f7fe640 1 --2- 172.21.15.46:0/1495752590 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f354015c2f0 0x7f354006bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:35.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.096+0000 7f353e7fc640 1 -- 172.21.15.46:0/1495752590 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f354006f210 msgr2=0x7f3540132810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:35.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.096+0000 7f353e7fc640 1 --2- 172.21.15.46:0/1495752590 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f354006f210 0x7f3540132810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:35.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.096+0000 7f353e7fc640 1 -- 172.21.15.46:0/1495752590 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f354015c2f0 msgr2=0x7f354006bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:35.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.096+0000 7f353e7fc640 1 --2- 172.21.15.46:0/1495752590 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f354015c2f0 0x7f354006bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:35.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.096+0000 7f353e7fc640 1 -- 172.21.15.46:0/1495752590 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3528009400 con 0x7f3540140d40 2023-12-02T00:31:35.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.096+0000 7f353e7fc640 1 --2- 172.21.15.46:0/1495752590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3540140d40 0x7f354006b990 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f352800ebe0 tx=0x7f352800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:35.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.096+0000 7f35377fe640 1 -- 172.21.15.46:0/1495752590 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3528009760 con 0x7f3540140d40 2023-12-02T00:31:35.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.096+0000 7f35377fe640 1 -- 172.21.15.46:0/1495752590 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f352800cc30 con 0x7f3540140d40 2023-12-02T00:31:35.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.096+0000 7f35377fe640 1 -- 172.21.15.46:0/1495752590 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f352800eee0 con 0x7f3540140d40 2023-12-02T00:31:35.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.096+0000 7f3545296640 1 -- 172.21.15.46:0/1495752590 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3530009a70 con 0x7f3540140d40 2023-12-02T00:31:35.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.096+0000 7f3545296640 1 -- 172.21.15.46:0/1495752590 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f354006cb10 con 0x7f3540140d40 2023-12-02T00:31:35.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.097+0000 7f3545296640 1 -- 172.21.15.46:0/1495752590 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3540069400 con 0x7f3540140d40 2023-12-02T00:31:35.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.098+0000 7f35377fe640 1 -- 172.21.15.46:0/1495752590 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3528013470 con 0x7f3540140d40 2023-12-02T00:31:35.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.098+0000 7f35377fe640 1 --2- 172.21.15.46:0/1495752590 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f352006ae40 0x7f352006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:35.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.098+0000 7f35377fe640 1 -- 172.21.15.46:0/1495752590 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f3528089880 con 0x7f3540140d40 2023-12-02T00:31:35.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.098+0000 7f353effd640 1 --2- 172.21.15.46:0/1495752590 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f352006ae40 0x7f352006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:35.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.099+0000 7f353effd640 1 --2- 172.21.15.46:0/1495752590 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f352006ae40 0x7f352006d300 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3538007fd0 tx=0x7f3538007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:35.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.102+0000 7f35377fe640 1 -- 172.21.15.46:0/1495752590 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f35280549a0 con 0x7f3540140d40 2023-12-02T00:31:35.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.283+0000 7f3545296640 1 -- 172.21.15.46:0/1495752590 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f354015b150 con 0x7f3540140d40 2023-12-02T00:31:35.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.284+0000 7f35377fe640 1 -- 172.21.15.46:0/1495752590 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+16585 (secure 0 0 0) 0x7f3528017090 con 0x7f3540140d40 2023-12-02T00:31:35.287 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:31:35.287 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":53,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:31:31.941814+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-12-02T00:30:59.469070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-12-02T00:31:00.174610+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.28","mappings":[{"from":0,"to":6}]},{"pgid":"8.38","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:31:35.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.286+0000 7f3545296640 1 -- 172.21.15.46:0/1495752590 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f352006ae40 msgr2=0x7f352006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:35.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.286+0000 7f3545296640 1 --2- 172.21.15.46:0/1495752590 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f352006ae40 0x7f352006d300 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3538007fd0 tx=0x7f3538007de0 comp rx=0 tx=0).stop 2023-12-02T00:31:35.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.286+0000 7f3545296640 1 -- 172.21.15.46:0/1495752590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3540140d40 msgr2=0x7f354006b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:35.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.286+0000 7f3545296640 1 --2- 172.21.15.46:0/1495752590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3540140d40 0x7f354006b990 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f352800ebe0 tx=0x7f352800c520 comp rx=0 tx=0).stop 2023-12-02T00:31:35.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.286+0000 7f3545296640 1 -- 172.21.15.46:0/1495752590 shutdown_connections 2023-12-02T00:31:35.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.286+0000 7f3545296640 1 --2- 172.21.15.46:0/1495752590 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f352006ae40 0x7f352006d300 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:35.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.286+0000 7f3545296640 1 --2- 172.21.15.46:0/1495752590 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f354015c2f0 0x7f354006bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:35.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.286+0000 7f3545296640 1 --2- 172.21.15.46:0/1495752590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3540140d40 0x7f354006b990 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:35.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.286+0000 7f3545296640 1 --2- 172.21.15.46:0/1495752590 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f354006f210 0x7f3540132810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:35.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.286+0000 7f3545296640 1 -- 172.21.15.46:0/1495752590 >> 172.21.15.46:0/1495752590 conn(0x7f354006a6b0 msgr2=0x7f354015ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:35.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.286+0000 7f3545296640 1 -- 172.21.15.46:0/1495752590 shutdown_connections 2023-12-02T00:31:35.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.286+0000 7f3545296640 1 -- 172.21.15.46:0/1495752590 wait complete. 2023-12-02T00:31:35.298 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-02T00:31:35.299 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-02T00:31:35.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.404+0000 7f537e5d7640 1 Processor -- start 2023-12-02T00:31:35.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.404+0000 7f537e5d7640 1 -- start start 2023-12-02T00:31:35.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.405+0000 7f537e5d7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f537815d340 0x7f537806b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:35.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.405+0000 7f537e5d7640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f53780ad640 0x7f537806ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:35.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.405+0000 7f537e5d7640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f537815d720 con 0x7f53780aeb80 2023-12-02T00:31:35.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.405+0000 7f537e5d7640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53780adae0 con 0x7f53780ad640 2023-12-02T00:31:35.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.405+0000 7f537e5d7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53780addc0 con 0x7f537815d340 2023-12-02T00:31:35.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.405+0000 7f537cb74640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f53780aeb80 0x7f5378160ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50848/0 (socket says 172.21.15.46:50848) 2023-12-02T00:31:35.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.405+0000 7f537cb74640 1 -- 172.21.15.46:0/364470783 learned_addr learned my addr 172.21.15.46:0/364470783 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:31:35.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.405+0000 7f5377fff640 1 --2- 172.21.15.46:0/364470783 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f537815d340 0x7f537806b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:35.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.406+0000 7f5376ffd640 1 -- 172.21.15.46:0/364470783 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3224042707 0 0) 0x7f537815d720 con 0x7f53780aeb80 2023-12-02T00:31:35.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.406+0000 7f53777fe640 1 --2- 172.21.15.46:0/364470783 >> v2:172.21.15.162:3300/0 conn(0x7f53780ad640 0x7f537806ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:35.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.406+0000 7f5376ffd640 1 -- 172.21.15.46:0/364470783 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5360003520 con 0x7f53780aeb80 2023-12-02T00:31:35.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.406+0000 7f5376ffd640 1 -- 172.21.15.46:0/364470783 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2741483555 0 0) 0x7f5360003520 con 0x7f53780aeb80 2023-12-02T00:31:35.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.406+0000 7f5376ffd640 1 -- 172.21.15.46:0/364470783 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f537815d340 msgr2=0x7f537806b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:35.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.406+0000 7f5376ffd640 1 --2- 172.21.15.46:0/364470783 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f537815d340 0x7f537806b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:35.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.406+0000 7f5376ffd640 1 -- 172.21.15.46:0/364470783 >> v2:172.21.15.162:3300/0 conn(0x7f53780ad640 msgr2=0x7f537806ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:35.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.406+0000 7f5376ffd640 1 --2- 172.21.15.46:0/364470783 >> v2:172.21.15.162:3300/0 conn(0x7f53780ad640 0x7f537806ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:35.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.406+0000 7f5376ffd640 1 -- 172.21.15.46:0/364470783 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53780ae0a0 con 0x7f53780aeb80 2023-12-02T00:31:35.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.406+0000 7f5376ffd640 1 -- 172.21.15.46:0/364470783 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7f53700019f0 con 0x7f53780aeb80 2023-12-02T00:31:35.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.406+0000 7f5376ffd640 1 -- 172.21.15.46:0/364470783 >> v1:172.21.15.46:6789/0 conn(0x7f53780aeb80 legacy=0x7f5378160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:35.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.406+0000 7f5376ffd640 1 --2- 172.21.15.46:0/364470783 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f537815d340 0x7f5360003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:35.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.406+0000 7f5376ffd640 1 --2- 172.21.15.46:0/364470783 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5360003ef0 0x7f53600047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:35.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.406+0000 7f5376ffd640 1 --2- 172.21.15.46:0/364470783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5360004d00 0x7f5360005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:35.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.406+0000 7f5376ffd640 1 -- 172.21.15.46:0/364470783 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53780adae0 con 0x7f5360004d00 2023-12-02T00:31:35.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.406+0000 7f5376ffd640 1 -- 172.21.15.46:0/364470783 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53780addc0 con 0x7f5360003ef0 2023-12-02T00:31:35.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.406+0000 7f5376ffd640 1 -- 172.21.15.46:0/364470783 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f536000b980 con 0x7f537815d340 2023-12-02T00:31:35.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.406+0000 7f5377fff640 1 --2- 172.21.15.46:0/364470783 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f537815d340 0x7f5360003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:35.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.406+0000 7f53777fe640 1 --2- 172.21.15.46:0/364470783 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5360003ef0 0x7f53600047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:35.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.406+0000 7f537cb74640 1 --2- 172.21.15.46:0/364470783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5360004d00 0x7f5360005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:35.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.406+0000 7f5377fff640 1 -- 172.21.15.46:0/364470783 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5360003ef0 msgr2=0x7f53600047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:35.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.406+0000 7f5377fff640 1 --2- 172.21.15.46:0/364470783 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5360003ef0 0x7f53600047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:35.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.406+0000 7f5377fff640 1 -- 172.21.15.46:0/364470783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5360004d00 msgr2=0x7f5360005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:35.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.406+0000 7f5377fff640 1 --2- 172.21.15.46:0/364470783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5360004d00 0x7f5360005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:35.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.406+0000 7f5377fff640 1 -- 172.21.15.46:0/364470783 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5360003520 con 0x7f537815d340 2023-12-02T00:31:35.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.406+0000 7f53777fe640 1 --2- 172.21.15.46:0/364470783 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5360003ef0 0x7f53600047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:31:35.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.406+0000 7f537cb74640 1 --2- 172.21.15.46:0/364470783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5360004d00 0x7f5360005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:31:35.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.407+0000 7f5377fff640 1 --2- 172.21.15.46:0/364470783 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f537815d340 0x7f5360003970 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f536c009ac0 tx=0x7f53600057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b905d1cf76277f6 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:35.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.407+0000 7f5376ffd640 1 -- 172.21.15.46:0/364470783 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536c02c710 con 0x7f537815d340 2023-12-02T00:31:35.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.407+0000 7f5376ffd640 1 -- 172.21.15.46:0/364470783 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f536c02c870 con 0x7f537815d340 2023-12-02T00:31:35.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.407+0000 7f5376ffd640 1 -- 172.21.15.46:0/364470783 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536c02cc00 con 0x7f537815d340 2023-12-02T00:31:35.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.408+0000 7f537e5d7640 1 -- 172.21.15.46:0/364470783 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f537815d340 msgr2=0x7f5360003970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:35.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.408+0000 7f537e5d7640 1 --2- 172.21.15.46:0/364470783 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f537815d340 0x7f5360003970 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f536c009ac0 tx=0x7f53600057f0 comp rx=0 tx=0).stop 2023-12-02T00:31:35.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.408+0000 7f537cb74640 1 -- 172.21.15.46:0/364470783 reap_dead start 2023-12-02T00:31:35.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.408+0000 7f537e5d7640 1 -- 172.21.15.46:0/364470783 shutdown_connections 2023-12-02T00:31:35.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.408+0000 7f537e5d7640 1 -- 172.21.15.46:0/364470783 >> 172.21.15.46:0/364470783 conn(0x7f5378070e70 msgr2=0x7f5378071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:35.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.408+0000 7f537e5d7640 1 -- 172.21.15.46:0/364470783 shutdown_connections 2023-12-02T00:31:35.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.408+0000 7f537e5d7640 1 -- 172.21.15.46:0/364470783 wait complete. 2023-12-02T00:31:35.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.409+0000 7f537e5d7640 1 Processor -- start 2023-12-02T00:31:35.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.409+0000 7f537e5d7640 1 -- start start 2023-12-02T00:31:35.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.410+0000 7f537e5d7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f537815d340 0x7f5378154aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:35.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.410+0000 7f537e5d7640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f53781f3520 0x7f53781f6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:35.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.410+0000 7f537e5d7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53781f39a0 0x7f53781f65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:35.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.410+0000 7f537e5d7640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5378154fe0 con 0x7f537815d340 2023-12-02T00:31:35.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.410+0000 7f537e5d7640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53780cc830 con 0x7f53781f3520 2023-12-02T00:31:35.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.410+0000 7f537e5d7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53780ccb10 con 0x7f53781f39a0 2023-12-02T00:31:35.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.410+0000 7f5377fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f537815d340 0x7f5378154aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:35.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.410+0000 7f5377fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f537815d340 0x7f5378154aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45090/0 (socket says 172.21.15.46:45090) 2023-12-02T00:31:35.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.410+0000 7f5377fff640 1 -- 172.21.15.46:0/3572952623 learned_addr learned my addr 172.21.15.46:0/3572952623 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:35.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.410+0000 7f53777fe640 1 --2- 172.21.15.46:0/3572952623 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f53781f3520 0x7f53781f6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:35.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.410+0000 7f537cb74640 1 --2- 172.21.15.46:0/3572952623 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53781f39a0 0x7f53781f65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:35.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.410+0000 7f5377fff640 1 -- 172.21.15.46:0/3572952623 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53781f39a0 msgr2=0x7f53781f65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:35.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.410+0000 7f5377fff640 1 --2- 172.21.15.46:0/3572952623 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53781f39a0 0x7f53781f65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:35.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.410+0000 7f5377fff640 1 -- 172.21.15.46:0/3572952623 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f53781f3520 msgr2=0x7f53781f6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:35.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.410+0000 7f5377fff640 1 --2- 172.21.15.46:0/3572952623 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f53781f3520 0x7f53781f6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:35.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.410+0000 7f5377fff640 1 -- 172.21.15.46:0/3572952623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53780ccdf0 con 0x7f537815d340 2023-12-02T00:31:35.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.411+0000 7f5377fff640 1 --2- 172.21.15.46:0/3572952623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f537815d340 0x7f5378154aa0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f536c009be0 tx=0x7f536c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:35.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.411+0000 7f5374ff9640 1 -- 172.21.15.46:0/3572952623 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536c003810 con 0x7f537815d340 2023-12-02T00:31:35.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.411+0000 7f5374ff9640 1 -- 172.21.15.46:0/3572952623 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f536c003970 con 0x7f537815d340 2023-12-02T00:31:35.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.411+0000 7f5374ff9640 1 -- 172.21.15.46:0/3572952623 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536c003d00 con 0x7f537815d340 2023-12-02T00:31:35.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.411+0000 7f537e5d7640 1 -- 172.21.15.46:0/3572952623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53780cd0d0 con 0x7f537815d340 2023-12-02T00:31:35.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.411+0000 7f537e5d7640 1 -- 172.21.15.46:0/3572952623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53780cd8d0 con 0x7f537815d340 2023-12-02T00:31:35.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.412+0000 7f537e5d7640 1 -- 172.21.15.46:0/3572952623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5378155150 con 0x7f537815d340 2023-12-02T00:31:35.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.412+0000 7f5374ff9640 1 -- 172.21.15.46:0/3572952623 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f536c03b070 con 0x7f537815d340 2023-12-02T00:31:35.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.413+0000 7f5374ff9640 1 --2- 172.21.15.46:0/3572952623 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f536406ae40 0x7f536406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:35.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.413+0000 7f5374ff9640 1 -- 172.21.15.46:0/3572952623 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f536c0ab3f0 con 0x7f537815d340 2023-12-02T00:31:35.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.413+0000 7f53777fe640 1 --2- 172.21.15.46:0/3572952623 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f536406ae40 0x7f536406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:35.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.413+0000 7f53777fe640 1 --2- 172.21.15.46:0/3572952623 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f536406ae40 0x7f536406d300 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5368009b20 tx=0x7f5368008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:35.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.417+0000 7f5374ff9640 1 -- 172.21.15.46:0/3572952623 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f536c0766c0 con 0x7f537815d340 2023-12-02T00:31:35.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.631+0000 7f537e5d7640 1 -- 172.21.15.46:0/3572952623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5378069400 con 0x7f537815d340 2023-12-02T00:31:35.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.631+0000 7f5374ff9640 1 -- 172.21.15.46:0/3572952623 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f536c003e60 con 0x7f537815d340 2023-12-02T00:31:35.633 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:31:35.633 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5189,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/4047806587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":4047806587},{"type":"v1","addr":"172.21.15.162:6837","nonce":4047806587}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5620,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/1418783715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1418783715},{"type":"v1","addr":"172.21.15.162:6836","nonce":1418783715}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5979,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/452915985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":452915985},{"type":"v1","addr":"172.21.15.46:6833","nonce":452915985}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-02T00:31:01.187535+0000","modified":"2023-12-02T00:31:04.211234+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5994":{"gid":5994,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/903694874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":903694874},{"type":"v1","addr":"172.21.15.46:6835","nonce":903694874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-12-02T00:31:35.633 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 24 2023-12-02T00:31:35.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.633+0000 7f537e5d7640 1 -- 172.21.15.46:0/3572952623 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f536406ae40 msgr2=0x7f536406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:35.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.633+0000 7f537e5d7640 1 --2- 172.21.15.46:0/3572952623 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f536406ae40 0x7f536406d300 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5368009b20 tx=0x7f5368008040 comp rx=0 tx=0).stop 2023-12-02T00:31:35.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.633+0000 7f537e5d7640 1 -- 172.21.15.46:0/3572952623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f537815d340 msgr2=0x7f5378154aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:35.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.633+0000 7f537e5d7640 1 --2- 172.21.15.46:0/3572952623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f537815d340 0x7f5378154aa0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f536c009be0 tx=0x7f536c004730 comp rx=0 tx=0).stop 2023-12-02T00:31:35.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.633+0000 7f537e5d7640 1 -- 172.21.15.46:0/3572952623 shutdown_connections 2023-12-02T00:31:35.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.633+0000 7f537e5d7640 1 --2- 172.21.15.46:0/3572952623 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f536406ae40 0x7f536406d300 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:35.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.633+0000 7f537e5d7640 1 --2- 172.21.15.46:0/3572952623 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53781f39a0 0x7f53781f65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:35.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.633+0000 7f537e5d7640 1 --2- 172.21.15.46:0/3572952623 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f53781f3520 0x7f53781f6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:35.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.633+0000 7f537e5d7640 1 --2- 172.21.15.46:0/3572952623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f537815d340 0x7f5378154aa0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:35.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.633+0000 7f537e5d7640 1 -- 172.21.15.46:0/3572952623 >> 172.21.15.46:0/3572952623 conn(0x7f5378070e70 msgr2=0x7f53780b0490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:35.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.633+0000 7f537e5d7640 1 -- 172.21.15.46:0/3572952623 shutdown_connections 2023-12-02T00:31:35.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.633+0000 7f537e5d7640 1 -- 172.21.15.46:0/3572952623 wait complete. 2023-12-02T00:31:35.645 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-02T00:31:35.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.751+0000 7f544152c640 1 Processor -- start 2023-12-02T00:31:35.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.751+0000 7f544152c640 1 -- start start 2023-12-02T00:31:35.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.751+0000 7f544152c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f543c0b1400 0x7f543c15a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:35.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.751+0000 7f544152c640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f543c15e2a0 con 0x7f543c15e730 2023-12-02T00:31:35.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.751+0000 7f544152c640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f543c15b240 con 0x7f543c143490 2023-12-02T00:31:35.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.751+0000 7f544152c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f543c15b520 con 0x7f543c0b1400 2023-12-02T00:31:35.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.752+0000 7f543b7fe640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f543c15e730 0x7f543c15aa20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50864/0 (socket says 172.21.15.46:50864) 2023-12-02T00:31:35.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.752+0000 7f543b7fe640 1 -- 172.21.15.46:0/2081117116 learned_addr learned my addr 172.21.15.46:0/2081117116 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:31:35.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.752+0000 7f543a7fc640 1 --2- 172.21.15.46:0/2081117116 >> v2:172.21.15.162:3300/0 conn(0x7f543c0b1400 0x7f543c15a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:35.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.752+0000 7f5439ffb640 1 -- 172.21.15.46:0/2081117116 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1728227024 0 0) 0x7f543c15e2a0 con 0x7f543c15e730 2023-12-02T00:31:35.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.752+0000 7f5439ffb640 1 -- 172.21.15.46:0/2081117116 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5424005730 con 0x7f543c15e730 2023-12-02T00:31:35.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.752+0000 7f5439ffb640 1 -- 172.21.15.46:0/2081117116 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1294654325 0 0) 0x7f5424005730 con 0x7f543c15e730 2023-12-02T00:31:35.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.752+0000 7f5439ffb640 1 -- 172.21.15.46:0/2081117116 >> v2:172.21.15.162:3300/0 conn(0x7f543c0b1400 msgr2=0x7f543c15a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:35.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.752+0000 7f5439ffb640 1 --2- 172.21.15.46:0/2081117116 >> v2:172.21.15.162:3300/0 conn(0x7f543c0b1400 0x7f543c15a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:35.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.752+0000 7f5439ffb640 1 -- 172.21.15.46:0/2081117116 >> v1:172.21.15.162:6789/0 conn(0x7f543c143490 legacy=0x7f543c0b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:35.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.752+0000 7f5439ffb640 1 -- 172.21.15.46:0/2081117116 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f543c15dc50 con 0x7f543c15e730 2023-12-02T00:31:35.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.752+0000 7f5439ffb640 1 -- 172.21.15.46:0/2081117116 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7f54340019f0 con 0x7f543c15e730 2023-12-02T00:31:35.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.752+0000 7f5439ffb640 1 -- 172.21.15.46:0/2081117116 >> v1:172.21.15.46:6789/0 conn(0x7f543c15e730 legacy=0x7f543c15aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:35.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.752+0000 7f5439ffb640 1 --2- 172.21.15.46:0/2081117116 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5424005b80 0x7f5424005f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:35.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.752+0000 7f5439ffb640 1 --2- 172.21.15.46:0/2081117116 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5424006580 0x7f5424006a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:35.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.752+0000 7f5439ffb640 1 --2- 172.21.15.46:0/2081117116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5424007250 0x7f542400db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:35.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.752+0000 7f5439ffb640 1 -- 172.21.15.46:0/2081117116 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543c15b520 con 0x7f5424007250 2023-12-02T00:31:35.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.752+0000 7f5439ffb640 1 -- 172.21.15.46:0/2081117116 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542400e1b0 con 0x7f5424005b80 2023-12-02T00:31:35.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.752+0000 7f5439ffb640 1 -- 172.21.15.46:0/2081117116 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f542400e370 con 0x7f5424006580 2023-12-02T00:31:35.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.752+0000 7f543b7fe640 1 --2- 172.21.15.46:0/2081117116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5424007250 0x7f542400db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:35.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.752+0000 7f543affd640 1 --2- 172.21.15.46:0/2081117116 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5424005b80 0x7f5424005f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:35.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.753+0000 7f543a7fc640 1 --2- 172.21.15.46:0/2081117116 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5424006580 0x7f5424006a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:35.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.753+0000 7f543b7fe640 1 -- 172.21.15.46:0/2081117116 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5424006580 msgr2=0x7f5424006a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:35.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.753+0000 7f543b7fe640 1 --2- 172.21.15.46:0/2081117116 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5424006580 0x7f5424006a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:35.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.753+0000 7f543b7fe640 1 -- 172.21.15.46:0/2081117116 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5424005b80 msgr2=0x7f5424005f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:35.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.753+0000 7f543b7fe640 1 --2- 172.21.15.46:0/2081117116 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5424005b80 0x7f5424005f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:35.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.753+0000 7f543b7fe640 1 -- 172.21.15.46:0/2081117116 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5424005730 con 0x7f5424007250 2023-12-02T00:31:35.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.753+0000 7f543b7fe640 1 -- 172.21.15.46:0/2081117116 reap_dead start 2023-12-02T00:31:35.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.754+0000 7f543b7fe640 1 --2- 172.21.15.46:0/2081117116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5424007250 0x7f542400db90 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f543400a830 tx=0x7f5424007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7258ef17100f9f2f server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:35.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.754+0000 7f5439ffb640 1 -- 172.21.15.46:0/2081117116 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f543402cea0 con 0x7f5424007250 2023-12-02T00:31:35.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.754+0000 7f5439ffb640 1 -- 172.21.15.46:0/2081117116 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f543402f070 con 0x7f5424007250 2023-12-02T00:31:35.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.754+0000 7f5439ffb640 1 -- 172.21.15.46:0/2081117116 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5434034890 con 0x7f5424007250 2023-12-02T00:31:35.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.754+0000 7f544152c640 1 -- 172.21.15.46:0/2081117116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5424007250 msgr2=0x7f542400db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:35.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.754+0000 7f544152c640 1 --2- 172.21.15.46:0/2081117116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5424007250 0x7f542400db90 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f543400a830 tx=0x7f5424007a00 comp rx=0 tx=0).stop 2023-12-02T00:31:35.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.754+0000 7f544152c640 1 -- 172.21.15.46:0/2081117116 shutdown_connections 2023-12-02T00:31:35.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.754+0000 7f544152c640 1 --2- 172.21.15.46:0/2081117116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5424007250 0x7f542400db90 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:35.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.754+0000 7f544152c640 1 -- 172.21.15.46:0/2081117116 >> 172.21.15.46:0/2081117116 conn(0x7f543c06a4d0 msgr2=0x7f543c06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:35.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.754+0000 7f544152c640 1 -- 172.21.15.46:0/2081117116 shutdown_connections 2023-12-02T00:31:35.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.754+0000 7f544152c640 1 -- 172.21.15.46:0/2081117116 wait complete. 2023-12-02T00:31:35.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.756+0000 7f544152c640 1 Processor -- start 2023-12-02T00:31:35.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.756+0000 7f544152c640 1 -- start start 2023-12-02T00:31:35.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.756+0000 7f544152c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5424007250 0x7f543c071e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:35.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.756+0000 7f544152c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f543c1388c0 0x7f543c13e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:35.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.756+0000 7f544152c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f543c070da0 0x7f543c071200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:35.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.756+0000 7f544152c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543c13cbc0 con 0x7f5424007250 2023-12-02T00:31:35.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.756+0000 7f544152c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543c0addd0 con 0x7f543c070da0 2023-12-02T00:31:35.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.756+0000 7f544152c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f543c13c600 con 0x7f543c1388c0 2023-12-02T00:31:35.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.756+0000 7f543affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5424007250 0x7f543c071e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:35.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.756+0000 7f543affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5424007250 0x7f543c071e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45098/0 (socket says 172.21.15.46:45098) 2023-12-02T00:31:35.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.756+0000 7f543affd640 1 -- 172.21.15.46:0/3637756501 learned_addr learned my addr 172.21.15.46:0/3637756501 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:35.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.757+0000 7f543a7fc640 1 --2- 172.21.15.46:0/3637756501 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f543c1388c0 0x7f543c13e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:35.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.757+0000 7f543b7fe640 1 --2- 172.21.15.46:0/3637756501 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f543c070da0 0x7f543c071200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:35.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.757+0000 7f543affd640 1 -- 172.21.15.46:0/3637756501 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f543c1388c0 msgr2=0x7f543c13e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:35.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.757+0000 7f543affd640 1 --2- 172.21.15.46:0/3637756501 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f543c1388c0 0x7f543c13e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:35.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.757+0000 7f543affd640 1 -- 172.21.15.46:0/3637756501 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f543c070da0 msgr2=0x7f543c071200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:35.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.757+0000 7f543affd640 1 --2- 172.21.15.46:0/3637756501 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f543c070da0 0x7f543c071200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:35.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.757+0000 7f543affd640 1 -- 172.21.15.46:0/3637756501 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f543c13c8e0 con 0x7f5424007250 2023-12-02T00:31:35.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.757+0000 7f543a7fc640 1 --2- 172.21.15.46:0/3637756501 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f543c1388c0 0x7f543c13e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:31:35.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.757+0000 7f543b7fe640 1 --2- 172.21.15.46:0/3637756501 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f543c070da0 0x7f543c071200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:31:35.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.757+0000 7f543affd640 1 --2- 172.21.15.46:0/3637756501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5424007250 0x7f543c071e80 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f542c00f7f0 tx=0x7f542c00fcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:35.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.757+0000 7f54237fe640 1 -- 172.21.15.46:0/3637756501 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542c00fea0 con 0x7f5424007250 2023-12-02T00:31:35.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.757+0000 7f54237fe640 1 -- 172.21.15.46:0/3637756501 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f542c0052e0 con 0x7f5424007250 2023-12-02T00:31:35.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.757+0000 7f544152c640 1 -- 172.21.15.46:0/3637756501 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f543c0ada60 con 0x7f5424007250 2023-12-02T00:31:35.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.757+0000 7f54237fe640 1 -- 172.21.15.46:0/3637756501 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542c00d970 con 0x7f5424007250 2023-12-02T00:31:35.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.757+0000 7f544152c640 1 -- 172.21.15.46:0/3637756501 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f543c13e600 con 0x7f5424007250 2023-12-02T00:31:35.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.758+0000 7f544152c640 1 -- 172.21.15.46:0/3637756501 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f543c069400 con 0x7f5424007250 2023-12-02T00:31:35.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.759+0000 7f54237fe640 1 -- 172.21.15.46:0/3637756501 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f542c00dad0 con 0x7f5424007250 2023-12-02T00:31:35.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.759+0000 7f54237fe640 1 --2- 172.21.15.46:0/3637756501 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f542806ae70 0x7f542806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:35.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.759+0000 7f54237fe640 1 -- 172.21.15.46:0/3637756501 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f542c0895f0 con 0x7f5424007250 2023-12-02T00:31:35.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.759+0000 7f543a7fc640 1 --2- 172.21.15.46:0/3637756501 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f542806ae70 0x7f542806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:35.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.760+0000 7f543a7fc640 1 --2- 172.21.15.46:0/3637756501 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f542806ae70 0x7f542806d330 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f54300093d0 tx=0x7f5430008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:35.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.763+0000 7f54237fe640 1 -- 172.21.15.46:0/3637756501 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f542c0546f0 con 0x7f5424007250 2023-12-02T00:31:35.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.979+0000 7f544152c640 1 -- 172.21.15.46:0/3637756501 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f543c0b2820 con 0x7f5424007250 2023-12-02T00:31:35.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.979+0000 7f54237fe640 1 -- 172.21.15.46:0/3637756501 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f542c057d10 con 0x7f5424007250 2023-12-02T00:31:35.981 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:31:35.982 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5189,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/4047806587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":4047806587},{"type":"v1","addr":"172.21.15.162:6837","nonce":4047806587}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5620,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/1418783715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1418783715},{"type":"v1","addr":"172.21.15.162:6836","nonce":1418783715}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5979,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/452915985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":452915985},{"type":"v1","addr":"172.21.15.46:6833","nonce":452915985}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-02T00:31:01.187535+0000","modified":"2023-12-02T00:31:04.211234+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5994":{"gid":5994,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/903694874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":903694874},{"type":"v1","addr":"172.21.15.46:6835","nonce":903694874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-12-02T00:31:35.983 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 24 2023-12-02T00:31:35.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.981+0000 7f544152c640 1 -- 172.21.15.46:0/3637756501 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f542806ae70 msgr2=0x7f542806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:35.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.981+0000 7f544152c640 1 --2- 172.21.15.46:0/3637756501 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f542806ae70 0x7f542806d330 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f54300093d0 tx=0x7f5430008040 comp rx=0 tx=0).stop 2023-12-02T00:31:35.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.981+0000 7f544152c640 1 -- 172.21.15.46:0/3637756501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5424007250 msgr2=0x7f543c071e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:35.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.981+0000 7f544152c640 1 --2- 172.21.15.46:0/3637756501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5424007250 0x7f543c071e80 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f542c00f7f0 tx=0x7f542c00fcb0 comp rx=0 tx=0).stop 2023-12-02T00:31:35.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.981+0000 7f544152c640 1 -- 172.21.15.46:0/3637756501 shutdown_connections 2023-12-02T00:31:35.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.981+0000 7f544152c640 1 --2- 172.21.15.46:0/3637756501 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f542806ae70 0x7f542806d330 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:35.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.981+0000 7f544152c640 1 --2- 172.21.15.46:0/3637756501 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f543c070da0 0x7f543c071200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:35.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.981+0000 7f544152c640 1 --2- 172.21.15.46:0/3637756501 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f543c1388c0 0x7f543c13e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:35.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.981+0000 7f544152c640 1 --2- 172.21.15.46:0/3637756501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5424007250 0x7f543c071e80 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:35.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.981+0000 7f544152c640 1 -- 172.21.15.46:0/3637756501 >> 172.21.15.46:0/3637756501 conn(0x7f543c06a4d0 msgr2=0x7f543c0ce420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:35.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.982+0000 7f544152c640 1 -- 172.21.15.46:0/3637756501 shutdown_connections 2023-12-02T00:31:35.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:35.982+0000 7f544152c640 1 -- 172.21.15.46:0/3637756501 wait complete. 2023-12-02T00:31:35.994 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:31:36.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.099+0000 7f8704ba5640 1 Processor -- start 2023-12-02T00:31:36.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.099+0000 7f8704ba5640 1 -- start start 2023-12-02T00:31:36.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.100+0000 7f8704ba5640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f870015d330 0x7f87000b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:36.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.100+0000 7f8704ba5640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f87000d3900 0x7f87000b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:36.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.100+0000 7f8704ba5640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87000b2b10 con 0x7f87000d2910 2023-12-02T00:31:36.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.100+0000 7f8704ba5640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87000d1590 con 0x7f87000d3900 2023-12-02T00:31:36.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.100+0000 7f8704ba5640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87000d1870 con 0x7f870015d330 2023-12-02T00:31:36.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.100+0000 7f86fed9d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f87000d3900 0x7f87000b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:36.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.100+0000 7f86fed9d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f87000d3900 0x7f87000b6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45104/0 (socket says 172.21.15.46:45104) 2023-12-02T00:31:36.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.100+0000 7f86fed9d640 1 -- 172.21.15.46:0/3475481635 learned_addr learned my addr 172.21.15.46:0/3475481635 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:36.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.100+0000 7f86fdd9b640 1 --1- 172.21.15.46:0/3475481635 >> v1:172.21.15.46:6789/0 conn(0x7f87000d2910 0x7f87000d31f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50872/0 (socket says 172.21.15.46:50872) 2023-12-02T00:31:36.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.101+0000 7f86fe59c640 1 --2- 172.21.15.46:0/3475481635 >> v2:172.21.15.162:3300/0 conn(0x7f870015d330 0x7f87000b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:36.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.101+0000 7f86fed9d640 1 -- 172.21.15.46:0/3475481635 >> v2:172.21.15.162:3300/0 conn(0x7f870015d330 msgr2=0x7f87000b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:36.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.101+0000 7f86fed9d640 1 --2- 172.21.15.46:0/3475481635 >> v2:172.21.15.162:3300/0 conn(0x7f870015d330 0x7f87000b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:36.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.101+0000 7f86fed9d640 1 -- 172.21.15.46:0/3475481635 >> v1:172.21.15.46:6789/0 conn(0x7f87000d2910 legacy=0x7f87000d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:36.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.101+0000 7f86fed9d640 1 -- 172.21.15.46:0/3475481635 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87000d1b50 con 0x7f87000d3900 2023-12-02T00:31:36.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.101+0000 7f86fe59c640 1 --2- 172.21.15.46:0/3475481635 >> v2:172.21.15.162:3300/0 conn(0x7f870015d330 0x7f87000b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:31:36.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.101+0000 7f86fed9d640 1 --2- 172.21.15.46:0/3475481635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87000d3900 0x7f87000b6390 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f87000d0d10 tx=0x7f86f002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7313c339609e9968 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:36.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.102+0000 7f86fd59a640 1 -- 172.21.15.46:0/3475481635 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f002ca80 con 0x7f87000d3900 2023-12-02T00:31:36.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.102+0000 7f86fd59a640 1 -- 172.21.15.46:0/3475481635 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86f002cbe0 con 0x7f87000d3900 2023-12-02T00:31:36.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.102+0000 7f86fd59a640 1 -- 172.21.15.46:0/3475481635 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f00343f0 con 0x7f87000d3900 2023-12-02T00:31:36.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.102+0000 7f8704ba5640 1 -- 172.21.15.46:0/3475481635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87000d3900 msgr2=0x7f87000b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:36.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.102+0000 7f8704ba5640 1 --2- 172.21.15.46:0/3475481635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87000d3900 0x7f87000b6390 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f87000d0d10 tx=0x7f86f002bee0 comp rx=0 tx=0).stop 2023-12-02T00:31:36.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.102+0000 7f8704ba5640 1 -- 172.21.15.46:0/3475481635 shutdown_connections 2023-12-02T00:31:36.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.102+0000 7f8704ba5640 1 --2- 172.21.15.46:0/3475481635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87000d3900 0x7f87000b6390 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:36.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.102+0000 7f8704ba5640 1 --2- 172.21.15.46:0/3475481635 >> v2:172.21.15.162:3300/0 conn(0x7f870015d330 0x7f87000b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:36.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.102+0000 7f8704ba5640 1 -- 172.21.15.46:0/3475481635 >> 172.21.15.46:0/3475481635 conn(0x7f8700072020 msgr2=0x7f8700072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:36.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.102+0000 7f8704ba5640 1 -- 172.21.15.46:0/3475481635 shutdown_connections 2023-12-02T00:31:36.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.102+0000 7f8704ba5640 1 -- 172.21.15.46:0/3475481635 wait complete. 2023-12-02T00:31:36.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.104+0000 7f8704ba5640 1 Processor -- start 2023-12-02T00:31:36.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.104+0000 7f8704ba5640 1 -- start start 2023-12-02T00:31:36.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.104+0000 7f8704ba5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f87000d2910 0x7f8700152910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:36.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.104+0000 7f8704ba5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87000d3900 0x7f87000c8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:36.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.104+0000 7f8704ba5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f870015d330 0x7f87000c7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:36.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.104+0000 7f8704ba5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f870015dee0 con 0x7f87000d3900 2023-12-02T00:31:36.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.104+0000 7f8704ba5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87000c94c0 con 0x7f870015d330 2023-12-02T00:31:36.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.104+0000 7f8704ba5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87000c9710 con 0x7f87000d2910 2023-12-02T00:31:36.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.105+0000 7f86fdd9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87000d3900 0x7f87000c8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:36.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.105+0000 7f86fdd9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87000d3900 0x7f87000c8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45114/0 (socket says 172.21.15.46:45114) 2023-12-02T00:31:36.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.105+0000 7f86fdd9b640 1 -- 172.21.15.46:0/3211002252 learned_addr learned my addr 172.21.15.46:0/3211002252 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:36.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.105+0000 7f86fe59c640 1 --2- 172.21.15.46:0/3211002252 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f87000d2910 0x7f8700152910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:36.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.105+0000 7f86fed9d640 1 --2- 172.21.15.46:0/3211002252 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f870015d330 0x7f87000c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:36.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.105+0000 7f86fdd9b640 1 -- 172.21.15.46:0/3211002252 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f87000d2910 msgr2=0x7f8700152910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:36.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.105+0000 7f86fdd9b640 1 --2- 172.21.15.46:0/3211002252 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f87000d2910 0x7f8700152910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:36.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.105+0000 7f86fdd9b640 1 -- 172.21.15.46:0/3211002252 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f870015d330 msgr2=0x7f87000c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:36.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.105+0000 7f86fdd9b640 1 --2- 172.21.15.46:0/3211002252 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f870015d330 0x7f87000c7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:36.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.105+0000 7f86fdd9b640 1 -- 172.21.15.46:0/3211002252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f0009b30 con 0x7f87000d3900 2023-12-02T00:31:36.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.105+0000 7f86fed9d640 1 --2- 172.21.15.46:0/3211002252 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f870015d330 0x7f87000c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:31:36.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.105+0000 7f86fdd9b640 1 --2- 172.21.15.46:0/3211002252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87000d3900 0x7f87000c8f80 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f86e800c4d0 tx=0x7f86e800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:36.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.105+0000 7f86f6ffd640 1 -- 172.21.15.46:0/3211002252 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e8007480 con 0x7f87000d3900 2023-12-02T00:31:36.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.105+0000 7f86f6ffd640 1 -- 172.21.15.46:0/3211002252 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e80075e0 con 0x7f87000d3900 2023-12-02T00:31:36.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.105+0000 7f8704ba5640 1 -- 172.21.15.46:0/3211002252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87000c99f0 con 0x7f87000d3900 2023-12-02T00:31:36.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.105+0000 7f86f6ffd640 1 -- 172.21.15.46:0/3211002252 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e80049e0 con 0x7f87000d3900 2023-12-02T00:31:36.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.105+0000 7f8704ba5640 1 -- 172.21.15.46:0/3211002252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87000c7a60 con 0x7f87000d3900 2023-12-02T00:31:36.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.106+0000 7f8704ba5640 1 -- 172.21.15.46:0/3211002252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f870015e020 con 0x7f87000d3900 2023-12-02T00:31:36.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.107+0000 7f86f6ffd640 1 -- 172.21.15.46:0/3211002252 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f86e8004b40 con 0x7f87000d3900 2023-12-02T00:31:36.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.107+0000 7f86f6ffd640 1 --2- 172.21.15.46:0/3211002252 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f86e006ae40 0x7f86e006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:36.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.107+0000 7f86f6ffd640 1 -- 172.21.15.46:0/3211002252 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f86e8088680 con 0x7f87000d3900 2023-12-02T00:31:36.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.107+0000 7f86fe59c640 1 --2- 172.21.15.46:0/3211002252 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f86e006ae40 0x7f86e006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:36.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.108+0000 7f86fe59c640 1 --2- 172.21.15.46:0/3211002252 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f86e006ae40 0x7f86e006d300 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f87000d3640 tx=0x7f86f8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:36.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.111+0000 7f86f6ffd640 1 -- 172.21.15.46:0/3211002252 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f870015e020 con 0x7f87000d3900 2023-12-02T00:31:36.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.290+0000 7f8704ba5640 1 -- 172.21.15.46:0/3211002252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f870015e020 con 0x7f87000d3900 2023-12-02T00:31:36.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.291+0000 7f86f6ffd640 1 -- 172.21.15.46:0/3211002252 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+16585 (secure 0 0 0) 0x7f870015e020 con 0x7f87000d3900 2023-12-02T00:31:36.293 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:31:36.294 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":53,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:31:31.941814+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-12-02T00:30:59.469070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-12-02T00:31:00.174610+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.28","mappings":[{"from":0,"to":6}]},{"pgid":"8.38","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:31:36.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.294+0000 7f8704ba5640 1 -- 172.21.15.46:0/3211002252 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f86e006ae40 msgr2=0x7f86e006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:36.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.294+0000 7f8704ba5640 1 --2- 172.21.15.46:0/3211002252 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f86e006ae40 0x7f86e006d300 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f87000d3640 tx=0x7f86f8008040 comp rx=0 tx=0).stop 2023-12-02T00:31:36.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.294+0000 7f8704ba5640 1 -- 172.21.15.46:0/3211002252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87000d3900 msgr2=0x7f87000c8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:36.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.294+0000 7f8704ba5640 1 --2- 172.21.15.46:0/3211002252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87000d3900 0x7f87000c8f80 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f86e800c4d0 tx=0x7f86e800c990 comp rx=0 tx=0).stop 2023-12-02T00:31:36.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.294+0000 7f8704ba5640 1 -- 172.21.15.46:0/3211002252 shutdown_connections 2023-12-02T00:31:36.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.294+0000 7f8704ba5640 1 --2- 172.21.15.46:0/3211002252 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f86e006ae40 0x7f86e006d300 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:36.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.294+0000 7f8704ba5640 1 --2- 172.21.15.46:0/3211002252 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f870015d330 0x7f87000c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:36.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.294+0000 7f8704ba5640 1 --2- 172.21.15.46:0/3211002252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87000d3900 0x7f87000c8f80 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:36.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.294+0000 7f8704ba5640 1 --2- 172.21.15.46:0/3211002252 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f87000d2910 0x7f8700152910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:36.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.294+0000 7f8704ba5640 1 -- 172.21.15.46:0/3211002252 >> 172.21.15.46:0/3211002252 conn(0x7f8700072020 msgr2=0x7f87000b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:36.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.294+0000 7f8704ba5640 1 -- 172.21.15.46:0/3211002252 shutdown_connections 2023-12-02T00:31:36.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.294+0000 7f8704ba5640 1 -- 172.21.15.46:0/3211002252 wait complete. 2023-12-02T00:31:36.308 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-02T00:31:36.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.414+0000 7f5233c48640 1 Processor -- start 2023-12-02T00:31:36.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.414+0000 7f5233c48640 1 -- start start 2023-12-02T00:31:36.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.415+0000 7f5233c48640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f522c15d1d0 0x7f522c06b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:36.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.415+0000 7f5233c48640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f522c161ce0 con 0x7f522c0ad640 2023-12-02T00:31:36.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.415+0000 7f5233c48640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f522c0adae0 con 0x7f522c0aeb80 2023-12-02T00:31:36.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.415+0000 7f5233c48640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f522c0addc0 con 0x7f522c15d1d0 2023-12-02T00:31:36.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.415+0000 7f52319e4640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f522c15d1d0 0x7f522c06b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:36.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.415+0000 7f52311e3640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f522c0ad640 0x7f522c160ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50880/0 (socket says 172.21.15.46:50880) 2023-12-02T00:31:36.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.415+0000 7f52311e3640 1 -- 172.21.15.46:0/271343107 learned_addr learned my addr 172.21.15.46:0/271343107 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:31:36.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.415+0000 7f52309e2640 1 -- 172.21.15.46:0/271343107 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1563626149 0 0) 0x7f522c161ce0 con 0x7f522c0ad640 2023-12-02T00:31:36.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.415+0000 7f52309e2640 1 -- 172.21.15.46:0/271343107 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5218003520 con 0x7f522c0ad640 2023-12-02T00:31:36.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.415+0000 7f52319e4640 1 -- 172.21.15.46:0/271343107 >> v1:172.21.15.162:6789/0 conn(0x7f522c0aeb80 legacy=0x7f522c1615d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:36.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.415+0000 7f52319e4640 1 -- 172.21.15.46:0/271343107 >> v1:172.21.15.46:6789/0 conn(0x7f522c0ad640 legacy=0x7f522c160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:36.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.415+0000 7f52319e4640 1 -- 172.21.15.46:0/271343107 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f522c0ae0a0 con 0x7f522c15d1d0 2023-12-02T00:31:36.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.416+0000 7f52319e4640 1 --2- 172.21.15.46:0/271343107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f522c15d1d0 0x7f522c06b4e0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f52280097a0 tx=0x7f522802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ba2ed261c74011a server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:36.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.416+0000 7f52309e2640 1 -- 172.21.15.46:0/271343107 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5228031830 con 0x7f522c15d1d0 2023-12-02T00:31:36.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.416+0000 7f52309e2640 1 -- 172.21.15.46:0/271343107 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5228031990 con 0x7f522c15d1d0 2023-12-02T00:31:36.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.416+0000 7f52309e2640 1 -- 172.21.15.46:0/271343107 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5228031d20 con 0x7f522c15d1d0 2023-12-02T00:31:36.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.417+0000 7f5233c48640 1 -- 172.21.15.46:0/271343107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f522c15d1d0 msgr2=0x7f522c06b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:36.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.417+0000 7f5233c48640 1 --2- 172.21.15.46:0/271343107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f522c15d1d0 0x7f522c06b4e0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f52280097a0 tx=0x7f522802b8d0 comp rx=0 tx=0).stop 2023-12-02T00:31:36.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.417+0000 7f5233c48640 1 -- 172.21.15.46:0/271343107 shutdown_connections 2023-12-02T00:31:36.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.417+0000 7f5233c48640 1 --2- 172.21.15.46:0/271343107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f522c15d1d0 0x7f522c06b4e0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:36.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.417+0000 7f5233c48640 1 -- 172.21.15.46:0/271343107 >> 172.21.15.46:0/271343107 conn(0x7f522c070e70 msgr2=0x7f522c071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:36.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.417+0000 7f5233c48640 1 -- 172.21.15.46:0/271343107 shutdown_connections 2023-12-02T00:31:36.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.417+0000 7f5233c48640 1 -- 172.21.15.46:0/271343107 wait complete. 2023-12-02T00:31:36.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.418+0000 7f5233c48640 1 Processor -- start 2023-12-02T00:31:36.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.418+0000 7f5233c48640 1 -- start start 2023-12-02T00:31:36.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.419+0000 7f5233c48640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f522c0ad640 0x7f522c140810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:36.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.419+0000 7f5233c48640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f522c0aeb80 0x7f522c0cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:36.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.419+0000 7f5233c48640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f522c15d1d0 0x7f522c0b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:36.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.419+0000 7f5233c48640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f522c0ce8b0 con 0x7f522c0aeb80 2023-12-02T00:31:36.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.419+0000 7f5233c48640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f522c0ceb90 con 0x7f522c15d1d0 2023-12-02T00:31:36.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.419+0000 7f5233c48640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f522c0cee70 con 0x7f522c0ad640 2023-12-02T00:31:36.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.419+0000 7f52311e3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f522c0aeb80 0x7f522c0cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:36.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.419+0000 7f52311e3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f522c0aeb80 0x7f522c0cdf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45128/0 (socket says 172.21.15.46:45128) 2023-12-02T00:31:36.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.419+0000 7f52311e3640 1 -- 172.21.15.46:0/4272823468 learned_addr learned my addr 172.21.15.46:0/4272823468 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:36.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.419+0000 7f52321e5640 1 --2- 172.21.15.46:0/4272823468 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f522c15d1d0 0x7f522c0b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:36.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.419+0000 7f52319e4640 1 --2- 172.21.15.46:0/4272823468 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f522c0ad640 0x7f522c140810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:36.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.419+0000 7f52311e3640 1 -- 172.21.15.46:0/4272823468 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f522c0ad640 msgr2=0x7f522c140810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:36.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.419+0000 7f52311e3640 1 --2- 172.21.15.46:0/4272823468 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f522c0ad640 0x7f522c140810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:36.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.419+0000 7f52311e3640 1 -- 172.21.15.46:0/4272823468 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f522c15d1d0 msgr2=0x7f522c0b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:36.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.419+0000 7f52311e3640 1 --2- 172.21.15.46:0/4272823468 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f522c15d1d0 0x7f522c0b2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:36.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.419+0000 7f52311e3640 1 -- 172.21.15.46:0/4272823468 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5228009400 con 0x7f522c0aeb80 2023-12-02T00:31:36.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.419+0000 7f52311e3640 1 --2- 172.21.15.46:0/4272823468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f522c0aeb80 0x7f522c0cdf40 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f521c00c500 tx=0x7f521c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:36.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.420+0000 7f52227fc640 1 -- 172.21.15.46:0/4272823468 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521c005ce0 con 0x7f522c0aeb80 2023-12-02T00:31:36.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.420+0000 7f52227fc640 1 -- 172.21.15.46:0/4272823468 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f521c005e40 con 0x7f522c0aeb80 2023-12-02T00:31:36.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.420+0000 7f5233c48640 1 -- 172.21.15.46:0/4272823468 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f522c0cf150 con 0x7f522c0aeb80 2023-12-02T00:31:36.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.420+0000 7f52227fc640 1 -- 172.21.15.46:0/4272823468 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521c007330 con 0x7f522c0aeb80 2023-12-02T00:31:36.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.420+0000 7f5233c48640 1 -- 172.21.15.46:0/4272823468 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f522c140d50 con 0x7f522c0aeb80 2023-12-02T00:31:36.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.421+0000 7f5233c48640 1 -- 172.21.15.46:0/4272823468 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f522c069400 con 0x7f522c0aeb80 2023-12-02T00:31:36.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.421+0000 7f52227fc640 1 -- 172.21.15.46:0/4272823468 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f521c007550 con 0x7f522c0aeb80 2023-12-02T00:31:36.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.422+0000 7f52227fc640 1 --2- 172.21.15.46:0/4272823468 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f521006ae40 0x7f521006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:36.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.422+0000 7f52227fc640 1 -- 172.21.15.46:0/4272823468 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f521c088c20 con 0x7f522c0aeb80 2023-12-02T00:31:36.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.422+0000 7f52319e4640 1 --2- 172.21.15.46:0/4272823468 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f521006ae40 0x7f521006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:36.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.422+0000 7f52319e4640 1 --2- 172.21.15.46:0/4272823468 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f521006ae40 0x7f521006d300 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5228036000 tx=0x7f522803c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:36.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.426+0000 7f52227fc640 1 -- 172.21.15.46:0/4272823468 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f521c053de0 con 0x7f522c0aeb80 2023-12-02T00:31:36.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.637+0000 7f5233c48640 1 -- 172.21.15.46:0/4272823468 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f522c15f920 con 0x7f522c0aeb80 2023-12-02T00:31:36.638 INFO:tasks.ceph.mon.a.smithi046.stderr:2023-12-02T00:31:36.637+0000 7f59c5f43640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-02T00:31:36.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.639+0000 7f52227fc640 1 -- 172.21.15.46:0/4272823468 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v25) v1 ==== 161+0+0 (secure 0 0 0) 0x7f521c017020 con 0x7f522c0aeb80 2023-12-02T00:31:36.640 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-02T00:31:36.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.641+0000 7f5233c48640 1 -- 172.21.15.46:0/4272823468 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f521006ae40 msgr2=0x7f521006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:36.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.641+0000 7f5233c48640 1 --2- 172.21.15.46:0/4272823468 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f521006ae40 0x7f521006d300 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5228036000 tx=0x7f522803c040 comp rx=0 tx=0).stop 2023-12-02T00:31:36.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.641+0000 7f5233c48640 1 -- 172.21.15.46:0/4272823468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f522c0aeb80 msgr2=0x7f522c0cdf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:36.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.641+0000 7f5233c48640 1 --2- 172.21.15.46:0/4272823468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f522c0aeb80 0x7f522c0cdf40 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f521c00c500 tx=0x7f521c00c9c0 comp rx=0 tx=0).stop 2023-12-02T00:31:36.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.641+0000 7f5233c48640 1 -- 172.21.15.46:0/4272823468 shutdown_connections 2023-12-02T00:31:36.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.641+0000 7f5233c48640 1 --2- 172.21.15.46:0/4272823468 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f521006ae40 0x7f521006d300 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:36.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.641+0000 7f5233c48640 1 --2- 172.21.15.46:0/4272823468 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f522c15d1d0 0x7f522c0b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:36.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.641+0000 7f5233c48640 1 --2- 172.21.15.46:0/4272823468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f522c0aeb80 0x7f522c0cdf40 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:36.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.641+0000 7f5233c48640 1 --2- 172.21.15.46:0/4272823468 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f522c0ad640 0x7f522c140810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:36.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.641+0000 7f5233c48640 1 -- 172.21.15.46:0/4272823468 >> 172.21.15.46:0/4272823468 conn(0x7f522c070e70 msgr2=0x7f522c1606d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:36.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.641+0000 7f5233c48640 1 -- 172.21.15.46:0/4272823468 shutdown_connections 2023-12-02T00:31:36.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.641+0000 7f5233c48640 1 -- 172.21.15.46:0/4272823468 wait complete. 2023-12-02T00:31:36.644 INFO:tasks.ceph.mds.a.smithi046.stderr: -2705> 2023-12-02T00:31:06.803+0000 7fb7a25a8640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-02T00:31:36.654 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-02T00:31:36.672 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2023-12-02T00:31:36.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.762+0000 7f0aa066e640 1 Processor -- start 2023-12-02T00:31:36.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.762+0000 7f0aa066e640 1 -- start start 2023-12-02T00:31:36.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.763+0000 7f0aa066e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0a98144980 0x7f0a9815c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:36.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.763+0000 7f0aa066e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0a980ceb70 0x7f0a980cc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:36.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.763+0000 7f0aa066e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a980b4370 0x7f0a980b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:36.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.763+0000 7f0aa066e640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a98071770 con 0x7f0a980ceb70 2023-12-02T00:31:36.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.763+0000 7f0aa066e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a980b4ce0 con 0x7f0a98144980 2023-12-02T00:31:36.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.763+0000 7f0aa066e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a980ccdc0 con 0x7f0a980b4370 2023-12-02T00:31:36.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.763+0000 7f0a9dc09640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0a980ceb70 0x7f0a980cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:36.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.763+0000 7f0a9dc09640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0a980ceb70 0x7f0a980cc450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45148/0 (socket says 172.21.15.46:45148) 2023-12-02T00:31:36.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.763+0000 7f0a9dc09640 1 -- 172.21.15.46:0/883694763 learned_addr learned my addr 172.21.15.46:0/883694763 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:36.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.763+0000 7f0a9ec0b640 1 --2- 172.21.15.46:0/883694763 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a980b4370 0x7f0a980b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:36.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.763+0000 7f0a9e40a640 1 --2- 172.21.15.46:0/883694763 >> v2:172.21.15.162:3300/0 conn(0x7f0a98144980 0x7f0a9815c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:36.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.763+0000 7f0a9dc09640 1 -- 172.21.15.46:0/883694763 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a980b4370 msgr2=0x7f0a980b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:36.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.763+0000 7f0a9dc09640 1 --2- 172.21.15.46:0/883694763 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a980b4370 0x7f0a980b47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:36.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.763+0000 7f0a9dc09640 1 -- 172.21.15.46:0/883694763 >> v2:172.21.15.162:3300/0 conn(0x7f0a98144980 msgr2=0x7f0a9815c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:36.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.764+0000 7f0a9dc09640 1 --2- 172.21.15.46:0/883694763 >> v2:172.21.15.162:3300/0 conn(0x7f0a98144980 0x7f0a9815c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:36.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.764+0000 7f0a9dc09640 1 -- 172.21.15.46:0/883694763 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a980cd0a0 con 0x7f0a980ceb70 2023-12-02T00:31:36.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.764+0000 7f0a9dc09640 1 --2- 172.21.15.46:0/883694763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a980ceb70 0x7f0a980cc450 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f0a980b4ec0 tx=0x7f0a8c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4f93b2159500302 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:36.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.764+0000 7f0a9d408640 1 -- 172.21.15.46:0/883694763 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c031870 con 0x7f0a980ceb70 2023-12-02T00:31:36.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.764+0000 7f0a9d408640 1 -- 172.21.15.46:0/883694763 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a8c0319d0 con 0x7f0a980ceb70 2023-12-02T00:31:36.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.765+0000 7f0a9d408640 1 -- 172.21.15.46:0/883694763 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c031d60 con 0x7f0a980ceb70 2023-12-02T00:31:36.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.765+0000 7f0aa066e640 1 -- 172.21.15.46:0/883694763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a980ceb70 msgr2=0x7f0a980cc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:36.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.765+0000 7f0aa066e640 1 --2- 172.21.15.46:0/883694763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a980ceb70 0x7f0a980cc450 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f0a980b4ec0 tx=0x7f0a8c02b910 comp rx=0 tx=0).stop 2023-12-02T00:31:36.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.765+0000 7f0aa066e640 1 -- 172.21.15.46:0/883694763 shutdown_connections 2023-12-02T00:31:36.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.765+0000 7f0aa066e640 1 --2- 172.21.15.46:0/883694763 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a980b4370 0x7f0a980b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:36.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.765+0000 7f0aa066e640 1 --2- 172.21.15.46:0/883694763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a980ceb70 0x7f0a980cc450 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:36.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.765+0000 7f0aa066e640 1 --2- 172.21.15.46:0/883694763 >> v2:172.21.15.162:3300/0 conn(0x7f0a98144980 0x7f0a9815c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:36.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.765+0000 7f0aa066e640 1 -- 172.21.15.46:0/883694763 >> 172.21.15.46:0/883694763 conn(0x7f0a9806a4d0 msgr2=0x7f0a9806a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:36.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.765+0000 7f0aa066e640 1 -- 172.21.15.46:0/883694763 shutdown_connections 2023-12-02T00:31:36.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.765+0000 7f0aa066e640 1 -- 172.21.15.46:0/883694763 wait complete. 2023-12-02T00:31:36.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.766+0000 7f0aa066e640 1 Processor -- start 2023-12-02T00:31:36.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.766+0000 7f0aa066e640 1 -- start start 2023-12-02T00:31:36.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.767+0000 7f0aa066e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a980b4370 0x7f0a980afd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:36.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.767+0000 7f0aa066e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a980ceb70 0x7f0a980b15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:36.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.767+0000 7f0aa066e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a98144980 0x7f0a981405f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:36.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.767+0000 7f0aa066e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a980b04c0 con 0x7f0a98144980 2023-12-02T00:31:36.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.767+0000 7f0aa066e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a980b0290 con 0x7f0a980ceb70 2023-12-02T00:31:36.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.767+0000 7f0aa066e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a980a9a20 con 0x7f0a980b4370 2023-12-02T00:31:36.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.767+0000 7f0a9ec0b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a98144980 0x7f0a981405f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:36.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.767+0000 7f0a9ec0b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a98144980 0x7f0a981405f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45150/0 (socket says 172.21.15.46:45150) 2023-12-02T00:31:36.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.767+0000 7f0a9ec0b640 1 -- 172.21.15.46:0/3403861652 learned_addr learned my addr 172.21.15.46:0/3403861652 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:36.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.767+0000 7f0a9e40a640 1 --2- 172.21.15.46:0/3403861652 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a980b4370 0x7f0a980afd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:36.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.767+0000 7f0a9dc09640 1 --2- 172.21.15.46:0/3403861652 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a980ceb70 0x7f0a980b15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:36.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.768+0000 7f0a9ec0b640 1 -- 172.21.15.46:0/3403861652 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a980b4370 msgr2=0x7f0a980afd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:36.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.768+0000 7f0a9ec0b640 1 --2- 172.21.15.46:0/3403861652 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a980b4370 0x7f0a980afd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:36.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.768+0000 7f0a9ec0b640 1 -- 172.21.15.46:0/3403861652 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a980ceb70 msgr2=0x7f0a980b15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:36.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.768+0000 7f0a9ec0b640 1 --2- 172.21.15.46:0/3403861652 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a980ceb70 0x7f0a980b15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:36.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.768+0000 7f0a9ec0b640 1 -- 172.21.15.46:0/3403861652 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a8c0094d0 con 0x7f0a98144980 2023-12-02T00:31:36.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.768+0000 7f0a9dc09640 1 --2- 172.21.15.46:0/3403861652 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a980ceb70 0x7f0a980b15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:31:36.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.768+0000 7f0a9ec0b640 1 --2- 172.21.15.46:0/3403861652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a98144980 0x7f0a981405f0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f0a88007e40 tx=0x7f0a8800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:36.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.768+0000 7f0a96ffd640 1 -- 172.21.15.46:0/3403861652 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a88014620 con 0x7f0a98144980 2023-12-02T00:31:36.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.768+0000 7f0a96ffd640 1 -- 172.21.15.46:0/3403861652 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a8800ee90 con 0x7f0a98144980 2023-12-02T00:31:36.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.768+0000 7f0a96ffd640 1 -- 172.21.15.46:0/3403861652 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a880149e0 con 0x7f0a98144980 2023-12-02T00:31:36.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.768+0000 7f0aa066e640 1 -- 172.21.15.46:0/3403861652 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a980b07e0 con 0x7f0a98144980 2023-12-02T00:31:36.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.768+0000 7f0aa066e640 1 -- 172.21.15.46:0/3403861652 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a980b0da0 con 0x7f0a98144980 2023-12-02T00:31:36.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.769+0000 7f0aa066e640 1 -- 172.21.15.46:0/3403861652 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a98069400 con 0x7f0a98144980 2023-12-02T00:31:36.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.769+0000 7f0a96ffd640 1 -- 172.21.15.46:0/3403861652 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0a88014b40 con 0x7f0a98144980 2023-12-02T00:31:36.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.770+0000 7f0a96ffd640 1 --2- 172.21.15.46:0/3403861652 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f0a8406ae40 0x7f0a8406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:36.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.770+0000 7f0a96ffd640 1 -- 172.21.15.46:0/3403861652 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f0a88089ff0 con 0x7f0a98144980 2023-12-02T00:31:36.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.770+0000 7f0a9e40a640 1 --2- 172.21.15.46:0/3403861652 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f0a8406ae40 0x7f0a8406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:36.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.770+0000 7f0a9e40a640 1 --2- 172.21.15.46:0/3403861652 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f0a8406ae40 0x7f0a8406d300 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0a90007920 tx=0x7f0a90008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:36.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.774+0000 7f0a96ffd640 1 -- 172.21.15.46:0/3403861652 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0a88055010 con 0x7f0a98144980 2023-12-02T00:31:36.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:36.986+0000 7f0aa066e640 1 -- 172.21.15.46:0/3403861652 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0a9815c920 con 0x7f0a98144980 2023-12-02T00:31:37.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.640+0000 7f0a96ffd640 1 -- 172.21.15.46:0/3403861652 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v26) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0a88058630 con 0x7f0a98144980 2023-12-02T00:31:37.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.643+0000 7f0aa066e640 1 -- 172.21.15.46:0/3403861652 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f0a8406ae40 msgr2=0x7f0a8406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:37.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.643+0000 7f0aa066e640 1 --2- 172.21.15.46:0/3403861652 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f0a8406ae40 0x7f0a8406d300 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0a90007920 tx=0x7f0a90008040 comp rx=0 tx=0).stop 2023-12-02T00:31:37.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.643+0000 7f0aa066e640 1 -- 172.21.15.46:0/3403861652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a98144980 msgr2=0x7f0a981405f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:37.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.643+0000 7f0aa066e640 1 --2- 172.21.15.46:0/3403861652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a98144980 0x7f0a981405f0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f0a88007e40 tx=0x7f0a8800e4b0 comp rx=0 tx=0).stop 2023-12-02T00:31:37.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.643+0000 7f0aa066e640 1 -- 172.21.15.46:0/3403861652 shutdown_connections 2023-12-02T00:31:37.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.643+0000 7f0aa066e640 1 --2- 172.21.15.46:0/3403861652 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f0a8406ae40 0x7f0a8406d300 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:37.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.643+0000 7f0aa066e640 1 --2- 172.21.15.46:0/3403861652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a98144980 0x7f0a981405f0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:37.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.643+0000 7f0aa066e640 1 --2- 172.21.15.46:0/3403861652 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a980ceb70 0x7f0a980b15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:37.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.643+0000 7f0aa066e640 1 --2- 172.21.15.46:0/3403861652 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a980b4370 0x7f0a980afd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:37.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.643+0000 7f0aa066e640 1 -- 172.21.15.46:0/3403861652 >> 172.21.15.46:0/3403861652 conn(0x7f0a9806a4d0 msgr2=0x7f0a9815b670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:37.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.643+0000 7f0aa066e640 1 -- 172.21.15.46:0/3403861652 shutdown_connections 2023-12-02T00:31:37.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.643+0000 7f0aa066e640 1 -- 172.21.15.46:0/3403861652 wait complete. 2023-12-02T00:31:37.658 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-02T00:31:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.816+0000 7fcf03b6e640 1 Processor -- start 2023-12-02T00:31:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.816+0000 7fcf03b6e640 1 -- start start 2023-12-02T00:31:37.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.817+0000 7fcf03b6e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcefc06ea50 0x7fcefc06ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:37.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.817+0000 7fcf03b6e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcefc0d0ed0 0x7fcefc0d1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:37.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.817+0000 7fcf03b6e640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcefc161d20 con 0x7fcefc160fb0 2023-12-02T00:31:37.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.817+0000 7fcf03b6e640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcefc06f370 con 0x7fcefc06ea50 2023-12-02T00:31:37.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.817+0000 7fcf03b6e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcefc15e760 con 0x7fcefc0d0ed0 2023-12-02T00:31:37.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.817+0000 7fcf0190a640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcefc06ea50 0x7fcefc06ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:37.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.817+0000 7fcf01109640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fcefc160fb0 0x7fcefc161440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50900/0 (socket says 172.21.15.46:50900) 2023-12-02T00:31:37.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.817+0000 7fcf01109640 1 -- 172.21.15.46:0/654383741 learned_addr learned my addr 172.21.15.46:0/654383741 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:31:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.817+0000 7fcf0210b640 1 --2- 172.21.15.46:0/654383741 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcefc0d0ed0 0x7fcefc0d1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.817+0000 7fcf0190a640 1 -- 172.21.15.46:0/654383741 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcefc0d0ed0 msgr2=0x7fcefc0d1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.817+0000 7fcf0190a640 1 --2- 172.21.15.46:0/654383741 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcefc0d0ed0 0x7fcefc0d1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.817+0000 7fcf0190a640 1 -- 172.21.15.46:0/654383741 >> v1:172.21.15.46:6789/0 conn(0x7fcefc160fb0 legacy=0x7fcefc161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.817+0000 7fcf00908640 1 -- 172.21.15.46:0/654383741 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3856700364 0 0) 0x7fcefc161d20 con 0x7fcefc160fb0 2023-12-02T00:31:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.817+0000 7fcf0190a640 1 -- 172.21.15.46:0/654383741 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcefc15d9a0 con 0x7fcefc06ea50 2023-12-02T00:31:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.818+0000 7fcf0190a640 1 --2- 172.21.15.46:0/654383741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcefc06ea50 0x7fcefc06ee30 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fcef80097a0 tx=0x7fcef802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1e04851efda4579 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.818+0000 7fcf00908640 1 -- 172.21.15.46:0/654383741 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef8031830 con 0x7fcefc06ea50 2023-12-02T00:31:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.818+0000 7fcf00908640 1 -- 172.21.15.46:0/654383741 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef8031990 con 0x7fcefc06ea50 2023-12-02T00:31:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.818+0000 7fcf00908640 1 -- 172.21.15.46:0/654383741 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef8031d20 con 0x7fcefc06ea50 2023-12-02T00:31:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.819+0000 7fcf03b6e640 1 -- 172.21.15.46:0/654383741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcefc06ea50 msgr2=0x7fcefc06ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.819+0000 7fcf03b6e640 1 --2- 172.21.15.46:0/654383741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcefc06ea50 0x7fcefc06ee30 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fcef80097a0 tx=0x7fcef802b8d0 comp rx=0 tx=0).stop 2023-12-02T00:31:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.819+0000 7fcf03b6e640 1 -- 172.21.15.46:0/654383741 shutdown_connections 2023-12-02T00:31:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.819+0000 7fcf03b6e640 1 --2- 172.21.15.46:0/654383741 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcefc0d0ed0 0x7fcefc0d1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.819+0000 7fcf03b6e640 1 --2- 172.21.15.46:0/654383741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcefc06ea50 0x7fcefc06ee30 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:37.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.819+0000 7fcf03b6e640 1 -- 172.21.15.46:0/654383741 >> 172.21.15.46:0/654383741 conn(0x7fcefc06a6b0 msgr2=0x7fcefc06aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:37.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.819+0000 7fcf03b6e640 1 -- 172.21.15.46:0/654383741 shutdown_connections 2023-12-02T00:31:37.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.819+0000 7fcf03b6e640 1 -- 172.21.15.46:0/654383741 wait complete. 2023-12-02T00:31:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.820+0000 7fcf03b6e640 1 Processor -- start 2023-12-02T00:31:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.820+0000 7fcf03b6e640 1 -- start start 2023-12-02T00:31:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.821+0000 7fcf03b6e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcefc06ea50 0x7fcefc157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.821+0000 7fcf03b6e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcefc0d0ed0 0x7fcefc0ce260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.821+0000 7fcf03b6e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcefc160fb0 0x7fcefc0cc270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.821+0000 7fcf03b6e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcefc0ce7a0 con 0x7fcefc160fb0 2023-12-02T00:31:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.821+0000 7fcf03b6e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcefc0ccac0 con 0x7fcefc06ea50 2023-12-02T00:31:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.821+0000 7fcf03b6e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcefc0ccda0 con 0x7fcefc0d0ed0 2023-12-02T00:31:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.821+0000 7fcf0210b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcefc160fb0 0x7fcefc0cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.821+0000 7fcf0210b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcefc160fb0 0x7fcefc0cc270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45158/0 (socket says 172.21.15.46:45158) 2023-12-02T00:31:37.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.821+0000 7fcf0210b640 1 -- 172.21.15.46:0/1983983353 learned_addr learned my addr 172.21.15.46:0/1983983353 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:37.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.821+0000 7fcf0190a640 1 --2- 172.21.15.46:0/1983983353 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcefc06ea50 0x7fcefc157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:37.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.821+0000 7fcf01109640 1 --2- 172.21.15.46:0/1983983353 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcefc0d0ed0 0x7fcefc0ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:37.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.821+0000 7fcf0210b640 1 -- 172.21.15.46:0/1983983353 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcefc0d0ed0 msgr2=0x7fcefc0ce260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:37.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.821+0000 7fcf0210b640 1 --2- 172.21.15.46:0/1983983353 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcefc0d0ed0 0x7fcefc0ce260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:37.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.821+0000 7fcf0210b640 1 -- 172.21.15.46:0/1983983353 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcefc06ea50 msgr2=0x7fcefc157150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:37.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.821+0000 7fcf0210b640 1 --2- 172.21.15.46:0/1983983353 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcefc06ea50 0x7fcefc157150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:37.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.821+0000 7fcf0210b640 1 -- 172.21.15.46:0/1983983353 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef8009400 con 0x7fcefc160fb0 2023-12-02T00:31:37.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.821+0000 7fcf01109640 1 --2- 172.21.15.46:0/1983983353 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcefc0d0ed0 0x7fcefc0ce260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:31:37.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.821+0000 7fcf0210b640 1 --2- 172.21.15.46:0/1983983353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcefc160fb0 0x7fcefc0cc270 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fcef4007d90 tx=0x7fcef400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:37.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.822+0000 7fcef27fc640 1 -- 172.21.15.46:0/1983983353 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef4014420 con 0x7fcefc160fb0 2023-12-02T00:31:37.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.822+0000 7fcef27fc640 1 -- 172.21.15.46:0/1983983353 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef40145a0 con 0x7fcefc160fb0 2023-12-02T00:31:37.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.822+0000 7fcef27fc640 1 -- 172.21.15.46:0/1983983353 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef4014950 con 0x7fcefc160fb0 2023-12-02T00:31:37.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.822+0000 7fcf03b6e640 1 -- 172.21.15.46:0/1983983353 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcefc158060 con 0x7fcefc160fb0 2023-12-02T00:31:37.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.822+0000 7fcf03b6e640 1 -- 172.21.15.46:0/1983983353 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcefc158620 con 0x7fcefc160fb0 2023-12-02T00:31:37.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.822+0000 7fcf03b6e640 1 -- 172.21.15.46:0/1983983353 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcefc157ec0 con 0x7fcefc160fb0 2023-12-02T00:31:37.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.823+0000 7fcef27fc640 1 -- 172.21.15.46:0/1983983353 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcef4014ab0 con 0x7fcefc160fb0 2023-12-02T00:31:37.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.823+0000 7fcef27fc640 1 --2- 172.21.15.46:0/1983983353 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fcee006ae40 0x7fcee006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:37.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.824+0000 7fcef27fc640 1 -- 172.21.15.46:0/1983983353 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fcef4089f80 con 0x7fcefc160fb0 2023-12-02T00:31:37.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.824+0000 7fcf0190a640 1 --2- 172.21.15.46:0/1983983353 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fcee006ae40 0x7fcee006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:37.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.824+0000 7fcf0190a640 1 --2- 172.21.15.46:0/1983983353 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fcee006ae40 0x7fcee006d300 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fcef8036000 tx=0x7fcef803c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:37.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:37.828+0000 7fcef27fc640 1 -- 172.21.15.46:0/1983983353 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcefc157ec0 con 0x7fcefc160fb0 2023-12-02T00:31:38.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:38.008+0000 7fcf03b6e640 1 -- 172.21.15.46:0/1983983353 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fcefc157ec0 con 0x7fcefc160fb0 2023-12-02T00:31:38.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:38.967+0000 7fcef27fc640 1 -- 172.21.15.46:0/1983983353 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v55) v1 ==== 181+0+0 (secure 0 0 0) 0x7fcefc157ec0 con 0x7fcefc160fb0 2023-12-02T00:31:38.968 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2023-12-02T00:31:38.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:38.971+0000 7fcf03b6e640 1 -- 172.21.15.46:0/1983983353 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fcee006ae40 msgr2=0x7fcee006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:38.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:38.971+0000 7fcf03b6e640 1 --2- 172.21.15.46:0/1983983353 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fcee006ae40 0x7fcee006d300 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fcef8036000 tx=0x7fcef803c040 comp rx=0 tx=0).stop 2023-12-02T00:31:38.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:38.971+0000 7fcf03b6e640 1 -- 172.21.15.46:0/1983983353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcefc160fb0 msgr2=0x7fcefc0cc270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:38.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:38.971+0000 7fcf03b6e640 1 --2- 172.21.15.46:0/1983983353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcefc160fb0 0x7fcefc0cc270 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fcef4007d90 tx=0x7fcef400e4b0 comp rx=0 tx=0).stop 2023-12-02T00:31:38.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:38.971+0000 7fcf03b6e640 1 -- 172.21.15.46:0/1983983353 shutdown_connections 2023-12-02T00:31:38.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:38.971+0000 7fcf03b6e640 1 --2- 172.21.15.46:0/1983983353 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fcee006ae40 0x7fcee006d300 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:38.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:38.971+0000 7fcf03b6e640 1 --2- 172.21.15.46:0/1983983353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcefc160fb0 0x7fcefc0cc270 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:38.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:38.971+0000 7fcf03b6e640 1 --2- 172.21.15.46:0/1983983353 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcefc0d0ed0 0x7fcefc0ce260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:38.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:38.971+0000 7fcf03b6e640 1 --2- 172.21.15.46:0/1983983353 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcefc06ea50 0x7fcefc157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:38.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:38.971+0000 7fcf03b6e640 1 -- 172.21.15.46:0/1983983353 >> 172.21.15.46:0/1983983353 conn(0x7fcefc06a6b0 msgr2=0x7fcefc06c890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:38.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:38.971+0000 7fcf03b6e640 1 -- 172.21.15.46:0/1983983353 shutdown_connections 2023-12-02T00:31:38.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:38.971+0000 7fcf03b6e640 1 -- 172.21.15.46:0/1983983353 wait complete. 2023-12-02T00:31:38.987 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-02T00:31:39.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.103+0000 7fe580b97640 1 Processor -- start 2023-12-02T00:31:39.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.103+0000 7fe580b97640 1 -- start start 2023-12-02T00:31:39.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.104+0000 7fe580b97640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe57c1415f0 0x7fe57c141e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:39.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.104+0000 7fe580b97640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe57c06bbf0 0x7fe57c0b3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:39.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.104+0000 7fe580b97640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe57c0b2ef0 con 0x7fe57c144a00 2023-12-02T00:31:39.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.104+0000 7fe580b97640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe57c0b3b70 con 0x7fe57c06bbf0 2023-12-02T00:31:39.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.104+0000 7fe580b97640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe57c0b31d0 con 0x7fe57c1415f0 2023-12-02T00:31:39.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.104+0000 7fe57ad9d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe57c06bbf0 0x7fe57c0b3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:39.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.104+0000 7fe57ad9d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe57c06bbf0 0x7fe57c0b3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45162/0 (socket says 172.21.15.46:45162) 2023-12-02T00:31:39.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.104+0000 7fe57ad9d640 1 -- 172.21.15.46:0/2651479951 learned_addr learned my addr 172.21.15.46:0/2651479951 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:39.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.104+0000 7fe57a59c640 1 --2- 172.21.15.46:0/2651479951 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe57c1415f0 0x7fe57c141e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:39.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.104+0000 7fe57ad9d640 1 -- 172.21.15.46:0/2651479951 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe57c1415f0 msgr2=0x7fe57c141e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:39.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.104+0000 7fe57ad9d640 1 --2- 172.21.15.46:0/2651479951 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe57c1415f0 0x7fe57c141e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:39.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.104+0000 7fe57ad9d640 1 -- 172.21.15.46:0/2651479951 >> v1:172.21.15.162:6789/0 conn(0x7fe57c144a00 legacy=0x7fe57c06b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:39.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.104+0000 7fe57ad9d640 1 -- 172.21.15.46:0/2651479951 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe57c0b34b0 con 0x7fe57c06bbf0 2023-12-02T00:31:39.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.105+0000 7fe57ad9d640 1 --2- 172.21.15.46:0/2651479951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe57c06bbf0 0x7fe57c0b3f90 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fe574009e50 tx=0x7fe57402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0c2dcef4f6634d4 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:39.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.105+0000 7fe579d9b640 1 -- 172.21.15.46:0/2651479951 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57402cb00 con 0x7fe57c06bbf0 2023-12-02T00:31:39.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.105+0000 7fe579d9b640 1 -- 172.21.15.46:0/2651479951 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe57402cc60 con 0x7fe57c06bbf0 2023-12-02T00:31:39.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.105+0000 7fe579d9b640 1 -- 172.21.15.46:0/2651479951 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5740343f0 con 0x7fe57c06bbf0 2023-12-02T00:31:39.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.106+0000 7fe580b97640 1 -- 172.21.15.46:0/2651479951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe57c06bbf0 msgr2=0x7fe57c0b3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:39.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.106+0000 7fe580b97640 1 --2- 172.21.15.46:0/2651479951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe57c06bbf0 0x7fe57c0b3f90 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fe574009e50 tx=0x7fe57402bf60 comp rx=0 tx=0).stop 2023-12-02T00:31:39.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.106+0000 7fe580b97640 1 -- 172.21.15.46:0/2651479951 shutdown_connections 2023-12-02T00:31:39.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.106+0000 7fe580b97640 1 --2- 172.21.15.46:0/2651479951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe57c06bbf0 0x7fe57c0b3f90 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:39.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.106+0000 7fe580b97640 1 --2- 172.21.15.46:0/2651479951 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe57c1415f0 0x7fe57c141e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:39.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.106+0000 7fe580b97640 1 -- 172.21.15.46:0/2651479951 >> 172.21.15.46:0/2651479951 conn(0x7fe57c070e70 msgr2=0x7fe57c071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:39.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.106+0000 7fe580b97640 1 -- 172.21.15.46:0/2651479951 shutdown_connections 2023-12-02T00:31:39.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.106+0000 7fe580b97640 1 -- 172.21.15.46:0/2651479951 wait complete. 2023-12-02T00:31:39.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.107+0000 7fe580b97640 1 Processor -- start 2023-12-02T00:31:39.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.107+0000 7fe580b97640 1 -- start start 2023-12-02T00:31:39.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.108+0000 7fe580b97640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe57c06bbf0 0x7fe57c132900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:39.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.108+0000 7fe580b97640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe57c1415f0 0x7fe57c12fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:39.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.108+0000 7fe580b97640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe57c144a00 0x7fe57c152980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:39.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.108+0000 7fe580b97640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe57c1302f0 con 0x7fe57c06bbf0 2023-12-02T00:31:39.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.108+0000 7fe580b97640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe57c1304e0 con 0x7fe57c144a00 2023-12-02T00:31:39.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.108+0000 7fe580b97640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe57c1307c0 con 0x7fe57c1415f0 2023-12-02T00:31:39.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.108+0000 7fe57a59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe57c06bbf0 0x7fe57c132900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:39.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.108+0000 7fe57a59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe57c06bbf0 0x7fe57c132900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45178/0 (socket says 172.21.15.46:45178) 2023-12-02T00:31:39.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.108+0000 7fe57a59c640 1 -- 172.21.15.46:0/719291240 learned_addr learned my addr 172.21.15.46:0/719291240 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:39.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.108+0000 7fe573fff640 1 --2- 172.21.15.46:0/719291240 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe57c1415f0 0x7fe57c12fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:39.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.108+0000 7fe57ad9d640 1 --2- 172.21.15.46:0/719291240 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe57c144a00 0x7fe57c152980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:39.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.108+0000 7fe57a59c640 1 -- 172.21.15.46:0/719291240 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe57c1415f0 msgr2=0x7fe57c12fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:39.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.108+0000 7fe57a59c640 1 --2- 172.21.15.46:0/719291240 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe57c1415f0 0x7fe57c12fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:39.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.108+0000 7fe57a59c640 1 -- 172.21.15.46:0/719291240 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe57c144a00 msgr2=0x7fe57c152980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:39.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.108+0000 7fe57a59c640 1 --2- 172.21.15.46:0/719291240 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe57c144a00 0x7fe57c152980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:39.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.108+0000 7fe57a59c640 1 -- 172.21.15.46:0/719291240 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe574009b30 con 0x7fe57c06bbf0 2023-12-02T00:31:39.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.108+0000 7fe57ad9d640 1 --2- 172.21.15.46:0/719291240 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe57c144a00 0x7fe57c152980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:31:39.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.108+0000 7fe57a59c640 1 --2- 172.21.15.46:0/719291240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe57c06bbf0 0x7fe57c132900 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fe56c00e9b0 tx=0x7fe56c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:39.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.109+0000 7fe572ffd640 1 -- 172.21.15.46:0/719291240 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56c009670 con 0x7fe57c06bbf0 2023-12-02T00:31:39.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.109+0000 7fe572ffd640 1 -- 172.21.15.46:0/719291240 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe56c00c9a0 con 0x7fe57c06bbf0 2023-12-02T00:31:39.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.109+0000 7fe580b97640 1 -- 172.21.15.46:0/719291240 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe57c130aa0 con 0x7fe57c06bbf0 2023-12-02T00:31:39.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.109+0000 7fe572ffd640 1 -- 172.21.15.46:0/719291240 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56c00cd50 con 0x7fe57c06bbf0 2023-12-02T00:31:39.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.109+0000 7fe580b97640 1 -- 172.21.15.46:0/719291240 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe57c131060 con 0x7fe57c06bbf0 2023-12-02T00:31:39.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.110+0000 7fe580b97640 1 -- 172.21.15.46:0/719291240 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe57c069400 con 0x7fe57c06bbf0 2023-12-02T00:31:39.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.110+0000 7fe572ffd640 1 -- 172.21.15.46:0/719291240 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe56c013470 con 0x7fe57c06bbf0 2023-12-02T00:31:39.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.111+0000 7fe572ffd640 1 --2- 172.21.15.46:0/719291240 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe55c06ae40 0x7fe55c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:39.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.111+0000 7fe572ffd640 1 -- 172.21.15.46:0/719291240 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5415+0+0 (secure 0 0 0) 0x7fe56c089710 con 0x7fe57c06bbf0 2023-12-02T00:31:39.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.111+0000 7fe573fff640 1 --2- 172.21.15.46:0/719291240 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe55c06ae40 0x7fe55c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:39.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.111+0000 7fe573fff640 1 --2- 172.21.15.46:0/719291240 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe55c06ae40 0x7fe55c06d300 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe57c1364c0 tx=0x7fe564009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:39.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.115+0000 7fe572ffd640 1 -- 172.21.15.46:0/719291240 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe56c054970 con 0x7fe57c06bbf0 2023-12-02T00:31:39.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.291+0000 7fe580b97640 1 -- 172.21.15.46:0/719291240 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe57c066760 con 0x7fe57c06bbf0 2023-12-02T00:31:39.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.981+0000 7fe572ffd640 1 -- 172.21.15.46:0/719291240 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v56) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe56c057f90 con 0x7fe57c06bbf0 2023-12-02T00:31:39.983 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2023-12-02T00:31:39.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.985+0000 7fe580b97640 1 -- 172.21.15.46:0/719291240 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe55c06ae40 msgr2=0x7fe55c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:39.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.985+0000 7fe580b97640 1 --2- 172.21.15.46:0/719291240 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe55c06ae40 0x7fe55c06d300 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe57c1364c0 tx=0x7fe564009040 comp rx=0 tx=0).stop 2023-12-02T00:31:39.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.985+0000 7fe580b97640 1 -- 172.21.15.46:0/719291240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe57c06bbf0 msgr2=0x7fe57c132900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:39.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.985+0000 7fe580b97640 1 --2- 172.21.15.46:0/719291240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe57c06bbf0 0x7fe57c132900 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fe56c00e9b0 tx=0x7fe56c00ee70 comp rx=0 tx=0).stop 2023-12-02T00:31:39.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.986+0000 7fe580b97640 1 -- 172.21.15.46:0/719291240 shutdown_connections 2023-12-02T00:31:39.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.986+0000 7fe580b97640 1 --2- 172.21.15.46:0/719291240 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe55c06ae40 0x7fe55c06d300 secure :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe57c1364c0 tx=0x7fe564009040 comp rx=0 tx=0).stop 2023-12-02T00:31:39.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.986+0000 7fe580b97640 1 --2- 172.21.15.46:0/719291240 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe57c144a00 0x7fe57c152980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:39.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.986+0000 7fe580b97640 1 --2- 172.21.15.46:0/719291240 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe57c1415f0 0x7fe57c12fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:39.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.986+0000 7fe580b97640 1 --2- 172.21.15.46:0/719291240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe57c06bbf0 0x7fe57c132900 secure :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fe56c00e9b0 tx=0x7fe56c00ee70 comp rx=0 tx=0).stop 2023-12-02T00:31:39.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.986+0000 7fe580b97640 1 -- 172.21.15.46:0/719291240 >> 172.21.15.46:0/719291240 conn(0x7fe57c070e70 msgr2=0x7fe57c1531d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:39.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.988+0000 7fe580b97640 1 -- 172.21.15.46:0/719291240 shutdown_connections 2023-12-02T00:31:39.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:39.989+0000 7fe580b97640 1 -- 172.21.15.46:0/719291240 wait complete. 2023-12-02T00:31:40.006 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-02T00:31:40.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.120+0000 7fb78ee7c640 1 Processor -- start 2023-12-02T00:31:40.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.120+0000 7fb78ee7c640 1 -- start start 2023-12-02T00:31:40.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.120+0000 7fb78ee7c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb7880d2da0 0x7fb7880d0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:40.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.120+0000 7fb78ee7c640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7880d12f0 con 0x7fb78815d3f0 2023-12-02T00:31:40.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.121+0000 7fb78ee7c640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb78815d7d0 con 0x7fb7880b4c60 2023-12-02T00:31:40.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.121+0000 7fb78ee7c640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7880b61e0 con 0x7fb7880d2da0 2023-12-02T00:31:40.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.121+0000 7fb787fff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb7880d2da0 0x7fb7880d0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:40.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.121+0000 7fb78cc18640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb78815d3f0 0x7fb7880d2690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50908/0 (socket says 172.21.15.46:50908) 2023-12-02T00:31:40.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.121+0000 7fb78cc18640 1 -- 172.21.15.46:0/131063152 learned_addr learned my addr 172.21.15.46:0/131063152 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:31:40.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.121+0000 7fb787fff640 1 -- 172.21.15.46:0/131063152 >> v1:172.21.15.162:6789/0 conn(0x7fb7880b4c60 legacy=0x7fb7880b6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:40.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.121+0000 7fb787fff640 1 -- 172.21.15.46:0/131063152 >> v1:172.21.15.46:6789/0 conn(0x7fb78815d3f0 legacy=0x7fb7880d2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:40.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.121+0000 7fb7877fe640 1 -- 172.21.15.46:0/131063152 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2186533866 0 0) 0x7fb7880d12f0 con 0x7fb78815d3f0 2023-12-02T00:31:40.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.121+0000 7fb787fff640 1 -- 172.21.15.46:0/131063152 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7880d1520 con 0x7fb7880d2da0 2023-12-02T00:31:40.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.122+0000 7fb787fff640 1 --2- 172.21.15.46:0/131063152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7880d2da0 0x7fb7880d0db0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fb7740097a0 tx=0x7fb77402b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adb1607779a5c866 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:40.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.122+0000 7fb7877fe640 1 -- 172.21.15.46:0/131063152 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77402f810 con 0x7fb7880d2da0 2023-12-02T00:31:40.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.122+0000 7fb7877fe640 1 -- 172.21.15.46:0/131063152 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb77402f970 con 0x7fb7880d2da0 2023-12-02T00:31:40.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.122+0000 7fb7877fe640 1 -- 172.21.15.46:0/131063152 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77402fd00 con 0x7fb7880d2da0 2023-12-02T00:31:40.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.122+0000 7fb78ee7c640 1 -- 172.21.15.46:0/131063152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7880d2da0 msgr2=0x7fb7880d0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:40.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.122+0000 7fb78ee7c640 1 --2- 172.21.15.46:0/131063152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7880d2da0 0x7fb7880d0db0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fb7740097a0 tx=0x7fb77402b9f0 comp rx=0 tx=0).stop 2023-12-02T00:31:40.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.123+0000 7fb78ee7c640 1 -- 172.21.15.46:0/131063152 shutdown_connections 2023-12-02T00:31:40.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.123+0000 7fb78ee7c640 1 --2- 172.21.15.46:0/131063152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7880d2da0 0x7fb7880d0db0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:40.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.123+0000 7fb78ee7c640 1 -- 172.21.15.46:0/131063152 >> 172.21.15.46:0/131063152 conn(0x7fb788072230 msgr2=0x7fb7880cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:40.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.123+0000 7fb78ee7c640 1 -- 172.21.15.46:0/131063152 shutdown_connections 2023-12-02T00:31:40.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.123+0000 7fb78ee7c640 1 -- 172.21.15.46:0/131063152 wait complete. 2023-12-02T00:31:40.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.124+0000 7fb78ee7c640 1 Processor -- start 2023-12-02T00:31:40.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.124+0000 7fb78ee7c640 1 -- start start 2023-12-02T00:31:40.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.125+0000 7fb78ee7c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7880b4c60 0x7fb78814a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:40.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.125+0000 7fb78ee7c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7880d2da0 0x7fb7881570d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:40.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.125+0000 7fb78ee7c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb78815d3f0 0x7fb7880c8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:40.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.125+0000 7fb78ee7c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7880b2c40 con 0x7fb78815d3f0 2023-12-02T00:31:40.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.125+0000 7fb78ee7c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7880b34e0 con 0x7fb7880d2da0 2023-12-02T00:31:40.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.125+0000 7fb78ee7c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb78814a9c0 con 0x7fb7880b4c60 2023-12-02T00:31:40.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.125+0000 7fb78d419640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb78815d3f0 0x7fb7880c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:40.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.125+0000 7fb78d419640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb78815d3f0 0x7fb7880c8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45196/0 (socket says 172.21.15.46:45196) 2023-12-02T00:31:40.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.125+0000 7fb78d419640 1 -- 172.21.15.46:0/3042619264 learned_addr learned my addr 172.21.15.46:0/3042619264 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:40.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.125+0000 7fb787fff640 1 --2- 172.21.15.46:0/3042619264 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7880d2da0 0x7fb7881570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:40.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.125+0000 7fb78cc18640 1 --2- 172.21.15.46:0/3042619264 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7880b4c60 0x7fb78814a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:40.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.125+0000 7fb78d419640 1 -- 172.21.15.46:0/3042619264 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7880b4c60 msgr2=0x7fb78814a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:40.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.125+0000 7fb78d419640 1 --2- 172.21.15.46:0/3042619264 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7880b4c60 0x7fb78814a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:40.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.125+0000 7fb78d419640 1 -- 172.21.15.46:0/3042619264 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7880d2da0 msgr2=0x7fb7881570d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:40.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.125+0000 7fb78d419640 1 --2- 172.21.15.46:0/3042619264 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7880d2da0 0x7fb7881570d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:40.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.125+0000 7fb78d419640 1 -- 172.21.15.46:0/3042619264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb774009400 con 0x7fb78815d3f0 2023-12-02T00:31:40.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.125+0000 7fb78d419640 1 --2- 172.21.15.46:0/3042619264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb78815d3f0 0x7fb7880c8330 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fb78000c710 tx=0x7fb78000cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:40.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.125+0000 7fb7857fa640 1 -- 172.21.15.46:0/3042619264 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb780003b70 con 0x7fb78815d3f0 2023-12-02T00:31:40.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.125+0000 7fb7857fa640 1 -- 172.21.15.46:0/3042619264 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb780003cf0 con 0x7fb78815d3f0 2023-12-02T00:31:40.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.125+0000 7fb78ee7c640 1 -- 172.21.15.46:0/3042619264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb78815a060 con 0x7fb78815d3f0 2023-12-02T00:31:40.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.125+0000 7fb7857fa640 1 -- 172.21.15.46:0/3042619264 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78000aa90 con 0x7fb78815d3f0 2023-12-02T00:31:40.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.126+0000 7fb78ee7c640 1 -- 172.21.15.46:0/3042619264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7880b3170 con 0x7fb78815d3f0 2023-12-02T00:31:40.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.127+0000 7fb78ee7c640 1 -- 172.21.15.46:0/3042619264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7880b3670 con 0x7fb78815d3f0 2023-12-02T00:31:40.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.127+0000 7fb7857fa640 1 -- 172.21.15.46:0/3042619264 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb78000abf0 con 0x7fb78815d3f0 2023-12-02T00:31:40.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.127+0000 7fb7857fa640 1 --2- 172.21.15.46:0/3042619264 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb76c06ae40 0x7fb76c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:40.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.127+0000 7fb7857fa640 1 -- 172.21.15.46:0/3042619264 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb780089020 con 0x7fb78815d3f0 2023-12-02T00:31:40.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.127+0000 7fb78cc18640 1 --2- 172.21.15.46:0/3042619264 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb76c06ae40 0x7fb76c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:40.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.128+0000 7fb78cc18640 1 --2- 172.21.15.46:0/3042619264 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb76c06ae40 0x7fb76c06d300 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb77c000f60 tx=0x7fb77c007b90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:40.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.132+0000 7fb7857fa640 1 -- 172.21.15.46:0/3042619264 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb780054460 con 0x7fb78815d3f0 2023-12-02T00:31:40.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.308+0000 7fb78ee7c640 1 -- 172.21.15.46:0/3042619264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb788069400 con 0x7fb78815d3f0 2023-12-02T00:31:40.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.311+0000 7fb7857fa640 1 -- 172.21.15.46:0/3042619264 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb780057a80 con 0x7fb78815d3f0 2023-12-02T00:31:40.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.313+0000 7fb78ee7c640 1 -- 172.21.15.46:0/3042619264 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb76c06ae40 msgr2=0x7fb76c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:40.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.313+0000 7fb78ee7c640 1 --2- 172.21.15.46:0/3042619264 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb76c06ae40 0x7fb76c06d300 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb77c000f60 tx=0x7fb77c007b90 comp rx=0 tx=0).stop 2023-12-02T00:31:40.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.313+0000 7fb78ee7c640 1 -- 172.21.15.46:0/3042619264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb78815d3f0 msgr2=0x7fb7880c8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:40.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.313+0000 7fb78ee7c640 1 --2- 172.21.15.46:0/3042619264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb78815d3f0 0x7fb7880c8330 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fb78000c710 tx=0x7fb78000cbd0 comp rx=0 tx=0).stop 2023-12-02T00:31:40.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.313+0000 7fb78ee7c640 1 -- 172.21.15.46:0/3042619264 shutdown_connections 2023-12-02T00:31:40.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.313+0000 7fb78ee7c640 1 --2- 172.21.15.46:0/3042619264 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb76c06ae40 0x7fb76c06d300 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:40.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.313+0000 7fb78ee7c640 1 --2- 172.21.15.46:0/3042619264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb78815d3f0 0x7fb7880c8330 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:40.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.313+0000 7fb78ee7c640 1 --2- 172.21.15.46:0/3042619264 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7880d2da0 0x7fb7881570d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:40.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.313+0000 7fb78ee7c640 1 --2- 172.21.15.46:0/3042619264 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7880b4c60 0x7fb78814a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:40.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.313+0000 7fb78ee7c640 1 -- 172.21.15.46:0/3042619264 >> 172.21.15.46:0/3042619264 conn(0x7fb788072230 msgr2=0x7fb7880b6320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:40.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.313+0000 7fb78ee7c640 1 -- 172.21.15.46:0/3042619264 shutdown_connections 2023-12-02T00:31:40.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.313+0000 7fb78ee7c640 1 -- 172.21.15.46:0/3042619264 wait complete. 2023-12-02T00:31:40.325 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm' 2023-12-02T00:31:40.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.431+0000 7f2e4bf7c640 1 Processor -- start 2023-12-02T00:31:40.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.431+0000 7f2e4bf7c640 1 -- start start 2023-12-02T00:31:40.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.432+0000 7f2e4bf7c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2e4415d1d0 0x7f2e4406b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:40.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.432+0000 7f2e4bf7c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2e440ad640 0x7f2e4406ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:40.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.432+0000 7f2e4bf7c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2e440aeb80 0x7f2e44160ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:40.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.432+0000 7f2e4bf7c640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e440adae0 con 0x7f2e440aeb80 2023-12-02T00:31:40.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.432+0000 7f2e4bf7c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e440addc0 con 0x7f2e4415d1d0 2023-12-02T00:31:40.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.432+0000 7f2e4bf7c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e440ae0a0 con 0x7f2e440ad640 2023-12-02T00:31:40.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.432+0000 7f2e4a519640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2e440aeb80 0x7f2e44160ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:40.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.432+0000 7f2e4a519640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2e440aeb80 0x7f2e44160ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45198/0 (socket says 172.21.15.46:45198) 2023-12-02T00:31:40.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.432+0000 7f2e4a519640 1 -- 172.21.15.46:0/2363766838 learned_addr learned my addr 172.21.15.46:0/2363766838 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:40.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.432+0000 7f2e49517640 1 --2- 172.21.15.46:0/2363766838 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2e440ad640 0x7f2e4406ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:40.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.432+0000 7f2e49d18640 1 --2- 172.21.15.46:0/2363766838 >> v2:172.21.15.162:3300/0 conn(0x7f2e4415d1d0 0x7f2e4406b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:40.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.432+0000 7f2e4a519640 1 -- 172.21.15.46:0/2363766838 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2e440ad640 msgr2=0x7f2e4406ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:40.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.432+0000 7f2e4a519640 1 --2- 172.21.15.46:0/2363766838 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2e440ad640 0x7f2e4406ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:40.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.432+0000 7f2e4a519640 1 -- 172.21.15.46:0/2363766838 >> v2:172.21.15.162:3300/0 conn(0x7f2e4415d1d0 msgr2=0x7f2e4406b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:40.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.432+0000 7f2e4a519640 1 --2- 172.21.15.46:0/2363766838 >> v2:172.21.15.162:3300/0 conn(0x7f2e4415d1d0 0x7f2e4406b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:40.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.432+0000 7f2e4a519640 1 -- 172.21.15.46:0/2363766838 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e440ae380 con 0x7f2e440aeb80 2023-12-02T00:31:40.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.432+0000 7f2e49d18640 1 --2- 172.21.15.46:0/2363766838 >> v2:172.21.15.162:3300/0 conn(0x7f2e4415d1d0 0x7f2e4406b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:31:40.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.433+0000 7f2e4a519640 1 --2- 172.21.15.46:0/2363766838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e440aeb80 0x7f2e44160ec0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f2e44072310 tx=0x7f2e3c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df8852f45fde8df4 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:40.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.433+0000 7f2e48d16640 1 -- 172.21.15.46:0/2363766838 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c02ca70 con 0x7f2e440aeb80 2023-12-02T00:31:40.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.433+0000 7f2e48d16640 1 -- 172.21.15.46:0/2363766838 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e3c02cbd0 con 0x7f2e440aeb80 2023-12-02T00:31:40.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.433+0000 7f2e48d16640 1 -- 172.21.15.46:0/2363766838 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c0343f0 con 0x7f2e440aeb80 2023-12-02T00:31:40.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.434+0000 7f2e4bf7c640 1 -- 172.21.15.46:0/2363766838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e440aeb80 msgr2=0x7f2e44160ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:40.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.434+0000 7f2e4bf7c640 1 --2- 172.21.15.46:0/2363766838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e440aeb80 0x7f2e44160ec0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f2e44072310 tx=0x7f2e3c02bed0 comp rx=0 tx=0).stop 2023-12-02T00:31:40.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.434+0000 7f2e4bf7c640 1 -- 172.21.15.46:0/2363766838 shutdown_connections 2023-12-02T00:31:40.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.434+0000 7f2e4bf7c640 1 --2- 172.21.15.46:0/2363766838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e440aeb80 0x7f2e44160ec0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:40.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.434+0000 7f2e4bf7c640 1 --2- 172.21.15.46:0/2363766838 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2e440ad640 0x7f2e4406ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:40.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.434+0000 7f2e4bf7c640 1 --2- 172.21.15.46:0/2363766838 >> v2:172.21.15.162:3300/0 conn(0x7f2e4415d1d0 0x7f2e4406b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:40.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.434+0000 7f2e4bf7c640 1 -- 172.21.15.46:0/2363766838 >> 172.21.15.46:0/2363766838 conn(0x7f2e44070e70 msgr2=0x7f2e44071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:40.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.434+0000 7f2e4bf7c640 1 -- 172.21.15.46:0/2363766838 shutdown_connections 2023-12-02T00:31:40.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.434+0000 7f2e4bf7c640 1 -- 172.21.15.46:0/2363766838 wait complete. 2023-12-02T00:31:40.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.435+0000 7f2e4bf7c640 1 Processor -- start 2023-12-02T00:31:40.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.435+0000 7f2e4bf7c640 1 -- start start 2023-12-02T00:31:40.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.436+0000 7f2e4bf7c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e440ad640 0x7f2e44149240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:40.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.436+0000 7f2e4bf7c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e440aeb80 0x7f2e44153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:40.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.436+0000 7f2e4bf7c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2e4415d1d0 0x7f2e4415f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:40.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.436+0000 7f2e4bf7c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e44155220 con 0x7f2e440aeb80 2023-12-02T00:31:40.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.436+0000 7f2e4bf7c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e44155640 con 0x7f2e440ad640 2023-12-02T00:31:40.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.436+0000 7f2e4bf7c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e441545a0 con 0x7f2e4415d1d0 2023-12-02T00:31:40.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.436+0000 7f2e49517640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e440aeb80 0x7f2e44153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:40.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.436+0000 7f2e49517640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e440aeb80 0x7f2e44153cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45202/0 (socket says 172.21.15.46:45202) 2023-12-02T00:31:40.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.436+0000 7f2e49517640 1 -- 172.21.15.46:0/2000784007 learned_addr learned my addr 172.21.15.46:0/2000784007 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:40.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.436+0000 7f2e4a519640 1 --2- 172.21.15.46:0/2000784007 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2e4415d1d0 0x7f2e4415f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:40.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.436+0000 7f2e49d18640 1 --2- 172.21.15.46:0/2000784007 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e440ad640 0x7f2e44149240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:40.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.436+0000 7f2e49517640 1 -- 172.21.15.46:0/2000784007 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2e4415d1d0 msgr2=0x7f2e4415f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:40.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.436+0000 7f2e49517640 1 --2- 172.21.15.46:0/2000784007 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2e4415d1d0 0x7f2e4415f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:40.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.436+0000 7f2e49517640 1 -- 172.21.15.46:0/2000784007 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e440ad640 msgr2=0x7f2e44149240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:40.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.436+0000 7f2e49517640 1 --2- 172.21.15.46:0/2000784007 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e440ad640 0x7f2e44149240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:40.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.436+0000 7f2e49517640 1 -- 172.21.15.46:0/2000784007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e3c009b30 con 0x7f2e440aeb80 2023-12-02T00:31:40.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.436+0000 7f2e49517640 1 --2- 172.21.15.46:0/2000784007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e440aeb80 0x7f2e44153cc0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f2e3400e890 tx=0x7f2e3400ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:40.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.436+0000 7f2e3a7fc640 1 -- 172.21.15.46:0/2000784007 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3400c940 con 0x7f2e440aeb80 2023-12-02T00:31:40.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.436+0000 7f2e3a7fc640 1 -- 172.21.15.46:0/2000784007 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e3400caa0 con 0x7f2e440aeb80 2023-12-02T00:31:40.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.437+0000 7f2e4bf7c640 1 -- 172.21.15.46:0/2000784007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e44154880 con 0x7f2e440aeb80 2023-12-02T00:31:40.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.437+0000 7f2e3a7fc640 1 -- 172.21.15.46:0/2000784007 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3400ce30 con 0x7f2e440aeb80 2023-12-02T00:31:40.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.437+0000 7f2e4bf7c640 1 -- 172.21.15.46:0/2000784007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e44154e40 con 0x7f2e440aeb80 2023-12-02T00:31:40.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.438+0000 7f2e4bf7c640 1 -- 172.21.15.46:0/2000784007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e44069400 con 0x7f2e440aeb80 2023-12-02T00:31:40.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.438+0000 7f2e3a7fc640 1 -- 172.21.15.46:0/2000784007 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2e34013470 con 0x7f2e440aeb80 2023-12-02T00:31:40.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.438+0000 7f2e3a7fc640 1 --2- 172.21.15.46:0/2000784007 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2e2806ae40 0x7f2e2806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:40.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.439+0000 7f2e3a7fc640 1 -- 172.21.15.46:0/2000784007 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2e340894d0 con 0x7f2e440aeb80 2023-12-02T00:31:40.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.439+0000 7f2e49d18640 1 --2- 172.21.15.46:0/2000784007 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2e2806ae40 0x7f2e2806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:40.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.439+0000 7f2e49d18640 1 --2- 172.21.15.46:0/2000784007 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2e2806ae40 0x7f2e2806d300 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2e40006fd0 tx=0x7f2e40008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:40.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.443+0000 7f2e3a7fc640 1 -- 172.21.15.46:0/2000784007 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2e340548a0 con 0x7f2e440aeb80 2023-12-02T00:31:40.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.685+0000 7f2e4bf7c640 1 -- 172.21.15.46:0/2000784007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]} v 0) v1 -- 0x7f2e441583f0 con 0x7f2e440aeb80 2023-12-02T00:31:40.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.982+0000 7f2e3a7fc640 1 -- 172.21.15.46:0/2000784007 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]}]=0 v178) v1 ==== 134+0+0 (secure 0 0 0) 0x7f2e340170c0 con 0x7f2e440aeb80 2023-12-02T00:31:40.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.984+0000 7f2e4bf7c640 1 -- 172.21.15.46:0/2000784007 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2e2806ae40 msgr2=0x7f2e2806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:40.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.984+0000 7f2e4bf7c640 1 --2- 172.21.15.46:0/2000784007 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2e2806ae40 0x7f2e2806d300 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2e40006fd0 tx=0x7f2e40008040 comp rx=0 tx=0).stop 2023-12-02T00:31:40.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.984+0000 7f2e4bf7c640 1 -- 172.21.15.46:0/2000784007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e440aeb80 msgr2=0x7f2e44153cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:40.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.984+0000 7f2e4bf7c640 1 --2- 172.21.15.46:0/2000784007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e440aeb80 0x7f2e44153cc0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f2e3400e890 tx=0x7f2e3400ed50 comp rx=0 tx=0).stop 2023-12-02T00:31:40.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.985+0000 7f2e4bf7c640 1 -- 172.21.15.46:0/2000784007 shutdown_connections 2023-12-02T00:31:40.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.985+0000 7f2e4bf7c640 1 --2- 172.21.15.46:0/2000784007 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2e2806ae40 0x7f2e2806d300 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:40.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.985+0000 7f2e4bf7c640 1 --2- 172.21.15.46:0/2000784007 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2e4415d1d0 0x7f2e4415f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:40.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.985+0000 7f2e4bf7c640 1 --2- 172.21.15.46:0/2000784007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e440aeb80 0x7f2e44153cc0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:40.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.985+0000 7f2e4bf7c640 1 --2- 172.21.15.46:0/2000784007 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e440ad640 0x7f2e44149240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:40.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.985+0000 7f2e4bf7c640 1 -- 172.21.15.46:0/2000784007 >> 172.21.15.46:0/2000784007 conn(0x7f2e44070e70 msgr2=0x7f2e4415bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:40.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.985+0000 7f2e4bf7c640 1 -- 172.21.15.46:0/2000784007 shutdown_connections 2023-12-02T00:31:40.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:40.985+0000 7f2e4bf7c640 1 -- 172.21.15.46:0/2000784007 wait complete. 2023-12-02T00:31:40.998 INFO:tasks.cephfs_test_runner:test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) ... ERROR 2023-12-02T00:31:40.998 INFO:tasks.cephfs_test_runner: 2023-12-02T00:31:40.999 INFO:tasks.cephfs_test_runner:====================================================================== 2023-12-02T00:31:40.999 INFO:tasks.cephfs_test_runner:ERROR: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2023-12-02T00:31:40.999 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-12-02T00:31:40.999 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-12-02T00:31:40.999 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_bed7681480ccb0192b91a35e691658867e1e96b7/qa/tasks/cephfs/test_pool_perm.py", line 52, in test_pool_perm 2023-12-02T00:31:40.999 INFO:tasks.cephfs_test_runner: self.mount_a.run_python(remote_script.format(path=file_path, check_read=str(True))) 2023-12-02T00:31:40.999 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_bed7681480ccb0192b91a35e691658867e1e96b7/qa/tasks/cephfs/mount.py", line 775, in run_python 2023-12-02T00:31:40.999 INFO:tasks.cephfs_test_runner: p.wait() 2023-12-02T00:31:40.999 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 161, in wait 2023-12-02T00:31:40.999 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-12-02T00:31:40.999 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-12-02T00:31:40.999 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-12-02T00:31:40.999 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi046 with status 1: 'stdin-killer -- python3 -c \'\nimport os\nimport errno\n\nfd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR)\ntry:\n if True:\n ret = os.read(fd, 1024)\n else:\n os.write(fd, b\'"\'"\'content\'"\'"\')\nexcept OSError as e:\n if e.errno != errno.EPERM:\n raise\nelse:\n raise RuntimeError("client does not check permission of data pool")\n\'' 2023-12-02T00:31:40.999 INFO:tasks.cephfs_test_runner: 2023-12-02T00:31:40.999 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-12-02T00:31:41.000 INFO:tasks.cephfs_test_runner:Ran 2 tests in 141.118s 2023-12-02T00:31:41.000 INFO:tasks.cephfs_test_runner: 2023-12-02T00:31:41.000 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2023-12-02T00:31:41.000 INFO:tasks.cephfs_test_runner: 2023-12-02T00:31:41.000 INFO:tasks.cephfs_test_runner:====================================================================== 2023-12-02T00:31:41.000 INFO:tasks.cephfs_test_runner:ERROR: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2023-12-02T00:31:41.000 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-12-02T00:31:41.000 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-12-02T00:31:41.000 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_bed7681480ccb0192b91a35e691658867e1e96b7/qa/tasks/cephfs/test_pool_perm.py", line 52, in test_pool_perm 2023-12-02T00:31:41.000 INFO:tasks.cephfs_test_runner: self.mount_a.run_python(remote_script.format(path=file_path, check_read=str(True))) 2023-12-02T00:31:41.000 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_bed7681480ccb0192b91a35e691658867e1e96b7/qa/tasks/cephfs/mount.py", line 775, in run_python 2023-12-02T00:31:41.000 INFO:tasks.cephfs_test_runner: p.wait() 2023-12-02T00:31:41.000 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 161, in wait 2023-12-02T00:31:41.000 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-12-02T00:31:41.000 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-12-02T00:31:41.000 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-12-02T00:31:41.001 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi046 with status 1: 'stdin-killer -- python3 -c \'\nimport os\nimport errno\n\nfd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR)\ntry:\n if True:\n ret = os.read(fd, 1024)\n else:\n os.write(fd, b\'"\'"\'content\'"\'"\')\nexcept OSError as e:\n if e.errno != errno.EPERM:\n raise\nelse:\n raise RuntimeError("client does not check permission of data pool")\n\'' 2023-12-02T00:31:41.001 INFO:tasks.cephfs_test_runner: 2023-12-02T00:31:41.001 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_bed7681480ccb0192b91a35e691658867e1e96b7/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2023-12-02T00:31:41.069 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=4c1c48699b5b4fe6b722addac570329c Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_bed7681480ccb0192b91a35e691658867e1e96b7/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2023-12-02T00:31:41.070 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-12-02T00:31:41.080 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-12-02T00:31:41.110 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-12-02T00:31:41.110 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-02T00:31:41.110 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-02T00:31:41.134 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057150,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498968k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19546 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249480k,nr_inodes=812370,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498968k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-02T00:31:41.134 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-02T00:31:41.134 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-02T00:31:41.189 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057150,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498968k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19546 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249480k,nr_inodes=812370,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498968k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-02T00:31:41.190 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-12-02T00:31:41.190 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-02T00:31:41.208 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056379,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497728k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15716 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248860k,nr_inodes=812215,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497728k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-02T00:31:41.208 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2023-12-02T00:31:41.208 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-02T00:31:41.267 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056379,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497728k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15716 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248860k,nr_inodes=812215,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497728k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-02T00:31:41.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:31:41.267 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2023-12-02T00:31:41.284 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-02T00:31:41.284 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-02T00:31:41.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:31:41.284 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-02T00:31:41.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:31:41.358 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-02T00:31:41.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:31:41.432 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link delete ceph-brx 2023-12-02T00:31:41.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:31:41.523 DEBUG:teuthology.orchestra.run.smithi162:> ip netns list 2023-12-02T00:31:41.541 INFO:teuthology.orchestra.run.smithi162.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-12-02T00:31:41.541 INFO:teuthology.orchestra.run.smithi162.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-12-02T00:31:41.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:31:41.542 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-12-02T00:31:41.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:31:41.619 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-02T00:31:41.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-02T00:31:41.689 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link delete ceph-brx 2023-12-02T00:31:41.788 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-12-02T00:31:41.802 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-12-02T00:31:41.802 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-02T00:31:41.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.914+0000 7f73c312c640 1 Processor -- start 2023-12-02T00:31:41.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.914+0000 7f73c312c640 1 -- start start 2023-12-02T00:31:41.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.914+0000 7f73c312c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f73bc0ceb70 0x7f73bc0cc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:41.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.914+0000 7f73c312c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f73bc0b4370 0x7f73bc0b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:41.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.915+0000 7f73c312c640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73bc071770 con 0x7f73bc144980 2023-12-02T00:31:41.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.915+0000 7f73c312c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73bc15ced0 con 0x7f73bc0b4370 2023-12-02T00:31:41.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.915+0000 7f73c312c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73bc0b4d20 con 0x7f73bc0ceb70 2023-12-02T00:31:41.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.915+0000 7f73bbfff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f73bc0ceb70 0x7f73bc0cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:41.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.915+0000 7f73bbfff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f73bc0ceb70 0x7f73bc0cc450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.46:60408/0 (socket says 172.21.15.46:60408) 2023-12-02T00:31:41.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.915+0000 7f73bbfff640 1 -- 172.21.15.46:0/2079630793 learned_addr learned my addr 172.21.15.46:0/2079630793 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:41.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.915+0000 7f73c16c9640 1 --2- 172.21.15.46:0/2079630793 >> v2:172.21.15.162:3300/0 conn(0x7f73bc0b4370 0x7f73bc0b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:41.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.915+0000 7f73bbfff640 1 -- 172.21.15.46:0/2079630793 >> v2:172.21.15.162:3300/0 conn(0x7f73bc0b4370 msgr2=0x7f73bc0b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:41.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.915+0000 7f73bbfff640 1 --2- 172.21.15.46:0/2079630793 >> v2:172.21.15.162:3300/0 conn(0x7f73bc0b4370 0x7f73bc0b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:41.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.915+0000 7f73bbfff640 1 -- 172.21.15.46:0/2079630793 >> v1:172.21.15.162:6789/0 conn(0x7f73bc144980 legacy=0x7f73bc15c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:41.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.916+0000 7f73bbfff640 1 -- 172.21.15.46:0/2079630793 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73bc0ccdc0 con 0x7f73bc0ceb70 2023-12-02T00:31:41.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.916+0000 7f73c16c9640 1 --2- 172.21.15.46:0/2079630793 >> v2:172.21.15.162:3300/0 conn(0x7f73bc0b4370 0x7f73bc0b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:31:41.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.916+0000 7f73bbfff640 1 --2- 172.21.15.46:0/2079630793 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f73bc0ceb70 0x7f73bc0cc450 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f73b0009870 tx=0x7f73b002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da6dbdf8cbf2c585 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:41.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.916+0000 7f73bb7fe640 1 -- 172.21.15.46:0/2079630793 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b0031640 con 0x7f73bc0ceb70 2023-12-02T00:31:41.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.917+0000 7f73bb7fe640 1 -- 172.21.15.46:0/2079630793 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73b00317a0 con 0x7f73bc0ceb70 2023-12-02T00:31:41.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.917+0000 7f73bb7fe640 1 -- 172.21.15.46:0/2079630793 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b0031b30 con 0x7f73bc0ceb70 2023-12-02T00:31:41.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.917+0000 7f73c312c640 1 -- 172.21.15.46:0/2079630793 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f73bc0ceb70 msgr2=0x7f73bc0cc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:41.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.917+0000 7f73c312c640 1 --2- 172.21.15.46:0/2079630793 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f73bc0ceb70 0x7f73bc0cc450 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f73b0009870 tx=0x7f73b002b8f0 comp rx=0 tx=0).stop 2023-12-02T00:31:41.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.917+0000 7f73c312c640 1 -- 172.21.15.46:0/2079630793 shutdown_connections 2023-12-02T00:31:41.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.917+0000 7f73c312c640 1 --2- 172.21.15.46:0/2079630793 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f73bc0ceb70 0x7f73bc0cc450 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:41.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.917+0000 7f73c312c640 1 --2- 172.21.15.46:0/2079630793 >> v2:172.21.15.162:3300/0 conn(0x7f73bc0b4370 0x7f73bc0b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:41.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.917+0000 7f73c312c640 1 -- 172.21.15.46:0/2079630793 >> 172.21.15.46:0/2079630793 conn(0x7f73bc06a4d0 msgr2=0x7f73bc06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:41.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.917+0000 7f73c312c640 1 -- 172.21.15.46:0/2079630793 shutdown_connections 2023-12-02T00:31:41.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.917+0000 7f73c312c640 1 -- 172.21.15.46:0/2079630793 wait complete. 2023-12-02T00:31:41.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.918+0000 7f73c312c640 1 Processor -- start 2023-12-02T00:31:41.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.918+0000 7f73c312c640 1 -- start start 2023-12-02T00:31:41.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.919+0000 7f73c312c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73bc0b4370 0x7f73bc13b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:41.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.919+0000 7f73c312c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f73bc0ceb70 0x7f73bc13fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:41.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.919+0000 7f73c312c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73bc144980 0x7f73bc13cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:41.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.919+0000 7f73c312c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73bc15bcc0 con 0x7f73bc0b4370 2023-12-02T00:31:41.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.919+0000 7f73c312c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73bc13be10 con 0x7f73bc144980 2023-12-02T00:31:41.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.919+0000 7f73c312c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73bc13c0b0 con 0x7f73bc0ceb70 2023-12-02T00:31:41.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.919+0000 7f73c0ec8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73bc0b4370 0x7f73bc13b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:41.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.919+0000 7f73c0ec8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73bc0b4370 0x7f73bc13b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45212/0 (socket says 172.21.15.46:45212) 2023-12-02T00:31:41.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.919+0000 7f73c0ec8640 1 -- 172.21.15.46:0/1491466229 learned_addr learned my addr 172.21.15.46:0/1491466229 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:41.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.919+0000 7f73bbfff640 1 --2- 172.21.15.46:0/1491466229 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f73bc0ceb70 0x7f73bc13fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:41.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.919+0000 7f73c16c9640 1 --2- 172.21.15.46:0/1491466229 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73bc144980 0x7f73bc13cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:41.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.920+0000 7f73c0ec8640 1 -- 172.21.15.46:0/1491466229 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f73bc0ceb70 msgr2=0x7f73bc13fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:41.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.920+0000 7f73c0ec8640 1 --2- 172.21.15.46:0/1491466229 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f73bc0ceb70 0x7f73bc13fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:41.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.920+0000 7f73c0ec8640 1 -- 172.21.15.46:0/1491466229 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73bc144980 msgr2=0x7f73bc13cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:41.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.920+0000 7f73c0ec8640 1 --2- 172.21.15.46:0/1491466229 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73bc144980 0x7f73bc13cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:41.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.920+0000 7f73c0ec8640 1 -- 172.21.15.46:0/1491466229 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b00094d0 con 0x7f73bc0b4370 2023-12-02T00:31:41.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.920+0000 7f73c0ec8640 1 --2- 172.21.15.46:0/1491466229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73bc0b4370 0x7f73bc13b8d0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f73b400c7c0 tx=0x7f73b400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:41.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.920+0000 7f73b97fa640 1 -- 172.21.15.46:0/1491466229 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b400aa50 con 0x7f73bc0b4370 2023-12-02T00:31:41.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.920+0000 7f73b97fa640 1 -- 172.21.15.46:0/1491466229 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73b400abb0 con 0x7f73bc0b4370 2023-12-02T00:31:41.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.920+0000 7f73b97fa640 1 -- 172.21.15.46:0/1491466229 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b4012420 con 0x7f73bc0b4370 2023-12-02T00:31:41.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.920+0000 7f73c312c640 1 -- 172.21.15.46:0/1491466229 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73bc140960 con 0x7f73bc0b4370 2023-12-02T00:31:41.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.920+0000 7f73c312c640 1 -- 172.21.15.46:0/1491466229 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73bc13c630 con 0x7f73bc0b4370 2023-12-02T00:31:41.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.921+0000 7f73c312c640 1 -- 172.21.15.46:0/1491466229 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73bc069400 con 0x7f73bc0b4370 2023-12-02T00:31:41.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.921+0000 7f73b97fa640 1 -- 172.21.15.46:0/1491466229 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f73b4012580 con 0x7f73bc0b4370 2023-12-02T00:31:41.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.922+0000 7f73b97fa640 1 --2- 172.21.15.46:0/1491466229 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f73a806ae40 0x7f73a806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:41.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.922+0000 7f73b97fa640 1 -- 172.21.15.46:0/1491466229 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f73b40890a0 con 0x7f73bc0b4370 2023-12-02T00:31:41.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.922+0000 7f73bbfff640 1 --2- 172.21.15.46:0/1491466229 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f73a806ae40 0x7f73a806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:41.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.923+0000 7f73bbfff640 1 --2- 172.21.15.46:0/1491466229 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f73a806ae40 0x7f73a806d300 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f73b0009870 tx=0x7f73b003c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:41.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:41.926+0000 7f73b97fa640 1 -- 172.21.15.46:0/1491466229 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f73b4054590 con 0x7f73bc0b4370 2023-12-02T00:31:42.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.105+0000 7f73c312c640 1 -- 172.21.15.46:0/1491466229 --> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f73bc071400 con 0x7f73a806ae40 2023-12-02T00:31:42.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.107+0000 7f73b97fa640 1 -- 172.21.15.46:0/1491466229 <== mgr.4108 v2:172.21.15.162:6832/3248644722 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+51061 (secure 0 0 0) 0x7f73b0031ae0 con 0x7f73a806ae40 2023-12-02T00:31:42.109 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:31:42.109 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-12-02T00:31:42.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.109+0000 7f73c312c640 1 -- 172.21.15.46:0/1491466229 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f73a806ae40 msgr2=0x7f73a806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:42.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.109+0000 7f73c312c640 1 --2- 172.21.15.46:0/1491466229 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f73a806ae40 0x7f73a806d300 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f73b0009870 tx=0x7f73b003c040 comp rx=0 tx=0).stop 2023-12-02T00:31:42.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.109+0000 7f73c312c640 1 -- 172.21.15.46:0/1491466229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73bc0b4370 msgr2=0x7f73bc13b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:42.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.109+0000 7f73c312c640 1 --2- 172.21.15.46:0/1491466229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73bc0b4370 0x7f73bc13b8d0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f73b400c7c0 tx=0x7f73b400cc80 comp rx=0 tx=0).stop 2023-12-02T00:31:42.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.109+0000 7f73c312c640 1 -- 172.21.15.46:0/1491466229 shutdown_connections 2023-12-02T00:31:42.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.109+0000 7f73c312c640 1 --2- 172.21.15.46:0/1491466229 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f73a806ae40 0x7f73a806d300 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:42.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.109+0000 7f73c312c640 1 --2- 172.21.15.46:0/1491466229 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73bc144980 0x7f73bc13cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:42.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.109+0000 7f73c312c640 1 --2- 172.21.15.46:0/1491466229 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f73bc0ceb70 0x7f73bc13fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:42.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.109+0000 7f73c312c640 1 --2- 172.21.15.46:0/1491466229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73bc0b4370 0x7f73bc13b8d0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:42.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.109+0000 7f73c312c640 1 -- 172.21.15.46:0/1491466229 >> 172.21.15.46:0/1491466229 conn(0x7f73bc06a4d0 msgr2=0x7f73bc071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:42.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.109+0000 7f73c312c640 1 -- 172.21.15.46:0/1491466229 shutdown_connections 2023-12-02T00:31:42.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.109+0000 7f73c312c640 1 -- 172.21.15.46:0/1491466229 wait complete. 2023-12-02T00:31:42.121 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2023-12-02T00:31:41.200793+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":184,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":239168,"kb_used_data":6352,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735552,"statfs":{"total":773094113280,"available":772849205248,"internally_reserved":0,"allocated":6504448,"data_stored":3406335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14792,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-62,"num_write_kb":-36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001285"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":198,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-12-02T00:31:37.840833+0000","last_change":"2023-12-02T00:28:39.379380+0000","last_active":"2023-12-02T00:31:37.840833+0000","last_peered":"2023-12-02T00:31:37.840833+0000","last_clean":"2023-12-02T00:31:37.840833+0000","last_became_active":"2023-12-02T00:28:39.378255+0000","last_became_peered":"2023-12-02T00:28:39.378255+0000","last_unstale":"2023-12-02T00:31:37.840833+0000","last_undegraded":"2023-12-02T00:31:37.840833+0000","last_fullsized":"2023-12-02T00:31:37.840833+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:38.177345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:38.177345+0000","last_clean_scrub_stamp":"2023-12-02T00:28:38.177345+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:21:08.582412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574888,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28244,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343596,"statfs":{"total":96636764160,"available":96607842304,"internally_reserved":0,"allocated":1253376,"data_stored":866623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.50900000000000001,"15min":0.499},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.52600000000000002}]},{"osd":1,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53400000000000003,"15min":0.53000000000000003},"min":{"1min":0.40400000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.54700000000000004,"15min":0.54400000000000004},"min":{"1min":0.46100000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.58999999999999997}]},{"osd":2,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54200000000000004,"15min":0.54400000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.67000000000000004,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.624},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.54300000000000004,"15min":0.53400000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.56299999999999994}]},{"osd":3,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.53000000000000003,"15min":0.52200000000000002},"min":{"1min":0.48099999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.57299999999999995,"15min":0.57499999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.74399999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.50800000000000001}]},{"osd":4,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.50700000000000001,"15min":0.51100000000000001},"min":{"1min":0.29799999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.72199999999999998,"5min":0.745,"15min":0.745},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.51500000000000001,"15min":0.52100000000000002},"min":{"1min":0.33800000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.61799999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.378}]},{"osd":5,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54700000000000004,"15min":0.55000000000000004},"min":{"1min":0.41899999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.78700000000000003,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.56100000000000005,"15min":0.56599999999999995},"min":{"1min":0.39500000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.63900000000000001,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.48199999999999998}]},{"osd":6,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.61799999999999999,"15min":0.627},"min":{"1min":0.40100000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.82899999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.44},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.60599999999999998,"15min":0.61299999999999999},"min":{"1min":0.44500000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76600000000000001,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.42999999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27684,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":675840,"data_stored":281854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.49199999999999999,"15min":0.48299999999999998},"min":{"1min":0.36499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.439},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.50700000000000001,"15min":0.501},"min":{"1min":0.40000000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.74099999999999999,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.60399999999999998}]},{"osd":1,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.51000000000000001,"15min":0.50600000000000001},"min":{"1min":0.33000000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.72599999999999998,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.53600000000000003,"15min":0.53300000000000003},"min":{"1min":0.39400000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.80900000000000005,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.38200000000000001}]},{"osd":2,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.52000000000000002,"15min":0.51200000000000001},"min":{"1min":0.38,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55500000000000005,"15min":0.55700000000000005},"min":{"1min":0.40300000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.77400000000000002,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.42099999999999999}]},{"osd":3,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57599999999999996,"15min":0.57299999999999995},"min":{"1min":0.45200000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.76300000000000001,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57199999999999995,"15min":0.56999999999999995},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.63500000000000001}]},{"osd":4,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.35399999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.71299999999999997,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52500000000000002,"15min":0.52300000000000002},"min":{"1min":0.34100000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73699999999999999,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.66900000000000004}]},{"osd":5,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.33400000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.502},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58799999999999997,"15min":0.59299999999999997},"min":{"1min":0.42399999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.61699999999999999}]},{"osd":7,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.35999999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.58899999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574889,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":659456,"data_stored":276255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.51700000000000002,"15min":0.50600000000000001},"min":{"1min":0.29099999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.73899999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.51600000000000001,"15min":0.51100000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.72899999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.871}]},{"osd":1,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.51400000000000001,"15min":0.505},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.754,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55300000000000005,"15min":0.55000000000000004},"min":{"1min":0.36199999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.69499999999999995,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.745}]},{"osd":2,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.56000000000000005,"15min":0.55100000000000005},"min":{"1min":0.41699999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.77500000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.53600000000000003,"15min":0.53300000000000003},"min":{"1min":0.34399999999999997,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.749,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.71299999999999997}]},{"osd":3,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58099999999999996,"15min":0.57699999999999996},"min":{"1min":0.39300000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.58299999999999996,"15min":0.58199999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.72499999999999998,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.82599999999999996}]},{"osd":4,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54300000000000004,"15min":0.54600000000000004},"min":{"1min":0.34799999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69899999999999995,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58699999999999997,"15min":0.58799999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.52800000000000002}]},{"osd":6,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.56599999999999995,"15min":0.57099999999999995},"min":{"1min":0.36899999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.72899999999999998,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.59199999999999997,"15min":0.59599999999999997},"min":{"1min":0.435,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.50600000000000001}]},{"osd":7,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.42699999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.82099999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.621},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.63100000000000001,"15min":0.63500000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.76700000000000002,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.55700000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574888,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27664,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":659456,"data_stored":276255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55300000000000005,"15min":0.54500000000000004},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.59399999999999997,"15min":0.58699999999999997},"min":{"1min":0.48699999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.83599999999999997,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.58299999999999996}]},{"osd":1,"last update":"Sat Dec 2 00:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.57999999999999996,"15min":0.56100000000000005},"min":{"1min":0.38400000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56399999999999995,"15min":0.55600000000000005},"min":{"1min":0.41099999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.73999999999999999,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.68100000000000005}]},{"osd":2,"last update":"Sat Dec 2 00:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.58399999999999996,"15min":0.57799999999999996},"min":{"1min":0.45500000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60799999999999998,"15min":0.60199999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.752,"5min":0.879,"15min":0.879},"last":0.64100000000000001}]},{"osd":3,"last update":"Sat Dec 2 00:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57399999999999995,"15min":0.56399999999999995},"min":{"1min":0.42299999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78100000000000003,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.622},{"interface":"front","average":{"1min":0.621,"5min":0.60899999999999999,"15min":0.60499999999999998},"min":{"1min":0.46800000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.80300000000000005,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.503}]},{"osd":5,"last update":"Sat Dec 2 00:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52700000000000002,"15min":0.52300000000000002},"min":{"1min":0.34899999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.79700000000000004,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.53200000000000003,"15min":0.53800000000000003},"min":{"1min":0.34399999999999997,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71399999999999997,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.72499999999999998}]},{"osd":6,"last update":"Sat Dec 2 00:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.53500000000000003,"15min":0.53400000000000003},"min":{"1min":0.32800000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.70399999999999996,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.57299999999999995,"15min":0.56899999999999995},"min":{"1min":0.41099999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72799999999999998,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.70199999999999996}]},{"osd":7,"last update":"Sat Dec 2 00:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55800000000000005,"15min":0.55500000000000005},"min":{"1min":0.40100000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.74399999999999999,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59199999999999997,"15min":0.58999999999999997},"min":{"1min":0.42599999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.70199999999999996,"5min":1.004,"15min":1.004},"last":0.41599999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574887,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31828,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340012,"statfs":{"total":96636764160,"available":96604172288,"internally_reserved":0,"allocated":659456,"data_stored":276255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.35699999999999998,"15min":0.35999999999999999},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.50800000000000001,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.38200000000000001,"15min":0.38100000000000001},"min":{"1min":0.27500000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.624,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.436}]},{"osd":1,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.379,"15min":0.38300000000000001},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.59999999999999998,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.25800000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.58699999999999997,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.46300000000000002}]},{"osd":2,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.40600000000000003,"15min":0.40999999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.52900000000000003,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.42199999999999999,"15min":0.42099999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.66500000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.58199999999999996}]},{"osd":4,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.60899999999999999,"15min":0.61099999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76600000000000001,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.622,"15min":0.623},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.68999999999999995}]},{"osd":5,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.58299999999999996,"15min":0.58699999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85599999999999998,"5min":1.004,"15min":1.004},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.61299999999999999,"15min":0.61499999999999999},"min":{"1min":0.39300000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.90600000000000003,"5min":1.163,"15min":1.163},"last":0.66800000000000004}]},{"osd":6,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63800000000000001,"15min":0.63900000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.83199999999999996,"5min":1.095,"15min":1.095},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.63200000000000001,"15min":0.63500000000000001},"min":{"1min":0.45100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.96399999999999997,"5min":1.044,"15min":1.044},"last":0.75600000000000001}]},{"osd":7,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.64700000000000002,"15min":0.65100000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.93899999999999995,"5min":1.177,"15min":1.177},"last":0.622},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.64500000000000002,"15min":0.65100000000000002},"min":{"1min":0.45200000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.84799999999999998,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.73099999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574888,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31840,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340000,"statfs":{"total":96636764160,"available":96604160000,"internally_reserved":0,"allocated":671744,"data_stored":281235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.377,"15min":0.36299999999999999},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":2.649,"5min":2.649,"15min":2.649},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.42299999999999999,"15min":0.40799999999999997},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":3.173,"5min":3.173,"15min":3.173},"last":0.254}]},{"osd":1,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.44800000000000001,"15min":0.41999999999999998},"min":{"1min":0.26300000000000001,"5min":0.246,"15min":0.246},"max":{"1min":4.6639999999999997,"5min":4.6639999999999997,"15min":4.6639999999999997},"last":0.375},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.433,"15min":0.39100000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":6.093,"5min":6.093,"15min":6.093},"last":0.31}]},{"osd":3,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.40300000000000002,"15min":0.376},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":4.1879999999999997,"5min":4.1879999999999997,"15min":4.1879999999999997},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.44600000000000001,"15min":0.433},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":2.4159999999999999,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":0.28899999999999998}]},{"osd":4,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.60999999999999999,"15min":0.59599999999999997},"min":{"1min":0.40999999999999998,"5min":0.38,"15min":0.38},"max":{"1min":2.6840000000000002,"5min":2.6840000000000002,"15min":2.6840000000000002},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.65800000000000003,"15min":0.64600000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":3.0419999999999998,"5min":3.0419999999999998,"15min":3.0419999999999998},"last":0.57799999999999996}]},{"osd":5,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.66100000000000003,"15min":0.65300000000000002},"min":{"1min":0.41599999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":3.093,"5min":3.093,"15min":3.093},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.64700000000000002,"15min":0.65100000000000002},"min":{"1min":0.41799999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.98499999999999999,"5min":1.111,"15min":1.111},"last":0.52800000000000002}]},{"osd":6,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.64000000000000001,"15min":0.63800000000000001},"min":{"1min":0.42499999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.67800000000000005,"15min":0.67200000000000004},"min":{"1min":0.47799999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":2.6030000000000002,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":0.55300000000000005}]},{"osd":7,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.65800000000000003,"15min":0.64800000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":3.1339999999999999,"5min":3.1339999999999999,"15min":3.1339999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.64200000000000002,"15min":0.64600000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.94199999999999995,"5min":1.2,"15min":1.2},"last":0.61199999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574887,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31824,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340016,"statfs":{"total":96636764160,"available":96604176384,"internally_reserved":0,"allocated":659456,"data_stored":276255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.33900000000000002,"15min":0.34100000000000003},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.436,"5min":0.48099999999999998,"15min":0.48099999999999998},"last":0.23899999999999999},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.39600000000000002,"15min":0.39800000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.58799999999999997,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.32800000000000001}]},{"osd":2,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.35099999999999998,"15min":0.34899999999999998},"min":{"1min":0.249,"5min":0.246,"15min":0.246},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.312},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.36299999999999999,"15min":0.36699999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.53500000000000003,"5min":0.626,"15min":0.626},"last":0.26400000000000001}]},{"osd":3,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.40100000000000002,"15min":0.40200000000000002},"min":{"1min":0.30499999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.371,"15min":0.376},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.5,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.36699999999999999}]},{"osd":4,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.55400000000000005,"15min":0.54800000000000004},"min":{"1min":0.377,"5min":0.376,"15min":0.376},"max":{"1min":0.77100000000000002,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.55800000000000005,"15min":0.55100000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.78000000000000003,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.55300000000000005}]},{"osd":5,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.58599999999999997,"15min":0.57899999999999996},"min":{"1min":0.45900000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.96799999999999997,"5min":1.107,"15min":1.107},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57299999999999995,"15min":0.56599999999999995},"min":{"1min":0.437,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.56999999999999995}]},{"osd":6,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60399999999999998,"15min":0.60199999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.76600000000000001,"5min":1.044,"15min":1.044},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.61199999999999999,"15min":0.60599999999999998},"min":{"1min":0.51000000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.85399999999999998,"5min":0.998,"15min":0.998},"last":0.60799999999999998}]},{"osd":7,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58999999999999997,"15min":0.58799999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.78900000000000003,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.627,"5min":0.60599999999999998,"15min":0.60199999999999998},"min":{"1min":0.45300000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.80000000000000004,"5min":1.151,"15min":1.151},"last":0.48799999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574889,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32420,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339420,"statfs":{"total":96636764160,"available":96603566080,"internally_reserved":0,"allocated":1265664,"data_stored":871603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Dec 2 00:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.57099999999999995,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.371,"15min":0.36499999999999999},"min":{"1min":0.26700000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.54000000000000004,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.45700000000000002}]},{"osd":2,"last update":"Sat Dec 2 00:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39300000000000002,"15min":0.39200000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.58499999999999996,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.436},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.40200000000000002,"15min":0.40100000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.56399999999999995,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.50600000000000001}]},{"osd":3,"last update":"Sat Dec 2 00:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.39100000000000001,"15min":0.39300000000000002},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.52700000000000002,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.42599999999999999,"15min":0.42699999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63900000000000001,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.47999999999999998}]},{"osd":4,"last update":"Sat Dec 2 00:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.40200000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.61499999999999999,"15min":0.61799999999999999},"min":{"1min":0.45300000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.78900000000000003,"5min":1.002,"15min":1.002},"last":0.78900000000000003}]},{"osd":5,"last update":"Sat Dec 2 00:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.629,"15min":0.627},"min":{"1min":0.49099999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":1.028},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64700000000000002,"15min":0.64600000000000002},"min":{"1min":0.47199999999999998,"5min":0.433,"15min":0.433},"max":{"1min":1.0129999999999999,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":1.0129999999999999}]},{"osd":6,"last update":"Sat Dec 2 00:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60999999999999999,"15min":0.60099999999999998},"min":{"1min":0.435,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88500000000000001,"5min":1.05,"15min":1.05},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.63300000000000001,"15min":0.63800000000000001},"min":{"1min":0.435,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.97399999999999998}]},{"osd":7,"last update":"Sat Dec 2 00:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.64000000000000001,"15min":0.63800000000000001},"min":{"1min":0.44500000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.86699999999999999,"5min":1.028,"15min":1.028},"last":0.754},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.624,"15min":0.626},"min":{"1min":0.432,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.91000000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-02T00:31:42.125 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-02T00:31:42.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.230+0000 7f62b4525640 1 Processor -- start 2023-12-02T00:31:42.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.230+0000 7f62b4525640 1 -- start start 2023-12-02T00:31:42.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.231+0000 7f62b4525640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f62ac143490 0x7f62ac0b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:42.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.231+0000 7f62b4525640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62ac15e2a0 con 0x7f62ac15abf0 2023-12-02T00:31:42.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.231+0000 7f62b4525640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62ac15b240 con 0x7f62ac15e730 2023-12-02T00:31:42.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.231+0000 7f62b4525640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62ac15b520 con 0x7f62ac143490 2023-12-02T00:31:42.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.231+0000 7f62b22c1640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f62ac143490 0x7f62ac0b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:42.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.231+0000 7f62b22c1640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f62ac143490 0x7f62ac0b0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45226/0 (socket says 172.21.15.46:45226) 2023-12-02T00:31:42.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.231+0000 7f62b22c1640 1 -- 172.21.15.46:0/2254168411 learned_addr learned my addr 172.21.15.46:0/2254168411 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:42.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.231+0000 7f62b2ac2640 1 --1- 172.21.15.46:0/2254168411 >> v1:172.21.15.46:6789/0 conn(0x7f62ac15abf0 0x7f62ac0b2f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50918/0 (socket says 172.21.15.46:50918) 2023-12-02T00:31:42.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.232+0000 7f62b22c1640 1 -- 172.21.15.46:0/2254168411 >> v1:172.21.15.162:6789/0 conn(0x7f62ac15e730 legacy=0x7f62ac15a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:42.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.232+0000 7f62b22c1640 1 -- 172.21.15.46:0/2254168411 >> v1:172.21.15.46:6789/0 conn(0x7f62ac15abf0 legacy=0x7f62ac0b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:42.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.232+0000 7f62b22c1640 1 -- 172.21.15.46:0/2254168411 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ac15dc50 con 0x7f62ac143490 2023-12-02T00:31:42.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.232+0000 7f62b12bf640 1 -- 172.21.15.46:0/2254168411 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3433057244 0 0) 0x7f62ac15e2a0 con 0x7f62ac15abf0 2023-12-02T00:31:42.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.232+0000 7f62b22c1640 1 --2- 172.21.15.46:0/2254168411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62ac143490 0x7f62ac0b0cf0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f62a4009e50 tx=0x7f62a402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e26d3a895890a6f server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:42.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.232+0000 7f62b12bf640 1 -- 172.21.15.46:0/2254168411 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a402cb20 con 0x7f62ac143490 2023-12-02T00:31:42.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.232+0000 7f62b12bf640 1 -- 172.21.15.46:0/2254168411 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62a402cc80 con 0x7f62ac143490 2023-12-02T00:31:42.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.232+0000 7f62b12bf640 1 -- 172.21.15.46:0/2254168411 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a40343f0 con 0x7f62ac143490 2023-12-02T00:31:42.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.233+0000 7f62b4525640 1 -- 172.21.15.46:0/2254168411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62ac143490 msgr2=0x7f62ac0b0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:42.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.233+0000 7f62b4525640 1 --2- 172.21.15.46:0/2254168411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62ac143490 0x7f62ac0b0cf0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f62a4009e50 tx=0x7f62a402bf80 comp rx=0 tx=0).stop 2023-12-02T00:31:42.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.233+0000 7f62b4525640 1 -- 172.21.15.46:0/2254168411 shutdown_connections 2023-12-02T00:31:42.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.233+0000 7f62b4525640 1 --2- 172.21.15.46:0/2254168411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62ac143490 0x7f62ac0b0cf0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:42.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.233+0000 7f62b4525640 1 -- 172.21.15.46:0/2254168411 >> 172.21.15.46:0/2254168411 conn(0x7f62ac06a4d0 msgr2=0x7f62ac06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:42.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.233+0000 7f62b4525640 1 -- 172.21.15.46:0/2254168411 shutdown_connections 2023-12-02T00:31:42.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.233+0000 7f62b4525640 1 -- 172.21.15.46:0/2254168411 wait complete. 2023-12-02T00:31:42.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.234+0000 7f62b4525640 1 Processor -- start 2023-12-02T00:31:42.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.234+0000 7f62b4525640 1 -- start start 2023-12-02T00:31:42.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.235+0000 7f62b4525640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62ac143490 0x7f62ac13a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:42.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.235+0000 7f62b4525640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62ac15abf0 0x7f62ac070500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:42.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.235+0000 7f62b4525640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f62ac15e730 0x7f62ac134450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:42.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.235+0000 7f62b4525640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ac138a70 con 0x7f62ac15abf0 2023-12-02T00:31:42.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.235+0000 7f62b4525640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ac138d50 con 0x7f62ac143490 2023-12-02T00:31:42.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.235+0000 7f62b4525640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62ac139030 con 0x7f62ac15e730 2023-12-02T00:31:42.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.235+0000 7f62b1ac0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62ac15abf0 0x7f62ac070500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:42.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.235+0000 7f62b1ac0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62ac15abf0 0x7f62ac070500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45230/0 (socket says 172.21.15.46:45230) 2023-12-02T00:31:42.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.235+0000 7f62b1ac0640 1 -- 172.21.15.46:0/351892134 learned_addr learned my addr 172.21.15.46:0/351892134 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:42.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.235+0000 7f62b22c1640 1 --2- 172.21.15.46:0/351892134 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62ac143490 0x7f62ac13a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:42.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.235+0000 7f62b2ac2640 1 --2- 172.21.15.46:0/351892134 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f62ac15e730 0x7f62ac134450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:42.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.235+0000 7f62b1ac0640 1 -- 172.21.15.46:0/351892134 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f62ac15e730 msgr2=0x7f62ac134450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:42.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.235+0000 7f62b1ac0640 1 --2- 172.21.15.46:0/351892134 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f62ac15e730 0x7f62ac134450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:42.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.235+0000 7f62b1ac0640 1 -- 172.21.15.46:0/351892134 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62ac143490 msgr2=0x7f62ac13a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:42.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.235+0000 7f62b1ac0640 1 --2- 172.21.15.46:0/351892134 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62ac143490 0x7f62ac13a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:42.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.236+0000 7f62b1ac0640 1 -- 172.21.15.46:0/351892134 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a4009b30 con 0x7f62ac15abf0 2023-12-02T00:31:42.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.236+0000 7f62b1ac0640 1 --2- 172.21.15.46:0/351892134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62ac15abf0 0x7f62ac070500 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f62a000b3f0 tx=0x7f62a000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:42.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.236+0000 7f62aaffd640 1 -- 172.21.15.46:0/351892134 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a0011830 con 0x7f62ac15abf0 2023-12-02T00:31:42.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.236+0000 7f62aaffd640 1 -- 172.21.15.46:0/351892134 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62a0011990 con 0x7f62ac15abf0 2023-12-02T00:31:42.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.236+0000 7f62aaffd640 1 -- 172.21.15.46:0/351892134 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a0011d40 con 0x7f62ac15abf0 2023-12-02T00:31:42.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.236+0000 7f62b4525640 1 -- 172.21.15.46:0/351892134 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62ac139310 con 0x7f62ac15abf0 2023-12-02T00:31:42.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.236+0000 7f62b4525640 1 -- 172.21.15.46:0/351892134 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62ac13a6c0 con 0x7f62ac15abf0 2023-12-02T00:31:42.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.237+0000 7f62b4525640 1 -- 172.21.15.46:0/351892134 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62ac0b4380 con 0x7f62ac15abf0 2023-12-02T00:31:42.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.237+0000 7f62aaffd640 1 -- 172.21.15.46:0/351892134 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f62a001a400 con 0x7f62ac15abf0 2023-12-02T00:31:42.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.238+0000 7f62aaffd640 1 --2- 172.21.15.46:0/351892134 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f629806ae70 0x7f629806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:42.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.238+0000 7f62aaffd640 1 -- 172.21.15.46:0/351892134 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f62a0089380 con 0x7f62ac15abf0 2023-12-02T00:31:42.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.238+0000 7f62b22c1640 1 --2- 172.21.15.46:0/351892134 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f629806ae70 0x7f629806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:42.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.239+0000 7f62b22c1640 1 --2- 172.21.15.46:0/351892134 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f629806ae70 0x7f629806d330 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f62a4035000 tx=0x7f62a4030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:42.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.242+0000 7f62aaffd640 1 -- 172.21.15.46:0/351892134 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f62a00547c0 con 0x7f62ac15abf0 2023-12-02T00:31:42.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.417+0000 7f62b4525640 1 -- 172.21.15.46:0/351892134 --> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f62ac138ec0 con 0x7f629806ae70 2023-12-02T00:31:42.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.420+0000 7f62aaffd640 1 -- 172.21.15.46:0/351892134 <== mgr.4108 v2:172.21.15.162:6832/3248644722 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+51061 (secure 0 0 0) 0x7f62ac138ec0 con 0x7f629806ae70 2023-12-02T00:31:42.421 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:31:42.422 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-12-02T00:31:42.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.422+0000 7f62b4525640 1 -- 172.21.15.46:0/351892134 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f629806ae70 msgr2=0x7f629806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:42.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.422+0000 7f62b4525640 1 --2- 172.21.15.46:0/351892134 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f629806ae70 0x7f629806d330 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f62a4035000 tx=0x7f62a4030000 comp rx=0 tx=0).stop 2023-12-02T00:31:42.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.422+0000 7f62b4525640 1 -- 172.21.15.46:0/351892134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62ac15abf0 msgr2=0x7f62ac070500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:42.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.422+0000 7f62b4525640 1 --2- 172.21.15.46:0/351892134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62ac15abf0 0x7f62ac070500 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f62a000b3f0 tx=0x7f62a000b8b0 comp rx=0 tx=0).stop 2023-12-02T00:31:42.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.422+0000 7f62b4525640 1 -- 172.21.15.46:0/351892134 shutdown_connections 2023-12-02T00:31:42.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.422+0000 7f62b4525640 1 --2- 172.21.15.46:0/351892134 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f629806ae70 0x7f629806d330 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:42.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.422+0000 7f62b4525640 1 --2- 172.21.15.46:0/351892134 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f62ac15e730 0x7f62ac134450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:42.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.422+0000 7f62b4525640 1 --2- 172.21.15.46:0/351892134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62ac15abf0 0x7f62ac070500 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:42.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.422+0000 7f62b4525640 1 --2- 172.21.15.46:0/351892134 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62ac143490 0x7f62ac13a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:42.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.422+0000 7f62b4525640 1 -- 172.21.15.46:0/351892134 >> 172.21.15.46:0/351892134 conn(0x7f62ac06a4d0 msgr2=0x7f62ac0aeba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:42.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.422+0000 7f62b4525640 1 -- 172.21.15.46:0/351892134 shutdown_connections 2023-12-02T00:31:42.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.422+0000 7f62b4525640 1 -- 172.21.15.46:0/351892134 wait complete. 2023-12-02T00:31:42.434 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2023-12-02T00:31:41.200793+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":184,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":239168,"kb_used_data":6352,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735552,"statfs":{"total":773094113280,"available":772849205248,"internally_reserved":0,"allocated":6504448,"data_stored":3406335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14792,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-62,"num_write_kb":-36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001285"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":198,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-12-02T00:31:37.840833+0000","last_change":"2023-12-02T00:28:39.379380+0000","last_active":"2023-12-02T00:31:37.840833+0000","last_peered":"2023-12-02T00:31:37.840833+0000","last_clean":"2023-12-02T00:31:37.840833+0000","last_became_active":"2023-12-02T00:28:39.378255+0000","last_became_peered":"2023-12-02T00:28:39.378255+0000","last_unstale":"2023-12-02T00:31:37.840833+0000","last_undegraded":"2023-12-02T00:31:37.840833+0000","last_fullsized":"2023-12-02T00:31:37.840833+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:38.177345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:38.177345+0000","last_clean_scrub_stamp":"2023-12-02T00:28:38.177345+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:21:08.582412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574888,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28244,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343596,"statfs":{"total":96636764160,"available":96607842304,"internally_reserved":0,"allocated":1253376,"data_stored":866623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.50900000000000001,"15min":0.499},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.52600000000000002}]},{"osd":1,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53400000000000003,"15min":0.53000000000000003},"min":{"1min":0.40400000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.54700000000000004,"15min":0.54400000000000004},"min":{"1min":0.46100000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.58999999999999997}]},{"osd":2,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54200000000000004,"15min":0.54400000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.67000000000000004,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.624},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.54300000000000004,"15min":0.53400000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.56299999999999994}]},{"osd":3,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.53000000000000003,"15min":0.52200000000000002},"min":{"1min":0.48099999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.57299999999999995,"15min":0.57499999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.74399999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.50800000000000001}]},{"osd":4,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.50700000000000001,"15min":0.51100000000000001},"min":{"1min":0.29799999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.72199999999999998,"5min":0.745,"15min":0.745},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.51500000000000001,"15min":0.52100000000000002},"min":{"1min":0.33800000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.61799999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.378}]},{"osd":5,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54700000000000004,"15min":0.55000000000000004},"min":{"1min":0.41899999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.78700000000000003,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.56100000000000005,"15min":0.56599999999999995},"min":{"1min":0.39500000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.63900000000000001,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.48199999999999998}]},{"osd":6,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.61799999999999999,"15min":0.627},"min":{"1min":0.40100000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.82899999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.44},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.60599999999999998,"15min":0.61299999999999999},"min":{"1min":0.44500000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76600000000000001,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.42999999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27684,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":675840,"data_stored":281854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.49199999999999999,"15min":0.48299999999999998},"min":{"1min":0.36499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.439},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.50700000000000001,"15min":0.501},"min":{"1min":0.40000000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.74099999999999999,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.60399999999999998}]},{"osd":1,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.51000000000000001,"15min":0.50600000000000001},"min":{"1min":0.33000000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.72599999999999998,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.53600000000000003,"15min":0.53300000000000003},"min":{"1min":0.39400000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.80900000000000005,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.38200000000000001}]},{"osd":2,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.52000000000000002,"15min":0.51200000000000001},"min":{"1min":0.38,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55500000000000005,"15min":0.55700000000000005},"min":{"1min":0.40300000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.77400000000000002,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.42099999999999999}]},{"osd":3,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57599999999999996,"15min":0.57299999999999995},"min":{"1min":0.45200000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.76300000000000001,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57199999999999995,"15min":0.56999999999999995},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.63500000000000001}]},{"osd":4,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.35399999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.71299999999999997,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52500000000000002,"15min":0.52300000000000002},"min":{"1min":0.34100000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73699999999999999,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.66900000000000004}]},{"osd":5,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.33400000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.502},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58799999999999997,"15min":0.59299999999999997},"min":{"1min":0.42399999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.61699999999999999}]},{"osd":7,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.35999999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.58899999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574889,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":659456,"data_stored":276255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.51700000000000002,"15min":0.50600000000000001},"min":{"1min":0.29099999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.73899999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.51600000000000001,"15min":0.51100000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.72899999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.871}]},{"osd":1,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.51400000000000001,"15min":0.505},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.754,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55300000000000005,"15min":0.55000000000000004},"min":{"1min":0.36199999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.69499999999999995,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.745}]},{"osd":2,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.56000000000000005,"15min":0.55100000000000005},"min":{"1min":0.41699999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.77500000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.53600000000000003,"15min":0.53300000000000003},"min":{"1min":0.34399999999999997,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.749,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.71299999999999997}]},{"osd":3,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58099999999999996,"15min":0.57699999999999996},"min":{"1min":0.39300000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.58299999999999996,"15min":0.58199999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.72499999999999998,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.82599999999999996}]},{"osd":4,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54300000000000004,"15min":0.54600000000000004},"min":{"1min":0.34799999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69899999999999995,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58699999999999997,"15min":0.58799999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.52800000000000002}]},{"osd":6,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.56599999999999995,"15min":0.57099999999999995},"min":{"1min":0.36899999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.72899999999999998,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.59199999999999997,"15min":0.59599999999999997},"min":{"1min":0.435,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.50600000000000001}]},{"osd":7,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.42699999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.82099999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.621},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.63100000000000001,"15min":0.63500000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.76700000000000002,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.55700000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574888,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27664,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":659456,"data_stored":276255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55300000000000005,"15min":0.54500000000000004},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.59399999999999997,"15min":0.58699999999999997},"min":{"1min":0.48699999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.83599999999999997,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.58299999999999996}]},{"osd":1,"last update":"Sat Dec 2 00:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.57999999999999996,"15min":0.56100000000000005},"min":{"1min":0.38400000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56399999999999995,"15min":0.55600000000000005},"min":{"1min":0.41099999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.73999999999999999,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.68100000000000005}]},{"osd":2,"last update":"Sat Dec 2 00:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.58399999999999996,"15min":0.57799999999999996},"min":{"1min":0.45500000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60799999999999998,"15min":0.60199999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.752,"5min":0.879,"15min":0.879},"last":0.64100000000000001}]},{"osd":3,"last update":"Sat Dec 2 00:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57399999999999995,"15min":0.56399999999999995},"min":{"1min":0.42299999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78100000000000003,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.622},{"interface":"front","average":{"1min":0.621,"5min":0.60899999999999999,"15min":0.60499999999999998},"min":{"1min":0.46800000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.80300000000000005,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.503}]},{"osd":5,"last update":"Sat Dec 2 00:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52700000000000002,"15min":0.52300000000000002},"min":{"1min":0.34899999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.79700000000000004,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.53200000000000003,"15min":0.53800000000000003},"min":{"1min":0.34399999999999997,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71399999999999997,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.72499999999999998}]},{"osd":6,"last update":"Sat Dec 2 00:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.53500000000000003,"15min":0.53400000000000003},"min":{"1min":0.32800000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.70399999999999996,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.57299999999999995,"15min":0.56899999999999995},"min":{"1min":0.41099999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72799999999999998,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.70199999999999996}]},{"osd":7,"last update":"Sat Dec 2 00:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55800000000000005,"15min":0.55500000000000005},"min":{"1min":0.40100000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.74399999999999999,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59199999999999997,"15min":0.58999999999999997},"min":{"1min":0.42599999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.70199999999999996,"5min":1.004,"15min":1.004},"last":0.41599999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574887,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31828,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340012,"statfs":{"total":96636764160,"available":96604172288,"internally_reserved":0,"allocated":659456,"data_stored":276255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.35699999999999998,"15min":0.35999999999999999},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.50800000000000001,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.38200000000000001,"15min":0.38100000000000001},"min":{"1min":0.27500000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.624,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.436}]},{"osd":1,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.379,"15min":0.38300000000000001},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.59999999999999998,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.25800000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.58699999999999997,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.46300000000000002}]},{"osd":2,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.40600000000000003,"15min":0.40999999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.52900000000000003,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.42199999999999999,"15min":0.42099999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.66500000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.58199999999999996}]},{"osd":4,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.60899999999999999,"15min":0.61099999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76600000000000001,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.622,"15min":0.623},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.68999999999999995}]},{"osd":5,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.58299999999999996,"15min":0.58699999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85599999999999998,"5min":1.004,"15min":1.004},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.61299999999999999,"15min":0.61499999999999999},"min":{"1min":0.39300000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.90600000000000003,"5min":1.163,"15min":1.163},"last":0.66800000000000004}]},{"osd":6,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63800000000000001,"15min":0.63900000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.83199999999999996,"5min":1.095,"15min":1.095},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.63200000000000001,"15min":0.63500000000000001},"min":{"1min":0.45100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.96399999999999997,"5min":1.044,"15min":1.044},"last":0.75600000000000001}]},{"osd":7,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.64700000000000002,"15min":0.65100000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.93899999999999995,"5min":1.177,"15min":1.177},"last":0.622},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.64500000000000002,"15min":0.65100000000000002},"min":{"1min":0.45200000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.84799999999999998,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.73099999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574888,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31840,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340000,"statfs":{"total":96636764160,"available":96604160000,"internally_reserved":0,"allocated":671744,"data_stored":281235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.377,"15min":0.36299999999999999},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":2.649,"5min":2.649,"15min":2.649},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.42299999999999999,"15min":0.40799999999999997},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":3.173,"5min":3.173,"15min":3.173},"last":0.254}]},{"osd":1,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.44800000000000001,"15min":0.41999999999999998},"min":{"1min":0.26300000000000001,"5min":0.246,"15min":0.246},"max":{"1min":4.6639999999999997,"5min":4.6639999999999997,"15min":4.6639999999999997},"last":0.375},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.433,"15min":0.39100000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":6.093,"5min":6.093,"15min":6.093},"last":0.31}]},{"osd":3,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.40300000000000002,"15min":0.376},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":4.1879999999999997,"5min":4.1879999999999997,"15min":4.1879999999999997},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.44600000000000001,"15min":0.433},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":2.4159999999999999,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":0.28899999999999998}]},{"osd":4,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.60999999999999999,"15min":0.59599999999999997},"min":{"1min":0.40999999999999998,"5min":0.38,"15min":0.38},"max":{"1min":2.6840000000000002,"5min":2.6840000000000002,"15min":2.6840000000000002},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.65800000000000003,"15min":0.64600000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":3.0419999999999998,"5min":3.0419999999999998,"15min":3.0419999999999998},"last":0.57799999999999996}]},{"osd":5,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.66100000000000003,"15min":0.65300000000000002},"min":{"1min":0.41599999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":3.093,"5min":3.093,"15min":3.093},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.64700000000000002,"15min":0.65100000000000002},"min":{"1min":0.41799999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.98499999999999999,"5min":1.111,"15min":1.111},"last":0.52800000000000002}]},{"osd":6,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.64000000000000001,"15min":0.63800000000000001},"min":{"1min":0.42499999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.67800000000000005,"15min":0.67200000000000004},"min":{"1min":0.47799999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":2.6030000000000002,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":0.55300000000000005}]},{"osd":7,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.65800000000000003,"15min":0.64800000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":3.1339999999999999,"5min":3.1339999999999999,"15min":3.1339999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.64200000000000002,"15min":0.64600000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.94199999999999995,"5min":1.2,"15min":1.2},"last":0.61199999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574887,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31824,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340016,"statfs":{"total":96636764160,"available":96604176384,"internally_reserved":0,"allocated":659456,"data_stored":276255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.33900000000000002,"15min":0.34100000000000003},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.436,"5min":0.48099999999999998,"15min":0.48099999999999998},"last":0.23899999999999999},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.39600000000000002,"15min":0.39800000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.58799999999999997,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.32800000000000001}]},{"osd":2,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.35099999999999998,"15min":0.34899999999999998},"min":{"1min":0.249,"5min":0.246,"15min":0.246},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.312},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.36299999999999999,"15min":0.36699999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.53500000000000003,"5min":0.626,"15min":0.626},"last":0.26400000000000001}]},{"osd":3,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.40100000000000002,"15min":0.40200000000000002},"min":{"1min":0.30499999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.371,"15min":0.376},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.5,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.36699999999999999}]},{"osd":4,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.55400000000000005,"15min":0.54800000000000004},"min":{"1min":0.377,"5min":0.376,"15min":0.376},"max":{"1min":0.77100000000000002,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.55800000000000005,"15min":0.55100000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.78000000000000003,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.55300000000000005}]},{"osd":5,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.58599999999999997,"15min":0.57899999999999996},"min":{"1min":0.45900000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.96799999999999997,"5min":1.107,"15min":1.107},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57299999999999995,"15min":0.56599999999999995},"min":{"1min":0.437,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.56999999999999995}]},{"osd":6,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60399999999999998,"15min":0.60199999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.76600000000000001,"5min":1.044,"15min":1.044},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.61199999999999999,"15min":0.60599999999999998},"min":{"1min":0.51000000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.85399999999999998,"5min":0.998,"15min":0.998},"last":0.60799999999999998}]},{"osd":7,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58999999999999997,"15min":0.58799999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.78900000000000003,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.627,"5min":0.60599999999999998,"15min":0.60199999999999998},"min":{"1min":0.45300000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.80000000000000004,"5min":1.151,"15min":1.151},"last":0.48799999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574889,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32420,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339420,"statfs":{"total":96636764160,"available":96603566080,"internally_reserved":0,"allocated":1265664,"data_stored":871603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Dec 2 00:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.57099999999999995,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.371,"15min":0.36499999999999999},"min":{"1min":0.26700000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.54000000000000004,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.45700000000000002}]},{"osd":2,"last update":"Sat Dec 2 00:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39300000000000002,"15min":0.39200000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.58499999999999996,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.436},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.40200000000000002,"15min":0.40100000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.56399999999999995,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.50600000000000001}]},{"osd":3,"last update":"Sat Dec 2 00:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.39100000000000001,"15min":0.39300000000000002},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.52700000000000002,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.42599999999999999,"15min":0.42699999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63900000000000001,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.47999999999999998}]},{"osd":4,"last update":"Sat Dec 2 00:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.40200000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.61499999999999999,"15min":0.61799999999999999},"min":{"1min":0.45300000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.78900000000000003,"5min":1.002,"15min":1.002},"last":0.78900000000000003}]},{"osd":5,"last update":"Sat Dec 2 00:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.629,"15min":0.627},"min":{"1min":0.49099999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":1.028},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64700000000000002,"15min":0.64600000000000002},"min":{"1min":0.47199999999999998,"5min":0.433,"15min":0.433},"max":{"1min":1.0129999999999999,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":1.0129999999999999}]},{"osd":6,"last update":"Sat Dec 2 00:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60999999999999999,"15min":0.60099999999999998},"min":{"1min":0.435,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88500000000000001,"5min":1.05,"15min":1.05},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.63300000000000001,"15min":0.63800000000000001},"min":{"1min":0.435,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.97399999999999998}]},{"osd":7,"last update":"Sat Dec 2 00:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.64000000000000001,"15min":0.63800000000000001},"min":{"1min":0.44500000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.86699999999999999,"5min":1.028,"15min":1.028},"last":0.754},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.624,"15min":0.626},"min":{"1min":0.432,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.91000000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-02T00:31:42.435 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-12-02T00:31:42.436 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-02T00:31:42.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.541+0000 7f2bd558a640 1 Processor -- start 2023-12-02T00:31:42.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.542+0000 7f2bd558a640 1 -- start start 2023-12-02T00:31:42.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.542+0000 7f2bd558a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2bd0143490 0x7f2bd00b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:42.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.542+0000 7f2bd558a640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bd015e2a0 con 0x7f2bd015e730 2023-12-02T00:31:42.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.542+0000 7f2bd558a640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bd015b240 con 0x7f2bd015abf0 2023-12-02T00:31:42.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.542+0000 7f2bd558a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd015b520 con 0x7f2bd0143490 2023-12-02T00:31:42.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.543+0000 7f2bce7fc640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2bd015e730 0x7f2bd015a4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50924/0 (socket says 172.21.15.46:50924) 2023-12-02T00:31:42.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.543+0000 7f2bce7fc640 1 -- 172.21.15.46:0/2071997123 learned_addr learned my addr 172.21.15.46:0/2071997123 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:31:42.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.543+0000 7f2bceffd640 1 --2- 172.21.15.46:0/2071997123 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2bd0143490 0x7f2bd00b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:42.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.543+0000 7f2bcdffb640 1 -- 172.21.15.46:0/2071997123 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2562333428 0 0) 0x7f2bd015e2a0 con 0x7f2bd015e730 2023-12-02T00:31:42.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.543+0000 7f2bcdffb640 1 -- 172.21.15.46:0/2071997123 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bb8003520 con 0x7f2bd015e730 2023-12-02T00:31:42.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.543+0000 7f2bcdffb640 1 -- 172.21.15.46:0/2071997123 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2781220328 0 0) 0x7f2bb8003520 con 0x7f2bd015e730 2023-12-02T00:31:42.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.543+0000 7f2bcdffb640 1 -- 172.21.15.46:0/2071997123 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2bd0143490 msgr2=0x7f2bd00b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:42.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.543+0000 7f2bcdffb640 1 --2- 172.21.15.46:0/2071997123 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2bd0143490 0x7f2bd00b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:42.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.543+0000 7f2bcdffb640 1 -- 172.21.15.46:0/2071997123 >> v1:172.21.15.162:6789/0 conn(0x7f2bd015abf0 legacy=0x7f2bd00b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:42.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.543+0000 7f2bcdffb640 1 -- 172.21.15.46:0/2071997123 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd015dc50 con 0x7f2bd015e730 2023-12-02T00:31:42.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.543+0000 7f2bcdffb640 1 -- 172.21.15.46:0/2071997123 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7f2bc4001670 con 0x7f2bd015e730 2023-12-02T00:31:42.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.543+0000 7f2bcdffb640 1 -- 172.21.15.46:0/2071997123 >> v1:172.21.15.46:6789/0 conn(0x7f2bd015e730 legacy=0x7f2bd015a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:42.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.543+0000 7f2bcdffb640 1 --2- 172.21.15.46:0/2071997123 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2bd0143490 0x7f2bb80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:42.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.543+0000 7f2bcdffb640 1 --2- 172.21.15.46:0/2071997123 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2bb8003ee0 0x7f2bb80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:42.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.543+0000 7f2bcdffb640 1 --2- 172.21.15.46:0/2071997123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bb8004d00 0x7f2bb8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:42.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.543+0000 7f2bcdffb640 1 -- 172.21.15.46:0/2071997123 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd015b520 con 0x7f2bb8004d00 2023-12-02T00:31:42.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.543+0000 7f2bcdffb640 1 -- 172.21.15.46:0/2071997123 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb800b9e0 con 0x7f2bb8003ee0 2023-12-02T00:31:42.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.543+0000 7f2bcdffb640 1 -- 172.21.15.46:0/2071997123 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb800bba0 con 0x7f2bd0143490 2023-12-02T00:31:42.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.543+0000 7f2bcf7fe640 1 --2- 172.21.15.46:0/2071997123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bb8004d00 0x7f2bb8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:42.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.543+0000 7f2bceffd640 1 --2- 172.21.15.46:0/2071997123 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2bd0143490 0x7f2bb80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:42.556 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.543+0000 7f2bce7fc640 1 --2- 172.21.15.46:0/2071997123 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2bb8003ee0 0x7f2bb80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:42.556 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.544+0000 7f2bcf7fe640 1 -- 172.21.15.46:0/2071997123 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2bd0143490 msgr2=0x7f2bb80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:42.556 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.544+0000 7f2bcf7fe640 1 --2- 172.21.15.46:0/2071997123 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2bd0143490 0x7f2bb80039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:42.556 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.544+0000 7f2bcf7fe640 1 -- 172.21.15.46:0/2071997123 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2bb8003ee0 msgr2=0x7f2bb80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:42.556 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.544+0000 7f2bcf7fe640 1 --2- 172.21.15.46:0/2071997123 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2bb8003ee0 0x7f2bb80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:42.556 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.544+0000 7f2bcf7fe640 1 -- 172.21.15.46:0/2071997123 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb8003520 con 0x7f2bb8004d00 2023-12-02T00:31:42.556 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.544+0000 7f2bcf7fe640 1 --2- 172.21.15.46:0/2071997123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bb8004d00 0x7f2bb8005180 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8002010 tx=0x7f2bb80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb901d6496b64dd8 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:42.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.544+0000 7f2bcdffb640 1 -- 172.21.15.46:0/2071997123 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc802c750 con 0x7f2bb8004d00 2023-12-02T00:31:42.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.545+0000 7f2bcdffb640 1 -- 172.21.15.46:0/2071997123 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc802c8b0 con 0x7f2bb8004d00 2023-12-02T00:31:42.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.545+0000 7f2bcdffb640 1 -- 172.21.15.46:0/2071997123 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc802cc80 con 0x7f2bb8004d00 2023-12-02T00:31:42.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.545+0000 7f2bd558a640 1 -- 172.21.15.46:0/2071997123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bb8004d00 msgr2=0x7f2bb8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:42.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.545+0000 7f2bd558a640 1 --2- 172.21.15.46:0/2071997123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bb8004d00 0x7f2bb8005180 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8002010 tx=0x7f2bb80057f0 comp rx=0 tx=0).stop 2023-12-02T00:31:42.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.545+0000 7f2bcf7fe640 1 -- 172.21.15.46:0/2071997123 reap_dead start 2023-12-02T00:31:42.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.545+0000 7f2bd558a640 1 -- 172.21.15.46:0/2071997123 shutdown_connections 2023-12-02T00:31:42.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.545+0000 7f2bd558a640 1 -- 172.21.15.46:0/2071997123 >> 172.21.15.46:0/2071997123 conn(0x7f2bd006a4d0 msgr2=0x7f2bd006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:42.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.545+0000 7f2bd558a640 1 -- 172.21.15.46:0/2071997123 shutdown_connections 2023-12-02T00:31:42.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.545+0000 7f2bd558a640 1 -- 172.21.15.46:0/2071997123 wait complete. 2023-12-02T00:31:42.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.546+0000 7f2bd558a640 1 Processor -- start 2023-12-02T00:31:42.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.546+0000 7f2bd558a640 1 -- start start 2023-12-02T00:31:42.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.547+0000 7f2bd558a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2bd00c78c0 0x7f2bd00c6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:42.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.547+0000 7f2bd558a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2bd013cbf0 0x7f2bd00c4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:42.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.547+0000 7f2bd558a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bd00c7ca0 0x7f2bd00c53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:42.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.547+0000 7f2bd558a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd013d120 con 0x7f2bd00c7ca0 2023-12-02T00:31:42.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.547+0000 7f2bd558a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd013d400 con 0x7f2bd00c78c0 2023-12-02T00:31:42.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.547+0000 7f2bd558a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd013d6e0 con 0x7f2bd013cbf0 2023-12-02T00:31:42.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.547+0000 7f2bcf7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bd00c7ca0 0x7f2bd00c53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:42.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.547+0000 7f2bcf7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bd00c7ca0 0x7f2bd00c53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45244/0 (socket says 172.21.15.46:45244) 2023-12-02T00:31:42.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.547+0000 7f2bcf7fe640 1 -- 172.21.15.46:0/1177617508 learned_addr learned my addr 172.21.15.46:0/1177617508 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:42.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.548+0000 7f2bceffd640 1 --2- 172.21.15.46:0/1177617508 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2bd00c78c0 0x7f2bd00c6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:42.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.548+0000 7f2bce7fc640 1 --2- 172.21.15.46:0/1177617508 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2bd013cbf0 0x7f2bd00c4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:42.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.548+0000 7f2bcf7fe640 1 -- 172.21.15.46:0/1177617508 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2bd013cbf0 msgr2=0x7f2bd00c4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:42.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.548+0000 7f2bcf7fe640 1 --2- 172.21.15.46:0/1177617508 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2bd013cbf0 0x7f2bd00c4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:42.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.548+0000 7f2bcf7fe640 1 -- 172.21.15.46:0/1177617508 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2bd00c78c0 msgr2=0x7f2bd00c6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:42.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.548+0000 7f2bcf7fe640 1 --2- 172.21.15.46:0/1177617508 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2bd00c78c0 0x7f2bd00c6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:42.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.548+0000 7f2bcf7fe640 1 -- 172.21.15.46:0/1177617508 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd013d9c0 con 0x7f2bd00c7ca0 2023-12-02T00:31:42.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.548+0000 7f2bceffd640 1 --2- 172.21.15.46:0/1177617508 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2bd00c78c0 0x7f2bd00c6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:31:42.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.548+0000 7f2bcf7fe640 1 --2- 172.21.15.46:0/1177617508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bd00c7ca0 0x7f2bd00c53f0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8037040 tx=0x7f2bc8001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:42.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.548+0000 7f2bb77fe640 1 -- 172.21.15.46:0/1177617508 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8003b70 con 0x7f2bd00c7ca0 2023-12-02T00:31:42.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.548+0000 7f2bb77fe640 1 -- 172.21.15.46:0/1177617508 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc8003cd0 con 0x7f2bd00c7ca0 2023-12-02T00:31:42.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.548+0000 7f2bb77fe640 1 -- 172.21.15.46:0/1177617508 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8005290 con 0x7f2bd00c7ca0 2023-12-02T00:31:42.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.548+0000 7f2bd558a640 1 -- 172.21.15.46:0/1177617508 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bd013dca0 con 0x7f2bd00c7ca0 2023-12-02T00:31:42.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.548+0000 7f2bd558a640 1 -- 172.21.15.46:0/1177617508 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bd00c5930 con 0x7f2bd00c7ca0 2023-12-02T00:31:42.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.549+0000 7f2bd558a640 1 -- 172.21.15.46:0/1177617508 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bd0069400 con 0x7f2bd00c7ca0 2023-12-02T00:31:42.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.549+0000 7f2bb77fe640 1 -- 172.21.15.46:0/1177617508 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2bc803b040 con 0x7f2bd00c7ca0 2023-12-02T00:31:42.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.550+0000 7f2bb77fe640 1 --2- 172.21.15.46:0/1177617508 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2bbc06ae40 0x7f2bbc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:42.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.550+0000 7f2bb77fe640 1 -- 172.21.15.46:0/1177617508 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2bc80a9bf0 con 0x7f2bd00c7ca0 2023-12-02T00:31:42.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.550+0000 7f2bceffd640 1 --2- 172.21.15.46:0/1177617508 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2bbc06ae40 0x7f2bbc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:42.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.550+0000 7f2bceffd640 1 --2- 172.21.15.46:0/1177617508 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2bbc06ae40 0x7f2bbc06d300 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2bd00b47d0 tx=0x7f2bc0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:42.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.554+0000 7f2bb77fe640 1 -- 172.21.15.46:0/1177617508 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2bc8075050 con 0x7f2bd00c7ca0 2023-12-02T00:31:42.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.726+0000 7f2bd558a640 1 -- 172.21.15.46:0/1177617508 --> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2bd013d570 con 0x7f2bbc06ae40 2023-12-02T00:31:42.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.728+0000 7f2bb77fe640 1 -- 172.21.15.46:0/1177617508 <== mgr.4108 v2:172.21.15.162:6832/3248644722 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+51061 (secure 0 0 0) 0x7f2bd013d570 con 0x7f2bbc06ae40 2023-12-02T00:31:42.804 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:31:42.805 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-12-02T00:31:42.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.730+0000 7f2bd558a640 1 -- 172.21.15.46:0/1177617508 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2bbc06ae40 msgr2=0x7f2bbc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:42.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.730+0000 7f2bd558a640 1 --2- 172.21.15.46:0/1177617508 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2bbc06ae40 0x7f2bbc06d300 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2bd00b47d0 tx=0x7f2bc0008040 comp rx=0 tx=0).stop 2023-12-02T00:31:42.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.730+0000 7f2bd558a640 1 -- 172.21.15.46:0/1177617508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bd00c7ca0 msgr2=0x7f2bd00c53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:42.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.730+0000 7f2bd558a640 1 --2- 172.21.15.46:0/1177617508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bd00c7ca0 0x7f2bd00c53f0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8037040 tx=0x7f2bc8001630 comp rx=0 tx=0).stop 2023-12-02T00:31:42.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.730+0000 7f2bd558a640 1 -- 172.21.15.46:0/1177617508 shutdown_connections 2023-12-02T00:31:42.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.730+0000 7f2bd558a640 1 --2- 172.21.15.46:0/1177617508 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f2bbc06ae40 0x7f2bbc06d300 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:42.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.730+0000 7f2bd558a640 1 --2- 172.21.15.46:0/1177617508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bd00c7ca0 0x7f2bd00c53f0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:42.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.730+0000 7f2bd558a640 1 --2- 172.21.15.46:0/1177617508 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2bd013cbf0 0x7f2bd00c4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:42.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.730+0000 7f2bd558a640 1 --2- 172.21.15.46:0/1177617508 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2bd00c78c0 0x7f2bd00c6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:42.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.730+0000 7f2bd558a640 1 -- 172.21.15.46:0/1177617508 >> 172.21.15.46:0/1177617508 conn(0x7f2bd006a4d0 msgr2=0x7f2bd00c3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:42.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.730+0000 7f2bd558a640 1 -- 172.21.15.46:0/1177617508 shutdown_connections 2023-12-02T00:31:42.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.730+0000 7f2bd558a640 1 -- 172.21.15.46:0/1177617508 wait complete. 2023-12-02T00:31:42.808 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2023-12-02T00:31:41.200793+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":184,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":239168,"kb_used_data":6352,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735552,"statfs":{"total":773094113280,"available":772849205248,"internally_reserved":0,"allocated":6504448,"data_stored":3406335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14792,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-62,"num_write_kb":-36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001285"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":198,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-12-02T00:31:37.840833+0000","last_change":"2023-12-02T00:28:39.379380+0000","last_active":"2023-12-02T00:31:37.840833+0000","last_peered":"2023-12-02T00:31:37.840833+0000","last_clean":"2023-12-02T00:31:37.840833+0000","last_became_active":"2023-12-02T00:28:39.378255+0000","last_became_peered":"2023-12-02T00:28:39.378255+0000","last_unstale":"2023-12-02T00:31:37.840833+0000","last_undegraded":"2023-12-02T00:31:37.840833+0000","last_fullsized":"2023-12-02T00:31:37.840833+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:38.177345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:38.177345+0000","last_clean_scrub_stamp":"2023-12-02T00:28:38.177345+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:21:08.582412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574888,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28244,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343596,"statfs":{"total":96636764160,"available":96607842304,"internally_reserved":0,"allocated":1253376,"data_stored":866623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.50900000000000001,"15min":0.499},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.52600000000000002}]},{"osd":1,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53400000000000003,"15min":0.53000000000000003},"min":{"1min":0.40400000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.54700000000000004,"15min":0.54400000000000004},"min":{"1min":0.46100000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.58999999999999997}]},{"osd":2,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54200000000000004,"15min":0.54400000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.67000000000000004,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.624},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.54300000000000004,"15min":0.53400000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.56299999999999994}]},{"osd":3,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.53000000000000003,"15min":0.52200000000000002},"min":{"1min":0.48099999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.57299999999999995,"15min":0.57499999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.74399999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.50800000000000001}]},{"osd":4,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.50700000000000001,"15min":0.51100000000000001},"min":{"1min":0.29799999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.72199999999999998,"5min":0.745,"15min":0.745},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.51500000000000001,"15min":0.52100000000000002},"min":{"1min":0.33800000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.61799999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.378}]},{"osd":5,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54700000000000004,"15min":0.55000000000000004},"min":{"1min":0.41899999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.78700000000000003,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.56100000000000005,"15min":0.56599999999999995},"min":{"1min":0.39500000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.63900000000000001,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.48199999999999998}]},{"osd":6,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.61799999999999999,"15min":0.627},"min":{"1min":0.40100000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.82899999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.44},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.60599999999999998,"15min":0.61299999999999999},"min":{"1min":0.44500000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76600000000000001,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.42999999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27684,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":675840,"data_stored":281854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.49199999999999999,"15min":0.48299999999999998},"min":{"1min":0.36499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.439},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.50700000000000001,"15min":0.501},"min":{"1min":0.40000000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.74099999999999999,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.60399999999999998}]},{"osd":1,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.51000000000000001,"15min":0.50600000000000001},"min":{"1min":0.33000000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.72599999999999998,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.53600000000000003,"15min":0.53300000000000003},"min":{"1min":0.39400000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.80900000000000005,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.38200000000000001}]},{"osd":2,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.52000000000000002,"15min":0.51200000000000001},"min":{"1min":0.38,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55500000000000005,"15min":0.55700000000000005},"min":{"1min":0.40300000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.77400000000000002,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.42099999999999999}]},{"osd":3,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57599999999999996,"15min":0.57299999999999995},"min":{"1min":0.45200000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.76300000000000001,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57199999999999995,"15min":0.56999999999999995},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.63500000000000001}]},{"osd":4,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.35399999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.71299999999999997,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52500000000000002,"15min":0.52300000000000002},"min":{"1min":0.34100000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73699999999999999,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.66900000000000004}]},{"osd":5,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.33400000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.502},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58799999999999997,"15min":0.59299999999999997},"min":{"1min":0.42399999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.61699999999999999}]},{"osd":7,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.35999999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.58899999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574889,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":659456,"data_stored":276255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.51700000000000002,"15min":0.50600000000000001},"min":{"1min":0.29099999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.73899999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.51600000000000001,"15min":0.51100000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.72899999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.871}]},{"osd":1,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.51400000000000001,"15min":0.505},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.754,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55300000000000005,"15min":0.55000000000000004},"min":{"1min":0.36199999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.69499999999999995,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.745}]},{"osd":2,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.56000000000000005,"15min":0.55100000000000005},"min":{"1min":0.41699999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.77500000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.53600000000000003,"15min":0.53300000000000003},"min":{"1min":0.34399999999999997,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.749,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.71299999999999997}]},{"osd":3,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58099999999999996,"15min":0.57699999999999996},"min":{"1min":0.39300000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.58299999999999996,"15min":0.58199999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.72499999999999998,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.82599999999999996}]},{"osd":4,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54300000000000004,"15min":0.54600000000000004},"min":{"1min":0.34799999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69899999999999995,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58699999999999997,"15min":0.58799999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.52800000000000002}]},{"osd":6,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.56599999999999995,"15min":0.57099999999999995},"min":{"1min":0.36899999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.72899999999999998,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.59199999999999997,"15min":0.59599999999999997},"min":{"1min":0.435,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.50600000000000001}]},{"osd":7,"last update":"Sat Dec 2 00:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.42699999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.82099999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.621},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.63100000000000001,"15min":0.63500000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.76700000000000002,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.55700000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574888,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27664,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":659456,"data_stored":276255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55300000000000005,"15min":0.54500000000000004},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.59399999999999997,"15min":0.58699999999999997},"min":{"1min":0.48699999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.83599999999999997,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.58299999999999996}]},{"osd":1,"last update":"Sat Dec 2 00:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.57999999999999996,"15min":0.56100000000000005},"min":{"1min":0.38400000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56399999999999995,"15min":0.55600000000000005},"min":{"1min":0.41099999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.73999999999999999,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.68100000000000005}]},{"osd":2,"last update":"Sat Dec 2 00:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.58399999999999996,"15min":0.57799999999999996},"min":{"1min":0.45500000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60799999999999998,"15min":0.60199999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.752,"5min":0.879,"15min":0.879},"last":0.64100000000000001}]},{"osd":3,"last update":"Sat Dec 2 00:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57399999999999995,"15min":0.56399999999999995},"min":{"1min":0.42299999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78100000000000003,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.622},{"interface":"front","average":{"1min":0.621,"5min":0.60899999999999999,"15min":0.60499999999999998},"min":{"1min":0.46800000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.80300000000000005,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.503}]},{"osd":5,"last update":"Sat Dec 2 00:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52700000000000002,"15min":0.52300000000000002},"min":{"1min":0.34899999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.79700000000000004,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.53200000000000003,"15min":0.53800000000000003},"min":{"1min":0.34399999999999997,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71399999999999997,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.72499999999999998}]},{"osd":6,"last update":"Sat Dec 2 00:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.53500000000000003,"15min":0.53400000000000003},"min":{"1min":0.32800000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.70399999999999996,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.57299999999999995,"15min":0.56899999999999995},"min":{"1min":0.41099999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72799999999999998,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.70199999999999996}]},{"osd":7,"last update":"Sat Dec 2 00:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55800000000000005,"15min":0.55500000000000005},"min":{"1min":0.40100000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.74399999999999999,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59199999999999997,"15min":0.58999999999999997},"min":{"1min":0.42599999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.70199999999999996,"5min":1.004,"15min":1.004},"last":0.41599999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574887,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31828,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340012,"statfs":{"total":96636764160,"available":96604172288,"internally_reserved":0,"allocated":659456,"data_stored":276255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.35699999999999998,"15min":0.35999999999999999},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.50800000000000001,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.38200000000000001,"15min":0.38100000000000001},"min":{"1min":0.27500000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.624,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.436}]},{"osd":1,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.379,"15min":0.38300000000000001},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.59999999999999998,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.25800000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.58699999999999997,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.46300000000000002}]},{"osd":2,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.40600000000000003,"15min":0.40999999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.52900000000000003,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.42199999999999999,"15min":0.42099999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.66500000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.58199999999999996}]},{"osd":4,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.60899999999999999,"15min":0.61099999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76600000000000001,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.622,"15min":0.623},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.68999999999999995}]},{"osd":5,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.58299999999999996,"15min":0.58699999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85599999999999998,"5min":1.004,"15min":1.004},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.61299999999999999,"15min":0.61499999999999999},"min":{"1min":0.39300000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.90600000000000003,"5min":1.163,"15min":1.163},"last":0.66800000000000004}]},{"osd":6,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63800000000000001,"15min":0.63900000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.83199999999999996,"5min":1.095,"15min":1.095},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.63200000000000001,"15min":0.63500000000000001},"min":{"1min":0.45100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.96399999999999997,"5min":1.044,"15min":1.044},"last":0.75600000000000001}]},{"osd":7,"last update":"Sat Dec 2 00:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.64700000000000002,"15min":0.65100000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.93899999999999995,"5min":1.177,"15min":1.177},"last":0.622},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.64500000000000002,"15min":0.65100000000000002},"min":{"1min":0.45200000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.84799999999999998,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.73099999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574888,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31840,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340000,"statfs":{"total":96636764160,"available":96604160000,"internally_reserved":0,"allocated":671744,"data_stored":281235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.377,"15min":0.36299999999999999},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":2.649,"5min":2.649,"15min":2.649},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.42299999999999999,"15min":0.40799999999999997},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":3.173,"5min":3.173,"15min":3.173},"last":0.254}]},{"osd":1,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.44800000000000001,"15min":0.41999999999999998},"min":{"1min":0.26300000000000001,"5min":0.246,"15min":0.246},"max":{"1min":4.6639999999999997,"5min":4.6639999999999997,"15min":4.6639999999999997},"last":0.375},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.433,"15min":0.39100000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":6.093,"5min":6.093,"15min":6.093},"last":0.31}]},{"osd":3,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.40300000000000002,"15min":0.376},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":4.1879999999999997,"5min":4.1879999999999997,"15min":4.1879999999999997},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.44600000000000001,"15min":0.433},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":2.4159999999999999,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":0.28899999999999998}]},{"osd":4,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.60999999999999999,"15min":0.59599999999999997},"min":{"1min":0.40999999999999998,"5min":0.38,"15min":0.38},"max":{"1min":2.6840000000000002,"5min":2.6840000000000002,"15min":2.6840000000000002},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.65800000000000003,"15min":0.64600000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":3.0419999999999998,"5min":3.0419999999999998,"15min":3.0419999999999998},"last":0.57799999999999996}]},{"osd":5,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.66100000000000003,"15min":0.65300000000000002},"min":{"1min":0.41599999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":3.093,"5min":3.093,"15min":3.093},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.64700000000000002,"15min":0.65100000000000002},"min":{"1min":0.41799999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.98499999999999999,"5min":1.111,"15min":1.111},"last":0.52800000000000002}]},{"osd":6,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.64000000000000001,"15min":0.63800000000000001},"min":{"1min":0.42499999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.67800000000000005,"15min":0.67200000000000004},"min":{"1min":0.47799999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":2.6030000000000002,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":0.55300000000000005}]},{"osd":7,"last update":"Sat Dec 2 00:30:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.65800000000000003,"15min":0.64800000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":3.1339999999999999,"5min":3.1339999999999999,"15min":3.1339999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.64200000000000002,"15min":0.64600000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.94199999999999995,"5min":1.2,"15min":1.2},"last":0.61199999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574887,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31824,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340016,"statfs":{"total":96636764160,"available":96604176384,"internally_reserved":0,"allocated":659456,"data_stored":276255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.33900000000000002,"15min":0.34100000000000003},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.436,"5min":0.48099999999999998,"15min":0.48099999999999998},"last":0.23899999999999999},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.39600000000000002,"15min":0.39800000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.58799999999999997,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.32800000000000001}]},{"osd":2,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.35099999999999998,"15min":0.34899999999999998},"min":{"1min":0.249,"5min":0.246,"15min":0.246},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.312},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.36299999999999999,"15min":0.36699999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.53500000000000003,"5min":0.626,"15min":0.626},"last":0.26400000000000001}]},{"osd":3,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.40100000000000002,"15min":0.40200000000000002},"min":{"1min":0.30499999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.371,"15min":0.376},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.5,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.36699999999999999}]},{"osd":4,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.55400000000000005,"15min":0.54800000000000004},"min":{"1min":0.377,"5min":0.376,"15min":0.376},"max":{"1min":0.77100000000000002,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.55800000000000005,"15min":0.55100000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.78000000000000003,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.55300000000000005}]},{"osd":5,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.58599999999999997,"15min":0.57899999999999996},"min":{"1min":0.45900000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.96799999999999997,"5min":1.107,"15min":1.107},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57299999999999995,"15min":0.56599999999999995},"min":{"1min":0.437,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.56999999999999995}]},{"osd":6,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60399999999999998,"15min":0.60199999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.76600000000000001,"5min":1.044,"15min":1.044},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.61199999999999999,"15min":0.60599999999999998},"min":{"1min":0.51000000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.85399999999999998,"5min":0.998,"15min":0.998},"last":0.60799999999999998}]},{"osd":7,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58999999999999997,"15min":0.58799999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.78900000000000003,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.627,"5min":0.60599999999999998,"15min":0.60199999999999998},"min":{"1min":0.45300000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.80000000000000004,"5min":1.151,"15min":1.151},"last":0.48799999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574889,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32420,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339420,"statfs":{"total":96636764160,"available":96603566080,"internally_reserved":0,"allocated":1265664,"data_stored":871603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Dec 2 00:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.57099999999999995,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.371,"15min":0.36499999999999999},"min":{"1min":0.26700000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.54000000000000004,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.45700000000000002}]},{"osd":2,"last update":"Sat Dec 2 00:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39300000000000002,"15min":0.39200000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.58499999999999996,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.436},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.40200000000000002,"15min":0.40100000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.56399999999999995,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.50600000000000001}]},{"osd":3,"last update":"Sat Dec 2 00:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.39100000000000001,"15min":0.39300000000000002},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.52700000000000002,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.42599999999999999,"15min":0.42699999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63900000000000001,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.47999999999999998}]},{"osd":4,"last update":"Sat Dec 2 00:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.40200000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.61499999999999999,"15min":0.61799999999999999},"min":{"1min":0.45300000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.78900000000000003,"5min":1.002,"15min":1.002},"last":0.78900000000000003}]},{"osd":5,"last update":"Sat Dec 2 00:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.629,"15min":0.627},"min":{"1min":0.49099999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":1.028},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64700000000000002,"15min":0.64600000000000002},"min":{"1min":0.47199999999999998,"5min":0.433,"15min":0.433},"max":{"1min":1.0129999999999999,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":1.0129999999999999}]},{"osd":6,"last update":"Sat Dec 2 00:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60999999999999999,"15min":0.60099999999999998},"min":{"1min":0.435,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88500000000000001,"5min":1.05,"15min":1.05},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.63300000000000001,"15min":0.63800000000000001},"min":{"1min":0.435,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.97399999999999998}]},{"osd":7,"last update":"Sat Dec 2 00:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.64000000000000001,"15min":0.63800000000000001},"min":{"1min":0.44500000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.86699999999999999,"5min":1.028,"15min":1.028},"last":0.754},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.624,"15min":0.626},"min":{"1min":0.432,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.91000000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-02T00:31:42.868 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-02T00:31:42.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.974+0000 7fe21896e640 1 Processor -- start 2023-12-02T00:31:42.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.974+0000 7fe21896e640 1 -- start start 2023-12-02T00:31:42.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.975+0000 7fe21896e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe21415d1d0 0x7fe21406b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:42.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.975+0000 7fe21896e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe2140ad640 0x7fe21406ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:42.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.975+0000 7fe21896e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2140adae0 con 0x7fe2140aeb80 2023-12-02T00:31:42.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.975+0000 7fe21896e640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2140addc0 con 0x7fe2140ad640 2023-12-02T00:31:42.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.975+0000 7fe21896e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2140ae0a0 con 0x7fe21415d1d0 2023-12-02T00:31:42.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.975+0000 7fe211d9b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe2140ad640 0x7fe21406ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:42.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.975+0000 7fe211d9b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe2140ad640 0x7fe21406ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48792/0 (socket says 172.21.15.46:48792) 2023-12-02T00:31:42.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.975+0000 7fe211d9b640 1 -- 172.21.15.46:0/1248206486 learned_addr learned my addr 172.21.15.46:0/1248206486 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:42.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.976+0000 7fe21259c640 1 --2- 172.21.15.46:0/1248206486 >> v2:172.21.15.162:3300/0 conn(0x7fe21415d1d0 0x7fe21406b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:42.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.976+0000 7fe211d9b640 1 -- 172.21.15.46:0/1248206486 >> v2:172.21.15.162:3300/0 conn(0x7fe21415d1d0 msgr2=0x7fe21406b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:42.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.976+0000 7fe211d9b640 1 --2- 172.21.15.46:0/1248206486 >> v2:172.21.15.162:3300/0 conn(0x7fe21415d1d0 0x7fe21406b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:42.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.976+0000 7fe211d9b640 1 -- 172.21.15.46:0/1248206486 >> v1:172.21.15.162:6789/0 conn(0x7fe2140aeb80 legacy=0x7fe214160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:42.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.976+0000 7fe211d9b640 1 -- 172.21.15.46:0/1248206486 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2140ae380 con 0x7fe2140ad640 2023-12-02T00:31:42.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.976+0000 7fe21259c640 1 --2- 172.21.15.46:0/1248206486 >> v2:172.21.15.162:3300/0 conn(0x7fe21415d1d0 0x7fe21406b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:31:42.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.976+0000 7fe211d9b640 1 --2- 172.21.15.46:0/1248206486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2140ad640 0x7fe21406ba20 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fe214072310 tx=0x7fe1fc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e8de371306796c6 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:42.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.977+0000 7fe21159a640 1 -- 172.21.15.46:0/1248206486 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc031870 con 0x7fe2140ad640 2023-12-02T00:31:42.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.977+0000 7fe21159a640 1 -- 172.21.15.46:0/1248206486 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1fc0319d0 con 0x7fe2140ad640 2023-12-02T00:31:42.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.977+0000 7fe21159a640 1 -- 172.21.15.46:0/1248206486 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc031d60 con 0x7fe2140ad640 2023-12-02T00:31:42.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.977+0000 7fe21896e640 1 -- 172.21.15.46:0/1248206486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2140ad640 msgr2=0x7fe21406ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:42.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.977+0000 7fe21896e640 1 --2- 172.21.15.46:0/1248206486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2140ad640 0x7fe21406ba20 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fe214072310 tx=0x7fe1fc02b910 comp rx=0 tx=0).stop 2023-12-02T00:31:42.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.977+0000 7fe21896e640 1 -- 172.21.15.46:0/1248206486 shutdown_connections 2023-12-02T00:31:42.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.977+0000 7fe21896e640 1 --2- 172.21.15.46:0/1248206486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2140ad640 0x7fe21406ba20 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:42.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.977+0000 7fe21896e640 1 --2- 172.21.15.46:0/1248206486 >> v2:172.21.15.162:3300/0 conn(0x7fe21415d1d0 0x7fe21406b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:42.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.977+0000 7fe21896e640 1 -- 172.21.15.46:0/1248206486 >> 172.21.15.46:0/1248206486 conn(0x7fe214070e70 msgr2=0x7fe214071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:42.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.977+0000 7fe21896e640 1 -- 172.21.15.46:0/1248206486 shutdown_connections 2023-12-02T00:31:42.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.977+0000 7fe21896e640 1 -- 172.21.15.46:0/1248206486 wait complete. 2023-12-02T00:31:42.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.979+0000 7fe21896e640 1 Processor -- start 2023-12-02T00:31:42.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.979+0000 7fe21896e640 1 -- start start 2023-12-02T00:31:42.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.979+0000 7fe21896e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2140ad640 0x7fe2141f5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:42.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.979+0000 7fe21896e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2140aeb80 0x7fe2140ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:42.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.979+0000 7fe21896e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe21415d1d0 0x7fe2141f3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:42.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.979+0000 7fe21896e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2140cd730 con 0x7fe2140ad640 2023-12-02T00:31:42.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.979+0000 7fe21896e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2141f5d40 con 0x7fe21415d1d0 2023-12-02T00:31:42.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.979+0000 7fe21896e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2141573e0 con 0x7fe2140aeb80 2023-12-02T00:31:42.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.979+0000 7fe21259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2140ad640 0x7fe2141f5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:42.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.979+0000 7fe21259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2140ad640 0x7fe2141f5800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48800/0 (socket says 172.21.15.46:48800) 2023-12-02T00:31:42.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.979+0000 7fe21259c640 1 -- 172.21.15.46:0/3716699296 learned_addr learned my addr 172.21.15.46:0/3716699296 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:42.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.980+0000 7fe211d9b640 1 --2- 172.21.15.46:0/3716699296 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2140aeb80 0x7fe2140ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:42.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.980+0000 7fe212d9d640 1 --2- 172.21.15.46:0/3716699296 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe21415d1d0 0x7fe2141f3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:42.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.980+0000 7fe21259c640 1 -- 172.21.15.46:0/3716699296 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2140aeb80 msgr2=0x7fe2140ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:42.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.980+0000 7fe21259c640 1 --2- 172.21.15.46:0/3716699296 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2140aeb80 0x7fe2140ccf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:42.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.980+0000 7fe21259c640 1 -- 172.21.15.46:0/3716699296 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe21415d1d0 msgr2=0x7fe2141f3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:42.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.980+0000 7fe21259c640 1 --2- 172.21.15.46:0/3716699296 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe21415d1d0 0x7fe2141f3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:42.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.980+0000 7fe21259c640 1 -- 172.21.15.46:0/3716699296 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1fc0094d0 con 0x7fe2140ad640 2023-12-02T00:31:42.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.980+0000 7fe211d9b640 1 --2- 172.21.15.46:0/3716699296 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2140aeb80 0x7fe2140ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:31:42.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.980+0000 7fe21259c640 1 --2- 172.21.15.46:0/3716699296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2140ad640 0x7fe2141f5800 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fe20c00c820 tx=0x7fe20c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:42.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.980+0000 7fe20affd640 1 -- 172.21.15.46:0/3716699296 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20c007a30 con 0x7fe2140ad640 2023-12-02T00:31:42.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.980+0000 7fe20affd640 1 -- 172.21.15.46:0/3716699296 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe20c007b90 con 0x7fe2140ad640 2023-12-02T00:31:42.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.980+0000 7fe21896e640 1 -- 172.21.15.46:0/3716699296 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2141576c0 con 0x7fe2140ad640 2023-12-02T00:31:42.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.980+0000 7fe20affd640 1 -- 172.21.15.46:0/3716699296 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20c00ce10 con 0x7fe2140ad640 2023-12-02T00:31:42.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.980+0000 7fe21896e640 1 -- 172.21.15.46:0/3716699296 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe214157c80 con 0x7fe2140ad640 2023-12-02T00:31:42.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.981+0000 7fe21896e640 1 -- 172.21.15.46:0/3716699296 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe214069400 con 0x7fe2140ad640 2023-12-02T00:31:42.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.981+0000 7fe20affd640 1 -- 172.21.15.46:0/3716699296 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe20c0113f0 con 0x7fe2140ad640 2023-12-02T00:31:42.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.982+0000 7fe20affd640 1 --2- 172.21.15.46:0/3716699296 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe1f406ae40 0x7fe1f406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:42.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.982+0000 7fe20affd640 1 -- 172.21.15.46:0/3716699296 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe20c089350 con 0x7fe2140ad640 2023-12-02T00:31:42.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.982+0000 7fe211d9b640 1 --2- 172.21.15.46:0/3716699296 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe1f406ae40 0x7fe1f406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:42.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.983+0000 7fe211d9b640 1 --2- 172.21.15.46:0/3716699296 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe1f406ae40 0x7fe1f406d300 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc036000 tx=0x7fe1fc03c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:42.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:42.986+0000 7fe20affd640 1 -- 172.21.15.46:0/3716699296 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe20c054790 con 0x7fe2140ad640 2023-12-02T00:31:43.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:43.165+0000 7fe21896e640 1 -- 172.21.15.46:0/3716699296 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe2140713f0 con 0x7fe2140ad640 2023-12-02T00:31:43.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:43.166+0000 7fe20affd640 1 -- 172.21.15.46:0/3716699296 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+12960 (secure 0 0 0) 0x7fe20c057db0 con 0x7fe2140ad640 2023-12-02T00:31:43.167 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:31:43.167 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":56,"fsid":"6d588070-5443-40f5-81f6-108a1e3f56c3","created":"2023-12-02T00:28:24.929551+0000","modified":"2023-12-02T00:31:39.977360+0000","last_up_change":"2023-12-02T00:28:35.681820+0000","last_in_change":"2023-12-02T00:28:27.213351+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-02T00:28:37.213645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f48f92eb-acb2-46f4-8721-3cb619b01bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6809","nonce":2057169608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6811","nonce":2057169608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6815","nonce":2057169608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2057169608},{"type":"v1","addr":"172.21.15.46:6813","nonce":2057169608}]},"public_addr":"172.21.15.46:6809/2057169608","cluster_addr":"172.21.15.46:6811/2057169608","heartbeat_back_addr":"172.21.15.46:6815/2057169608","heartbeat_front_addr":"172.21.15.46:6813/2057169608","state":["exists","up"]},{"osd":1,"uuid":"52634887-f882-4f8d-b25b-b78368c750ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6801","nonce":10535129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6803","nonce":10535129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6807","nonce":10535129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":10535129},{"type":"v1","addr":"172.21.15.46:6805","nonce":10535129}]},"public_addr":"172.21.15.46:6801/10535129","cluster_addr":"172.21.15.46:6803/10535129","heartbeat_back_addr":"172.21.15.46:6807/10535129","heartbeat_front_addr":"172.21.15.46:6805/10535129","state":["exists","up"]},{"osd":2,"uuid":"f8d40b7c-3802-453a-ad40-9322d0e1a3f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6825","nonce":878860153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6827","nonce":878860153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6831","nonce":878860153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":878860153},{"type":"v1","addr":"172.21.15.46:6829","nonce":878860153}]},"public_addr":"172.21.15.46:6825/878860153","cluster_addr":"172.21.15.46:6827/878860153","heartbeat_back_addr":"172.21.15.46:6831/878860153","heartbeat_front_addr":"172.21.15.46:6829/878860153","state":["exists","up"]},{"osd":3,"uuid":"c99837cf-79d2-4b7e-9829-62697b43ff06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6817","nonce":1799228239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6819","nonce":1799228239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6823","nonce":1799228239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1799228239},{"type":"v1","addr":"172.21.15.46:6821","nonce":1799228239}]},"public_addr":"172.21.15.46:6817/1799228239","cluster_addr":"172.21.15.46:6819/1799228239","heartbeat_back_addr":"172.21.15.46:6823/1799228239","heartbeat_front_addr":"172.21.15.46:6821/1799228239","state":["exists","up"]},{"osd":4,"uuid":"b745b29b-2d14-4f6f-adc7-54ce56d0d8ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6809","nonce":4132149803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6811","nonce":4132149803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6815","nonce":4132149803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":4132149803},{"type":"v1","addr":"172.21.15.162:6813","nonce":4132149803}]},"public_addr":"172.21.15.162:6809/4132149803","cluster_addr":"172.21.15.162:6811/4132149803","heartbeat_back_addr":"172.21.15.162:6815/4132149803","heartbeat_front_addr":"172.21.15.162:6813/4132149803","state":["exists","up"]},{"osd":5,"uuid":"b90c10b2-c5ff-4fe7-964b-419d1bb28bee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6817","nonce":3463332362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6819","nonce":3463332362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6823","nonce":3463332362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3463332362},{"type":"v1","addr":"172.21.15.162:6821","nonce":3463332362}]},"public_addr":"172.21.15.162:6817/3463332362","cluster_addr":"172.21.15.162:6819/3463332362","heartbeat_back_addr":"172.21.15.162:6823/3463332362","heartbeat_front_addr":"172.21.15.162:6821/3463332362","state":["exists","up"]},{"osd":6,"uuid":"2294203e-5c5a-4bec-9dcd-c5a576ff3cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6801","nonce":3225626420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6803","nonce":3225626420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6807","nonce":3225626420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":3225626420},{"type":"v1","addr":"172.21.15.162:6805","nonce":3225626420}]},"public_addr":"172.21.15.162:6801/3225626420","cluster_addr":"172.21.15.162:6803/3225626420","heartbeat_back_addr":"172.21.15.162:6807/3225626420","heartbeat_front_addr":"172.21.15.162:6805/3225626420","state":["exists","up"]},{"osd":7,"uuid":"ca4e6592-a26f-4360-981b-3bcad8251164","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6825","nonce":4107689713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6827","nonce":4107689713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6831","nonce":4107689713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":4107689713},{"type":"v1","addr":"172.21.15.162:6829","nonce":4107689713}]},"public_addr":"172.21.15.162:6825/4107689713","cluster_addr":"172.21.15.162:6827/4107689713","heartbeat_back_addr":"172.21.15.162:6831/4107689713","heartbeat_front_addr":"172.21.15.162:6829/4107689713","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.341118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.366466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.373906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.403702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.384310+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.334912+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.383650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-02T00:28:32.415380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/903694874":"2023-12-03T00:31:36.638370+0000","172.21.15.46:6834/903694874":"2023-12-03T00:31:36.638370+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-02T00:31:43.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:43.167+0000 7fe21896e640 1 -- 172.21.15.46:0/3716699296 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe1f406ae40 msgr2=0x7fe1f406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:43.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:43.167+0000 7fe21896e640 1 --2- 172.21.15.46:0/3716699296 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe1f406ae40 0x7fe1f406d300 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc036000 tx=0x7fe1fc03c040 comp rx=0 tx=0).stop 2023-12-02T00:31:43.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:43.167+0000 7fe21896e640 1 -- 172.21.15.46:0/3716699296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2140ad640 msgr2=0x7fe2141f5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:43.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:43.167+0000 7fe21896e640 1 --2- 172.21.15.46:0/3716699296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2140ad640 0x7fe2141f5800 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fe20c00c820 tx=0x7fe20c00cce0 comp rx=0 tx=0).stop 2023-12-02T00:31:43.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:43.167+0000 7fe21896e640 1 -- 172.21.15.46:0/3716699296 shutdown_connections 2023-12-02T00:31:43.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:43.167+0000 7fe21896e640 1 --2- 172.21.15.46:0/3716699296 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fe1f406ae40 0x7fe1f406d300 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:43.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:43.167+0000 7fe21896e640 1 --2- 172.21.15.46:0/3716699296 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe21415d1d0 0x7fe2141f3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:43.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:43.167+0000 7fe21896e640 1 --2- 172.21.15.46:0/3716699296 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2140aeb80 0x7fe2140ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:43.170 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:43.167+0000 7fe21896e640 1 --2- 172.21.15.46:0/3716699296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2140ad640 0x7fe2141f5800 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:43.170 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:43.167+0000 7fe21896e640 1 -- 172.21.15.46:0/3716699296 >> 172.21.15.46:0/3716699296 conn(0x7fe214070e70 msgr2=0x7fe21406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:43.170 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:43.168+0000 7fe21896e640 1 -- 172.21.15.46:0/3716699296 shutdown_connections 2023-12-02T00:31:43.170 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:43.168+0000 7fe21896e640 1 -- 172.21.15.46:0/3716699296 wait complete. 2023-12-02T00:31:44.180 INFO:tasks.ceph:Scrubbing osd.0 2023-12-02T00:31:44.180 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-12-02T00:31:44.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.302+0000 7f502684f640 1 Processor -- start 2023-12-02T00:31:44.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.302+0000 7f502684f640 1 -- start start 2023-12-02T00:31:44.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.303+0000 7f502684f640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f502015e620 0x7f502006eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:44.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.303+0000 7f502684f640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f502006f050 0x7f502013f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:44.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.303+0000 7f502684f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f502013fda0 0x7f502015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:44.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.303+0000 7f502684f640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f502006f4c0 con 0x7f502006f050 2023-12-02T00:31:44.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.303+0000 7f502684f640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f502015ea00 con 0x7f502015e620 2023-12-02T00:31:44.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.303+0000 7f502684f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f502013e7e0 con 0x7f502013fda0 2023-12-02T00:31:44.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.303+0000 7f501f7fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f502006f050 0x7f502013f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:44.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.303+0000 7f501f7fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f502006f050 0x7f502013f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48802/0 (socket says 172.21.15.46:48802) 2023-12-02T00:31:44.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.303+0000 7f501f7fe640 1 -- 172.21.15.46:0/3425934900 learned_addr learned my addr 172.21.15.46:0/3425934900 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:44.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.303+0000 7f5024dec640 1 --2- 172.21.15.46:0/3425934900 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f502013fda0 0x7f502015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:44.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.303+0000 7f501ffff640 1 --2- 172.21.15.46:0/3425934900 >> v2:172.21.15.162:3300/0 conn(0x7f502015e620 0x7f502006eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:44.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.304+0000 7f501f7fe640 1 -- 172.21.15.46:0/3425934900 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f502013fda0 msgr2=0x7f502015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:44.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.304+0000 7f501f7fe640 1 --2- 172.21.15.46:0/3425934900 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f502013fda0 0x7f502015bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:44.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.304+0000 7f501f7fe640 1 -- 172.21.15.46:0/3425934900 >> v2:172.21.15.162:3300/0 conn(0x7f502015e620 msgr2=0x7f502006eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:44.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.304+0000 7f501f7fe640 1 --2- 172.21.15.46:0/3425934900 >> v2:172.21.15.162:3300/0 conn(0x7f502015e620 0x7f502006eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:44.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.304+0000 7f501f7fe640 1 -- 172.21.15.46:0/3425934900 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f502013eaa0 con 0x7f502006f050 2023-12-02T00:31:44.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.304+0000 7f501ffff640 1 --2- 172.21.15.46:0/3425934900 >> v2:172.21.15.162:3300/0 conn(0x7f502015e620 0x7f502006eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:31:44.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.304+0000 7f5024dec640 1 --2- 172.21.15.46:0/3425934900 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f502013fda0 0x7f502015bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:31:44.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.304+0000 7f501f7fe640 1 --2- 172.21.15.46:0/3425934900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f502006f050 0x7f502013f860 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f502006f750 tx=0x7f501002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91269ab636563b0a server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:44.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.304+0000 7f501effd640 1 -- 172.21.15.46:0/3425934900 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5010031640 con 0x7f502006f050 2023-12-02T00:31:44.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.304+0000 7f501effd640 1 -- 172.21.15.46:0/3425934900 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50100317a0 con 0x7f502006f050 2023-12-02T00:31:44.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.304+0000 7f501effd640 1 -- 172.21.15.46:0/3425934900 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5010031b30 con 0x7f502006f050 2023-12-02T00:31:44.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.305+0000 7f502684f640 1 -- 172.21.15.46:0/3425934900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f502006f050 msgr2=0x7f502013f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:44.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.305+0000 7f502684f640 1 --2- 172.21.15.46:0/3425934900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f502006f050 0x7f502013f860 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f502006f750 tx=0x7f501002b8f0 comp rx=0 tx=0).stop 2023-12-02T00:31:44.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.305+0000 7f502684f640 1 -- 172.21.15.46:0/3425934900 shutdown_connections 2023-12-02T00:31:44.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.305+0000 7f502684f640 1 --2- 172.21.15.46:0/3425934900 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f502013fda0 0x7f502015bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:44.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.305+0000 7f502684f640 1 --2- 172.21.15.46:0/3425934900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f502006f050 0x7f502013f860 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:44.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.305+0000 7f502684f640 1 --2- 172.21.15.46:0/3425934900 >> v2:172.21.15.162:3300/0 conn(0x7f502015e620 0x7f502006eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:44.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.305+0000 7f502684f640 1 -- 172.21.15.46:0/3425934900 >> 172.21.15.46:0/3425934900 conn(0x7f502006a6b0 msgr2=0x7f502006aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:44.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.305+0000 7f502684f640 1 -- 172.21.15.46:0/3425934900 shutdown_connections 2023-12-02T00:31:44.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.305+0000 7f502684f640 1 -- 172.21.15.46:0/3425934900 wait complete. 2023-12-02T00:31:44.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.306+0000 7f502684f640 1 Processor -- start 2023-12-02T00:31:44.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.307+0000 7f502684f640 1 -- start start 2023-12-02T00:31:44.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.307+0000 7f502684f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f502006f050 0x7f50201549f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:44.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.307+0000 7f502684f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f502013fda0 0x7f502013cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:44.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.307+0000 7f502684f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f502015e620 0x7f50200b4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:44.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.307+0000 7f502684f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5020153fa0 con 0x7f502015e620 2023-12-02T00:31:44.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.307+0000 7f502684f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50200b4af0 con 0x7f502013fda0 2023-12-02T00:31:44.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.307+0000 7f502684f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50200b4dd0 con 0x7f502006f050 2023-12-02T00:31:44.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.307+0000 7f5024dec640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f502015e620 0x7f50200b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:44.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.307+0000 7f5024dec640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f502015e620 0x7f50200b4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48816/0 (socket says 172.21.15.46:48816) 2023-12-02T00:31:44.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.307+0000 7f5024dec640 1 -- 172.21.15.46:0/3579625416 learned_addr learned my addr 172.21.15.46:0/3579625416 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:44.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.308+0000 7f501ffff640 1 --2- 172.21.15.46:0/3579625416 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f502006f050 0x7f50201549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:44.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.308+0000 7f501f7fe640 1 --2- 172.21.15.46:0/3579625416 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f502013fda0 0x7f502013cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:44.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.308+0000 7f5024dec640 1 -- 172.21.15.46:0/3579625416 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f502006f050 msgr2=0x7f50201549f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:44.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.308+0000 7f5024dec640 1 --2- 172.21.15.46:0/3579625416 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f502006f050 0x7f50201549f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:44.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.308+0000 7f5024dec640 1 -- 172.21.15.46:0/3579625416 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f502013fda0 msgr2=0x7f502013cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:44.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.308+0000 7f5024dec640 1 --2- 172.21.15.46:0/3579625416 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f502013fda0 0x7f502013cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:44.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.308+0000 7f5024dec640 1 -- 172.21.15.46:0/3579625416 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50100094d0 con 0x7f502015e620 2023-12-02T00:31:44.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.308+0000 7f501f7fe640 1 --2- 172.21.15.46:0/3579625416 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f502013fda0 0x7f502013cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:31:44.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.308+0000 7f5024dec640 1 --2- 172.21.15.46:0/3579625416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f502015e620 0x7f50200b4110 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f501800cef0 tx=0x7f501800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:44.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.308+0000 7f501cff9640 1 -- 172.21.15.46:0/3579625416 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501800d840 con 0x7f502015e620 2023-12-02T00:31:44.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.308+0000 7f501cff9640 1 -- 172.21.15.46:0/3579625416 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f501800d9a0 con 0x7f502015e620 2023-12-02T00:31:44.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.308+0000 7f502684f640 1 -- 172.21.15.46:0/3579625416 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50201541f0 con 0x7f502015e620 2023-12-02T00:31:44.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.308+0000 7f501cff9640 1 -- 172.21.15.46:0/3579625416 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501800dd50 con 0x7f502015e620 2023-12-02T00:31:44.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.308+0000 7f502684f640 1 -- 172.21.15.46:0/3579625416 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50200b4650 con 0x7f502015e620 2023-12-02T00:31:44.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.309+0000 7f502684f640 1 -- 172.21.15.46:0/3579625416 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f502013f340 con 0x7f502015e620 2023-12-02T00:31:44.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.310+0000 7f501cff9640 1 -- 172.21.15.46:0/3579625416 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f501801e3b0 con 0x7f502015e620 2023-12-02T00:31:44.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.310+0000 7f501cff9640 1 --2- 172.21.15.46:0/3579625416 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f500006ae40 0x7f500006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:44.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.310+0000 7f501cff9640 1 -- 172.21.15.46:0/3579625416 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5018089cc0 con 0x7f502015e620 2023-12-02T00:31:44.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.310+0000 7f501cff9640 1 --2- 172.21.15.46:0/3579625416 >> [v2:172.21.15.46:6808/2057169608,v1:172.21.15.46:6809/2057169608] conn(0x7f5000070ca0 0x7f50000730c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:44.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.310+0000 7f501cff9640 1 -- 172.21.15.46:0/3579625416 --> [v2:172.21.15.46:6808/2057169608,v1:172.21.15.46:6809/2057169608] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5000073770 con 0x7f5000070ca0 2023-12-02T00:31:44.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.310+0000 7f501cff9640 1 -- 172.21.15.46:0/3579625416 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f501808c2a0 con 0x7f502015e620 2023-12-02T00:31:44.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.310+0000 7f501ffff640 1 --2- 172.21.15.46:0/3579625416 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f500006ae40 0x7f500006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:44.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.311+0000 7f501f7fe640 1 --2- 172.21.15.46:0/3579625416 >> [v2:172.21.15.46:6808/2057169608,v1:172.21.15.46:6809/2057169608] conn(0x7f5000070ca0 0x7f50000730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:44.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.311+0000 7f501f7fe640 1 --2- 172.21.15.46:0/3579625416 >> [v2:172.21.15.46:6808/2057169608,v1:172.21.15.46:6809/2057169608] conn(0x7f5000070ca0 0x7f50000730c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:44.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.311+0000 7f501ffff640 1 --2- 172.21.15.46:0/3579625416 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f500006ae40 0x7f500006d300 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f502006d1d0 tx=0x7f5008008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:44.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.312+0000 7f501cff9640 1 -- 172.21.15.46:0/3579625416 <== osd.0 v2:172.21.15.46:6808/2057169608 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f5000073770 con 0x7f5000070ca0 2023-12-02T00:31:44.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.325+0000 7f502684f640 1 -- 172.21.15.46:0/3579625416 --> [v2:172.21.15.46:6808/2057169608,v1:172.21.15.46:6809/2057169608] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f502013d7f0 con 0x7f5000070ca0 2023-12-02T00:31:44.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.325+0000 7f501cff9640 1 -- 172.21.15.46:0/3579625416 <== osd.0 v2:172.21.15.46:6808/2057169608 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f502013d7f0 con 0x7f5000070ca0 2023-12-02T00:31:44.326 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-12-02T00:31:44.327 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-12-02T00:31:44.327 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-12-02T00:31:44.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.325+0000 7f502684f640 1 -- 172.21.15.46:0/3579625416 >> [v2:172.21.15.46:6808/2057169608,v1:172.21.15.46:6809/2057169608] conn(0x7f5000070ca0 msgr2=0x7f50000730c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:44.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.325+0000 7f502684f640 1 --2- 172.21.15.46:0/3579625416 >> [v2:172.21.15.46:6808/2057169608,v1:172.21.15.46:6809/2057169608] conn(0x7f5000070ca0 0x7f50000730c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:44.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.325+0000 7f502684f640 1 -- 172.21.15.46:0/3579625416 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f500006ae40 msgr2=0x7f500006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:44.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.325+0000 7f502684f640 1 --2- 172.21.15.46:0/3579625416 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f500006ae40 0x7f500006d300 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f502006d1d0 tx=0x7f5008008040 comp rx=0 tx=0).stop 2023-12-02T00:31:44.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.325+0000 7f502684f640 1 -- 172.21.15.46:0/3579625416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f502015e620 msgr2=0x7f50200b4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:44.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.325+0000 7f502684f640 1 --2- 172.21.15.46:0/3579625416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f502015e620 0x7f50200b4110 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f501800cef0 tx=0x7f501800a710 comp rx=0 tx=0).stop 2023-12-02T00:31:44.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.325+0000 7f5024dec640 1 -- 172.21.15.46:0/3579625416 reap_dead start 2023-12-02T00:31:44.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.325+0000 7f502684f640 1 -- 172.21.15.46:0/3579625416 shutdown_connections 2023-12-02T00:31:44.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.325+0000 7f502684f640 1 -- 172.21.15.46:0/3579625416 >> 172.21.15.46:0/3579625416 conn(0x7f502006a6b0 msgr2=0x7f50200b0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:44.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.326+0000 7f502684f640 1 -- 172.21.15.46:0/3579625416 shutdown_connections 2023-12-02T00:31:44.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.326+0000 7f502684f640 1 -- 172.21.15.46:0/3579625416 wait complete. 2023-12-02T00:31:44.337 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-12-02T00:31:44.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.444+0000 7f55b1348640 1 Processor -- start 2023-12-02T00:31:44.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.444+0000 7f55b1348640 1 -- start start 2023-12-02T00:31:44.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.445+0000 7f55b1348640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55ac06ea50 0x7f55ac06ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:44.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.445+0000 7f55b1348640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f55ac160fb0 0x7f55ac161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:44.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.445+0000 7f55b1348640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55ac161d20 con 0x7f55ac0b3ff0 2023-12-02T00:31:44.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.445+0000 7f55b1348640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55ac0d1ee0 con 0x7f55ac160fb0 2023-12-02T00:31:44.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.445+0000 7f55b1348640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55ac06f370 con 0x7f55ac06ea50 2023-12-02T00:31:44.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.445+0000 7f55a27fc640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f55ac160fb0 0x7f55ac161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:44.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.445+0000 7f55aaffd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55ac06ea50 0x7f55ac06ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:44.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.445+0000 7f55a27fc640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f55ac160fb0 0x7f55ac161410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.46:53712/0 (socket says 172.21.15.46:53712) 2023-12-02T00:31:44.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.445+0000 7f55a27fc640 1 -- 172.21.15.46:0/983360221 learned_addr learned my addr 172.21.15.46:0/983360221 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:44.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.445+0000 7f55ab7fe640 1 --1- 172.21.15.46:0/983360221 >> v1:172.21.15.162:6789/0 conn(0x7f55ac0b3ff0 0x7f55ac0b4460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.46:55988/0 (socket says 172.21.15.46:55988) 2023-12-02T00:31:44.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.445+0000 7f55a27fc640 1 -- 172.21.15.46:0/983360221 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55ac06ea50 msgr2=0x7f55ac06ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:44.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.445+0000 7f55a27fc640 1 --2- 172.21.15.46:0/983360221 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55ac06ea50 0x7f55ac06ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:44.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.445+0000 7f55a27fc640 1 -- 172.21.15.46:0/983360221 >> v1:172.21.15.162:6789/0 conn(0x7f55ac0b3ff0 legacy=0x7f55ac0b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:44.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.445+0000 7f55a27fc640 1 -- 172.21.15.46:0/983360221 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55ac15e760 con 0x7f55ac160fb0 2023-12-02T00:31:44.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.445+0000 7f55aa7fc640 1 -- 172.21.15.46:0/983360221 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 46292463 0 0) 0x7f55ac161d20 con 0x7f55ac0b3ff0 2023-12-02T00:31:44.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.446+0000 7f55a27fc640 1 --2- 172.21.15.46:0/983360221 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55ac160fb0 0x7f55ac161410 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5594009870 tx=0x7f559402b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6a0ba4cc9ceade7 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:44.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.446+0000 7f55aa7fc640 1 -- 172.21.15.46:0/983360221 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5594031640 con 0x7f55ac160fb0 2023-12-02T00:31:44.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.446+0000 7f55aa7fc640 1 -- 172.21.15.46:0/983360221 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55940317a0 con 0x7f55ac160fb0 2023-12-02T00:31:44.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.446+0000 7f55aa7fc640 1 -- 172.21.15.46:0/983360221 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5594031b30 con 0x7f55ac160fb0 2023-12-02T00:31:44.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.447+0000 7f55b1348640 1 -- 172.21.15.46:0/983360221 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55ac160fb0 msgr2=0x7f55ac161410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:44.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.447+0000 7f55b1348640 1 --2- 172.21.15.46:0/983360221 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55ac160fb0 0x7f55ac161410 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5594009870 tx=0x7f559402b8f0 comp rx=0 tx=0).stop 2023-12-02T00:31:44.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.447+0000 7f55b1348640 1 -- 172.21.15.46:0/983360221 shutdown_connections 2023-12-02T00:31:44.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.447+0000 7f55b1348640 1 --2- 172.21.15.46:0/983360221 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55ac160fb0 0x7f55ac161410 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:44.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.447+0000 7f55b1348640 1 --2- 172.21.15.46:0/983360221 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55ac06ea50 0x7f55ac06ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:44.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.447+0000 7f55b1348640 1 -- 172.21.15.46:0/983360221 >> 172.21.15.46:0/983360221 conn(0x7f55ac06a6b0 msgr2=0x7f55ac06aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:44.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.447+0000 7f55b1348640 1 -- 172.21.15.46:0/983360221 shutdown_connections 2023-12-02T00:31:44.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.447+0000 7f55b1348640 1 -- 172.21.15.46:0/983360221 wait complete. 2023-12-02T00:31:44.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.448+0000 7f55b1348640 1 Processor -- start 2023-12-02T00:31:44.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.448+0000 7f55b1348640 1 -- start start 2023-12-02T00:31:44.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.449+0000 7f55b1348640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55ac06ea50 0x7f55ac13c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:44.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.449+0000 7f55b1348640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55ac0b3ff0 0x7f55ac13edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:44.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.449+0000 7f55b1348640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55ac160fb0 0x7f55ac15a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:44.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.449+0000 7f55b1348640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55ac159200 con 0x7f55ac0b3ff0 2023-12-02T00:31:44.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.449+0000 7f55b1348640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55ac159450 con 0x7f55ac160fb0 2023-12-02T00:31:44.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.449+0000 7f55b1348640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55ac159730 con 0x7f55ac06ea50 2023-12-02T00:31:44.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.449+0000 7f55a27fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55ac0b3ff0 0x7f55ac13edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:44.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.449+0000 7f55a27fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55ac0b3ff0 0x7f55ac13edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48824/0 (socket says 172.21.15.46:48824) 2023-12-02T00:31:44.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.449+0000 7f55a27fc640 1 -- 172.21.15.46:0/553311443 learned_addr learned my addr 172.21.15.46:0/553311443 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:44.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.449+0000 7f55aaffd640 1 --2- 172.21.15.46:0/553311443 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55ac06ea50 0x7f55ac13c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:44.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.449+0000 7f55ab7fe640 1 --2- 172.21.15.46:0/553311443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55ac160fb0 0x7f55ac15a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:44.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.450+0000 7f55a27fc640 1 -- 172.21.15.46:0/553311443 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55ac06ea50 msgr2=0x7f55ac13c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:44.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.450+0000 7f55a27fc640 1 --2- 172.21.15.46:0/553311443 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55ac06ea50 0x7f55ac13c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:44.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.450+0000 7f55a27fc640 1 -- 172.21.15.46:0/553311443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55ac160fb0 msgr2=0x7f55ac15a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:44.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.450+0000 7f55a27fc640 1 --2- 172.21.15.46:0/553311443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55ac160fb0 0x7f55ac15a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:44.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.450+0000 7f55a27fc640 1 -- 172.21.15.46:0/553311443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55940094d0 con 0x7f55ac0b3ff0 2023-12-02T00:31:44.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.450+0000 7f55a27fc640 1 --2- 172.21.15.46:0/553311443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55ac0b3ff0 0x7f55ac13edd0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f5594009870 tx=0x7f55940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:44.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.450+0000 7f55a3fff640 1 -- 172.21.15.46:0/553311443 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5594004230 con 0x7f55ac0b3ff0 2023-12-02T00:31:44.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.450+0000 7f55a3fff640 1 -- 172.21.15.46:0/553311443 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5594004390 con 0x7f55ac0b3ff0 2023-12-02T00:31:44.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.450+0000 7f55b1348640 1 -- 172.21.15.46:0/553311443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55ac159a10 con 0x7f55ac0b3ff0 2023-12-02T00:31:44.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.450+0000 7f55a3fff640 1 -- 172.21.15.46:0/553311443 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5594003850 con 0x7f55ac0b3ff0 2023-12-02T00:31:44.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.450+0000 7f55b1348640 1 -- 172.21.15.46:0/553311443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55ac13c8c0 con 0x7f55ac0b3ff0 2023-12-02T00:31:44.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.451+0000 7f55b1348640 1 -- 172.21.15.46:0/553311443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55ac159f30 con 0x7f55ac0b3ff0 2023-12-02T00:31:44.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.451+0000 7f55a3fff640 1 -- 172.21.15.46:0/553311443 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5594043070 con 0x7f55ac0b3ff0 2023-12-02T00:31:44.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.452+0000 7f55a3fff640 1 --2- 172.21.15.46:0/553311443 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f558c06ae40 0x7f558c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:44.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.452+0000 7f55a3fff640 1 -- 172.21.15.46:0/553311443 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f55940aae20 con 0x7f55ac0b3ff0 2023-12-02T00:31:44.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.452+0000 7f55aaffd640 1 --2- 172.21.15.46:0/553311443 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f558c06ae40 0x7f558c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:44.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.453+0000 7f55aaffd640 1 --2- 172.21.15.46:0/553311443 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f558c06ae40 0x7f558c06d300 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f559c009aa0 tx=0x7f559c009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:44.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.456+0000 7f55a3fff640 1 -- 172.21.15.46:0/553311443 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5594076310 con 0x7f55ac0b3ff0 2023-12-02T00:31:44.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.633+0000 7f55b1348640 1 -- 172.21.15.46:0/553311443 --> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f55ac15df60 con 0x7f558c06ae40 2023-12-02T00:31:44.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.634+0000 7f55a3fff640 1 -- 172.21.15.46:0/553311443 <== mgr.4108 v2:172.21.15.162:6832/3248644722 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f559c011a00 con 0x7f558c06ae40 2023-12-02T00:31:44.635 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 0 to deep-scrub 2023-12-02T00:31:44.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.636+0000 7f55b1348640 1 -- 172.21.15.46:0/553311443 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f558c06ae40 msgr2=0x7f558c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:44.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.636+0000 7f55b1348640 1 --2- 172.21.15.46:0/553311443 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f558c06ae40 0x7f558c06d300 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f559c009aa0 tx=0x7f559c009190 comp rx=0 tx=0).stop 2023-12-02T00:31:44.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.636+0000 7f55b1348640 1 -- 172.21.15.46:0/553311443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55ac0b3ff0 msgr2=0x7f55ac13edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:44.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.636+0000 7f55b1348640 1 --2- 172.21.15.46:0/553311443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55ac0b3ff0 0x7f55ac13edd0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f5594009870 tx=0x7f55940046c0 comp rx=0 tx=0).stop 2023-12-02T00:31:44.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.636+0000 7f55b1348640 1 -- 172.21.15.46:0/553311443 shutdown_connections 2023-12-02T00:31:44.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.636+0000 7f55b1348640 1 --2- 172.21.15.46:0/553311443 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f558c06ae40 0x7f558c06d300 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:44.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.636+0000 7f55b1348640 1 --2- 172.21.15.46:0/553311443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55ac160fb0 0x7f55ac15a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:44.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.636+0000 7f55b1348640 1 --2- 172.21.15.46:0/553311443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55ac0b3ff0 0x7f55ac13edd0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:44.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.636+0000 7f55b1348640 1 --2- 172.21.15.46:0/553311443 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55ac06ea50 0x7f55ac13c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:44.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.636+0000 7f55b1348640 1 -- 172.21.15.46:0/553311443 >> 172.21.15.46:0/553311443 conn(0x7f55ac06a6b0 msgr2=0x7f55ac0b4460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:44.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.636+0000 7f55b1348640 1 -- 172.21.15.46:0/553311443 shutdown_connections 2023-12-02T00:31:44.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.636+0000 7f55b1348640 1 -- 172.21.15.46:0/553311443 wait complete. 2023-12-02T00:31:44.648 INFO:tasks.ceph:Scrubbing osd.1 2023-12-02T00:31:44.648 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-12-02T00:31:44.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.757+0000 7f402e1d0640 1 Processor -- start 2023-12-02T00:31:44.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.757+0000 7f402e1d0640 1 -- start start 2023-12-02T00:31:44.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.758+0000 7f402e1d0640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f40281448a0 0x7f40280ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:44.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.758+0000 7f402e1d0640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f402806f780 con 0x7f402806a4d0 2023-12-02T00:31:44.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.758+0000 7f402e1d0640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40280afb00 con 0x7f402815c9d0 2023-12-02T00:31:44.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.758+0000 7f402e1d0640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40280717d0 con 0x7f40281448a0 2023-12-02T00:31:44.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.758+0000 7f40277fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f40281448a0 0x7f40280ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:44.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.758+0000 7f4026ffd640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f402806a4d0 0x7f402806e960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49046/0 (socket says 172.21.15.46:49046) 2023-12-02T00:31:44.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.758+0000 7f4026ffd640 1 -- 172.21.15.46:0/928164188 learned_addr learned my addr 172.21.15.46:0/928164188 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:31:44.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.758+0000 7f40277fe640 1 -- 172.21.15.46:0/928164188 >> v1:172.21.15.162:6789/0 conn(0x7f402815c9d0 legacy=0x7f402806f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:44.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.758+0000 7f40277fe640 1 -- 172.21.15.46:0/928164188 >> v1:172.21.15.46:6789/0 conn(0x7f402806a4d0 legacy=0x7f402806e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:44.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.758+0000 7f40277fe640 1 -- 172.21.15.46:0/928164188 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4028071ab0 con 0x7f40281448a0 2023-12-02T00:31:44.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.758+0000 7f40267fc640 1 -- 172.21.15.46:0/928164188 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 363940320 0 0) 0x7f402806f780 con 0x7f402806a4d0 2023-12-02T00:31:44.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.759+0000 7f40277fe640 1 --2- 172.21.15.46:0/928164188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40281448a0 0x7f40280ce0e0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f40180097a0 tx=0x7f401802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35ab0804088b1781 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:44.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.759+0000 7f40267fc640 1 -- 172.21.15.46:0/928164188 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4018030610 con 0x7f40281448a0 2023-12-02T00:31:44.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.759+0000 7f40267fc640 1 -- 172.21.15.46:0/928164188 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4018030770 con 0x7f40281448a0 2023-12-02T00:31:44.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.759+0000 7f40267fc640 1 -- 172.21.15.46:0/928164188 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4018030b00 con 0x7f40281448a0 2023-12-02T00:31:44.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.760+0000 7f402e1d0640 1 -- 172.21.15.46:0/928164188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40281448a0 msgr2=0x7f40280ce0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:44.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.760+0000 7f402e1d0640 1 --2- 172.21.15.46:0/928164188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40281448a0 0x7f40280ce0e0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f40180097a0 tx=0x7f401802b960 comp rx=0 tx=0).stop 2023-12-02T00:31:44.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.760+0000 7f402e1d0640 1 -- 172.21.15.46:0/928164188 shutdown_connections 2023-12-02T00:31:44.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.760+0000 7f402e1d0640 1 --2- 172.21.15.46:0/928164188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40281448a0 0x7f40280ce0e0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:44.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.760+0000 7f402e1d0640 1 -- 172.21.15.46:0/928164188 >> 172.21.15.46:0/928164188 conn(0x7f40280b1e50 msgr2=0x7f40280b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:44.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.760+0000 7f402e1d0640 1 -- 172.21.15.46:0/928164188 shutdown_connections 2023-12-02T00:31:44.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.760+0000 7f402e1d0640 1 -- 172.21.15.46:0/928164188 wait complete. 2023-12-02T00:31:44.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.761+0000 7f402e1d0640 1 Processor -- start 2023-12-02T00:31:44.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.761+0000 7f402e1d0640 1 -- start start 2023-12-02T00:31:44.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.762+0000 7f402e1d0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f402806a4d0 0x7f40280b3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:44.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.762+0000 7f402e1d0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40281448a0 0x7f402813bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:44.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.762+0000 7f402e1d0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f402815c9d0 0x7f40280ae1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:44.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.762+0000 7f402e1d0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402813c0a0 con 0x7f40281448a0 2023-12-02T00:31:44.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.762+0000 7f402e1d0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40280b38e0 con 0x7f402806a4d0 2023-12-02T00:31:44.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.762+0000 7f402e1d0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40280b3bc0 con 0x7f402815c9d0 2023-12-02T00:31:44.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.762+0000 7f4026ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40281448a0 0x7f402813bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:44.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.762+0000 7f4026ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40281448a0 0x7f402813bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48838/0 (socket says 172.21.15.46:48838) 2023-12-02T00:31:44.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.762+0000 7f4026ffd640 1 -- 172.21.15.46:0/3747695877 learned_addr learned my addr 172.21.15.46:0/3747695877 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:44.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.762+0000 7f40277fe640 1 --2- 172.21.15.46:0/3747695877 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f402806a4d0 0x7f40280b3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:44.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.762+0000 7f4027fff640 1 --2- 172.21.15.46:0/3747695877 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f402815c9d0 0x7f40280ae1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:44.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.762+0000 7f4026ffd640 1 -- 172.21.15.46:0/3747695877 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f402815c9d0 msgr2=0x7f40280ae1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:44.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.762+0000 7f4026ffd640 1 --2- 172.21.15.46:0/3747695877 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f402815c9d0 0x7f40280ae1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:44.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.762+0000 7f4026ffd640 1 -- 172.21.15.46:0/3747695877 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f402806a4d0 msgr2=0x7f40280b3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:44.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.762+0000 7f4026ffd640 1 --2- 172.21.15.46:0/3747695877 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f402806a4d0 0x7f40280b3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:44.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.762+0000 7f4026ffd640 1 -- 172.21.15.46:0/3747695877 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4018009400 con 0x7f40281448a0 2023-12-02T00:31:44.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.762+0000 7f4027fff640 1 --2- 172.21.15.46:0/3747695877 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f402815c9d0 0x7f40280ae1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:31:44.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.762+0000 7f40277fe640 1 --2- 172.21.15.46:0/3747695877 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f402806a4d0 0x7f40280b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:31:44.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.763+0000 7f4026ffd640 1 --2- 172.21.15.46:0/3747695877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40281448a0 0x7f402813bb60 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f401c00c4d0 tx=0x7f401c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:44.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.763+0000 7f400ffff640 1 -- 172.21.15.46:0/3747695877 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401c0074e0 con 0x7f40281448a0 2023-12-02T00:31:44.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.763+0000 7f400ffff640 1 -- 172.21.15.46:0/3747695877 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f401c007660 con 0x7f40281448a0 2023-12-02T00:31:44.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.763+0000 7f402e1d0640 1 -- 172.21.15.46:0/3747695877 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40280b3ea0 con 0x7f40281448a0 2023-12-02T00:31:44.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.763+0000 7f400ffff640 1 -- 172.21.15.46:0/3747695877 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401c004a20 con 0x7f40281448a0 2023-12-02T00:31:44.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.763+0000 7f402e1d0640 1 -- 172.21.15.46:0/3747695877 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40280b4180 con 0x7f40281448a0 2023-12-02T00:31:44.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.764+0000 7f402e1d0640 1 -- 172.21.15.46:0/3747695877 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4028072350 con 0x7f40281448a0 2023-12-02T00:31:44.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.764+0000 7f400ffff640 1 -- 172.21.15.46:0/3747695877 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f401c004b80 con 0x7f40281448a0 2023-12-02T00:31:44.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.765+0000 7f400ffff640 1 --2- 172.21.15.46:0/3747695877 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f401406ae40 0x7f401406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:44.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.765+0000 7f400ffff640 1 -- 172.21.15.46:0/3747695877 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f401c0887b0 con 0x7f40281448a0 2023-12-02T00:31:44.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.765+0000 7f400ffff640 1 --2- 172.21.15.46:0/3747695877 >> [v2:172.21.15.46:6800/10535129,v1:172.21.15.46:6801/10535129] conn(0x7f4014070ca0 0x7f40140730c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:44.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.765+0000 7f400ffff640 1 -- 172.21.15.46:0/3747695877 --> [v2:172.21.15.46:6800/10535129,v1:172.21.15.46:6801/10535129] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4014073770 con 0x7f4014070ca0 2023-12-02T00:31:44.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.765+0000 7f400ffff640 1 -- 172.21.15.46:0/3747695877 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f401c08ae70 con 0x7f40281448a0 2023-12-02T00:31:44.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.765+0000 7f40277fe640 1 --2- 172.21.15.46:0/3747695877 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f401406ae40 0x7f401406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:44.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.765+0000 7f4027fff640 1 --2- 172.21.15.46:0/3747695877 >> [v2:172.21.15.46:6800/10535129,v1:172.21.15.46:6801/10535129] conn(0x7f4014070ca0 0x7f40140730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:44.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.765+0000 7f4027fff640 1 --2- 172.21.15.46:0/3747695877 >> [v2:172.21.15.46:6800/10535129,v1:172.21.15.46:6801/10535129] conn(0x7f4014070ca0 0x7f40140730c0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:44.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.765+0000 7f40277fe640 1 --2- 172.21.15.46:0/3747695877 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f401406ae40 0x7f401406d300 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4018036000 tx=0x7f401803b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:44.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.766+0000 7f400ffff640 1 -- 172.21.15.46:0/3747695877 <== osd.1 v2:172.21.15.46:6800/10535129 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f4014073770 con 0x7f4014070ca0 2023-12-02T00:31:44.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.779+0000 7f402e1d0640 1 -- 172.21.15.46:0/3747695877 --> [v2:172.21.15.46:6800/10535129,v1:172.21.15.46:6801/10535129] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f4028072070 con 0x7f4014070ca0 2023-12-02T00:31:44.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.779+0000 7f400ffff640 1 -- 172.21.15.46:0/3747695877 <== osd.1 v2:172.21.15.46:6800/10535129 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f4020007ac0 con 0x7f4014070ca0 2023-12-02T00:31:44.781 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-12-02T00:31:44.781 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-12-02T00:31:44.781 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-12-02T00:31:44.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.780+0000 7f402e1d0640 1 -- 172.21.15.46:0/3747695877 >> [v2:172.21.15.46:6800/10535129,v1:172.21.15.46:6801/10535129] conn(0x7f4014070ca0 msgr2=0x7f40140730c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:44.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.780+0000 7f402e1d0640 1 --2- 172.21.15.46:0/3747695877 >> [v2:172.21.15.46:6800/10535129,v1:172.21.15.46:6801/10535129] conn(0x7f4014070ca0 0x7f40140730c0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:44.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.780+0000 7f402e1d0640 1 -- 172.21.15.46:0/3747695877 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f401406ae40 msgr2=0x7f401406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:44.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.780+0000 7f402e1d0640 1 --2- 172.21.15.46:0/3747695877 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f401406ae40 0x7f401406d300 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4018036000 tx=0x7f401803b040 comp rx=0 tx=0).stop 2023-12-02T00:31:44.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.780+0000 7f402e1d0640 1 -- 172.21.15.46:0/3747695877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40281448a0 msgr2=0x7f402813bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:44.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.780+0000 7f402e1d0640 1 --2- 172.21.15.46:0/3747695877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40281448a0 0x7f402813bb60 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f401c00c4d0 tx=0x7f401c00c990 comp rx=0 tx=0).stop 2023-12-02T00:31:44.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.780+0000 7f4027fff640 1 -- 172.21.15.46:0/3747695877 reap_dead start 2023-12-02T00:31:44.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.780+0000 7f402e1d0640 1 -- 172.21.15.46:0/3747695877 shutdown_connections 2023-12-02T00:31:44.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.780+0000 7f402e1d0640 1 -- 172.21.15.46:0/3747695877 >> 172.21.15.46:0/3747695877 conn(0x7f40280b1e50 msgr2=0x7f40280cdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:44.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.780+0000 7f402e1d0640 1 -- 172.21.15.46:0/3747695877 shutdown_connections 2023-12-02T00:31:44.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.780+0000 7f402e1d0640 1 -- 172.21.15.46:0/3747695877 wait complete. 2023-12-02T00:31:44.791 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-12-02T00:31:44.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.898+0000 7f86035de640 1 Processor -- start 2023-12-02T00:31:44.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.898+0000 7f86035de640 1 -- start start 2023-12-02T00:31:44.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.899+0000 7f86035de640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f85fc143490 0x7f85fc0b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:44.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.899+0000 7f86035de640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f85fc15abf0 0x7f85fc0b1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:44.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.899+0000 7f86035de640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85fc15e2a0 con 0x7f85fc15e730 2023-12-02T00:31:44.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.899+0000 7f86035de640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85fc15b520 con 0x7f85fc143490 2023-12-02T00:31:44.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.899+0000 7f86035de640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85fc143870 con 0x7f85fc15abf0 2023-12-02T00:31:44.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.899+0000 7f860137a640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f85fc143490 0x7f85fc0b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:44.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.899+0000 7f860137a640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f85fc143490 0x7f85fc0b0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48848/0 (socket says 172.21.15.46:48848) 2023-12-02T00:31:44.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.899+0000 7f860137a640 1 -- 172.21.15.46:0/690644563 learned_addr learned my addr 172.21.15.46:0/690644563 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:44.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.899+0000 7f8601b7b640 1 --2- 172.21.15.46:0/690644563 >> v2:172.21.15.162:3300/0 conn(0x7f85fc15abf0 0x7f85fc0b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:44.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.899+0000 7f860137a640 1 -- 172.21.15.46:0/690644563 >> v2:172.21.15.162:3300/0 conn(0x7f85fc15abf0 msgr2=0x7f85fc0b1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:44.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.899+0000 7f860137a640 1 --2- 172.21.15.46:0/690644563 >> v2:172.21.15.162:3300/0 conn(0x7f85fc15abf0 0x7f85fc0b1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:44.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.899+0000 7f860137a640 1 -- 172.21.15.46:0/690644563 >> v1:172.21.15.162:6789/0 conn(0x7f85fc15e730 legacy=0x7f85fc15a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:44.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.899+0000 7f860137a640 1 -- 172.21.15.46:0/690644563 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85fc15b240 con 0x7f85fc143490 2023-12-02T00:31:44.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.900+0000 7f860137a640 1 --2- 172.21.15.46:0/690644563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85fc143490 0x7f85fc0b0cf0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f85fc0b4ca0 tx=0x7f85f802e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63551c97c43d786d server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:44.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.900+0000 7f85f3fff640 1 -- 172.21.15.46:0/690644563 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f802ee90 con 0x7f85fc143490 2023-12-02T00:31:44.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.900+0000 7f85f3fff640 1 -- 172.21.15.46:0/690644563 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f8036440 con 0x7f85fc143490 2023-12-02T00:31:44.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.900+0000 7f85f3fff640 1 -- 172.21.15.46:0/690644563 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f80367d0 con 0x7f85fc143490 2023-12-02T00:31:44.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.901+0000 7f86035de640 1 -- 172.21.15.46:0/690644563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85fc143490 msgr2=0x7f85fc0b0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:44.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.901+0000 7f86035de640 1 --2- 172.21.15.46:0/690644563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85fc143490 0x7f85fc0b0cf0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f85fc0b4ca0 tx=0x7f85f802e0e0 comp rx=0 tx=0).stop 2023-12-02T00:31:44.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.901+0000 7f86035de640 1 -- 172.21.15.46:0/690644563 shutdown_connections 2023-12-02T00:31:44.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.901+0000 7f86035de640 1 --2- 172.21.15.46:0/690644563 >> v2:172.21.15.162:3300/0 conn(0x7f85fc15abf0 0x7f85fc0b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:44.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.901+0000 7f86035de640 1 --2- 172.21.15.46:0/690644563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85fc143490 0x7f85fc0b0cf0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:44.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.901+0000 7f86035de640 1 -- 172.21.15.46:0/690644563 >> 172.21.15.46:0/690644563 conn(0x7f85fc06a4d0 msgr2=0x7f85fc06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:44.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.901+0000 7f86035de640 1 -- 172.21.15.46:0/690644563 shutdown_connections 2023-12-02T00:31:44.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.901+0000 7f86035de640 1 -- 172.21.15.46:0/690644563 wait complete. 2023-12-02T00:31:44.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.902+0000 7f86035de640 1 Processor -- start 2023-12-02T00:31:44.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.902+0000 7f86035de640 1 -- start start 2023-12-02T00:31:44.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.903+0000 7f86035de640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85fc143490 0x7f85fc138930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:44.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.903+0000 7f86035de640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85fc15abf0 0x7f85fc140d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:44.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.903+0000 7f86035de640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85fc15e730 0x7f85fc13aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:44.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.903+0000 7f86035de640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85fc13cd30 con 0x7f85fc15e730 2023-12-02T00:31:44.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.903+0000 7f86035de640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85fc13ba30 con 0x7f85fc15abf0 2023-12-02T00:31:44.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.903+0000 7f86035de640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85fc13bd10 con 0x7f85fc143490 2023-12-02T00:31:44.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.903+0000 7f8601b7b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85fc15e730 0x7f85fc13aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:44.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.903+0000 7f8601b7b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85fc15e730 0x7f85fc13aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48858/0 (socket says 172.21.15.46:48858) 2023-12-02T00:31:44.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.903+0000 7f8601b7b640 1 -- 172.21.15.46:0/709695305 learned_addr learned my addr 172.21.15.46:0/709695305 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:44.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.903+0000 7f860137a640 1 --2- 172.21.15.46:0/709695305 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85fc143490 0x7f85fc138930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:44.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.903+0000 7f8600b79640 1 --2- 172.21.15.46:0/709695305 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85fc15abf0 0x7f85fc140d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:44.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.903+0000 7f8601b7b640 1 -- 172.21.15.46:0/709695305 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85fc143490 msgr2=0x7f85fc138930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:44.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.903+0000 7f8601b7b640 1 --2- 172.21.15.46:0/709695305 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85fc143490 0x7f85fc138930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:44.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.903+0000 7f8601b7b640 1 -- 172.21.15.46:0/709695305 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85fc15abf0 msgr2=0x7f85fc140d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:44.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.903+0000 7f8601b7b640 1 --2- 172.21.15.46:0/709695305 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85fc15abf0 0x7f85fc140d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:44.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.903+0000 7f8601b7b640 1 -- 172.21.15.46:0/709695305 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f800b770 con 0x7f85fc15e730 2023-12-02T00:31:44.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.903+0000 7f8601b7b640 1 --2- 172.21.15.46:0/709695305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85fc15e730 0x7f85fc13aab0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f85f400cd70 tx=0x7f85f400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:44.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.903+0000 7f85f1ffb640 1 -- 172.21.15.46:0/709695305 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f4003b90 con 0x7f85fc15e730 2023-12-02T00:31:44.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.903+0000 7f85f1ffb640 1 -- 172.21.15.46:0/709695305 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f4003cf0 con 0x7f85fc15e730 2023-12-02T00:31:44.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.903+0000 7f86035de640 1 -- 172.21.15.46:0/709695305 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85fc13bff0 con 0x7f85fc15e730 2023-12-02T00:31:44.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.903+0000 7f85f1ffb640 1 -- 172.21.15.46:0/709695305 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f4004240 con 0x7f85fc15e730 2023-12-02T00:31:44.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.904+0000 7f86035de640 1 -- 172.21.15.46:0/709695305 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85fc138e70 con 0x7f85fc15e730 2023-12-02T00:31:44.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.905+0000 7f86035de640 1 -- 172.21.15.46:0/709695305 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85fc0b2610 con 0x7f85fc15e730 2023-12-02T00:31:44.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.905+0000 7f85f1ffb640 1 -- 172.21.15.46:0/709695305 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f85f40043a0 con 0x7f85fc15e730 2023-12-02T00:31:44.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.905+0000 7f85f1ffb640 1 --2- 172.21.15.46:0/709695305 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f85e006ae40 0x7f85e006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:44.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.905+0000 7f85f1ffb640 1 -- 172.21.15.46:0/709695305 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f85f4012070 con 0x7f85fc15e730 2023-12-02T00:31:44.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.905+0000 7f860137a640 1 --2- 172.21.15.46:0/709695305 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f85e006ae40 0x7f85e006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:44.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.906+0000 7f860137a640 1 --2- 172.21.15.46:0/709695305 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f85e006ae40 0x7f85e006d300 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f85f8038000 tx=0x7f85f8032000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:44.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:44.909+0000 7f85f1ffb640 1 -- 172.21.15.46:0/709695305 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f85f4055060 con 0x7f85fc15e730 2023-12-02T00:31:45.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.086+0000 7f86035de640 1 -- 172.21.15.46:0/709695305 --> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f85fc15df30 con 0x7f85e006ae40 2023-12-02T00:31:45.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.087+0000 7f85f1ffb640 1 -- 172.21.15.46:0/709695305 <== mgr.4108 v2:172.21.15.162:6832/3248644722 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f85f8036730 con 0x7f85e006ae40 2023-12-02T00:31:45.088 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 1 to deep-scrub 2023-12-02T00:31:45.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.089+0000 7f86035de640 1 -- 172.21.15.46:0/709695305 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f85e006ae40 msgr2=0x7f85e006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:45.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.089+0000 7f86035de640 1 --2- 172.21.15.46:0/709695305 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f85e006ae40 0x7f85e006d300 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f85f8038000 tx=0x7f85f8032000 comp rx=0 tx=0).stop 2023-12-02T00:31:45.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.089+0000 7f86035de640 1 -- 172.21.15.46:0/709695305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85fc15e730 msgr2=0x7f85fc13aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:45.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.089+0000 7f86035de640 1 --2- 172.21.15.46:0/709695305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85fc15e730 0x7f85fc13aab0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f85f400cd70 tx=0x7f85f400a660 comp rx=0 tx=0).stop 2023-12-02T00:31:45.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.089+0000 7f86035de640 1 -- 172.21.15.46:0/709695305 shutdown_connections 2023-12-02T00:31:45.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.089+0000 7f86035de640 1 --2- 172.21.15.46:0/709695305 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f85e006ae40 0x7f85e006d300 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:45.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.089+0000 7f86035de640 1 --2- 172.21.15.46:0/709695305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85fc15e730 0x7f85fc13aab0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:45.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.089+0000 7f86035de640 1 --2- 172.21.15.46:0/709695305 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85fc15abf0 0x7f85fc140d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:45.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.089+0000 7f86035de640 1 --2- 172.21.15.46:0/709695305 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85fc143490 0x7f85fc138930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:45.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.089+0000 7f86035de640 1 -- 172.21.15.46:0/709695305 >> 172.21.15.46:0/709695305 conn(0x7f85fc06a4d0 msgr2=0x7f85fc1426a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:45.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.089+0000 7f86035de640 1 -- 172.21.15.46:0/709695305 shutdown_connections 2023-12-02T00:31:45.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.089+0000 7f86035de640 1 -- 172.21.15.46:0/709695305 wait complete. 2023-12-02T00:31:45.101 INFO:tasks.ceph:Scrubbing osd.2 2023-12-02T00:31:45.102 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-12-02T00:31:45.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.211+0000 7f6d6e2c9640 1 Processor -- start 2023-12-02T00:31:45.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.211+0000 7f6d6e2c9640 1 -- start start 2023-12-02T00:31:45.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.212+0000 7f6d6e2c9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d68160bb0 0x7f6d680d3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:45.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.212+0000 7f6d6e2c9640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6d680ce440 0x7f6d680d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:45.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.212+0000 7f6d6e2c9640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d680b3450 con 0x7f6d680b3fc0 2023-12-02T00:31:45.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.212+0000 7f6d6e2c9640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d680ce8b0 con 0x7f6d680ce440 2023-12-02T00:31:45.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.212+0000 7f6d6e2c9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d680b45b0 con 0x7f6d68160bb0 2023-12-02T00:31:45.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.212+0000 7f6d677fe640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6d680b3fc0 0x7f6d680cdd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49060/0 (socket says 172.21.15.46:49060) 2023-12-02T00:31:45.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.212+0000 7f6d677fe640 1 -- 172.21.15.46:0/3436635512 learned_addr learned my addr 172.21.15.46:0/3436635512 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:31:45.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.213+0000 7f6d67fff640 1 --2- 172.21.15.46:0/3436635512 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d68160bb0 0x7f6d680d3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:45.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.213+0000 7f6d6c866640 1 --2- 172.21.15.46:0/3436635512 >> v2:172.21.15.162:3300/0 conn(0x7f6d680ce440 0x7f6d680d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:45.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.213+0000 7f6d66ffd640 1 -- 172.21.15.46:0/3436635512 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1138964823 0 0) 0x7f6d680b3450 con 0x7f6d680b3fc0 2023-12-02T00:31:45.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.213+0000 7f6d66ffd640 1 -- 172.21.15.46:0/3436635512 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d50003520 con 0x7f6d680b3fc0 2023-12-02T00:31:45.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.213+0000 7f6d66ffd640 1 -- 172.21.15.46:0/3436635512 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2641870379 0 0) 0x7f6d50003520 con 0x7f6d680b3fc0 2023-12-02T00:31:45.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.213+0000 7f6d66ffd640 1 -- 172.21.15.46:0/3436635512 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d68160bb0 msgr2=0x7f6d680d3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:45.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.213+0000 7f6d66ffd640 1 --2- 172.21.15.46:0/3436635512 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d68160bb0 0x7f6d680d3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:45.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.213+0000 7f6d66ffd640 1 -- 172.21.15.46:0/3436635512 >> v2:172.21.15.162:3300/0 conn(0x7f6d680ce440 msgr2=0x7f6d680d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:45.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.213+0000 7f6d66ffd640 1 --2- 172.21.15.46:0/3436635512 >> v2:172.21.15.162:3300/0 conn(0x7f6d680ce440 0x7f6d680d1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:45.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.213+0000 7f6d66ffd640 1 -- 172.21.15.46:0/3436635512 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d680b4890 con 0x7f6d680b3fc0 2023-12-02T00:31:45.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.213+0000 7f6d66ffd640 1 -- 172.21.15.46:0/3436635512 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7f6d5c001670 con 0x7f6d680b3fc0 2023-12-02T00:31:45.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.213+0000 7f6d66ffd640 1 -- 172.21.15.46:0/3436635512 >> v1:172.21.15.46:6789/0 conn(0x7f6d680b3fc0 legacy=0x7f6d680cdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:45.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.213+0000 7f6d66ffd640 1 --2- 172.21.15.46:0/3436635512 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d50003970 0x7f6d50003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:45.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.213+0000 7f6d66ffd640 1 --2- 172.21.15.46:0/3436635512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d500047c0 0x7f6d50004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:45.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.213+0000 7f6d66ffd640 1 --2- 172.21.15.46:0/3436635512 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d68160bb0 0x7f6d50005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:45.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.213+0000 7f6d66ffd640 1 -- 172.21.15.46:0/3436635512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d680ce8b0 con 0x7f6d500047c0 2023-12-02T00:31:45.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.213+0000 7f6d66ffd640 1 -- 172.21.15.46:0/3436635512 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d680b45b0 con 0x7f6d50003970 2023-12-02T00:31:45.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.213+0000 7f6d66ffd640 1 -- 172.21.15.46:0/3436635512 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5000b980 con 0x7f6d68160bb0 2023-12-02T00:31:45.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.213+0000 7f6d677fe640 1 --2- 172.21.15.46:0/3436635512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d500047c0 0x7f6d50004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:45.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.213+0000 7f6d67fff640 1 --2- 172.21.15.46:0/3436635512 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d50003970 0x7f6d50003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:45.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.213+0000 7f6d6c866640 1 --2- 172.21.15.46:0/3436635512 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d68160bb0 0x7f6d50005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:45.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.213+0000 7f6d677fe640 1 -- 172.21.15.46:0/3436635512 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d68160bb0 msgr2=0x7f6d50005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:45.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.213+0000 7f6d677fe640 1 --2- 172.21.15.46:0/3436635512 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d68160bb0 0x7f6d50005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:45.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.213+0000 7f6d677fe640 1 -- 172.21.15.46:0/3436635512 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d50003970 msgr2=0x7f6d50003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:45.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.213+0000 7f6d677fe640 1 --2- 172.21.15.46:0/3436635512 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d50003970 0x7f6d50003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:45.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.213+0000 7f6d677fe640 1 -- 172.21.15.46:0/3436635512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d50003520 con 0x7f6d500047c0 2023-12-02T00:31:45.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.214+0000 7f6d677fe640 1 --2- 172.21.15.46:0/3436635512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d500047c0 0x7f6d50004c40 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c00c290 tx=0x7f6d500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da8c4a7f76b03805 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:45.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.214+0000 7f6d66ffd640 1 -- 172.21.15.46:0/3436635512 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c005ad0 con 0x7f6d500047c0 2023-12-02T00:31:45.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.214+0000 7f6d66ffd640 1 -- 172.21.15.46:0/3436635512 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d5c005c30 con 0x7f6d500047c0 2023-12-02T00:31:45.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.214+0000 7f6d66ffd640 1 -- 172.21.15.46:0/3436635512 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c02c8d0 con 0x7f6d500047c0 2023-12-02T00:31:45.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.215+0000 7f6d6e2c9640 1 -- 172.21.15.46:0/3436635512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d500047c0 msgr2=0x7f6d50004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:45.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.215+0000 7f6d6e2c9640 1 --2- 172.21.15.46:0/3436635512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d500047c0 0x7f6d50004c40 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c00c290 tx=0x7f6d500057f0 comp rx=0 tx=0).stop 2023-12-02T00:31:45.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.215+0000 7f6d6c866640 1 -- 172.21.15.46:0/3436635512 reap_dead start 2023-12-02T00:31:45.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.215+0000 7f6d6e2c9640 1 -- 172.21.15.46:0/3436635512 shutdown_connections 2023-12-02T00:31:45.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.215+0000 7f6d6e2c9640 1 -- 172.21.15.46:0/3436635512 >> 172.21.15.46:0/3436635512 conn(0x7f6d6809c590 msgr2=0x7f6d680d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:45.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.215+0000 7f6d6e2c9640 1 -- 172.21.15.46:0/3436635512 shutdown_connections 2023-12-02T00:31:45.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.215+0000 7f6d6e2c9640 1 -- 172.21.15.46:0/3436635512 wait complete. 2023-12-02T00:31:45.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.216+0000 7f6d6e2c9640 1 Processor -- start 2023-12-02T00:31:45.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.216+0000 7f6d6e2c9640 1 -- start start 2023-12-02T00:31:45.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.217+0000 7f6d6e2c9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d500047c0 0x7f6d681542a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:45.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.217+0000 7f6d6e2c9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d681463e0 0x7f6d680b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:45.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.217+0000 7f6d6e2c9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d68146840 0x7f6d680b8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:45.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.217+0000 7f6d6e2c9640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6815c5d0 con 0x7f6d681463e0 2023-12-02T00:31:45.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.217+0000 7f6d6e2c9640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d680b8d40 con 0x7f6d68146840 2023-12-02T00:31:45.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.217+0000 7f6d6e2c9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6815c9d0 con 0x7f6d500047c0 2023-12-02T00:31:45.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.217+0000 7f6d677fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d681463e0 0x7f6d680b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:45.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.217+0000 7f6d677fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d681463e0 0x7f6d680b82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48868/0 (socket says 172.21.15.46:48868) 2023-12-02T00:31:45.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.217+0000 7f6d677fe640 1 -- 172.21.15.46:0/2720454634 learned_addr learned my addr 172.21.15.46:0/2720454634 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:45.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.217+0000 7f6d67fff640 1 --2- 172.21.15.46:0/2720454634 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d500047c0 0x7f6d681542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:45.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.217+0000 7f6d6c866640 1 --2- 172.21.15.46:0/2720454634 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d68146840 0x7f6d680b8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:45.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.218+0000 7f6d677fe640 1 -- 172.21.15.46:0/2720454634 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d500047c0 msgr2=0x7f6d681542a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:45.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.218+0000 7f6d677fe640 1 --2- 172.21.15.46:0/2720454634 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d500047c0 0x7f6d681542a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:45.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.218+0000 7f6d677fe640 1 -- 172.21.15.46:0/2720454634 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d68146840 msgr2=0x7f6d680b8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:45.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.218+0000 7f6d677fe640 1 --2- 172.21.15.46:0/2720454634 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d68146840 0x7f6d680b8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:45.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.218+0000 7f6d677fe640 1 -- 172.21.15.46:0/2720454634 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d6815b950 con 0x7f6d681463e0 2023-12-02T00:31:45.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.218+0000 7f6d6c866640 1 --2- 172.21.15.46:0/2720454634 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d68146840 0x7f6d680b8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:31:45.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.218+0000 7f6d677fe640 1 --2- 172.21.15.46:0/2720454634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d681463e0 0x7f6d680b82c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c02c470 tx=0x7f6d5c000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:45.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.218+0000 7f6d64ff9640 1 -- 172.21.15.46:0/2720454634 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c03c040 con 0x7f6d681463e0 2023-12-02T00:31:45.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.218+0000 7f6d64ff9640 1 -- 172.21.15.46:0/2720454634 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d5c02ddb0 con 0x7f6d681463e0 2023-12-02T00:31:45.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.218+0000 7f6d6e2c9640 1 -- 172.21.15.46:0/2720454634 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d6815bc30 con 0x7f6d681463e0 2023-12-02T00:31:45.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.218+0000 7f6d64ff9640 1 -- 172.21.15.46:0/2720454634 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c0072b0 con 0x7f6d681463e0 2023-12-02T00:31:45.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.218+0000 7f6d6e2c9640 1 -- 172.21.15.46:0/2720454634 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d6815c1f0 con 0x7f6d681463e0 2023-12-02T00:31:45.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.219+0000 7f6d6e2c9640 1 -- 172.21.15.46:0/2720454634 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6d68160f90 con 0x7f6d681463e0 2023-12-02T00:31:45.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.219+0000 7f6d64ff9640 1 -- 172.21.15.46:0/2720454634 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6d5c032070 con 0x7f6d681463e0 2023-12-02T00:31:45.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.220+0000 7f6d64ff9640 1 --2- 172.21.15.46:0/2720454634 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f6d5406ae70 0x7f6d5406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:45.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.220+0000 7f6d64ff9640 1 -- 172.21.15.46:0/2720454634 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6d5c0aa480 con 0x7f6d681463e0 2023-12-02T00:31:45.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.220+0000 7f6d64ff9640 1 --2- 172.21.15.46:0/2720454634 >> [v2:172.21.15.46:6824/878860153,v1:172.21.15.46:6825/878860153] conn(0x7f6d54070cd0 0x7f6d540730f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:45.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.220+0000 7f6d64ff9640 1 -- 172.21.15.46:0/2720454634 --> [v2:172.21.15.46:6824/878860153,v1:172.21.15.46:6825/878860153] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6d540737a0 con 0x7f6d54070cd0 2023-12-02T00:31:45.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.220+0000 7f6d64ff9640 1 -- 172.21.15.46:0/2720454634 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d5c0aca60 con 0x7f6d681463e0 2023-12-02T00:31:45.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.220+0000 7f6d6c866640 1 --2- 172.21.15.46:0/2720454634 >> [v2:172.21.15.46:6824/878860153,v1:172.21.15.46:6825/878860153] conn(0x7f6d54070cd0 0x7f6d540730f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:45.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.220+0000 7f6d67fff640 1 --2- 172.21.15.46:0/2720454634 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f6d5406ae70 0x7f6d5406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:45.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.221+0000 7f6d6c866640 1 --2- 172.21.15.46:0/2720454634 >> [v2:172.21.15.46:6824/878860153,v1:172.21.15.46:6825/878860153] conn(0x7f6d54070cd0 0x7f6d540730f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:45.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.221+0000 7f6d67fff640 1 --2- 172.21.15.46:0/2720454634 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f6d5406ae70 0x7f6d5406d330 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6d58009a60 tx=0x7f6d58009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:45.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.221+0000 7f6d64ff9640 1 -- 172.21.15.46:0/2720454634 <== osd.2 v2:172.21.15.46:6824/878860153 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f6d540737a0 con 0x7f6d54070cd0 2023-12-02T00:31:45.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.234+0000 7f6d6e2c9640 1 -- 172.21.15.46:0/2720454634 --> [v2:172.21.15.46:6824/878860153,v1:172.21.15.46:6825/878860153] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f6d680b3980 con 0x7f6d54070cd0 2023-12-02T00:31:45.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.234+0000 7f6d64ff9640 1 -- 172.21.15.46:0/2720454634 <== osd.2 v2:172.21.15.46:6824/878860153 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f6d6000f040 con 0x7f6d54070cd0 2023-12-02T00:31:45.236 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-12-02T00:31:45.236 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-12-02T00:31:45.236 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-12-02T00:31:45.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.235+0000 7f6d6e2c9640 1 -- 172.21.15.46:0/2720454634 >> [v2:172.21.15.46:6824/878860153,v1:172.21.15.46:6825/878860153] conn(0x7f6d54070cd0 msgr2=0x7f6d540730f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:45.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.235+0000 7f6d6e2c9640 1 --2- 172.21.15.46:0/2720454634 >> [v2:172.21.15.46:6824/878860153,v1:172.21.15.46:6825/878860153] conn(0x7f6d54070cd0 0x7f6d540730f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:45.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.235+0000 7f6d6e2c9640 1 -- 172.21.15.46:0/2720454634 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f6d5406ae70 msgr2=0x7f6d5406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:45.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.235+0000 7f6d6e2c9640 1 --2- 172.21.15.46:0/2720454634 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f6d5406ae70 0x7f6d5406d330 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6d58009a60 tx=0x7f6d58009190 comp rx=0 tx=0).stop 2023-12-02T00:31:45.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.235+0000 7f6d6e2c9640 1 -- 172.21.15.46:0/2720454634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d681463e0 msgr2=0x7f6d680b82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:45.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.235+0000 7f6d6e2c9640 1 --2- 172.21.15.46:0/2720454634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d681463e0 0x7f6d680b82c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c02c470 tx=0x7f6d5c000fe0 comp rx=0 tx=0).stop 2023-12-02T00:31:45.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.235+0000 7f6d6c866640 1 -- 172.21.15.46:0/2720454634 reap_dead start 2023-12-02T00:31:45.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.235+0000 7f6d6e2c9640 1 -- 172.21.15.46:0/2720454634 shutdown_connections 2023-12-02T00:31:45.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.235+0000 7f6d6e2c9640 1 -- 172.21.15.46:0/2720454634 >> 172.21.15.46:0/2720454634 conn(0x7f6d6809c590 msgr2=0x7f6d680d1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:45.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.235+0000 7f6d6e2c9640 1 -- 172.21.15.46:0/2720454634 shutdown_connections 2023-12-02T00:31:45.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.235+0000 7f6d6e2c9640 1 -- 172.21.15.46:0/2720454634 wait complete. 2023-12-02T00:31:45.247 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-12-02T00:31:45.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.352+0000 7f817c3d0640 1 Processor -- start 2023-12-02T00:31:45.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.352+0000 7f817c3d0640 1 -- start start 2023-12-02T00:31:45.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.353+0000 7f817c3d0640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f817415d330 0x7f81740b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:45.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.353+0000 7f817c3d0640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f81740d2910 0x7f81740d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:45.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.353+0000 7f817c3d0640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81740b2b10 con 0x7f81740d3730 2023-12-02T00:31:45.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.353+0000 7f817c3d0640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81740d3bc0 con 0x7f817415d330 2023-12-02T00:31:45.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.353+0000 7f817c3d0640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81740d1590 con 0x7f81740d2910 2023-12-02T00:31:45.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.353+0000 7f817a16c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f817415d330 0x7f81740b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:45.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.353+0000 7f817a16c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f817415d330 0x7f81740b8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48870/0 (socket says 172.21.15.46:48870) 2023-12-02T00:31:45.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.353+0000 7f817a16c640 1 -- 172.21.15.46:0/3178871617 learned_addr learned my addr 172.21.15.46:0/3178871617 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:45.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.354+0000 7f817996b640 1 --2- 172.21.15.46:0/3178871617 >> v2:172.21.15.162:3300/0 conn(0x7f81740d2910 0x7f81740d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:45.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.354+0000 7f817a16c640 1 -- 172.21.15.46:0/3178871617 >> v2:172.21.15.162:3300/0 conn(0x7f81740d2910 msgr2=0x7f81740d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:45.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.354+0000 7f817a16c640 1 --2- 172.21.15.46:0/3178871617 >> v2:172.21.15.162:3300/0 conn(0x7f81740d2910 0x7f81740d31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:45.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.354+0000 7f817a16c640 1 -- 172.21.15.46:0/3178871617 >> v1:172.21.15.162:6789/0 conn(0x7f81740d3730 legacy=0x7f81740b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:45.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.354+0000 7f817a16c640 1 -- 172.21.15.46:0/3178871617 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81740d1870 con 0x7f817415d330 2023-12-02T00:31:45.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.354+0000 7f817a16c640 1 --2- 172.21.15.46:0/3178871617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f817415d330 0x7f81740b8b80 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f81740d1540 tx=0x7f816c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cd82e06b60cab54 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:45.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.355+0000 7f817916a640 1 -- 172.21.15.46:0/3178871617 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816c0317e0 con 0x7f817415d330 2023-12-02T00:31:45.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.355+0000 7f817916a640 1 -- 172.21.15.46:0/3178871617 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f816c031940 con 0x7f817415d330 2023-12-02T00:31:45.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.355+0000 7f817916a640 1 -- 172.21.15.46:0/3178871617 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816c031cd0 con 0x7f817415d330 2023-12-02T00:31:45.362 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.355+0000 7f817c3d0640 1 -- 172.21.15.46:0/3178871617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f817415d330 msgr2=0x7f81740b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:45.362 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.355+0000 7f817c3d0640 1 --2- 172.21.15.46:0/3178871617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f817415d330 0x7f81740b8b80 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f81740d1540 tx=0x7f816c02b880 comp rx=0 tx=0).stop 2023-12-02T00:31:45.362 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.355+0000 7f817c3d0640 1 -- 172.21.15.46:0/3178871617 shutdown_connections 2023-12-02T00:31:45.362 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.355+0000 7f817c3d0640 1 --2- 172.21.15.46:0/3178871617 >> v2:172.21.15.162:3300/0 conn(0x7f81740d2910 0x7f81740d31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:45.362 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.355+0000 7f817c3d0640 1 --2- 172.21.15.46:0/3178871617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f817415d330 0x7f81740b8b80 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:45.362 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.355+0000 7f817c3d0640 1 -- 172.21.15.46:0/3178871617 >> 172.21.15.46:0/3178871617 conn(0x7f8174072020 msgr2=0x7f8174072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:45.362 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.355+0000 7f817c3d0640 1 -- 172.21.15.46:0/3178871617 shutdown_connections 2023-12-02T00:31:45.362 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.355+0000 7f817c3d0640 1 -- 172.21.15.46:0/3178871617 wait complete. 2023-12-02T00:31:45.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.356+0000 7f817c3d0640 1 Processor -- start 2023-12-02T00:31:45.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.357+0000 7f817c3d0640 1 -- start start 2023-12-02T00:31:45.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.357+0000 7f817c3d0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81740d2910 0x7f81741f6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:45.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.357+0000 7f817c3d0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81740d3730 0x7f81740c7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:45.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.357+0000 7f817c3d0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f817415d330 0x7f8174157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:45.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.357+0000 7f817c3d0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81740c80b0 con 0x7f81740d2910 2023-12-02T00:31:45.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.357+0000 7f817c3d0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81740c7590 con 0x7f81740d3730 2023-12-02T00:31:45.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.357+0000 7f817c3d0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81740c7810 con 0x7f817415d330 2023-12-02T00:31:45.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.357+0000 7f817a16c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81740d2910 0x7f81741f6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:45.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.357+0000 7f817a16c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81740d2910 0x7f81741f6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48884/0 (socket says 172.21.15.46:48884) 2023-12-02T00:31:45.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.357+0000 7f817a16c640 1 -- 172.21.15.46:0/3868598930 learned_addr learned my addr 172.21.15.46:0/3868598930 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:45.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.358+0000 7f817996b640 1 --2- 172.21.15.46:0/3868598930 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81740d3730 0x7f81740c7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:45.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.358+0000 7f817a96d640 1 --2- 172.21.15.46:0/3868598930 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f817415d330 0x7f8174157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:45.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.358+0000 7f817a16c640 1 -- 172.21.15.46:0/3868598930 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f817415d330 msgr2=0x7f8174157ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:45.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.358+0000 7f817a16c640 1 --2- 172.21.15.46:0/3868598930 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f817415d330 0x7f8174157ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:45.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.358+0000 7f817a16c640 1 -- 172.21.15.46:0/3868598930 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81740d3730 msgr2=0x7f81740c7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:45.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.358+0000 7f817a16c640 1 --2- 172.21.15.46:0/3868598930 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81740d3730 0x7f81740c7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:45.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.358+0000 7f817a16c640 1 -- 172.21.15.46:0/3868598930 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f816c0094d0 con 0x7f81740d2910 2023-12-02T00:31:45.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.358+0000 7f817996b640 1 --2- 172.21.15.46:0/3868598930 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81740d3730 0x7f81740c7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:31:45.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.358+0000 7f817a16c640 1 --2- 172.21.15.46:0/3868598930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81740d2910 0x7f81741f6f90 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f816c02b8c0 tx=0x7f816c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:45.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.358+0000 7f8172ffd640 1 -- 172.21.15.46:0/3868598930 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816c032880 con 0x7f81740d2910 2023-12-02T00:31:45.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.358+0000 7f8172ffd640 1 -- 172.21.15.46:0/3868598930 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f816c0329e0 con 0x7f81740d2910 2023-12-02T00:31:45.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.358+0000 7f817c3d0640 1 -- 172.21.15.46:0/3868598930 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81740c7af0 con 0x7f81740d2910 2023-12-02T00:31:45.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.358+0000 7f8172ffd640 1 -- 172.21.15.46:0/3868598930 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816c032d90 con 0x7f81740d2910 2023-12-02T00:31:45.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.358+0000 7f817c3d0640 1 -- 172.21.15.46:0/3868598930 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81740c8600 con 0x7f81740d2910 2023-12-02T00:31:45.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.359+0000 7f817c3d0640 1 -- 172.21.15.46:0/3868598930 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81740c8460 con 0x7f81740d2910 2023-12-02T00:31:45.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.360+0000 7f8172ffd640 1 -- 172.21.15.46:0/3868598930 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f816c04e450 con 0x7f81740d2910 2023-12-02T00:31:45.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.360+0000 7f8172ffd640 1 --2- 172.21.15.46:0/3868598930 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f816006ae40 0x7f816006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:45.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.360+0000 7f8172ffd640 1 -- 172.21.15.46:0/3868598930 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f816c0ab410 con 0x7f81740d2910 2023-12-02T00:31:45.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.360+0000 7f817996b640 1 --2- 172.21.15.46:0/3868598930 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f816006ae40 0x7f816006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:45.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.361+0000 7f817996b640 1 --2- 172.21.15.46:0/3868598930 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f816006ae40 0x7f816006d300 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f81740cd420 tx=0x7f8168008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:45.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.364+0000 7f8172ffd640 1 -- 172.21.15.46:0/3868598930 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f81740c8460 con 0x7f81740d2910 2023-12-02T00:31:45.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.543+0000 7f817c3d0640 1 -- 172.21.15.46:0/3868598930 --> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f817406bf20 con 0x7f816006ae40 2023-12-02T00:31:45.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.543+0000 7f8172ffd640 1 -- 172.21.15.46:0/3868598930 <== mgr.4108 v2:172.21.15.162:6832/3248644722 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f817406bf20 con 0x7f816006ae40 2023-12-02T00:31:45.544 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 2 to deep-scrub 2023-12-02T00:31:45.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.545+0000 7f817c3d0640 1 -- 172.21.15.46:0/3868598930 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f816006ae40 msgr2=0x7f816006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:45.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.545+0000 7f817c3d0640 1 --2- 172.21.15.46:0/3868598930 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f816006ae40 0x7f816006d300 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f81740cd420 tx=0x7f8168008040 comp rx=0 tx=0).stop 2023-12-02T00:31:45.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.545+0000 7f817c3d0640 1 -- 172.21.15.46:0/3868598930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81740d2910 msgr2=0x7f81741f6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:45.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.545+0000 7f817c3d0640 1 --2- 172.21.15.46:0/3868598930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81740d2910 0x7f81741f6f90 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f816c02b8c0 tx=0x7f816c0046c0 comp rx=0 tx=0).stop 2023-12-02T00:31:45.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.545+0000 7f817c3d0640 1 -- 172.21.15.46:0/3868598930 shutdown_connections 2023-12-02T00:31:45.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.545+0000 7f817c3d0640 1 --2- 172.21.15.46:0/3868598930 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f816006ae40 0x7f816006d300 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:45.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.545+0000 7f817c3d0640 1 --2- 172.21.15.46:0/3868598930 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f817415d330 0x7f8174157ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:45.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.545+0000 7f817c3d0640 1 --2- 172.21.15.46:0/3868598930 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81740d3730 0x7f81740c7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:45.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.545+0000 7f817c3d0640 1 --2- 172.21.15.46:0/3868598930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81740d2910 0x7f81741f6f90 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:45.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.545+0000 7f817c3d0640 1 -- 172.21.15.46:0/3868598930 >> 172.21.15.46:0/3868598930 conn(0x7f8174072020 msgr2=0x7f81740b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:45.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.545+0000 7f817c3d0640 1 -- 172.21.15.46:0/3868598930 shutdown_connections 2023-12-02T00:31:45.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.545+0000 7f817c3d0640 1 -- 172.21.15.46:0/3868598930 wait complete. 2023-12-02T00:31:45.557 INFO:tasks.ceph:Scrubbing osd.3 2023-12-02T00:31:45.557 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-12-02T00:31:45.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.666+0000 7f259c390640 1 Processor -- start 2023-12-02T00:31:45.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.666+0000 7f259c390640 1 -- start start 2023-12-02T00:31:45.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.667+0000 7f259c390640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f259415d580 0x7f25940b8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:45.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.667+0000 7f259c390640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2594160ed0 0x7f25940cded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:45.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.667+0000 7f259c390640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25940d1800 con 0x7f25940ce410 2023-12-02T00:31:45.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.667+0000 7f259c390640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25940ce8a0 con 0x7f2594160ed0 2023-12-02T00:31:45.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.667+0000 7f259c390640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25940b6330 con 0x7f259415d580 2023-12-02T00:31:45.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.667+0000 7f259992b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2594160ed0 0x7f25940cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:45.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.667+0000 7f259992b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2594160ed0 0x7f25940cded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48900/0 (socket says 172.21.15.46:48900) 2023-12-02T00:31:45.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.667+0000 7f259992b640 1 -- 172.21.15.46:0/899848601 learned_addr learned my addr 172.21.15.46:0/899848601 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:45.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.667+0000 7f259a12c640 1 --2- 172.21.15.46:0/899848601 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f259415d580 0x7f25940b8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:45.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.667+0000 7f259992b640 1 -- 172.21.15.46:0/899848601 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f259415d580 msgr2=0x7f25940b8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:45.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.667+0000 7f259992b640 1 --2- 172.21.15.46:0/899848601 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f259415d580 0x7f25940b8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:45.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.667+0000 7f259992b640 1 -- 172.21.15.46:0/899848601 >> v1:172.21.15.162:6789/0 conn(0x7f25940ce410 legacy=0x7f259406b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:45.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.667+0000 7f259992b640 1 -- 172.21.15.46:0/899848601 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2594161640 con 0x7f2594160ed0 2023-12-02T00:31:45.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.667+0000 7f259a12c640 1 --2- 172.21.15.46:0/899848601 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f259415d580 0x7f25940b8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:31:45.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.668+0000 7f259992b640 1 --2- 172.21.15.46:0/899848601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2594160ed0 0x7f25940cded0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f2584009870 tx=0x7f258402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fda8ba0329f965d7 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:45.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.668+0000 7f259912a640 1 -- 172.21.15.46:0/899848601 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2584031640 con 0x7f2594160ed0 2023-12-02T00:31:45.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.668+0000 7f259912a640 1 -- 172.21.15.46:0/899848601 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25840317a0 con 0x7f2594160ed0 2023-12-02T00:31:45.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.668+0000 7f259912a640 1 -- 172.21.15.46:0/899848601 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2584031b30 con 0x7f2594160ed0 2023-12-02T00:31:45.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.669+0000 7f259c390640 1 -- 172.21.15.46:0/899848601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2594160ed0 msgr2=0x7f25940cded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:45.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.669+0000 7f259c390640 1 --2- 172.21.15.46:0/899848601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2594160ed0 0x7f25940cded0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f2584009870 tx=0x7f258402b8f0 comp rx=0 tx=0).stop 2023-12-02T00:31:45.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.669+0000 7f259c390640 1 -- 172.21.15.46:0/899848601 shutdown_connections 2023-12-02T00:31:45.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.669+0000 7f259c390640 1 --2- 172.21.15.46:0/899848601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2594160ed0 0x7f25940cded0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:45.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.669+0000 7f259c390640 1 --2- 172.21.15.46:0/899848601 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f259415d580 0x7f25940b8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:45.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.669+0000 7f259c390640 1 -- 172.21.15.46:0/899848601 >> 172.21.15.46:0/899848601 conn(0x7f2594072200 msgr2=0x7f25940cfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:45.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.669+0000 7f259c390640 1 -- 172.21.15.46:0/899848601 shutdown_connections 2023-12-02T00:31:45.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.669+0000 7f259c390640 1 -- 172.21.15.46:0/899848601 wait complete. 2023-12-02T00:31:45.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.670+0000 7f259c390640 1 Processor -- start 2023-12-02T00:31:45.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.670+0000 7f259c390640 1 -- start start 2023-12-02T00:31:45.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.671+0000 7f259c390640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25940ce410 0x7f259415f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:45.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.671+0000 7f259c390640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f259415d580 0x7f25940ca540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:45.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.671+0000 7f259c390640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2594160ed0 0x7f2594151270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:45.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.671+0000 7f259c390640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25940c72c0 con 0x7f25940ce410 2023-12-02T00:31:45.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.671+0000 7f259c390640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259415fc30 con 0x7f2594160ed0 2023-12-02T00:31:45.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.671+0000 7f259c390640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f259415ff10 con 0x7f259415d580 2023-12-02T00:31:45.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.671+0000 7f259a12c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25940ce410 0x7f259415f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:45.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.671+0000 7f259a12c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25940ce410 0x7f259415f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48908/0 (socket says 172.21.15.46:48908) 2023-12-02T00:31:45.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.671+0000 7f259a12c640 1 -- 172.21.15.46:0/3380449047 learned_addr learned my addr 172.21.15.46:0/3380449047 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:45.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.671+0000 7f259992b640 1 --2- 172.21.15.46:0/3380449047 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f259415d580 0x7f25940ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:45.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.671+0000 7f259a92d640 1 --2- 172.21.15.46:0/3380449047 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2594160ed0 0x7f2594151270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:45.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.671+0000 7f259a12c640 1 -- 172.21.15.46:0/3380449047 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f259415d580 msgr2=0x7f25940ca540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:45.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.671+0000 7f259a12c640 1 --2- 172.21.15.46:0/3380449047 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f259415d580 0x7f25940ca540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:45.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.671+0000 7f259a12c640 1 -- 172.21.15.46:0/3380449047 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2594160ed0 msgr2=0x7f2594151270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:45.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.671+0000 7f259a12c640 1 --2- 172.21.15.46:0/3380449047 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2594160ed0 0x7f2594151270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:45.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.671+0000 7f259a12c640 1 -- 172.21.15.46:0/3380449047 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25840094d0 con 0x7f25940ce410 2023-12-02T00:31:45.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.672+0000 7f259a12c640 1 --2- 172.21.15.46:0/3380449047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25940ce410 0x7f259415f3c0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f259000c7f0 tx=0x7f259000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:45.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.672+0000 7f258affd640 1 -- 172.21.15.46:0/3380449047 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2590007a30 con 0x7f25940ce410 2023-12-02T00:31:45.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.672+0000 7f258affd640 1 -- 172.21.15.46:0/3380449047 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2590007b90 con 0x7f25940ce410 2023-12-02T00:31:45.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.672+0000 7f259c390640 1 -- 172.21.15.46:0/3380449047 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25941601f0 con 0x7f25940ce410 2023-12-02T00:31:45.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.672+0000 7f258affd640 1 -- 172.21.15.46:0/3380449047 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259000ce40 con 0x7f25940ce410 2023-12-02T00:31:45.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.672+0000 7f259c390640 1 -- 172.21.15.46:0/3380449047 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f259415f900 con 0x7f25940ce410 2023-12-02T00:31:45.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.673+0000 7f259c390640 1 -- 172.21.15.46:0/3380449047 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f25940cfa40 con 0x7f25940ce410 2023-12-02T00:31:45.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.673+0000 7f258affd640 1 -- 172.21.15.46:0/3380449047 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f25900113f0 con 0x7f25940ce410 2023-12-02T00:31:45.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.674+0000 7f258affd640 1 --2- 172.21.15.46:0/3380449047 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f257806ae40 0x7f257806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:45.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.674+0000 7f258affd640 1 -- 172.21.15.46:0/3380449047 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2590089540 con 0x7f25940ce410 2023-12-02T00:31:45.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.674+0000 7f258affd640 1 --2- 172.21.15.46:0/3380449047 >> [v2:172.21.15.46:6816/1799228239,v1:172.21.15.46:6817/1799228239] conn(0x7f2578070ca0 0x7f25780730c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:45.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.674+0000 7f259992b640 1 --2- 172.21.15.46:0/3380449047 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f257806ae40 0x7f257806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:45.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.674+0000 7f258affd640 1 -- 172.21.15.46:0/3380449047 --> [v2:172.21.15.46:6816/1799228239,v1:172.21.15.46:6817/1799228239] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2578073770 con 0x7f2578070ca0 2023-12-02T00:31:45.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.674+0000 7f258affd640 1 -- 172.21.15.46:0/3380449047 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f259008bc00 con 0x7f25940ce410 2023-12-02T00:31:45.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.674+0000 7f259a92d640 1 --2- 172.21.15.46:0/3380449047 >> [v2:172.21.15.46:6816/1799228239,v1:172.21.15.46:6817/1799228239] conn(0x7f2578070ca0 0x7f25780730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:45.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.674+0000 7f259a92d640 1 --2- 172.21.15.46:0/3380449047 >> [v2:172.21.15.46:6816/1799228239,v1:172.21.15.46:6817/1799228239] conn(0x7f2578070ca0 0x7f25780730c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:45.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.674+0000 7f259992b640 1 --2- 172.21.15.46:0/3380449047 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f257806ae40 0x7f257806d300 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2584036000 tx=0x7f258403c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:45.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.675+0000 7f258affd640 1 -- 172.21.15.46:0/3380449047 <== osd.3 v2:172.21.15.46:6816/1799228239 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f2578073770 con 0x7f2578070ca0 2023-12-02T00:31:45.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.688+0000 7f259c390640 1 -- 172.21.15.46:0/3380449047 --> [v2:172.21.15.46:6816/1799228239,v1:172.21.15.46:6817/1799228239] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2594161330 con 0x7f2578070ca0 2023-12-02T00:31:45.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.688+0000 7f258affd640 1 -- 172.21.15.46:0/3380449047 <== osd.3 v2:172.21.15.46:6816/1799228239 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f258c005650 con 0x7f2578070ca0 2023-12-02T00:31:45.690 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-12-02T00:31:45.690 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-12-02T00:31:45.690 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-12-02T00:31:45.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.689+0000 7f259c390640 1 -- 172.21.15.46:0/3380449047 >> [v2:172.21.15.46:6816/1799228239,v1:172.21.15.46:6817/1799228239] conn(0x7f2578070ca0 msgr2=0x7f25780730c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:45.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.689+0000 7f259c390640 1 --2- 172.21.15.46:0/3380449047 >> [v2:172.21.15.46:6816/1799228239,v1:172.21.15.46:6817/1799228239] conn(0x7f2578070ca0 0x7f25780730c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:45.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.689+0000 7f259c390640 1 -- 172.21.15.46:0/3380449047 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f257806ae40 msgr2=0x7f257806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:45.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.689+0000 7f259c390640 1 --2- 172.21.15.46:0/3380449047 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f257806ae40 0x7f257806d300 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2584036000 tx=0x7f258403c040 comp rx=0 tx=0).stop 2023-12-02T00:31:45.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.689+0000 7f259c390640 1 -- 172.21.15.46:0/3380449047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25940ce410 msgr2=0x7f259415f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:45.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.689+0000 7f259c390640 1 --2- 172.21.15.46:0/3380449047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25940ce410 0x7f259415f3c0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f259000c7f0 tx=0x7f259000cbf0 comp rx=0 tx=0).stop 2023-12-02T00:31:45.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.689+0000 7f259a92d640 1 -- 172.21.15.46:0/3380449047 reap_dead start 2023-12-02T00:31:45.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.689+0000 7f259c390640 1 -- 172.21.15.46:0/3380449047 shutdown_connections 2023-12-02T00:31:45.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.689+0000 7f259c390640 1 -- 172.21.15.46:0/3380449047 >> 172.21.15.46:0/3380449047 conn(0x7f2594072200 msgr2=0x7f25940b6470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:45.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.689+0000 7f259c390640 1 -- 172.21.15.46:0/3380449047 shutdown_connections 2023-12-02T00:31:45.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.689+0000 7f259c390640 1 -- 172.21.15.46:0/3380449047 wait complete. 2023-12-02T00:31:45.700 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-12-02T00:31:45.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.807+0000 7f100ad1d640 1 Processor -- start 2023-12-02T00:31:45.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.807+0000 7f100ad1d640 1 -- start start 2023-12-02T00:31:45.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.807+0000 7f100ad1d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f10040d2910 0x7f10040d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:45.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.807+0000 7f100ad1d640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10040b2b10 con 0x7f10040d3730 2023-12-02T00:31:45.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.807+0000 7f100ad1d640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10040d1590 con 0x7f100415d330 2023-12-02T00:31:45.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.807+0000 7f100ad1d640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10040d3bc0 con 0x7f10040d2910 2023-12-02T00:31:45.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.808+0000 7f10092ba640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f10040d3730 0x7f10040b6390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49062/0 (socket says 172.21.15.46:49062) 2023-12-02T00:31:45.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.808+0000 7f10092ba640 1 -- 172.21.15.46:0/2934936243 learned_addr learned my addr 172.21.15.46:0/2934936243 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:31:45.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.808+0000 7f1003fff640 1 --2- 172.21.15.46:0/2934936243 >> v2:172.21.15.162:3300/0 conn(0x7f10040d2910 0x7f10040d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:45.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.808+0000 7f10037fe640 1 -- 172.21.15.46:0/2934936243 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2139765102 0 0) 0x7f10040b2b10 con 0x7f10040d3730 2023-12-02T00:31:45.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.808+0000 7f10037fe640 1 -- 172.21.15.46:0/2934936243 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ff4003520 con 0x7f10040d3730 2023-12-02T00:31:45.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.808+0000 7f10037fe640 1 -- 172.21.15.46:0/2934936243 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1153568562 0 0) 0x7f0ff4003520 con 0x7f10040d3730 2023-12-02T00:31:45.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.808+0000 7f10037fe640 1 -- 172.21.15.46:0/2934936243 >> v2:172.21.15.162:3300/0 conn(0x7f10040d2910 msgr2=0x7f10040d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:45.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.808+0000 7f10037fe640 1 --2- 172.21.15.46:0/2934936243 >> v2:172.21.15.162:3300/0 conn(0x7f10040d2910 0x7f10040d31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:45.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.808+0000 7f10037fe640 1 -- 172.21.15.46:0/2934936243 >> v1:172.21.15.162:6789/0 conn(0x7f100415d330 legacy=0x7f10040b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:45.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.808+0000 7f10037fe640 1 -- 172.21.15.46:0/2934936243 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10040d1870 con 0x7f10040d3730 2023-12-02T00:31:45.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.808+0000 7f10037fe640 1 -- 172.21.15.46:0/2934936243 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2793898007 0 0) 0x7f0ffc0019f0 con 0x7f10040d3730 2023-12-02T00:31:45.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.808+0000 7f10037fe640 1 -- 172.21.15.46:0/2934936243 >> v1:172.21.15.46:6789/0 conn(0x7f10040d3730 legacy=0x7f10040b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:45.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.808+0000 7f10037fe640 1 --2- 172.21.15.46:0/2934936243 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ff4003970 0x7f0ff4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:45.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.808+0000 7f10037fe640 1 --2- 172.21.15.46:0/2934936243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff40047c0 0x7f0ff4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:45.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.808+0000 7f10037fe640 1 --2- 172.21.15.46:0/2934936243 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ff4005180 0x7f0ff400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:45.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.808+0000 7f10037fe640 1 -- 172.21.15.46:0/2934936243 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10040d3bc0 con 0x7f0ff40047c0 2023-12-02T00:31:45.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.808+0000 7f10037fe640 1 -- 172.21.15.46:0/2934936243 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff400c120 con 0x7f0ff4003970 2023-12-02T00:31:45.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.808+0000 7f10037fe640 1 -- 172.21.15.46:0/2934936243 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff400c2e0 con 0x7f0ff4005180 2023-12-02T00:31:45.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.808+0000 7f1003fff640 1 --2- 172.21.15.46:0/2934936243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff40047c0 0x7f0ff4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:45.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.808+0000 7f1008ab9640 1 --2- 172.21.15.46:0/2934936243 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ff4003970 0x7f0ff4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:45.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.809+0000 7f10092ba640 1 --2- 172.21.15.46:0/2934936243 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ff4005180 0x7f0ff400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:45.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.809+0000 7f1003fff640 1 -- 172.21.15.46:0/2934936243 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ff4005180 msgr2=0x7f0ff400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:45.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.809+0000 7f1003fff640 1 --2- 172.21.15.46:0/2934936243 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ff4005180 0x7f0ff400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:45.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.809+0000 7f1003fff640 1 -- 172.21.15.46:0/2934936243 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ff4003970 msgr2=0x7f0ff4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:45.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.809+0000 7f1003fff640 1 --2- 172.21.15.46:0/2934936243 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ff4003970 0x7f0ff4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:45.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.809+0000 7f1003fff640 1 -- 172.21.15.46:0/2934936243 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff4003520 con 0x7f0ff40047c0 2023-12-02T00:31:45.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.809+0000 7f10092ba640 1 -- 172.21.15.46:0/2934936243 reap_dead start 2023-12-02T00:31:45.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.809+0000 7f1003fff640 1 --2- 172.21.15.46:0/2934936243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff40047c0 0x7f0ff4004c40 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0009ac0 tx=0x7f0ff40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a7e135e8c5c47d3 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:45.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.810+0000 7f10037fe640 1 -- 172.21.15.46:0/2934936243 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff002c6b0 con 0x7f0ff40047c0 2023-12-02T00:31:45.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.810+0000 7f10037fe640 1 -- 172.21.15.46:0/2934936243 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ff002c810 con 0x7f0ff40047c0 2023-12-02T00:31:45.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.810+0000 7f10037fe640 1 -- 172.21.15.46:0/2934936243 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff002cba0 con 0x7f0ff40047c0 2023-12-02T00:31:45.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.810+0000 7f100ad1d640 1 -- 172.21.15.46:0/2934936243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff40047c0 msgr2=0x7f0ff4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:45.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.810+0000 7f100ad1d640 1 --2- 172.21.15.46:0/2934936243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff40047c0 0x7f0ff4004c40 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0009ac0 tx=0x7f0ff40057f0 comp rx=0 tx=0).stop 2023-12-02T00:31:45.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.810+0000 7f100ad1d640 1 -- 172.21.15.46:0/2934936243 shutdown_connections 2023-12-02T00:31:45.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.810+0000 7f100ad1d640 1 --2- 172.21.15.46:0/2934936243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff40047c0 0x7f0ff4004c40 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:45.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.810+0000 7f100ad1d640 1 -- 172.21.15.46:0/2934936243 >> 172.21.15.46:0/2934936243 conn(0x7f1004072020 msgr2=0x7f1004072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:45.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.810+0000 7f100ad1d640 1 -- 172.21.15.46:0/2934936243 shutdown_connections 2023-12-02T00:31:45.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.810+0000 7f100ad1d640 1 -- 172.21.15.46:0/2934936243 wait complete. 2023-12-02T00:31:45.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.811+0000 7f100ad1d640 1 Processor -- start 2023-12-02T00:31:45.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.811+0000 7f100ad1d640 1 -- start start 2023-12-02T00:31:45.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.812+0000 7f100ad1d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff40047c0 0x7f1004153920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:45.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.812+0000 7f100ad1d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1004155280 0x7f10041f66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:45.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.812+0000 7f100ad1d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10041f6c20 0x7f10041f70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:45.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.812+0000 7f100ad1d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1004153e60 con 0x7f0ff40047c0 2023-12-02T00:31:45.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.812+0000 7f100ad1d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1004156620 con 0x7f10041f6c20 2023-12-02T00:31:45.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.812+0000 7f100ad1d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1004143480 con 0x7f1004155280 2023-12-02T00:31:45.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.812+0000 7f1008ab9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff40047c0 0x7f1004153920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:45.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.812+0000 7f1008ab9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff40047c0 0x7f1004153920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48932/0 (socket says 172.21.15.46:48932) 2023-12-02T00:31:45.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.812+0000 7f1008ab9640 1 -- 172.21.15.46:0/1055432475 learned_addr learned my addr 172.21.15.46:0/1055432475 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:45.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.813+0000 7f10092ba640 1 --2- 172.21.15.46:0/1055432475 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10041f6c20 0x7f10041f70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:45.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.812+0000 7f1003fff640 1 --2- 172.21.15.46:0/1055432475 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1004155280 0x7f10041f66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:45.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.813+0000 7f1008ab9640 1 -- 172.21.15.46:0/1055432475 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1004155280 msgr2=0x7f10041f66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:45.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.813+0000 7f1008ab9640 1 --2- 172.21.15.46:0/1055432475 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1004155280 0x7f10041f66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:45.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.813+0000 7f1008ab9640 1 -- 172.21.15.46:0/1055432475 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10041f6c20 msgr2=0x7f10041f70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:45.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.813+0000 7f1008ab9640 1 --2- 172.21.15.46:0/1055432475 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10041f6c20 0x7f10041f70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:45.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.813+0000 7f1008ab9640 1 -- 172.21.15.46:0/1055432475 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1004155a10 con 0x7f0ff40047c0 2023-12-02T00:31:45.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.813+0000 7f10092ba640 1 --2- 172.21.15.46:0/1055432475 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10041f6c20 0x7f10041f70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:31:45.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.813+0000 7f1008ab9640 1 --2- 172.21.15.46:0/1055432475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff40047c0 0x7f1004153920 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8007810 tx=0x7f0ff800fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:45.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.813+0000 7f10017fa640 1 -- 172.21.15.46:0/1055432475 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff80050d0 con 0x7f0ff40047c0 2023-12-02T00:31:45.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.813+0000 7f10017fa640 1 -- 172.21.15.46:0/1055432475 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ff8005230 con 0x7f0ff40047c0 2023-12-02T00:31:45.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.813+0000 7f100ad1d640 1 -- 172.21.15.46:0/1055432475 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1004155cf0 con 0x7f0ff40047c0 2023-12-02T00:31:45.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.813+0000 7f10017fa640 1 -- 172.21.15.46:0/1055432475 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff800d730 con 0x7f0ff40047c0 2023-12-02T00:31:45.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.813+0000 7f100ad1d640 1 -- 172.21.15.46:0/1055432475 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10041562b0 con 0x7f0ff40047c0 2023-12-02T00:31:45.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.814+0000 7f100ad1d640 1 -- 172.21.15.46:0/1055432475 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1004069400 con 0x7f0ff40047c0 2023-12-02T00:31:45.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.814+0000 7f10017fa640 1 -- 172.21.15.46:0/1055432475 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0ff800d890 con 0x7f0ff40047c0 2023-12-02T00:31:45.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.815+0000 7f10017fa640 1 --2- 172.21.15.46:0/1055432475 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f0fe806ae40 0x7f0fe806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:45.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.815+0000 7f10017fa640 1 -- 172.21.15.46:0/1055432475 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0ff8089090 con 0x7f0ff40047c0 2023-12-02T00:31:45.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.815+0000 7f1003fff640 1 --2- 172.21.15.46:0/1055432475 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f0fe806ae40 0x7f0fe806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:45.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.816+0000 7f1003fff640 1 --2- 172.21.15.46:0/1055432475 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f0fe806ae40 0x7f0fe806d300 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0036000 tx=0x7f0ff0030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:45.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.819+0000 7f10017fa640 1 -- 172.21.15.46:0/1055432475 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0ff80544d0 con 0x7f0ff40047c0 2023-12-02T00:31:45.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.996+0000 7f100ad1d640 1 -- 172.21.15.46:0/1055432475 --> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f1004156470 con 0x7f0fe806ae40 2023-12-02T00:31:45.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.996+0000 7f10017fa640 1 -- 172.21.15.46:0/1055432475 <== mgr.4108 v2:172.21.15.162:6832/3248644722 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f1004156470 con 0x7f0fe806ae40 2023-12-02T00:31:45.998 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 3 to deep-scrub 2023-12-02T00:31:46.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.999+0000 7f100ad1d640 1 -- 172.21.15.46:0/1055432475 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f0fe806ae40 msgr2=0x7f0fe806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:46.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.999+0000 7f100ad1d640 1 --2- 172.21.15.46:0/1055432475 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f0fe806ae40 0x7f0fe806d300 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0036000 tx=0x7f0ff0030000 comp rx=0 tx=0).stop 2023-12-02T00:31:46.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.999+0000 7f100ad1d640 1 -- 172.21.15.46:0/1055432475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff40047c0 msgr2=0x7f1004153920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:46.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.999+0000 7f100ad1d640 1 --2- 172.21.15.46:0/1055432475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff40047c0 0x7f1004153920 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8007810 tx=0x7f0ff800fc00 comp rx=0 tx=0).stop 2023-12-02T00:31:46.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.999+0000 7f100ad1d640 1 -- 172.21.15.46:0/1055432475 shutdown_connections 2023-12-02T00:31:46.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.999+0000 7f100ad1d640 1 --2- 172.21.15.46:0/1055432475 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f0fe806ae40 0x7f0fe806d300 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:46.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.999+0000 7f100ad1d640 1 --2- 172.21.15.46:0/1055432475 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10041f6c20 0x7f10041f70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:46.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.999+0000 7f100ad1d640 1 --2- 172.21.15.46:0/1055432475 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1004155280 0x7f10041f66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:46.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.999+0000 7f100ad1d640 1 --2- 172.21.15.46:0/1055432475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff40047c0 0x7f1004153920 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:46.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.999+0000 7f100ad1d640 1 -- 172.21.15.46:0/1055432475 >> 172.21.15.46:0/1055432475 conn(0x7f1004072020 msgr2=0x7f10040c6ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:46.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.999+0000 7f100ad1d640 1 -- 172.21.15.46:0/1055432475 shutdown_connections 2023-12-02T00:31:46.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:45.999+0000 7f100ad1d640 1 -- 172.21.15.46:0/1055432475 wait complete. 2023-12-02T00:31:46.012 INFO:tasks.ceph:Scrubbing osd.4 2023-12-02T00:31:46.012 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-12-02T00:31:46.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.121+0000 7f1a4c6a2640 1 Processor -- start 2023-12-02T00:31:46.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.121+0000 7f1a4c6a2640 1 -- start start 2023-12-02T00:31:46.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.122+0000 7f1a4c6a2640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1a440b3030 0x7f1a440b3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:46.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.122+0000 7f1a4c6a2640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a440b4560 con 0x7f1a440b11f0 2023-12-02T00:31:46.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.122+0000 7f1a4c6a2640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a440b4840 con 0x7f1a440b0d60 2023-12-02T00:31:46.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.122+0000 7f1a4c6a2640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a440b4b20 con 0x7f1a440b3030 2023-12-02T00:31:46.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.122+0000 7f1a4a43e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1a440b3030 0x7f1a440b3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:46.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.122+0000 7f1a4ac3f640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1a440b11f0 0x7f1a4415a5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49068/0 (socket says 172.21.15.46:49068) 2023-12-02T00:31:46.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.122+0000 7f1a4a43e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1a440b3030 0x7f1a440b3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48940/0 (socket says 172.21.15.46:48940) 2023-12-02T00:31:46.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.122+0000 7f1a4ac3f640 1 -- 172.21.15.46:0/1408427887 learned_addr learned my addr 172.21.15.46:0/1408427887 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:31:46.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.122+0000 7f1a4a43e640 1 -- 172.21.15.46:0/1408427887 >> v1:172.21.15.162:6789/0 conn(0x7f1a440b0d60 legacy=0x7f1a44159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:46.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.122+0000 7f1a4a43e640 1 -- 172.21.15.46:0/1408427887 >> v1:172.21.15.46:6789/0 conn(0x7f1a440b11f0 legacy=0x7f1a4415a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:46.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.122+0000 7f1a4a43e640 1 -- 172.21.15.46:0/1408427887 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a440b4e00 con 0x7f1a440b3030 2023-12-02T00:31:46.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.122+0000 7f1a4943c640 1 -- 172.21.15.46:0/1408427887 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3586881184 0 0) 0x7f1a440b4560 con 0x7f1a440b11f0 2023-12-02T00:31:46.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.123+0000 7f1a4a43e640 1 --2- 172.21.15.46:0/1408427887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a440b3030 0x7f1a440b3e60 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c0097a0 tx=0x7f1a3c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53a45db349774cc8 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:46.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.123+0000 7f1a4943c640 1 -- 172.21.15.46:0/1408427887 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c031830 con 0x7f1a440b3030 2023-12-02T00:31:46.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.123+0000 7f1a4943c640 1 -- 172.21.15.46:0/1408427887 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a3c031990 con 0x7f1a440b3030 2023-12-02T00:31:46.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.123+0000 7f1a4943c640 1 -- 172.21.15.46:0/1408427887 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c031d20 con 0x7f1a440b3030 2023-12-02T00:31:46.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.124+0000 7f1a4c6a2640 1 -- 172.21.15.46:0/1408427887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a440b3030 msgr2=0x7f1a440b3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:46.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.124+0000 7f1a4c6a2640 1 --2- 172.21.15.46:0/1408427887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a440b3030 0x7f1a440b3e60 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c0097a0 tx=0x7f1a3c02b8d0 comp rx=0 tx=0).stop 2023-12-02T00:31:46.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.124+0000 7f1a4c6a2640 1 -- 172.21.15.46:0/1408427887 shutdown_connections 2023-12-02T00:31:46.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.124+0000 7f1a4c6a2640 1 --2- 172.21.15.46:0/1408427887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a440b3030 0x7f1a440b3e60 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:46.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.124+0000 7f1a4c6a2640 1 -- 172.21.15.46:0/1408427887 >> 172.21.15.46:0/1408427887 conn(0x7f1a4406a4d0 msgr2=0x7f1a4406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:46.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.124+0000 7f1a4c6a2640 1 -- 172.21.15.46:0/1408427887 shutdown_connections 2023-12-02T00:31:46.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.124+0000 7f1a4c6a2640 1 -- 172.21.15.46:0/1408427887 wait complete. 2023-12-02T00:31:46.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.125+0000 7f1a4c6a2640 1 Processor -- start 2023-12-02T00:31:46.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.125+0000 7f1a4c6a2640 1 -- start start 2023-12-02T00:31:46.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.126+0000 7f1a4c6a2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a440b0d60 0x7f1a440ac0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:46.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.126+0000 7f1a4c6a2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a440b11f0 0x7f1a4414da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:46.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.126+0000 7f1a4c6a2640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a440b3030 0x7f1a440aa3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:46.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.126+0000 7f1a4c6a2640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a441f1b10 con 0x7f1a440b0d60 2023-12-02T00:31:46.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.126+0000 7f1a4c6a2640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a441f0f90 con 0x7f1a440b3030 2023-12-02T00:31:46.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.126+0000 7f1a4c6a2640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a441f1270 con 0x7f1a440b11f0 2023-12-02T00:31:46.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.126+0000 7f1a4a43e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a440b0d60 0x7f1a440ac0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:46.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.126+0000 7f1a4a43e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a440b0d60 0x7f1a440ac0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48954/0 (socket says 172.21.15.46:48954) 2023-12-02T00:31:46.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.126+0000 7f1a4a43e640 1 -- 172.21.15.46:0/828315309 learned_addr learned my addr 172.21.15.46:0/828315309 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:46.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.126+0000 7f1a49c3d640 1 --2- 172.21.15.46:0/828315309 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a440b11f0 0x7f1a4414da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:46.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.126+0000 7f1a4ac3f640 1 --2- 172.21.15.46:0/828315309 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a440b3030 0x7f1a440aa3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:46.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.126+0000 7f1a4a43e640 1 -- 172.21.15.46:0/828315309 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a440b11f0 msgr2=0x7f1a4414da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:46.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.126+0000 7f1a4a43e640 1 --2- 172.21.15.46:0/828315309 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a440b11f0 0x7f1a4414da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:46.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.126+0000 7f1a4a43e640 1 -- 172.21.15.46:0/828315309 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a440b3030 msgr2=0x7f1a440aa3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:46.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.126+0000 7f1a4a43e640 1 --2- 172.21.15.46:0/828315309 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a440b3030 0x7f1a440aa3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:46.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.126+0000 7f1a4a43e640 1 -- 172.21.15.46:0/828315309 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a3c009400 con 0x7f1a440b0d60 2023-12-02T00:31:46.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.126+0000 7f1a49c3d640 1 --2- 172.21.15.46:0/828315309 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a440b11f0 0x7f1a4414da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:31:46.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.126+0000 7f1a4ac3f640 1 --2- 172.21.15.46:0/828315309 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a440b3030 0x7f1a440aa3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:31:46.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.127+0000 7f1a4a43e640 1 --2- 172.21.15.46:0/828315309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a440b0d60 0x7f1a440ac0d0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c000c00 tx=0x7f1a3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:46.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.127+0000 7f1a3affd640 1 -- 172.21.15.46:0/828315309 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c03aa60 con 0x7f1a440b0d60 2023-12-02T00:31:46.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.127+0000 7f1a3affd640 1 -- 172.21.15.46:0/828315309 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a3c03abc0 con 0x7f1a440b0d60 2023-12-02T00:31:46.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.127+0000 7f1a4c6a2640 1 -- 172.21.15.46:0/828315309 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a441f1550 con 0x7f1a440b0d60 2023-12-02T00:31:46.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.127+0000 7f1a3affd640 1 -- 172.21.15.46:0/828315309 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c0443f0 con 0x7f1a440b0d60 2023-12-02T00:31:46.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.127+0000 7f1a4c6a2640 1 -- 172.21.15.46:0/828315309 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a440ac610 con 0x7f1a440b0d60 2023-12-02T00:31:46.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.127+0000 7f1a4c6a2640 1 -- 172.21.15.46:0/828315309 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1a440b3410 con 0x7f1a440b0d60 2023-12-02T00:31:46.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.128+0000 7f1a3affd640 1 -- 172.21.15.46:0/828315309 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1a3c04d450 con 0x7f1a440b0d60 2023-12-02T00:31:46.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.129+0000 7f1a3affd640 1 --2- 172.21.15.46:0/828315309 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f1a2806ae40 0x7f1a2806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:46.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.129+0000 7f1a3affd640 1 -- 172.21.15.46:0/828315309 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1a3c0abb50 con 0x7f1a440b0d60 2023-12-02T00:31:46.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.129+0000 7f1a3affd640 1 --2- 172.21.15.46:0/828315309 >> [v2:172.21.15.162:6808/4132149803,v1:172.21.15.162:6809/4132149803] conn(0x7f1a28070ca0 0x7f1a280730c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:46.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.129+0000 7f1a3affd640 1 -- 172.21.15.46:0/828315309 --> [v2:172.21.15.162:6808/4132149803,v1:172.21.15.162:6809/4132149803] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1a28073770 con 0x7f1a28070ca0 2023-12-02T00:31:46.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.129+0000 7f1a3affd640 1 -- 172.21.15.46:0/828315309 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a3c0abfa0 con 0x7f1a440b0d60 2023-12-02T00:31:46.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.129+0000 7f1a49c3d640 1 --2- 172.21.15.46:0/828315309 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f1a2806ae40 0x7f1a2806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:46.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.129+0000 7f1a4ac3f640 1 --2- 172.21.15.46:0/828315309 >> [v2:172.21.15.162:6808/4132149803,v1:172.21.15.162:6809/4132149803] conn(0x7f1a28070ca0 0x7f1a280730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:46.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.130+0000 7f1a49c3d640 1 --2- 172.21.15.46:0/828315309 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f1a2806ae40 0x7f1a2806d300 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1a440b0260 tx=0x7f1a34009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:46.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.130+0000 7f1a4ac3f640 1 --2- 172.21.15.46:0/828315309 >> [v2:172.21.15.162:6808/4132149803,v1:172.21.15.162:6809/4132149803] conn(0x7f1a28070ca0 0x7f1a280730c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:46.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.130+0000 7f1a3affd640 1 -- 172.21.15.46:0/828315309 <== osd.4 v2:172.21.15.162:6808/4132149803 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f1a28073770 con 0x7f1a28070ca0 2023-12-02T00:31:46.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.143+0000 7f1a4c6a2640 1 -- 172.21.15.46:0/828315309 --> [v2:172.21.15.162:6808/4132149803,v1:172.21.15.162:6809/4132149803] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1a44069400 con 0x7f1a28070ca0 2023-12-02T00:31:46.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.144+0000 7f1a3affd640 1 -- 172.21.15.46:0/828315309 <== osd.4 v2:172.21.15.162:6808/4132149803 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f1a44069400 con 0x7f1a28070ca0 2023-12-02T00:31:46.145 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-12-02T00:31:46.145 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-12-02T00:31:46.146 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-12-02T00:31:46.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.144+0000 7f1a4c6a2640 1 -- 172.21.15.46:0/828315309 >> [v2:172.21.15.162:6808/4132149803,v1:172.21.15.162:6809/4132149803] conn(0x7f1a28070ca0 msgr2=0x7f1a280730c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:46.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.144+0000 7f1a4c6a2640 1 --2- 172.21.15.46:0/828315309 >> [v2:172.21.15.162:6808/4132149803,v1:172.21.15.162:6809/4132149803] conn(0x7f1a28070ca0 0x7f1a280730c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:46.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.144+0000 7f1a4c6a2640 1 -- 172.21.15.46:0/828315309 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f1a2806ae40 msgr2=0x7f1a2806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:46.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.144+0000 7f1a4c6a2640 1 --2- 172.21.15.46:0/828315309 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f1a2806ae40 0x7f1a2806d300 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1a440b0260 tx=0x7f1a34009040 comp rx=0 tx=0).stop 2023-12-02T00:31:46.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.144+0000 7f1a4c6a2640 1 -- 172.21.15.46:0/828315309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a440b0d60 msgr2=0x7f1a440ac0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:46.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.144+0000 7f1a4c6a2640 1 --2- 172.21.15.46:0/828315309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a440b0d60 0x7f1a440ac0d0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c000c00 tx=0x7f1a3c0046c0 comp rx=0 tx=0).stop 2023-12-02T00:31:46.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.144+0000 7f1a4ac3f640 1 -- 172.21.15.46:0/828315309 reap_dead start 2023-12-02T00:31:46.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.144+0000 7f1a4c6a2640 1 -- 172.21.15.46:0/828315309 shutdown_connections 2023-12-02T00:31:46.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.144+0000 7f1a4c6a2640 1 -- 172.21.15.46:0/828315309 >> 172.21.15.46:0/828315309 conn(0x7f1a4406a4d0 msgr2=0x7f1a44151d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:46.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.144+0000 7f1a4c6a2640 1 -- 172.21.15.46:0/828315309 shutdown_connections 2023-12-02T00:31:46.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.144+0000 7f1a4c6a2640 1 -- 172.21.15.46:0/828315309 wait complete. 2023-12-02T00:31:46.157 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-12-02T00:31:46.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.264+0000 7f9ecf3b6640 1 Processor -- start 2023-12-02T00:31:46.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.264+0000 7f9ecf3b6640 1 -- start start 2023-12-02T00:31:46.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.265+0000 7f9ecf3b6640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9ec815d1d0 0x7f9ec806b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:46.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.265+0000 7f9ecf3b6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9ec80ad640 0x7f9ec806ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:46.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.265+0000 7f9ecf3b6640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ec80adae0 con 0x7f9ec80aeb80 2023-12-02T00:31:46.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.265+0000 7f9ecf3b6640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ec80addc0 con 0x7f9ec815d1d0 2023-12-02T00:31:46.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.265+0000 7f9ecf3b6640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec80ae0a0 con 0x7f9ec80ad640 2023-12-02T00:31:46.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.265+0000 7f9ecd152640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9ec815d1d0 0x7f9ec806b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:46.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.265+0000 7f9ecd953640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9ec80aeb80 0x7f9ec8160ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49080/0 (socket says 172.21.15.46:49080) 2023-12-02T00:31:46.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.265+0000 7f9ecd953640 1 -- 172.21.15.46:0/1785635737 learned_addr learned my addr 172.21.15.46:0/1785635737 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:31:46.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.265+0000 7f9ecd152640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9ec815d1d0 0x7f9ec806b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48958/0 (socket says 172.21.15.46:48958) 2023-12-02T00:31:46.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.265+0000 7f9ecc951640 1 --2- 172.21.15.46:0/1785635737 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9ec80ad640 0x7f9ec806ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:46.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.265+0000 7f9ecd152640 1 -- 172.21.15.46:0/1785635737 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9ec80ad640 msgr2=0x7f9ec806ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:46.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.265+0000 7f9ecd152640 1 --2- 172.21.15.46:0/1785635737 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9ec80ad640 0x7f9ec806ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:46.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.265+0000 7f9ecd152640 1 -- 172.21.15.46:0/1785635737 >> v1:172.21.15.46:6789/0 conn(0x7f9ec80aeb80 legacy=0x7f9ec8160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:46.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.265+0000 7f9ebffff640 1 -- 172.21.15.46:0/1785635737 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 169979472 0 0) 0x7f9ec80adae0 con 0x7f9ec80aeb80 2023-12-02T00:31:46.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.265+0000 7f9ecd152640 1 -- 172.21.15.46:0/1785635737 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec80ae380 con 0x7f9ec815d1d0 2023-12-02T00:31:46.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.266+0000 7f9ecd152640 1 --2- 172.21.15.46:0/1785635737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ec815d1d0 0x7f9ec806b4e0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f9ec40097a0 tx=0x7f9ec402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1626bcf83a211ce server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:46.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.266+0000 7f9ebffff640 1 -- 172.21.15.46:0/1785635737 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4031830 con 0x7f9ec815d1d0 2023-12-02T00:31:46.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.266+0000 7f9ebffff640 1 -- 172.21.15.46:0/1785635737 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec4031990 con 0x7f9ec815d1d0 2023-12-02T00:31:46.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.266+0000 7f9ebffff640 1 -- 172.21.15.46:0/1785635737 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4031d20 con 0x7f9ec815d1d0 2023-12-02T00:31:46.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.267+0000 7f9ecf3b6640 1 -- 172.21.15.46:0/1785635737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ec815d1d0 msgr2=0x7f9ec806b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:46.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.267+0000 7f9ecf3b6640 1 --2- 172.21.15.46:0/1785635737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ec815d1d0 0x7f9ec806b4e0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f9ec40097a0 tx=0x7f9ec402b8d0 comp rx=0 tx=0).stop 2023-12-02T00:31:46.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.267+0000 7f9ecf3b6640 1 -- 172.21.15.46:0/1785635737 shutdown_connections 2023-12-02T00:31:46.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.267+0000 7f9ecf3b6640 1 --2- 172.21.15.46:0/1785635737 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9ec80ad640 0x7f9ec806ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:46.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.267+0000 7f9ecf3b6640 1 --2- 172.21.15.46:0/1785635737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ec815d1d0 0x7f9ec806b4e0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:46.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.267+0000 7f9ecf3b6640 1 -- 172.21.15.46:0/1785635737 >> 172.21.15.46:0/1785635737 conn(0x7f9ec8070e70 msgr2=0x7f9ec8071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:46.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.267+0000 7f9ecf3b6640 1 -- 172.21.15.46:0/1785635737 shutdown_connections 2023-12-02T00:31:46.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.267+0000 7f9ecf3b6640 1 -- 172.21.15.46:0/1785635737 wait complete. 2023-12-02T00:31:46.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.268+0000 7f9ecf3b6640 1 Processor -- start 2023-12-02T00:31:46.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.268+0000 7f9ecf3b6640 1 -- start start 2023-12-02T00:31:46.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.269+0000 7f9ecf3b6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9ec80ad640 0x7f9ec8152950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:46.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.269+0000 7f9ecf3b6640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ec80aeb80 0x7f9ec80cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:46.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.269+0000 7f9ecf3b6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ec815d1d0 0x7f9ec80ce430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:46.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.269+0000 7f9ecf3b6640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec80ca900 con 0x7f9ec815d1d0 2023-12-02T00:31:46.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.269+0000 7f9ecf3b6640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec80cab80 con 0x7f9ec80aeb80 2023-12-02T00:31:46.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.269+0000 7f9ecf3b6640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec80cae60 con 0x7f9ec80ad640 2023-12-02T00:31:46.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.269+0000 7f9ecd953640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ec815d1d0 0x7f9ec80ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:46.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.269+0000 7f9ecd953640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ec815d1d0 0x7f9ec80ce430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48970/0 (socket says 172.21.15.46:48970) 2023-12-02T00:31:46.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.269+0000 7f9ecd953640 1 -- 172.21.15.46:0/4010726296 learned_addr learned my addr 172.21.15.46:0/4010726296 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:46.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.269+0000 7f9ecd152640 1 --2- 172.21.15.46:0/4010726296 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9ec80ad640 0x7f9ec8152950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:46.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.269+0000 7f9ecc951640 1 --2- 172.21.15.46:0/4010726296 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ec80aeb80 0x7f9ec80cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:46.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.269+0000 7f9ecd953640 1 -- 172.21.15.46:0/4010726296 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9ec80ad640 msgr2=0x7f9ec8152950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:46.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.269+0000 7f9ecd953640 1 --2- 172.21.15.46:0/4010726296 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9ec80ad640 0x7f9ec8152950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:46.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.269+0000 7f9ecd953640 1 -- 172.21.15.46:0/4010726296 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ec80aeb80 msgr2=0x7f9ec80cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:46.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.269+0000 7f9ecd953640 1 --2- 172.21.15.46:0/4010726296 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ec80aeb80 0x7f9ec80cdef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:46.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.269+0000 7f9ecd953640 1 -- 172.21.15.46:0/4010726296 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec4009400 con 0x7f9ec815d1d0 2023-12-02T00:31:46.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.269+0000 7f9ecd953640 1 --2- 172.21.15.46:0/4010726296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ec815d1d0 0x7f9ec80ce430 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f9ec000c7e0 tx=0x7f9ec000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:46.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.269+0000 7f9ebdffb640 1 -- 172.21.15.46:0/4010726296 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec000a640 con 0x7f9ec815d1d0 2023-12-02T00:31:46.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.269+0000 7f9ebdffb640 1 -- 172.21.15.46:0/4010726296 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec000a7c0 con 0x7f9ec815d1d0 2023-12-02T00:31:46.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.269+0000 7f9ecf3b6640 1 -- 172.21.15.46:0/4010726296 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ec80cb140 con 0x7f9ec815d1d0 2023-12-02T00:31:46.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.269+0000 7f9ebdffb640 1 -- 172.21.15.46:0/4010726296 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec000ab70 con 0x7f9ec815d1d0 2023-12-02T00:31:46.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.270+0000 7f9ecf3b6640 1 -- 172.21.15.46:0/4010726296 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ec80cb700 con 0x7f9ec815d1d0 2023-12-02T00:31:46.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.271+0000 7f9ecf3b6640 1 -- 172.21.15.46:0/4010726296 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ec80cb8f0 con 0x7f9ec815d1d0 2023-12-02T00:31:46.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.271+0000 7f9ebdffb640 1 -- 172.21.15.46:0/4010726296 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9ec000acd0 con 0x7f9ec815d1d0 2023-12-02T00:31:46.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.272+0000 7f9ebdffb640 1 --2- 172.21.15.46:0/4010726296 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9eac06ae40 0x7f9eac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:46.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.272+0000 7f9ebdffb640 1 -- 172.21.15.46:0/4010726296 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9ec0088c70 con 0x7f9ec815d1d0 2023-12-02T00:31:46.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.272+0000 7f9ecd152640 1 --2- 172.21.15.46:0/4010726296 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9eac06ae40 0x7f9eac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:46.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.272+0000 7f9ecd152640 1 --2- 172.21.15.46:0/4010726296 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9eac06ae40 0x7f9eac06d300 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4036000 tx=0x7f9ec403c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:46.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.276+0000 7f9ebdffb640 1 -- 172.21.15.46:0/4010726296 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9ec0053f80 con 0x7f9ec815d1d0 2023-12-02T00:31:46.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.454+0000 7f9ecf3b6640 1 -- 172.21.15.46:0/4010726296 --> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ec80b2b10 con 0x7f9eac06ae40 2023-12-02T00:31:46.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.455+0000 7f9ebdffb640 1 -- 172.21.15.46:0/4010726296 <== mgr.4108 v2:172.21.15.162:6832/3248644722 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f9ec4031b00 con 0x7f9eac06ae40 2023-12-02T00:31:46.456 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 4 to deep-scrub 2023-12-02T00:31:46.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.456+0000 7f9ecf3b6640 1 -- 172.21.15.46:0/4010726296 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9eac06ae40 msgr2=0x7f9eac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:46.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.456+0000 7f9ecf3b6640 1 --2- 172.21.15.46:0/4010726296 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9eac06ae40 0x7f9eac06d300 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4036000 tx=0x7f9ec403c040 comp rx=0 tx=0).stop 2023-12-02T00:31:46.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.456+0000 7f9ecf3b6640 1 -- 172.21.15.46:0/4010726296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ec815d1d0 msgr2=0x7f9ec80ce430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:46.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.456+0000 7f9ecf3b6640 1 --2- 172.21.15.46:0/4010726296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ec815d1d0 0x7f9ec80ce430 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f9ec000c7e0 tx=0x7f9ec000cca0 comp rx=0 tx=0).stop 2023-12-02T00:31:46.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.456+0000 7f9ecf3b6640 1 -- 172.21.15.46:0/4010726296 shutdown_connections 2023-12-02T00:31:46.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.456+0000 7f9ecf3b6640 1 --2- 172.21.15.46:0/4010726296 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9eac06ae40 0x7f9eac06d300 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:46.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.456+0000 7f9ecf3b6640 1 --2- 172.21.15.46:0/4010726296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ec815d1d0 0x7f9ec80ce430 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:46.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.456+0000 7f9ecf3b6640 1 --2- 172.21.15.46:0/4010726296 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ec80aeb80 0x7f9ec80cdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:46.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.456+0000 7f9ecf3b6640 1 --2- 172.21.15.46:0/4010726296 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9ec80ad640 0x7f9ec8152950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:46.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.456+0000 7f9ecf3b6640 1 -- 172.21.15.46:0/4010726296 >> 172.21.15.46:0/4010726296 conn(0x7f9ec8070e70 msgr2=0x7f9ec806a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:46.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.457+0000 7f9ecf3b6640 1 -- 172.21.15.46:0/4010726296 shutdown_connections 2023-12-02T00:31:46.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.457+0000 7f9ecf3b6640 1 -- 172.21.15.46:0/4010726296 wait complete. 2023-12-02T00:31:46.468 INFO:tasks.ceph:Scrubbing osd.5 2023-12-02T00:31:46.469 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-12-02T00:31:46.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.578+0000 7f8a95c36640 1 Processor -- start 2023-12-02T00:31:46.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.578+0000 7f8a95c36640 1 -- start start 2023-12-02T00:31:46.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.579+0000 7f8a95c36640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8a90140310 0x7f8a90161840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:46.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.579+0000 7f8a95c36640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a90161d80 con 0x7f8a90142900 2023-12-02T00:31:46.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.579+0000 7f8a95c36640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a901406f0 con 0x7f8a9006abe0 2023-12-02T00:31:46.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.579+0000 7f8a95c36640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a90143960 con 0x7f8a90140310 2023-12-02T00:31:46.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.579+0000 7f8a8f7fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8a90140310 0x7f8a90161840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:46.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.579+0000 7f8a8effd640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8a90142900 0x7f8a9006a4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49094/0 (socket says 172.21.15.46:49094) 2023-12-02T00:31:46.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.579+0000 7f8a8effd640 1 -- 172.21.15.46:0/2768542888 learned_addr learned my addr 172.21.15.46:0/2768542888 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:31:46.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.579+0000 7f8a8f7fe640 1 -- 172.21.15.46:0/2768542888 >> v1:172.21.15.162:6789/0 conn(0x7f8a9006abe0 legacy=0x7f8a9006eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:46.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.579+0000 7f8a8f7fe640 1 -- 172.21.15.46:0/2768542888 >> v1:172.21.15.46:6789/0 conn(0x7f8a90142900 legacy=0x7f8a9006a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:46.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.579+0000 7f8a8f7fe640 1 -- 172.21.15.46:0/2768542888 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a90142de0 con 0x7f8a90140310 2023-12-02T00:31:46.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.579+0000 7f8a8e7fc640 1 -- 172.21.15.46:0/2768542888 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3518297097 0 0) 0x7f8a90161d80 con 0x7f8a90142900 2023-12-02T00:31:46.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.580+0000 7f8a8f7fe640 1 --2- 172.21.15.46:0/2768542888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a90140310 0x7f8a90161840 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f8a80009720 tx=0x7f8a8002b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2af239be5d91dbeb server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:46.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.580+0000 7f8a8e7fc640 1 -- 172.21.15.46:0/2768542888 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a80030710 con 0x7f8a90140310 2023-12-02T00:31:46.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.580+0000 7f8a8e7fc640 1 -- 172.21.15.46:0/2768542888 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a8002e050 con 0x7f8a90140310 2023-12-02T00:31:46.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.580+0000 7f8a8e7fc640 1 -- 172.21.15.46:0/2768542888 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a80030a90 con 0x7f8a90140310 2023-12-02T00:31:46.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.581+0000 7f8a95c36640 1 -- 172.21.15.46:0/2768542888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a90140310 msgr2=0x7f8a90161840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:46.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.581+0000 7f8a95c36640 1 --2- 172.21.15.46:0/2768542888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a90140310 0x7f8a90161840 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f8a80009720 tx=0x7f8a8002b6a0 comp rx=0 tx=0).stop 2023-12-02T00:31:46.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.581+0000 7f8a95c36640 1 -- 172.21.15.46:0/2768542888 shutdown_connections 2023-12-02T00:31:46.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.581+0000 7f8a95c36640 1 --2- 172.21.15.46:0/2768542888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a90140310 0x7f8a90161840 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:46.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.581+0000 7f8a95c36640 1 -- 172.21.15.46:0/2768542888 >> 172.21.15.46:0/2768542888 conn(0x7f8a900b1e50 msgr2=0x7f8a900b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:46.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.581+0000 7f8a95c36640 1 -- 172.21.15.46:0/2768542888 shutdown_connections 2023-12-02T00:31:46.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.581+0000 7f8a95c36640 1 -- 172.21.15.46:0/2768542888 wait complete. 2023-12-02T00:31:46.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.582+0000 7f8a95c36640 1 Processor -- start 2023-12-02T00:31:46.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.582+0000 7f8a95c36640 1 -- start start 2023-12-02T00:31:46.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.583+0000 7f8a95c36640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8a9006abe0 0x7f8a901f5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:46.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.583+0000 7f8a95c36640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a90140310 0x7f8a901f3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:46.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.583+0000 7f8a95c36640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a90142900 0x7f8a9015a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:46.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.583+0000 7f8a95c36640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a901f4420 con 0x7f8a90140310 2023-12-02T00:31:46.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.583+0000 7f8a95c36640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a901f3ca0 con 0x7f8a90142900 2023-12-02T00:31:46.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.583+0000 7f8a95c36640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a90157f70 con 0x7f8a9006abe0 2023-12-02T00:31:46.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.583+0000 7f8a8effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a90140310 0x7f8a901f3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:46.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.583+0000 7f8a8effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a90140310 0x7f8a901f3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48986/0 (socket says 172.21.15.46:48986) 2023-12-02T00:31:46.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.583+0000 7f8a8effd640 1 -- 172.21.15.46:0/1070664295 learned_addr learned my addr 172.21.15.46:0/1070664295 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:46.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.583+0000 7f8a8effd640 1 -- 172.21.15.46:0/1070664295 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8a9006abe0 msgr2=0x7f8a901f5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:46.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.583+0000 7f8a8effd640 1 --2- 172.21.15.46:0/1070664295 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8a9006abe0 0x7f8a901f5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:46.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.583+0000 7f8a8effd640 1 -- 172.21.15.46:0/1070664295 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a90142900 msgr2=0x7f8a9015a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:46.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.583+0000 7f8a8ffff640 1 --2- 172.21.15.46:0/1070664295 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a90142900 0x7f8a9015a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:46.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.583+0000 7f8a8effd640 1 --2- 172.21.15.46:0/1070664295 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a90142900 0x7f8a9015a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:46.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.583+0000 7f8a8effd640 1 -- 172.21.15.46:0/1070664295 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a80009400 con 0x7f8a90140310 2023-12-02T00:31:46.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.584+0000 7f8a8effd640 1 --2- 172.21.15.46:0/1070664295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a90140310 0x7f8a901f3ee0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f8a7800c450 tx=0x7f8a7800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:46.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.584+0000 7f8a87fff640 1 -- 172.21.15.46:0/1070664295 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a780074e0 con 0x7f8a90140310 2023-12-02T00:31:46.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.584+0000 7f8a87fff640 1 -- 172.21.15.46:0/1070664295 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a78007640 con 0x7f8a90140310 2023-12-02T00:31:46.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.584+0000 7f8a95c36640 1 -- 172.21.15.46:0/1070664295 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a90158250 con 0x7f8a90140310 2023-12-02T00:31:46.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.584+0000 7f8a87fff640 1 -- 172.21.15.46:0/1070664295 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a78004a20 con 0x7f8a90140310 2023-12-02T00:31:46.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.584+0000 7f8a95c36640 1 -- 172.21.15.46:0/1070664295 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a90158810 con 0x7f8a90140310 2023-12-02T00:31:46.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.585+0000 7f8a95c36640 1 -- 172.21.15.46:0/1070664295 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8a90143680 con 0x7f8a90140310 2023-12-02T00:31:46.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.585+0000 7f8a87fff640 1 -- 172.21.15.46:0/1070664295 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8a78004b80 con 0x7f8a90140310 2023-12-02T00:31:46.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.585+0000 7f8a87fff640 1 --2- 172.21.15.46:0/1070664295 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8a7006ae40 0x7f8a7006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:46.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.586+0000 7f8a87fff640 1 -- 172.21.15.46:0/1070664295 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8a78088490 con 0x7f8a90140310 2023-12-02T00:31:46.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.586+0000 7f8a87fff640 1 --2- 172.21.15.46:0/1070664295 >> [v2:172.21.15.162:6816/3463332362,v1:172.21.15.162:6817/3463332362] conn(0x7f8a70070ca0 0x7f8a700730c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:46.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.586+0000 7f8a87fff640 1 -- 172.21.15.46:0/1070664295 --> [v2:172.21.15.162:6816/3463332362,v1:172.21.15.162:6817/3463332362] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8a70073770 con 0x7f8a70070ca0 2023-12-02T00:31:46.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.586+0000 7f8a87fff640 1 -- 172.21.15.46:0/1070664295 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a7808ab50 con 0x7f8a90140310 2023-12-02T00:31:46.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.586+0000 7f8a8f7fe640 1 --2- 172.21.15.46:0/1070664295 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8a7006ae40 0x7f8a7006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:46.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.586+0000 7f8a8ffff640 1 --2- 172.21.15.46:0/1070664295 >> [v2:172.21.15.162:6816/3463332362,v1:172.21.15.162:6817/3463332362] conn(0x7f8a70070ca0 0x7f8a700730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:46.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.586+0000 7f8a8f7fe640 1 --2- 172.21.15.46:0/1070664295 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8a7006ae40 0x7f8a7006d300 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8a80034000 tx=0x7f8a8002f000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:46.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.587+0000 7f8a8ffff640 1 --2- 172.21.15.46:0/1070664295 >> [v2:172.21.15.162:6816/3463332362,v1:172.21.15.162:6817/3463332362] conn(0x7f8a70070ca0 0x7f8a700730c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:46.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.587+0000 7f8a87fff640 1 -- 172.21.15.46:0/1070664295 <== osd.5 v2:172.21.15.162:6816/3463332362 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f8a70073770 con 0x7f8a70070ca0 2023-12-02T00:31:46.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.600+0000 7f8a95c36640 1 -- 172.21.15.46:0/1070664295 --> [v2:172.21.15.162:6816/3463332362,v1:172.21.15.162:6817/3463332362] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f8a901580e0 con 0x7f8a70070ca0 2023-12-02T00:31:46.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.601+0000 7f8a87fff640 1 -- 172.21.15.46:0/1070664295 <== osd.5 v2:172.21.15.162:6816/3463332362 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f8a901580e0 con 0x7f8a70070ca0 2023-12-02T00:31:46.603 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-12-02T00:31:46.603 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-12-02T00:31:46.603 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-12-02T00:31:46.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.601+0000 7f8a95c36640 1 -- 172.21.15.46:0/1070664295 >> [v2:172.21.15.162:6816/3463332362,v1:172.21.15.162:6817/3463332362] conn(0x7f8a70070ca0 msgr2=0x7f8a700730c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:46.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.601+0000 7f8a95c36640 1 --2- 172.21.15.46:0/1070664295 >> [v2:172.21.15.162:6816/3463332362,v1:172.21.15.162:6817/3463332362] conn(0x7f8a70070ca0 0x7f8a700730c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:46.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.601+0000 7f8a95c36640 1 -- 172.21.15.46:0/1070664295 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8a7006ae40 msgr2=0x7f8a7006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:46.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.601+0000 7f8a95c36640 1 --2- 172.21.15.46:0/1070664295 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f8a7006ae40 0x7f8a7006d300 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8a80034000 tx=0x7f8a8002f000 comp rx=0 tx=0).stop 2023-12-02T00:31:46.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.601+0000 7f8a95c36640 1 -- 172.21.15.46:0/1070664295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a90140310 msgr2=0x7f8a901f3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:46.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.601+0000 7f8a95c36640 1 --2- 172.21.15.46:0/1070664295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a90140310 0x7f8a901f3ee0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f8a7800c450 tx=0x7f8a7800c910 comp rx=0 tx=0).stop 2023-12-02T00:31:46.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.602+0000 7f8a8ffff640 1 -- 172.21.15.46:0/1070664295 reap_dead start 2023-12-02T00:31:46.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.602+0000 7f8a95c36640 1 -- 172.21.15.46:0/1070664295 shutdown_connections 2023-12-02T00:31:46.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.602+0000 7f8a95c36640 1 -- 172.21.15.46:0/1070664295 >> 172.21.15.46:0/1070664295 conn(0x7f8a900b1e50 msgr2=0x7f8a90160160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:46.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.602+0000 7f8a95c36640 1 -- 172.21.15.46:0/1070664295 shutdown_connections 2023-12-02T00:31:46.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.602+0000 7f8a95c36640 1 -- 172.21.15.46:0/1070664295 wait complete. 2023-12-02T00:31:46.613 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-12-02T00:31:46.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.719+0000 7f56f302b640 1 Processor -- start 2023-12-02T00:31:46.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.719+0000 7f56f302b640 1 -- start start 2023-12-02T00:31:46.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.720+0000 7f56f302b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f56ec06bbf0 0x7f56ec06d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:46.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.720+0000 7f56f302b640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56ec0adae0 con 0x7f56ec0ad640 2023-12-02T00:31:46.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.720+0000 7f56f302b640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56ec0addc0 con 0x7f56ec15d1d0 2023-12-02T00:31:46.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.720+0000 7f56f302b640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56ec0ae0a0 con 0x7f56ec06bbf0 2023-12-02T00:31:46.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.720+0000 7f56f15c8640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f56ec06bbf0 0x7f56ec06d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:46.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.720+0000 7f56f15c8640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f56ec06bbf0 0x7f56ec06d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49002/0 (socket says 172.21.15.46:49002) 2023-12-02T00:31:46.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.720+0000 7f56f15c8640 1 -- 172.21.15.46:0/1099575719 learned_addr learned my addr 172.21.15.46:0/1099575719 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:46.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.720+0000 7f56ebfff640 1 --1- 172.21.15.46:0/1099575719 >> v1:172.21.15.46:6789/0 conn(0x7f56ec0ad640 0x7f56ec06d0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49106/0 (socket says 172.21.15.46:49106) 2023-12-02T00:31:46.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.720+0000 7f56f15c8640 1 -- 172.21.15.46:0/1099575719 >> v1:172.21.15.162:6789/0 conn(0x7f56ec15d1d0 legacy=0x7f56ec06b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:46.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.720+0000 7f56f15c8640 1 -- 172.21.15.46:0/1099575719 >> v1:172.21.15.46:6789/0 conn(0x7f56ec0ad640 legacy=0x7f56ec06d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:46.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.720+0000 7f56f15c8640 1 -- 172.21.15.46:0/1099575719 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56ec0ae380 con 0x7f56ec06bbf0 2023-12-02T00:31:46.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.721+0000 7f56f15c8640 1 --2- 172.21.15.46:0/1099575719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56ec06bbf0 0x7f56ec06d7c0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f56e4009e50 tx=0x7f56e402c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=121e4147b8270b0d server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:46.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.721+0000 7f56eb7fe640 1 -- 172.21.15.46:0/1099575719 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e402cbb0 con 0x7f56ec06bbf0 2023-12-02T00:31:46.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.721+0000 7f56eb7fe640 1 -- 172.21.15.46:0/1099575719 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e402cd10 con 0x7f56ec06bbf0 2023-12-02T00:31:46.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.721+0000 7f56eb7fe640 1 -- 172.21.15.46:0/1099575719 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e4034610 con 0x7f56ec06bbf0 2023-12-02T00:31:46.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.722+0000 7f56f302b640 1 -- 172.21.15.46:0/1099575719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56ec06bbf0 msgr2=0x7f56ec06d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:46.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.722+0000 7f56f302b640 1 --2- 172.21.15.46:0/1099575719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56ec06bbf0 0x7f56ec06d7c0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f56e4009e50 tx=0x7f56e402c010 comp rx=0 tx=0).stop 2023-12-02T00:31:46.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.722+0000 7f56f302b640 1 -- 172.21.15.46:0/1099575719 shutdown_connections 2023-12-02T00:31:46.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.722+0000 7f56f302b640 1 --2- 172.21.15.46:0/1099575719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56ec06bbf0 0x7f56ec06d7c0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:46.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.722+0000 7f56f302b640 1 -- 172.21.15.46:0/1099575719 >> 172.21.15.46:0/1099575719 conn(0x7f56ec070e70 msgr2=0x7f56ec071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:46.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.722+0000 7f56f302b640 1 -- 172.21.15.46:0/1099575719 shutdown_connections 2023-12-02T00:31:46.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.722+0000 7f56f302b640 1 -- 172.21.15.46:0/1099575719 wait complete. 2023-12-02T00:31:46.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.723+0000 7f56f302b640 1 Processor -- start 2023-12-02T00:31:46.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.723+0000 7f56f302b640 1 -- start start 2023-12-02T00:31:46.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.724+0000 7f56f302b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56ec06bbf0 0x7f56ec1453a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:46.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.724+0000 7f56f302b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56ec0ad640 0x7f56ec157240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:46.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.724+0000 7f56f302b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56ec15d1d0 0x7f56ec14f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:46.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.724+0000 7f56f302b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec1458e0 con 0x7f56ec15d1d0 2023-12-02T00:31:46.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.724+0000 7f56f302b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec158900 con 0x7f56ec0ad640 2023-12-02T00:31:46.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.724+0000 7f56f302b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec157d00 con 0x7f56ec06bbf0 2023-12-02T00:31:46.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.724+0000 7f56f15c8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56ec15d1d0 0x7f56ec14f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:46.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.724+0000 7f56f15c8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56ec15d1d0 0x7f56ec14f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49010/0 (socket says 172.21.15.46:49010) 2023-12-02T00:31:46.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.724+0000 7f56f15c8640 1 -- 172.21.15.46:0/1834715137 learned_addr learned my addr 172.21.15.46:0/1834715137 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:46.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.724+0000 7f56f0dc7640 1 --2- 172.21.15.46:0/1834715137 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56ec06bbf0 0x7f56ec1453a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:46.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.724+0000 7f56ebfff640 1 --2- 172.21.15.46:0/1834715137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56ec0ad640 0x7f56ec157240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:46.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.724+0000 7f56f15c8640 1 -- 172.21.15.46:0/1834715137 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56ec06bbf0 msgr2=0x7f56ec1453a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:46.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.724+0000 7f56f15c8640 1 --2- 172.21.15.46:0/1834715137 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56ec06bbf0 0x7f56ec1453a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:46.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.724+0000 7f56f15c8640 1 -- 172.21.15.46:0/1834715137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56ec0ad640 msgr2=0x7f56ec157240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:46.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.724+0000 7f56f15c8640 1 --2- 172.21.15.46:0/1834715137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56ec0ad640 0x7f56ec157240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:46.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.724+0000 7f56f15c8640 1 -- 172.21.15.46:0/1834715137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e4009b30 con 0x7f56ec15d1d0 2023-12-02T00:31:46.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.724+0000 7f56ebfff640 1 --2- 172.21.15.46:0/1834715137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56ec0ad640 0x7f56ec157240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:31:46.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.724+0000 7f56f15c8640 1 --2- 172.21.15.46:0/1834715137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56ec15d1d0 0x7f56ec14f6f0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f56e4009e50 tx=0x7f56e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:46.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.725+0000 7f56e97fa640 1 -- 172.21.15.46:0/1834715137 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e4009510 con 0x7f56ec15d1d0 2023-12-02T00:31:46.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.725+0000 7f56e97fa640 1 -- 172.21.15.46:0/1834715137 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e403e450 con 0x7f56ec15d1d0 2023-12-02T00:31:46.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.725+0000 7f56f302b640 1 -- 172.21.15.46:0/1834715137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56ec157fe0 con 0x7f56ec15d1d0 2023-12-02T00:31:46.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.725+0000 7f56e97fa640 1 -- 172.21.15.46:0/1834715137 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e403e800 con 0x7f56ec15d1d0 2023-12-02T00:31:46.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.725+0000 7f56f302b640 1 -- 172.21.15.46:0/1834715137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56ec0cf300 con 0x7f56ec15d1d0 2023-12-02T00:31:46.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.726+0000 7f56f302b640 1 -- 172.21.15.46:0/1834715137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56ec1586d0 con 0x7f56ec15d1d0 2023-12-02T00:31:46.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.726+0000 7f56e97fa640 1 -- 172.21.15.46:0/1834715137 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f56e403e960 con 0x7f56ec15d1d0 2023-12-02T00:31:46.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.726+0000 7f56e97fa640 1 --2- 172.21.15.46:0/1834715137 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f56d006ae40 0x7f56d006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:46.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.727+0000 7f56e97fa640 1 -- 172.21.15.46:0/1834715137 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f56e40ac140 con 0x7f56ec15d1d0 2023-12-02T00:31:46.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.727+0000 7f56f0dc7640 1 --2- 172.21.15.46:0/1834715137 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f56d006ae40 0x7f56d006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:46.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.727+0000 7f56f0dc7640 1 --2- 172.21.15.46:0/1834715137 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f56d006ae40 0x7f56d006d300 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f56e0000f60 tx=0x7f56e0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:46.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.731+0000 7f56e97fa640 1 -- 172.21.15.46:0/1834715137 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f56ec1586d0 con 0x7f56ec15d1d0 2023-12-02T00:31:46.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.908+0000 7f56f302b640 1 -- 172.21.15.46:0/1834715137 --> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f56ec06dd00 con 0x7f56d006ae40 2023-12-02T00:31:46.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.909+0000 7f56e97fa640 1 -- 172.21.15.46:0/1834715137 <== mgr.4108 v2:172.21.15.162:6832/3248644722 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f56e00050d0 con 0x7f56d006ae40 2023-12-02T00:31:46.910 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 5 to deep-scrub 2023-12-02T00:31:46.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.910+0000 7f56f302b640 1 -- 172.21.15.46:0/1834715137 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f56d006ae40 msgr2=0x7f56d006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:46.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.910+0000 7f56f302b640 1 --2- 172.21.15.46:0/1834715137 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f56d006ae40 0x7f56d006d300 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f56e0000f60 tx=0x7f56e0009040 comp rx=0 tx=0).stop 2023-12-02T00:31:46.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.910+0000 7f56f302b640 1 -- 172.21.15.46:0/1834715137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56ec15d1d0 msgr2=0x7f56ec14f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:46.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.910+0000 7f56f302b640 1 --2- 172.21.15.46:0/1834715137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56ec15d1d0 0x7f56ec14f6f0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f56e4009e50 tx=0x7f56e4004a40 comp rx=0 tx=0).stop 2023-12-02T00:31:46.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.910+0000 7f56f302b640 1 -- 172.21.15.46:0/1834715137 shutdown_connections 2023-12-02T00:31:46.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.910+0000 7f56f302b640 1 --2- 172.21.15.46:0/1834715137 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f56d006ae40 0x7f56d006d300 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:46.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.911+0000 7f56f302b640 1 --2- 172.21.15.46:0/1834715137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56ec15d1d0 0x7f56ec14f6f0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:46.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.911+0000 7f56f302b640 1 --2- 172.21.15.46:0/1834715137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56ec0ad640 0x7f56ec157240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:46.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.911+0000 7f56f302b640 1 --2- 172.21.15.46:0/1834715137 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56ec06bbf0 0x7f56ec1453a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:46.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.911+0000 7f56f302b640 1 -- 172.21.15.46:0/1834715137 >> 172.21.15.46:0/1834715137 conn(0x7f56ec070e70 msgr2=0x7f56ec06d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:46.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.911+0000 7f56f302b640 1 -- 172.21.15.46:0/1834715137 shutdown_connections 2023-12-02T00:31:46.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:46.911+0000 7f56f302b640 1 -- 172.21.15.46:0/1834715137 wait complete. 2023-12-02T00:31:46.922 INFO:tasks.ceph:Scrubbing osd.6 2023-12-02T00:31:46.923 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-12-02T00:31:47.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.032+0000 7f5053d51640 1 Processor -- start 2023-12-02T00:31:47.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.032+0000 7f5053d51640 1 -- start start 2023-12-02T00:31:47.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.033+0000 7f5053d51640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f504c15d580 0x7f504c0b8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:47.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.033+0000 7f5053d51640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f504c160ed0 0x7f504c0cded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:47.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.033+0000 7f5053d51640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f504c0d1800 con 0x7f504c0ce410 2023-12-02T00:31:47.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.033+0000 7f5053d51640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f504c0b6330 con 0x7f504c15d580 2023-12-02T00:31:47.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.033+0000 7f5053d51640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f504c0ce880 con 0x7f504c160ed0 2023-12-02T00:31:47.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.033+0000 7f5051aed640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f504c15d580 0x7f504c0b8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:47.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.033+0000 7f50522ee640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f504c0ce410 0x7f504c06b570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49114/0 (socket says 172.21.15.46:49114) 2023-12-02T00:31:47.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.033+0000 7f50522ee640 1 -- 172.21.15.46:0/333854438 learned_addr learned my addr 172.21.15.46:0/333854438 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:31:47.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.034+0000 7f50512ec640 1 --2- 172.21.15.46:0/333854438 >> v2:172.21.15.162:3300/0 conn(0x7f504c160ed0 0x7f504c0cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:47.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.034+0000 7f5051aed640 1 -- 172.21.15.46:0/333854438 >> v2:172.21.15.162:3300/0 conn(0x7f504c160ed0 msgr2=0x7f504c0cded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:47.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.034+0000 7f5051aed640 1 --2- 172.21.15.46:0/333854438 >> v2:172.21.15.162:3300/0 conn(0x7f504c160ed0 0x7f504c0cded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:47.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.034+0000 7f5051aed640 1 -- 172.21.15.46:0/333854438 >> v1:172.21.15.46:6789/0 conn(0x7f504c0ce410 legacy=0x7f504c06b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:47.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.034+0000 7f5051aed640 1 -- 172.21.15.46:0/333854438 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f504c161640 con 0x7f504c15d580 2023-12-02T00:31:47.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.034+0000 7f5050aeb640 1 -- 172.21.15.46:0/333854438 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 612740225 0 0) 0x7f504c0d1800 con 0x7f504c0ce410 2023-12-02T00:31:47.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.034+0000 7f5051aed640 1 --2- 172.21.15.46:0/333854438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f504c15d580 0x7f504c0b8d40 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f504c072be0 tx=0x7f504402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48dcd55010feea76 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:47.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.035+0000 7f5050aeb640 1 -- 172.21.15.46:0/333854438 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5044031830 con 0x7f504c15d580 2023-12-02T00:31:47.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.035+0000 7f5050aeb640 1 -- 172.21.15.46:0/333854438 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5044031990 con 0x7f504c15d580 2023-12-02T00:31:47.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.035+0000 7f5050aeb640 1 -- 172.21.15.46:0/333854438 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5044031d20 con 0x7f504c15d580 2023-12-02T00:31:47.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.035+0000 7f5053d51640 1 -- 172.21.15.46:0/333854438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f504c15d580 msgr2=0x7f504c0b8d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:47.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.035+0000 7f5053d51640 1 --2- 172.21.15.46:0/333854438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f504c15d580 0x7f504c0b8d40 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f504c072be0 tx=0x7f504402b8d0 comp rx=0 tx=0).stop 2023-12-02T00:31:47.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.035+0000 7f5053d51640 1 -- 172.21.15.46:0/333854438 shutdown_connections 2023-12-02T00:31:47.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.035+0000 7f5053d51640 1 --2- 172.21.15.46:0/333854438 >> v2:172.21.15.162:3300/0 conn(0x7f504c160ed0 0x7f504c0cded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:47.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.035+0000 7f5053d51640 1 --2- 172.21.15.46:0/333854438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f504c15d580 0x7f504c0b8d40 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:47.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.035+0000 7f5053d51640 1 -- 172.21.15.46:0/333854438 >> 172.21.15.46:0/333854438 conn(0x7f504c072200 msgr2=0x7f504c0cfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:47.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.035+0000 7f5053d51640 1 -- 172.21.15.46:0/333854438 shutdown_connections 2023-12-02T00:31:47.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.035+0000 7f5053d51640 1 -- 172.21.15.46:0/333854438 wait complete. 2023-12-02T00:31:47.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.036+0000 7f5053d51640 1 Processor -- start 2023-12-02T00:31:47.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.036+0000 7f5053d51640 1 -- start start 2023-12-02T00:31:47.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.037+0000 7f5053d51640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f504c0ce410 0x7f504c0c97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:47.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.037+0000 7f5053d51640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f504c15d580 0x7f504c0c6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:47.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.037+0000 7f5053d51640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f504c160ed0 0x7f504c0b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:47.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.037+0000 7f5053d51640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f504c0c7080 con 0x7f504c160ed0 2023-12-02T00:31:47.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.037+0000 7f5053d51640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f504c0c7ea0 con 0x7f504c15d580 2023-12-02T00:31:47.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.037+0000 7f5053d51640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f504c0c7290 con 0x7f504c0ce410 2023-12-02T00:31:47.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.037+0000 7f50522ee640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f504c160ed0 0x7f504c0b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:47.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.037+0000 7f50522ee640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f504c160ed0 0x7f504c0b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49034/0 (socket says 172.21.15.46:49034) 2023-12-02T00:31:47.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.037+0000 7f50522ee640 1 -- 172.21.15.46:0/1315886141 learned_addr learned my addr 172.21.15.46:0/1315886141 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:47.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.037+0000 7f50512ec640 1 --2- 172.21.15.46:0/1315886141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f504c15d580 0x7f504c0c6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:47.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.037+0000 7f5051aed640 1 --2- 172.21.15.46:0/1315886141 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f504c0ce410 0x7f504c0c97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:47.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.038+0000 7f50522ee640 1 -- 172.21.15.46:0/1315886141 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f504c0ce410 msgr2=0x7f504c0c97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:47.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.038+0000 7f50522ee640 1 --2- 172.21.15.46:0/1315886141 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f504c0ce410 0x7f504c0c97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:47.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.038+0000 7f50522ee640 1 -- 172.21.15.46:0/1315886141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f504c15d580 msgr2=0x7f504c0c6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:47.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.038+0000 7f50522ee640 1 --2- 172.21.15.46:0/1315886141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f504c15d580 0x7f504c0c6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:47.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.038+0000 7f50522ee640 1 -- 172.21.15.46:0/1315886141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5044009400 con 0x7f504c160ed0 2023-12-02T00:31:47.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.038+0000 7f50522ee640 1 --2- 172.21.15.46:0/1315886141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f504c160ed0 0x7f504c0b3f30 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f503c00c7a0 tx=0x7f503c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:47.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.038+0000 7f504a7fc640 1 -- 172.21.15.46:0/1315886141 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503c00a640 con 0x7f504c160ed0 2023-12-02T00:31:47.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.038+0000 7f504a7fc640 1 -- 172.21.15.46:0/1315886141 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f503c00a7a0 con 0x7f504c160ed0 2023-12-02T00:31:47.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.038+0000 7f5053d51640 1 -- 172.21.15.46:0/1315886141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f504c0c7570 con 0x7f504c160ed0 2023-12-02T00:31:47.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.038+0000 7f504a7fc640 1 -- 172.21.15.46:0/1315886141 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503c00abd0 con 0x7f504c160ed0 2023-12-02T00:31:47.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.038+0000 7f5053d51640 1 -- 172.21.15.46:0/1315886141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f504c0c7b30 con 0x7f504c160ed0 2023-12-02T00:31:47.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.039+0000 7f5053d51640 1 -- 172.21.15.46:0/1315886141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f504c0b5ad0 con 0x7f504c160ed0 2023-12-02T00:31:47.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.040+0000 7f504a7fc640 1 -- 172.21.15.46:0/1315886141 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f503c00ad30 con 0x7f504c160ed0 2023-12-02T00:31:47.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.040+0000 7f504a7fc640 1 --2- 172.21.15.46:0/1315886141 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f503806ae40 0x7f503806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:47.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.040+0000 7f504a7fc640 1 -- 172.21.15.46:0/1315886141 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f503c088a60 con 0x7f504c160ed0 2023-12-02T00:31:47.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.040+0000 7f504a7fc640 1 --2- 172.21.15.46:0/1315886141 >> [v2:172.21.15.162:6800/3225626420,v1:172.21.15.162:6801/3225626420] conn(0x7f5038070ca0 0x7f50380730c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:47.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.040+0000 7f5051aed640 1 --2- 172.21.15.46:0/1315886141 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f503806ae40 0x7f503806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:47.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.040+0000 7f504a7fc640 1 -- 172.21.15.46:0/1315886141 --> [v2:172.21.15.162:6800/3225626420,v1:172.21.15.162:6801/3225626420] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5038073770 con 0x7f5038070ca0 2023-12-02T00:31:47.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.040+0000 7f504a7fc640 1 -- 172.21.15.46:0/1315886141 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f503c08d2c0 con 0x7f504c160ed0 2023-12-02T00:31:47.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.040+0000 7f50512ec640 1 --2- 172.21.15.46:0/1315886141 >> [v2:172.21.15.162:6800/3225626420,v1:172.21.15.162:6801/3225626420] conn(0x7f5038070ca0 0x7f50380730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:47.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.041+0000 7f5051aed640 1 --2- 172.21.15.46:0/1315886141 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f503806ae40 0x7f503806d300 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5044036000 tx=0x7f504403e040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:47.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.041+0000 7f50512ec640 1 --2- 172.21.15.46:0/1315886141 >> [v2:172.21.15.162:6800/3225626420,v1:172.21.15.162:6801/3225626420] conn(0x7f5038070ca0 0x7f50380730c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:47.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.042+0000 7f504a7fc640 1 -- 172.21.15.46:0/1315886141 <== osd.6 v2:172.21.15.162:6800/3225626420 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f5038073770 con 0x7f5038070ca0 2023-12-02T00:31:47.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.055+0000 7f5053d51640 1 -- 172.21.15.46:0/1315886141 --> [v2:172.21.15.162:6800/3225626420,v1:172.21.15.162:6801/3225626420] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f504c161330 con 0x7f5038070ca0 2023-12-02T00:31:47.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.055+0000 7f504a7fc640 1 -- 172.21.15.46:0/1315886141 <== osd.6 v2:172.21.15.162:6800/3225626420 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f50400075f0 con 0x7f5038070ca0 2023-12-02T00:31:47.057 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-12-02T00:31:47.058 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-12-02T00:31:47.058 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-12-02T00:31:47.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.056+0000 7f5053d51640 1 -- 172.21.15.46:0/1315886141 >> [v2:172.21.15.162:6800/3225626420,v1:172.21.15.162:6801/3225626420] conn(0x7f5038070ca0 msgr2=0x7f50380730c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:47.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.056+0000 7f5053d51640 1 --2- 172.21.15.46:0/1315886141 >> [v2:172.21.15.162:6800/3225626420,v1:172.21.15.162:6801/3225626420] conn(0x7f5038070ca0 0x7f50380730c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:47.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.056+0000 7f5053d51640 1 -- 172.21.15.46:0/1315886141 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f503806ae40 msgr2=0x7f503806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:47.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.056+0000 7f5053d51640 1 --2- 172.21.15.46:0/1315886141 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f503806ae40 0x7f503806d300 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5044036000 tx=0x7f504403e040 comp rx=0 tx=0).stop 2023-12-02T00:31:47.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.056+0000 7f5053d51640 1 -- 172.21.15.46:0/1315886141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f504c160ed0 msgr2=0x7f504c0b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:47.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.056+0000 7f5053d51640 1 --2- 172.21.15.46:0/1315886141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f504c160ed0 0x7f504c0b3f30 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f503c00c7a0 tx=0x7f503c00cc60 comp rx=0 tx=0).stop 2023-12-02T00:31:47.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.056+0000 7f50522ee640 1 -- 172.21.15.46:0/1315886141 reap_dead start 2023-12-02T00:31:47.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.056+0000 7f5053d51640 1 -- 172.21.15.46:0/1315886141 shutdown_connections 2023-12-02T00:31:47.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.056+0000 7f5053d51640 1 -- 172.21.15.46:0/1315886141 >> 172.21.15.46:0/1315886141 conn(0x7f504c072200 msgr2=0x7f504c15e090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:47.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.056+0000 7f5053d51640 1 -- 172.21.15.46:0/1315886141 shutdown_connections 2023-12-02T00:31:47.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.056+0000 7f5053d51640 1 -- 172.21.15.46:0/1315886141 wait complete. 2023-12-02T00:31:47.068 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-12-02T00:31:47.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.174+0000 7f9e2f24f640 1 Processor -- start 2023-12-02T00:31:47.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.174+0000 7f9e2f24f640 1 -- start start 2023-12-02T00:31:47.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.175+0000 7f9e2f24f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e28160fb0 0x7f9e28161440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:47.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.175+0000 7f9e2f24f640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9e280d0ed0 0x7f9e280d1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:47.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.175+0000 7f9e2f24f640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e28161d20 con 0x7f9e2806ea50 2023-12-02T00:31:47.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.175+0000 7f9e2f24f640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e2815e760 con 0x7f9e280d0ed0 2023-12-02T00:31:47.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.175+0000 7f9e2f24f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2815d9a0 con 0x7f9e28160fb0 2023-12-02T00:31:47.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.175+0000 7f9e2cfeb640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9e2806ea50 0x7f9e2806ee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49122/0 (socket says 172.21.15.46:49122) 2023-12-02T00:31:47.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.175+0000 7f9e2cfeb640 1 -- 172.21.15.46:0/2196621250 learned_addr learned my addr 172.21.15.46:0/2196621250 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:31:47.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.175+0000 7f9e27fff640 1 --2- 172.21.15.46:0/2196621250 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e28160fb0 0x7f9e28161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:47.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.175+0000 7f9e2d7ec640 1 --2- 172.21.15.46:0/2196621250 >> v2:172.21.15.162:3300/0 conn(0x7f9e280d0ed0 0x7f9e280d1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:47.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.175+0000 7f9e277fe640 1 -- 172.21.15.46:0/2196621250 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 442537795 0 0) 0x7f9e28161d20 con 0x7f9e2806ea50 2023-12-02T00:31:47.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.176+0000 7f9e277fe640 1 -- 172.21.15.46:0/2196621250 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e10003520 con 0x7f9e2806ea50 2023-12-02T00:31:47.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.176+0000 7f9e277fe640 1 -- 172.21.15.46:0/2196621250 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3607182690 0 0) 0x7f9e10003520 con 0x7f9e2806ea50 2023-12-02T00:31:47.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.176+0000 7f9e2d7ec640 1 -- 172.21.15.46:0/2196621250 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e28160fb0 msgr2=0x7f9e28161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:47.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.176+0000 7f9e2d7ec640 1 --2- 172.21.15.46:0/2196621250 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e28160fb0 0x7f9e28161440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:47.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.176+0000 7f9e2d7ec640 1 -- 172.21.15.46:0/2196621250 >> v1:172.21.15.46:6789/0 conn(0x7f9e2806ea50 legacy=0x7f9e2806ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:47.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.176+0000 7f9e2d7ec640 1 -- 172.21.15.46:0/2196621250 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e2815dc80 con 0x7f9e280d0ed0 2023-12-02T00:31:47.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.177+0000 7f9e2d7ec640 1 --2- 172.21.15.46:0/2196621250 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e280d0ed0 0x7f9e280d1360 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9e180097a0 tx=0x7f9e1802b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97a3313f576221f1 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:47.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.177+0000 7f9e277fe640 1 -- 172.21.15.46:0/2196621250 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e200019f0 con 0x7f9e280d0ed0 2023-12-02T00:31:47.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.177+0000 7f9e277fe640 1 -- 172.21.15.46:0/2196621250 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e1802e050 con 0x7f9e280d0ed0 2023-12-02T00:31:47.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.177+0000 7f9e277fe640 1 -- 172.21.15.46:0/2196621250 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e18030730 con 0x7f9e280d0ed0 2023-12-02T00:31:47.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.177+0000 7f9e2f24f640 1 -- 172.21.15.46:0/2196621250 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e280d0ed0 msgr2=0x7f9e280d1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:47.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.177+0000 7f9e2f24f640 1 --2- 172.21.15.46:0/2196621250 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e280d0ed0 0x7f9e280d1360 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9e180097a0 tx=0x7f9e1802b790 comp rx=0 tx=0).stop 2023-12-02T00:31:47.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.177+0000 7f9e2f24f640 1 -- 172.21.15.46:0/2196621250 shutdown_connections 2023-12-02T00:31:47.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.177+0000 7f9e2f24f640 1 --2- 172.21.15.46:0/2196621250 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e280d0ed0 0x7f9e280d1360 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:47.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.177+0000 7f9e2f24f640 1 --2- 172.21.15.46:0/2196621250 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e28160fb0 0x7f9e28161440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:47.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.177+0000 7f9e2f24f640 1 -- 172.21.15.46:0/2196621250 >> 172.21.15.46:0/2196621250 conn(0x7f9e2806a6b0 msgr2=0x7f9e2806aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:47.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.177+0000 7f9e2f24f640 1 -- 172.21.15.46:0/2196621250 shutdown_connections 2023-12-02T00:31:47.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.177+0000 7f9e2f24f640 1 -- 172.21.15.46:0/2196621250 wait complete. 2023-12-02T00:31:47.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.178+0000 7f9e2f24f640 1 Processor -- start 2023-12-02T00:31:47.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.179+0000 7f9e2f24f640 1 -- start start 2023-12-02T00:31:47.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.179+0000 7f9e2f24f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e2806ea50 0x7f9e280d0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:47.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.179+0000 7f9e2f24f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e280d0ed0 0x7f9e2806bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:47.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.179+0000 7f9e2f24f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e28160fb0 0x7f9e2806c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:47.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.179+0000 7f9e2f24f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2815a0b0 con 0x7f9e28160fb0 2023-12-02T00:31:47.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.179+0000 7f9e2f24f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e281f30c0 con 0x7f9e280d0ed0 2023-12-02T00:31:47.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.179+0000 7f9e2f24f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e28159530 con 0x7f9e2806ea50 2023-12-02T00:31:47.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.179+0000 7f9e2d7ec640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e28160fb0 0x7f9e2806c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:47.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.180+0000 7f9e2d7ec640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e28160fb0 0x7f9e2806c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49048/0 (socket says 172.21.15.46:49048) 2023-12-02T00:31:47.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.180+0000 7f9e2d7ec640 1 -- 172.21.15.46:0/3797249570 learned_addr learned my addr 172.21.15.46:0/3797249570 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:47.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.180+0000 7f9e2cfeb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e2806ea50 0x7f9e280d0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:47.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.180+0000 7f9e27fff640 1 --2- 172.21.15.46:0/3797249570 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e280d0ed0 0x7f9e2806bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:47.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.180+0000 7f9e2d7ec640 1 -- 172.21.15.46:0/3797249570 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e2806ea50 msgr2=0x7f9e280d0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:47.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.180+0000 7f9e2d7ec640 1 --2- 172.21.15.46:0/3797249570 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e2806ea50 0x7f9e280d0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:47.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.180+0000 7f9e2d7ec640 1 -- 172.21.15.46:0/3797249570 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e280d0ed0 msgr2=0x7f9e2806bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:47.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.180+0000 7f9e2d7ec640 1 --2- 172.21.15.46:0/3797249570 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e280d0ed0 0x7f9e2806bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:47.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.180+0000 7f9e2d7ec640 1 -- 172.21.15.46:0/3797249570 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e18009400 con 0x7f9e28160fb0 2023-12-02T00:31:47.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.180+0000 7f9e2d7ec640 1 --2- 172.21.15.46:0/3797249570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e28160fb0 0x7f9e2806c4b0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f9e180097a0 tx=0x7f9e180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:47.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.180+0000 7f9e257fa640 1 -- 172.21.15.46:0/3797249570 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1803b810 con 0x7f9e28160fb0 2023-12-02T00:31:47.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.180+0000 7f9e257fa640 1 -- 172.21.15.46:0/3797249570 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e1803b970 con 0x7f9e28160fb0 2023-12-02T00:31:47.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.180+0000 7f9e2f24f640 1 -- 172.21.15.46:0/3797249570 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e28159810 con 0x7f9e28160fb0 2023-12-02T00:31:47.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.180+0000 7f9e257fa640 1 -- 172.21.15.46:0/3797249570 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1803bce0 con 0x7f9e28160fb0 2023-12-02T00:31:47.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.180+0000 7f9e2f24f640 1 -- 172.21.15.46:0/3797249570 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e28159dd0 con 0x7f9e28160fb0 2023-12-02T00:31:47.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.181+0000 7f9e2f24f640 1 -- 172.21.15.46:0/3797249570 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e281f3230 con 0x7f9e28160fb0 2023-12-02T00:31:47.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.182+0000 7f9e257fa640 1 -- 172.21.15.46:0/3797249570 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9e18039040 con 0x7f9e28160fb0 2023-12-02T00:31:47.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.182+0000 7f9e257fa640 1 --2- 172.21.15.46:0/3797249570 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9e1406ae40 0x7f9e1406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:47.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.182+0000 7f9e257fa640 1 -- 172.21.15.46:0/3797249570 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9e180aa980 con 0x7f9e28160fb0 2023-12-02T00:31:47.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.182+0000 7f9e2cfeb640 1 --2- 172.21.15.46:0/3797249570 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9e1406ae40 0x7f9e1406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:47.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.183+0000 7f9e2cfeb640 1 --2- 172.21.15.46:0/3797249570 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9e1406ae40 0x7f9e1406d300 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9e20005c00 tx=0x7f9e20009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:47.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.186+0000 7f9e257fa640 1 -- 172.21.15.46:0/3797249570 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9e281f3230 con 0x7f9e28160fb0 2023-12-02T00:31:47.362 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.361+0000 7f9e2f24f640 1 -- 172.21.15.46:0/3797249570 --> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f9e2815e240 con 0x7f9e1406ae40 2023-12-02T00:31:47.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.362+0000 7f9e257fa640 1 -- 172.21.15.46:0/3797249570 <== mgr.4108 v2:172.21.15.162:6832/3248644722 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f9e200019b0 con 0x7f9e1406ae40 2023-12-02T00:31:47.363 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 6 to deep-scrub 2023-12-02T00:31:47.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.363+0000 7f9e2f24f640 1 -- 172.21.15.46:0/3797249570 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9e1406ae40 msgr2=0x7f9e1406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:47.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.363+0000 7f9e2f24f640 1 --2- 172.21.15.46:0/3797249570 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9e1406ae40 0x7f9e1406d300 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9e20005c00 tx=0x7f9e20009040 comp rx=0 tx=0).stop 2023-12-02T00:31:47.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.363+0000 7f9e2f24f640 1 -- 172.21.15.46:0/3797249570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e28160fb0 msgr2=0x7f9e2806c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:47.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.363+0000 7f9e2f24f640 1 --2- 172.21.15.46:0/3797249570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e28160fb0 0x7f9e2806c4b0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f9e180097a0 tx=0x7f9e180046c0 comp rx=0 tx=0).stop 2023-12-02T00:31:47.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.363+0000 7f9e2f24f640 1 -- 172.21.15.46:0/3797249570 shutdown_connections 2023-12-02T00:31:47.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.363+0000 7f9e2f24f640 1 --2- 172.21.15.46:0/3797249570 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f9e1406ae40 0x7f9e1406d300 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:47.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.363+0000 7f9e2f24f640 1 --2- 172.21.15.46:0/3797249570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e28160fb0 0x7f9e2806c4b0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:47.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.363+0000 7f9e2f24f640 1 --2- 172.21.15.46:0/3797249570 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e280d0ed0 0x7f9e2806bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:47.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.363+0000 7f9e2f24f640 1 --2- 172.21.15.46:0/3797249570 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e2806ea50 0x7f9e280d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:47.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.363+0000 7f9e2f24f640 1 -- 172.21.15.46:0/3797249570 >> 172.21.15.46:0/3797249570 conn(0x7f9e2806a6b0 msgr2=0x7f9e2806aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:47.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.363+0000 7f9e2f24f640 1 -- 172.21.15.46:0/3797249570 shutdown_connections 2023-12-02T00:31:47.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.363+0000 7f9e2f24f640 1 -- 172.21.15.46:0/3797249570 wait complete. 2023-12-02T00:31:47.375 INFO:tasks.ceph:Scrubbing osd.7 2023-12-02T00:31:47.376 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-12-02T00:31:47.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.485+0000 7fc59a5ec640 1 Processor -- start 2023-12-02T00:31:47.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.485+0000 7fc59a5ec640 1 -- start start 2023-12-02T00:31:47.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.486+0000 7fc59a5ec640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc59406f1b0 0x7fc594168230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:47.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.486+0000 7fc59a5ec640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc594142960 con 0x7fc594140400 2023-12-02T00:31:47.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.486+0000 7fc59a5ec640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc594142ba0 con 0x7fc5941409e0 2023-12-02T00:31:47.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.486+0000 7fc59a5ec640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc594142e80 con 0x7fc59406f1b0 2023-12-02T00:31:47.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.486+0000 7fc593fff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc594140400 0x7fc5940b45e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49134/0 (socket says 172.21.15.46:49134) 2023-12-02T00:31:47.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.486+0000 7fc593fff640 1 -- 172.21.15.46:0/1868418733 learned_addr learned my addr 172.21.15.46:0/1868418733 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:31:47.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.486+0000 7fc598b89640 1 --2- 172.21.15.46:0/1868418733 >> v2:172.21.15.162:3300/0 conn(0x7fc59406f1b0 0x7fc594168230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:47.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.486+0000 7fc592ffd640 1 -- 172.21.15.46:0/1868418733 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1441439031 0 0) 0x7fc584001110 con 0x7fc594140400 2023-12-02T00:31:47.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.486+0000 7fc592ffd640 1 -- 172.21.15.46:0/1868418733 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc57c003520 con 0x7fc594140400 2023-12-02T00:31:47.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.486+0000 7fc598b89640 1 -- 172.21.15.46:0/1868418733 >> v1:172.21.15.162:6789/0 conn(0x7fc5941409e0 legacy=0x7fc59406eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:47.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.486+0000 7fc592ffd640 1 -- 172.21.15.46:0/1868418733 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1150594823 0 0) 0x7fc57c003520 con 0x7fc594140400 2023-12-02T00:31:47.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.486+0000 7fc598b89640 1 -- 172.21.15.46:0/1868418733 >> v1:172.21.15.46:6789/0 conn(0x7fc594140400 legacy=0x7fc5940b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:47.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.486+0000 7fc598b89640 1 -- 172.21.15.46:0/1868418733 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc594143160 con 0x7fc59406f1b0 2023-12-02T00:31:47.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.487+0000 7fc598b89640 1 --2- 172.21.15.46:0/1868418733 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc59406f1b0 0x7fc594168230 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc58c009d90 tx=0x7fc58c02be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bbeada1d7f9de01 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:47.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.487+0000 7fc592ffd640 1 -- 172.21.15.46:0/1868418733 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc584001800 con 0x7fc59406f1b0 2023-12-02T00:31:47.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.487+0000 7fc592ffd640 1 -- 172.21.15.46:0/1868418733 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc58c02c940 con 0x7fc59406f1b0 2023-12-02T00:31:47.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.487+0000 7fc592ffd640 1 -- 172.21.15.46:0/1868418733 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc58c02ccb0 con 0x7fc59406f1b0 2023-12-02T00:31:47.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.488+0000 7fc59a5ec640 1 -- 172.21.15.46:0/1868418733 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc59406f1b0 msgr2=0x7fc594168230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:47.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.488+0000 7fc59a5ec640 1 --2- 172.21.15.46:0/1868418733 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc59406f1b0 0x7fc594168230 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc58c009d90 tx=0x7fc58c02be10 comp rx=0 tx=0).stop 2023-12-02T00:31:47.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.488+0000 7fc59a5ec640 1 -- 172.21.15.46:0/1868418733 shutdown_connections 2023-12-02T00:31:47.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.488+0000 7fc59a5ec640 1 --2- 172.21.15.46:0/1868418733 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc59406f1b0 0x7fc594168230 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:47.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.488+0000 7fc59a5ec640 1 -- 172.21.15.46:0/1868418733 >> 172.21.15.46:0/1868418733 conn(0x7fc59406a6b0 msgr2=0x7fc59406aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:47.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.488+0000 7fc59a5ec640 1 -- 172.21.15.46:0/1868418733 shutdown_connections 2023-12-02T00:31:47.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.488+0000 7fc59a5ec640 1 -- 172.21.15.46:0/1868418733 wait complete. 2023-12-02T00:31:47.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.489+0000 7fc59a5ec640 1 Processor -- start 2023-12-02T00:31:47.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.489+0000 7fc59a5ec640 1 -- start start 2023-12-02T00:31:47.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.490+0000 7fc59a5ec640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc59406f1b0 0x7fc594158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:47.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.490+0000 7fc59a5ec640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc594140400 0x7fc59413ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:47.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.490+0000 7fc59a5ec640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5941409e0 0x7fc59406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:47.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.490+0000 7fc59a5ec640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5941e92a0 con 0x7fc59406f1b0 2023-12-02T00:31:47.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.490+0000 7fc59a5ec640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5941e8a00 con 0x7fc594140400 2023-12-02T00:31:47.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.490+0000 7fc59a5ec640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5941e8ce0 con 0x7fc5941409e0 2023-12-02T00:31:47.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.490+0000 7fc593fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc59406f1b0 0x7fc594158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:47.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.490+0000 7fc593fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc59406f1b0 0x7fc594158bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49062/0 (socket says 172.21.15.46:49062) 2023-12-02T00:31:47.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.490+0000 7fc593fff640 1 -- 172.21.15.46:0/820087586 learned_addr learned my addr 172.21.15.46:0/820087586 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:47.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.490+0000 7fc598b89640 1 --2- 172.21.15.46:0/820087586 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5941409e0 0x7fc59406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:47.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.490+0000 7fc5937fe640 1 --2- 172.21.15.46:0/820087586 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc594140400 0x7fc59413ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:47.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.490+0000 7fc593fff640 1 -- 172.21.15.46:0/820087586 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5941409e0 msgr2=0x7fc59406c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:47.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.490+0000 7fc593fff640 1 --2- 172.21.15.46:0/820087586 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5941409e0 0x7fc59406c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:47.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.490+0000 7fc593fff640 1 -- 172.21.15.46:0/820087586 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc594140400 msgr2=0x7fc59413ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:47.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.490+0000 7fc593fff640 1 --2- 172.21.15.46:0/820087586 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc594140400 0x7fc59413ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:47.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.490+0000 7fc593fff640 1 -- 172.21.15.46:0/820087586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc58c009a70 con 0x7fc59406f1b0 2023-12-02T00:31:47.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.490+0000 7fc593fff640 1 --2- 172.21.15.46:0/820087586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc59406f1b0 0x7fc594158bf0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fc58400c750 tx=0x7fc58400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:47.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.491+0000 7fc590ff9640 1 -- 172.21.15.46:0/820087586 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc584005ad0 con 0x7fc59406f1b0 2023-12-02T00:31:47.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.491+0000 7fc590ff9640 1 -- 172.21.15.46:0/820087586 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc584005c50 con 0x7fc59406f1b0 2023-12-02T00:31:47.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.491+0000 7fc59a5ec640 1 -- 172.21.15.46:0/820087586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5941e8fc0 con 0x7fc59406f1b0 2023-12-02T00:31:47.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.491+0000 7fc590ff9640 1 -- 172.21.15.46:0/820087586 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc584007070 con 0x7fc59406f1b0 2023-12-02T00:31:47.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.491+0000 7fc59a5ec640 1 -- 172.21.15.46:0/820087586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5941e8570 con 0x7fc59406f1b0 2023-12-02T00:31:47.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.491+0000 7fc59a5ec640 1 -- 172.21.15.46:0/820087586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc594143720 con 0x7fc59406f1b0 2023-12-02T00:31:47.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.492+0000 7fc590ff9640 1 -- 172.21.15.46:0/820087586 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc5840071d0 con 0x7fc59406f1b0 2023-12-02T00:31:47.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.492+0000 7fc590ff9640 1 --2- 172.21.15.46:0/820087586 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc58006ae40 0x7fc58006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:47.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.493+0000 7fc590ff9640 1 -- 172.21.15.46:0/820087586 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc584088840 con 0x7fc59406f1b0 2023-12-02T00:31:47.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.493+0000 7fc590ff9640 1 --2- 172.21.15.46:0/820087586 >> [v2:172.21.15.162:6824/4107689713,v1:172.21.15.162:6825/4107689713] conn(0x7fc580070ca0 0x7fc5800730c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:47.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.493+0000 7fc590ff9640 1 -- 172.21.15.46:0/820087586 --> [v2:172.21.15.162:6824/4107689713,v1:172.21.15.162:6825/4107689713] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc580073770 con 0x7fc580070ca0 2023-12-02T00:31:47.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.493+0000 7fc590ff9640 1 -- 172.21.15.46:0/820087586 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc58408d2c0 con 0x7fc59406f1b0 2023-12-02T00:31:47.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.493+0000 7fc5937fe640 1 --2- 172.21.15.46:0/820087586 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc58006ae40 0x7fc58006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:47.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.493+0000 7fc598b89640 1 --2- 172.21.15.46:0/820087586 >> [v2:172.21.15.162:6824/4107689713,v1:172.21.15.162:6825/4107689713] conn(0x7fc580070ca0 0x7fc5800730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:47.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.493+0000 7fc5937fe640 1 --2- 172.21.15.46:0/820087586 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc58006ae40 0x7fc58006d300 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc59413e8e0 tx=0x7fc588009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:47.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.493+0000 7fc598b89640 1 --2- 172.21.15.46:0/820087586 >> [v2:172.21.15.162:6824/4107689713,v1:172.21.15.162:6825/4107689713] conn(0x7fc580070ca0 0x7fc5800730c0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:47.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.494+0000 7fc590ff9640 1 -- 172.21.15.46:0/820087586 <== osd.7 v2:172.21.15.162:6824/4107689713 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fc580073770 con 0x7fc580070ca0 2023-12-02T00:31:47.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.508+0000 7fc59a5ec640 1 -- 172.21.15.46:0/820087586 --> [v2:172.21.15.162:6824/4107689713,v1:172.21.15.162:6825/4107689713] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc5940b0930 con 0x7fc580070ca0 2023-12-02T00:31:47.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.508+0000 7fc590ff9640 1 -- 172.21.15.46:0/820087586 <== osd.7 v2:172.21.15.162:6824/4107689713 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fc5940b0930 con 0x7fc580070ca0 2023-12-02T00:31:47.509 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-12-02T00:31:47.510 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-12-02T00:31:47.510 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-12-02T00:31:47.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.509+0000 7fc59a5ec640 1 -- 172.21.15.46:0/820087586 >> [v2:172.21.15.162:6824/4107689713,v1:172.21.15.162:6825/4107689713] conn(0x7fc580070ca0 msgr2=0x7fc5800730c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:47.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.509+0000 7fc59a5ec640 1 --2- 172.21.15.46:0/820087586 >> [v2:172.21.15.162:6824/4107689713,v1:172.21.15.162:6825/4107689713] conn(0x7fc580070ca0 0x7fc5800730c0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:47.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.509+0000 7fc59a5ec640 1 -- 172.21.15.46:0/820087586 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc58006ae40 msgr2=0x7fc58006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:47.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.509+0000 7fc59a5ec640 1 --2- 172.21.15.46:0/820087586 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fc58006ae40 0x7fc58006d300 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc59413e8e0 tx=0x7fc588009040 comp rx=0 tx=0).stop 2023-12-02T00:31:47.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.509+0000 7fc59a5ec640 1 -- 172.21.15.46:0/820087586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc59406f1b0 msgr2=0x7fc594158bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:47.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.509+0000 7fc59a5ec640 1 --2- 172.21.15.46:0/820087586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc59406f1b0 0x7fc594158bf0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fc58400c750 tx=0x7fc58400cc10 comp rx=0 tx=0).stop 2023-12-02T00:31:47.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.509+0000 7fc598b89640 1 -- 172.21.15.46:0/820087586 reap_dead start 2023-12-02T00:31:47.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.509+0000 7fc59a5ec640 1 -- 172.21.15.46:0/820087586 shutdown_connections 2023-12-02T00:31:47.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.509+0000 7fc59a5ec640 1 -- 172.21.15.46:0/820087586 >> 172.21.15.46:0/820087586 conn(0x7fc59406a6b0 msgr2=0x7fc59406aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:47.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.509+0000 7fc59a5ec640 1 -- 172.21.15.46:0/820087586 shutdown_connections 2023-12-02T00:31:47.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.509+0000 7fc59a5ec640 1 -- 172.21.15.46:0/820087586 wait complete. 2023-12-02T00:31:47.520 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-12-02T00:31:47.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.626+0000 7fd578b39640 1 Processor -- start 2023-12-02T00:31:47.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.626+0000 7fd578b39640 1 -- start start 2023-12-02T00:31:47.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.627+0000 7fd578b39640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd57406abe0 0x7fd5741618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:47.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.627+0000 7fd578b39640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd5741409d0 0x7fd57406efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:47.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.627+0000 7fd578b39640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5741406f0 con 0x7fd574140310 2023-12-02T00:31:47.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.627+0000 7fd578b39640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd574161e20 con 0x7fd5741409d0 2023-12-02T00:31:47.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.627+0000 7fd578b39640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd574143960 con 0x7fd57406abe0 2023-12-02T00:31:47.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.627+0000 7fd572d9d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd5741409d0 0x7fd57406efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:47.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.627+0000 7fd57259c640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd574140310 0x7fd57406a4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49150/0 (socket says 172.21.15.46:49150) 2023-12-02T00:31:47.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.627+0000 7fd572d9d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd5741409d0 0x7fd57406efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49072/0 (socket says 172.21.15.46:49072) 2023-12-02T00:31:47.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.627+0000 7fd57259c640 1 -- 172.21.15.46:0/2474852096 learned_addr learned my addr 172.21.15.46:0/2474852096 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:31:47.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.627+0000 7fd572d9d640 1 -- 172.21.15.46:0/2474852096 >> v2:172.21.15.162:3300/0 conn(0x7fd57406abe0 msgr2=0x7fd5741618e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:47.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.627+0000 7fd571d9b640 1 --2- 172.21.15.46:0/2474852096 >> v2:172.21.15.162:3300/0 conn(0x7fd57406abe0 0x7fd5741618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:47.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.627+0000 7fd57159a640 1 -- 172.21.15.46:0/2474852096 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3627175117 0 0) 0x7fd5741406f0 con 0x7fd574140310 2023-12-02T00:31:47.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.627+0000 7fd572d9d640 1 --2- 172.21.15.46:0/2474852096 >> v2:172.21.15.162:3300/0 conn(0x7fd57406abe0 0x7fd5741618e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:47.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.627+0000 7fd572d9d640 1 -- 172.21.15.46:0/2474852096 >> v1:172.21.15.46:6789/0 conn(0x7fd574140310 legacy=0x7fd57406a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:47.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.627+0000 7fd572d9d640 1 -- 172.21.15.46:0/2474852096 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd574142de0 con 0x7fd5741409d0 2023-12-02T00:31:47.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.627+0000 7fd571d9b640 1 --2- 172.21.15.46:0/2474852096 >> v2:172.21.15.162:3300/0 conn(0x7fd57406abe0 0x7fd5741618e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-02T00:31:47.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.628+0000 7fd572d9d640 1 --2- 172.21.15.46:0/2474852096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5741409d0 0x7fd57406efb0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fd57406e720 tx=0x7fd56402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4321589a77fbf79e server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:47.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.628+0000 7fd57159a640 1 -- 172.21.15.46:0/2474852096 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56402cbf0 con 0x7fd5741409d0 2023-12-02T00:31:47.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.628+0000 7fd57159a640 1 -- 172.21.15.46:0/2474852096 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd56402cd50 con 0x7fd5741409d0 2023-12-02T00:31:47.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.628+0000 7fd57159a640 1 -- 172.21.15.46:0/2474852096 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd564034660 con 0x7fd5741409d0 2023-12-02T00:31:47.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.629+0000 7fd578b39640 1 -- 172.21.15.46:0/2474852096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5741409d0 msgr2=0x7fd57406efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:47.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.629+0000 7fd578b39640 1 --2- 172.21.15.46:0/2474852096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5741409d0 0x7fd57406efb0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fd57406e720 tx=0x7fd56402be40 comp rx=0 tx=0).stop 2023-12-02T00:31:47.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.629+0000 7fd578b39640 1 -- 172.21.15.46:0/2474852096 shutdown_connections 2023-12-02T00:31:47.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.629+0000 7fd578b39640 1 --2- 172.21.15.46:0/2474852096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5741409d0 0x7fd57406efb0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:47.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.629+0000 7fd578b39640 1 --2- 172.21.15.46:0/2474852096 >> v2:172.21.15.162:3300/0 conn(0x7fd57406abe0 0x7fd5741618e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:47.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.629+0000 7fd578b39640 1 -- 172.21.15.46:0/2474852096 >> 172.21.15.46:0/2474852096 conn(0x7fd5740b1e50 msgr2=0x7fd5740b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:47.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.629+0000 7fd578b39640 1 -- 172.21.15.46:0/2474852096 shutdown_connections 2023-12-02T00:31:47.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.629+0000 7fd578b39640 1 -- 172.21.15.46:0/2474852096 wait complete. 2023-12-02T00:31:47.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.630+0000 7fd578b39640 1 Processor -- start 2023-12-02T00:31:47.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.631+0000 7fd578b39640 1 -- start start 2023-12-02T00:31:47.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.631+0000 7fd578b39640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd57406abe0 0x7fd5741543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:47.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.631+0000 7fd578b39640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd574140310 0x7fd5740b1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:47.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.631+0000 7fd578b39640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd5741409d0 0x7fd574131e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:47.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.631+0000 7fd578b39640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5740b0ce0 con 0x7fd574140310 2023-12-02T00:31:47.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.631+0000 7fd578b39640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5740afdc0 con 0x7fd5741409d0 2023-12-02T00:31:47.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.631+0000 7fd578b39640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5740b0060 con 0x7fd57406abe0 2023-12-02T00:31:47.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.631+0000 7fd571d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd574140310 0x7fd5740b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:47.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.631+0000 7fd571d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd574140310 0x7fd5740b1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49078/0 (socket says 172.21.15.46:49078) 2023-12-02T00:31:47.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.631+0000 7fd571d9b640 1 -- 172.21.15.46:0/1569585684 learned_addr learned my addr 172.21.15.46:0/1569585684 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:47.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.631+0000 7fd57259c640 1 --2- 172.21.15.46:0/1569585684 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd57406abe0 0x7fd5741543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:47.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.632+0000 7fd572d9d640 1 --2- 172.21.15.46:0/1569585684 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd5741409d0 0x7fd574131e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:47.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.632+0000 7fd571d9b640 1 -- 172.21.15.46:0/1569585684 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd57406abe0 msgr2=0x7fd5741543b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:47.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.632+0000 7fd571d9b640 1 --2- 172.21.15.46:0/1569585684 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd57406abe0 0x7fd5741543b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:47.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.632+0000 7fd571d9b640 1 -- 172.21.15.46:0/1569585684 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd5741409d0 msgr2=0x7fd574131e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:47.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.632+0000 7fd571d9b640 1 --2- 172.21.15.46:0/1569585684 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd5741409d0 0x7fd574131e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:47.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.632+0000 7fd571d9b640 1 -- 172.21.15.46:0/1569585684 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd564009a70 con 0x7fd574140310 2023-12-02T00:31:47.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.632+0000 7fd571d9b640 1 --2- 172.21.15.46:0/1569585684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd574140310 0x7fd5740b1100 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fd55c00c7f0 tx=0x7fd55c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:47.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.632+0000 7fd56affd640 1 -- 172.21.15.46:0/1569585684 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55c007820 con 0x7fd574140310 2023-12-02T00:31:47.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.632+0000 7fd56affd640 1 -- 172.21.15.46:0/1569585684 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd55c007980 con 0x7fd574140310 2023-12-02T00:31:47.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.632+0000 7fd578b39640 1 -- 172.21.15.46:0/1569585684 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5740b0340 con 0x7fd574140310 2023-12-02T00:31:47.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.632+0000 7fd56affd640 1 -- 172.21.15.46:0/1569585684 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55c007cf0 con 0x7fd574140310 2023-12-02T00:31:47.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.632+0000 7fd578b39640 1 -- 172.21.15.46:0/1569585684 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5740b0900 con 0x7fd574140310 2023-12-02T00:31:47.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.633+0000 7fd578b39640 1 -- 172.21.15.46:0/1569585684 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5740b0e20 con 0x7fd574140310 2023-12-02T00:31:47.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.634+0000 7fd56affd640 1 -- 172.21.15.46:0/1569585684 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd55c0113f0 con 0x7fd574140310 2023-12-02T00:31:47.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.634+0000 7fd56affd640 1 --2- 172.21.15.46:0/1569585684 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd55406ae40 0x7fd55406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:47.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.634+0000 7fd56affd640 1 -- 172.21.15.46:0/1569585684 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd55c0895a0 con 0x7fd574140310 2023-12-02T00:31:47.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.634+0000 7fd57259c640 1 --2- 172.21.15.46:0/1569585684 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd55406ae40 0x7fd55406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:47.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.635+0000 7fd57259c640 1 --2- 172.21.15.46:0/1569585684 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd55406ae40 0x7fd55406d300 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd56c000f60 tx=0x7fd56c007d50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:47.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.638+0000 7fd56affd640 1 -- 172.21.15.46:0/1569585684 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd55c054a00 con 0x7fd574140310 2023-12-02T00:31:47.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.814+0000 7fd578b39640 1 -- 172.21.15.46:0/1569585684 --> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fd574143680 con 0x7fd55406ae40 2023-12-02T00:31:47.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.815+0000 7fd56affd640 1 -- 172.21.15.46:0/1569585684 <== mgr.4108 v2:172.21.15.162:6832/3248644722 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd56c004c70 con 0x7fd55406ae40 2023-12-02T00:31:47.816 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 7 to deep-scrub 2023-12-02T00:31:47.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.817+0000 7fd578b39640 1 -- 172.21.15.46:0/1569585684 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd55406ae40 msgr2=0x7fd55406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:47.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.817+0000 7fd578b39640 1 --2- 172.21.15.46:0/1569585684 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd55406ae40 0x7fd55406d300 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd56c000f60 tx=0x7fd56c007d50 comp rx=0 tx=0).stop 2023-12-02T00:31:47.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.817+0000 7fd578b39640 1 -- 172.21.15.46:0/1569585684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd574140310 msgr2=0x7fd5740b1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:47.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.817+0000 7fd578b39640 1 --2- 172.21.15.46:0/1569585684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd574140310 0x7fd5740b1100 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fd55c00c7f0 tx=0x7fd55c00ccb0 comp rx=0 tx=0).stop 2023-12-02T00:31:47.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.817+0000 7fd578b39640 1 -- 172.21.15.46:0/1569585684 shutdown_connections 2023-12-02T00:31:47.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.817+0000 7fd578b39640 1 --2- 172.21.15.46:0/1569585684 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fd55406ae40 0x7fd55406d300 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:47.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.817+0000 7fd578b39640 1 --2- 172.21.15.46:0/1569585684 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd5741409d0 0x7fd574131e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:47.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.817+0000 7fd578b39640 1 --2- 172.21.15.46:0/1569585684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd574140310 0x7fd5740b1100 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:47.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.817+0000 7fd578b39640 1 --2- 172.21.15.46:0/1569585684 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd57406abe0 0x7fd5741543b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:47.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.817+0000 7fd578b39640 1 -- 172.21.15.46:0/1569585684 >> 172.21.15.46:0/1569585684 conn(0x7fd5740b1e50 msgr2=0x7fd57406f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:47.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.817+0000 7fd578b39640 1 -- 172.21.15.46:0/1569585684 shutdown_connections 2023-12-02T00:31:47.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.817+0000 7fd578b39640 1 -- 172.21.15.46:0/1569585684 wait complete. 2023-12-02T00:31:47.829 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-02T00:31:47.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.934+0000 7f07cfa32640 1 Processor -- start 2023-12-02T00:31:47.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.934+0000 7f07cfa32640 1 -- start start 2023-12-02T00:31:47.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.935+0000 7f07cfa32640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07c806ea50 0x7f07c806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:47.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.935+0000 7f07cfa32640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f07c80d0ed0 0x7f07c80d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:47.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.935+0000 7f07cfa32640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07c8161d20 con 0x7f07c8160fb0 2023-12-02T00:31:47.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.935+0000 7f07cfa32640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07c8161b20 con 0x7f07c80d0ed0 2023-12-02T00:31:47.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.935+0000 7f07cfa32640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07c806f370 con 0x7f07c806ea50 2023-12-02T00:31:47.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.935+0000 7f07ccfcd640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f07c8160fb0 0x7f07c8161410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49152/0 (socket says 172.21.15.46:49152) 2023-12-02T00:31:47.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.935+0000 7f07ccfcd640 1 -- 172.21.15.46:0/3273928005 learned_addr learned my addr 172.21.15.46:0/3273928005 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-02T00:31:47.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.935+0000 7f07cd7ce640 1 --2- 172.21.15.46:0/3273928005 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07c806ea50 0x7f07c806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:47.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.935+0000 7f07cdfcf640 1 --2- 172.21.15.46:0/3273928005 >> v2:172.21.15.162:3300/0 conn(0x7f07c80d0ed0 0x7f07c80d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:47.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.936+0000 7f07c7fff640 1 -- 172.21.15.46:0/3273928005 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2493174659 0 0) 0x7f07c8161d20 con 0x7f07c8160fb0 2023-12-02T00:31:47.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.936+0000 7f07c7fff640 1 -- 172.21.15.46:0/3273928005 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07b0003520 con 0x7f07c8160fb0 2023-12-02T00:31:47.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.936+0000 7f07cdfcf640 1 -- 172.21.15.46:0/3273928005 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07c806ea50 msgr2=0x7f07c806ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:31:47.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.936+0000 7f07c7fff640 1 -- 172.21.15.46:0/3273928005 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1549524069 0 0) 0x7f07b0003520 con 0x7f07c8160fb0 2023-12-02T00:31:47.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.936+0000 7f07cdfcf640 1 --2- 172.21.15.46:0/3273928005 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07c806ea50 0x7f07c806ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:47.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.936+0000 7f07cdfcf640 1 -- 172.21.15.46:0/3273928005 >> v1:172.21.15.46:6789/0 conn(0x7f07c8160fb0 legacy=0x7f07c8161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:47.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.936+0000 7f07cdfcf640 1 -- 172.21.15.46:0/3273928005 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07c815e760 con 0x7f07c80d0ed0 2023-12-02T00:31:47.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.936+0000 7f07cd7ce640 1 --2- 172.21.15.46:0/3273928005 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07c806ea50 0x7f07c806ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-02T00:31:47.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.937+0000 7f07cdfcf640 1 --2- 172.21.15.46:0/3273928005 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07c80d0ed0 0x7f07c80d1340 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f07b8009d90 tx=0x7f07b802bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5dd2a608e9dc0070 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:47.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.937+0000 7f07c7fff640 1 -- 172.21.15.46:0/3273928005 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07bc001670 con 0x7f07c80d0ed0 2023-12-02T00:31:47.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.937+0000 7f07c7fff640 1 -- 172.21.15.46:0/3273928005 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07b802c8a0 con 0x7f07c80d0ed0 2023-12-02T00:31:47.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.937+0000 7f07c7fff640 1 -- 172.21.15.46:0/3273928005 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b802cc30 con 0x7f07c80d0ed0 2023-12-02T00:31:47.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.937+0000 7f07cfa32640 1 -- 172.21.15.46:0/3273928005 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07c80d0ed0 msgr2=0x7f07c80d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:47.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.937+0000 7f07cfa32640 1 --2- 172.21.15.46:0/3273928005 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07c80d0ed0 0x7f07c80d1340 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f07b8009d90 tx=0x7f07b802bd70 comp rx=0 tx=0).stop 2023-12-02T00:31:47.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.937+0000 7f07cfa32640 1 -- 172.21.15.46:0/3273928005 shutdown_connections 2023-12-02T00:31:47.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.937+0000 7f07cfa32640 1 --2- 172.21.15.46:0/3273928005 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07c80d0ed0 0x7f07c80d1340 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:47.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.937+0000 7f07cfa32640 1 --2- 172.21.15.46:0/3273928005 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07c806ea50 0x7f07c806ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:47.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.937+0000 7f07cfa32640 1 -- 172.21.15.46:0/3273928005 >> 172.21.15.46:0/3273928005 conn(0x7f07c806a6b0 msgr2=0x7f07c806aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:47.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.938+0000 7f07cfa32640 1 -- 172.21.15.46:0/3273928005 shutdown_connections 2023-12-02T00:31:47.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.938+0000 7f07cfa32640 1 -- 172.21.15.46:0/3273928005 wait complete. 2023-12-02T00:31:47.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.939+0000 7f07cfa32640 1 Processor -- start 2023-12-02T00:31:47.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.939+0000 7f07cfa32640 1 -- start start 2023-12-02T00:31:47.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.939+0000 7f07cfa32640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07c806ea50 0x7f07c80cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:47.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.939+0000 7f07cfa32640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07c80d0ed0 0x7f07c80cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:47.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.939+0000 7f07cfa32640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07c8160fb0 0x7f07c8156560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:47.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.939+0000 7f07cfa32640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c8158a70 con 0x7f07c806ea50 2023-12-02T00:31:47.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.939+0000 7f07cfa32640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c813fdb0 con 0x7f07c80d0ed0 2023-12-02T00:31:47.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.939+0000 7f07cfa32640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07c8157ec0 con 0x7f07c8160fb0 2023-12-02T00:31:47.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.940+0000 7f07cd7ce640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07c806ea50 0x7f07c80cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:47.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.940+0000 7f07cd7ce640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07c806ea50 0x7f07c80cdbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49082/0 (socket says 172.21.15.46:49082) 2023-12-02T00:31:47.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.940+0000 7f07cd7ce640 1 -- 172.21.15.46:0/3805620414 learned_addr learned my addr 172.21.15.46:0/3805620414 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:31:47.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.940+0000 7f07cdfcf640 1 --2- 172.21.15.46:0/3805620414 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07c8160fb0 0x7f07c8156560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:47.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.940+0000 7f07ccfcd640 1 --2- 172.21.15.46:0/3805620414 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07c80d0ed0 0x7f07c80cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:47.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.940+0000 7f07cd7ce640 1 -- 172.21.15.46:0/3805620414 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07c8160fb0 msgr2=0x7f07c8156560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:47.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.940+0000 7f07cd7ce640 1 --2- 172.21.15.46:0/3805620414 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07c8160fb0 0x7f07c8156560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:47.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.940+0000 7f07cd7ce640 1 -- 172.21.15.46:0/3805620414 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07c80d0ed0 msgr2=0x7f07c80cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:47.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.940+0000 7f07cd7ce640 1 --2- 172.21.15.46:0/3805620414 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07c80d0ed0 0x7f07c80cbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:47.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.940+0000 7f07cd7ce640 1 -- 172.21.15.46:0/3805620414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07c0009400 con 0x7f07c806ea50 2023-12-02T00:31:47.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.940+0000 7f07cdfcf640 1 --2- 172.21.15.46:0/3805620414 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07c8160fb0 0x7f07c8156560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-02T00:31:47.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.940+0000 7f07cd7ce640 1 --2- 172.21.15.46:0/3805620414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07c806ea50 0x7f07c80cdbd0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f07c000e960 tx=0x7f07c000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:47.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.940+0000 7f07c5ffb640 1 -- 172.21.15.46:0/3805620414 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c000ca00 con 0x7f07c806ea50 2023-12-02T00:31:47.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.940+0000 7f07c5ffb640 1 -- 172.21.15.46:0/3805620414 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07c000cb60 con 0x7f07c806ea50 2023-12-02T00:31:47.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.940+0000 7f07cfa32640 1 -- 172.21.15.46:0/3805620414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07b8009a70 con 0x7f07c806ea50 2023-12-02T00:31:47.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.940+0000 7f07c5ffb640 1 -- 172.21.15.46:0/3805620414 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c0013470 con 0x7f07c806ea50 2023-12-02T00:31:47.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.941+0000 7f07cfa32640 1 -- 172.21.15.46:0/3805620414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07c81583b0 con 0x7f07c806ea50 2023-12-02T00:31:47.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.942+0000 7f07cfa32640 1 -- 172.21.15.46:0/3805620414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07c8158ce0 con 0x7f07c806ea50 2023-12-02T00:31:47.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.942+0000 7f07c5ffb640 1 -- 172.21.15.46:0/3805620414 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f07c00135d0 con 0x7f07c806ea50 2023-12-02T00:31:47.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.942+0000 7f07c5ffb640 1 --2- 172.21.15.46:0/3805620414 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f07b406ae40 0x7f07b406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:31:47.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.942+0000 7f07c5ffb640 1 -- 172.21.15.46:0/3805620414 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f07c0089560 con 0x7f07c806ea50 2023-12-02T00:31:47.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.942+0000 7f07ccfcd640 1 --2- 172.21.15.46:0/3805620414 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f07b406ae40 0x7f07b406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:31:47.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.943+0000 7f07ccfcd640 1 --2- 172.21.15.46:0/3805620414 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f07b406ae40 0x7f07b406d300 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f07bc001b90 tx=0x7f07bc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:31:47.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:47.947+0000 7f07c5ffb640 1 -- 172.21.15.46:0/3805620414 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f07c0054910 con 0x7f07c806ea50 2023-12-02T00:31:48.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:48.123+0000 7f07cfa32640 1 -- 172.21.15.46:0/3805620414 --> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f07c815df60 con 0x7f07b406ae40 2023-12-02T00:31:48.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:48.125+0000 7f07c5ffb640 1 -- 172.21.15.46:0/3805620414 <== mgr.4108 v2:172.21.15.162:6832/3248644722 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50806 (secure 0 0 0) 0x7f07bc0058d0 con 0x7f07b406ae40 2023-12-02T00:31:48.127 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:31:48.128 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-12-02T00:31:48.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:48.127+0000 7f07cfa32640 1 -- 172.21.15.46:0/3805620414 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f07b406ae40 msgr2=0x7f07b406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:48.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:48.127+0000 7f07cfa32640 1 --2- 172.21.15.46:0/3805620414 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f07b406ae40 0x7f07b406d300 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f07bc001b90 tx=0x7f07bc009040 comp rx=0 tx=0).stop 2023-12-02T00:31:48.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:48.127+0000 7f07cfa32640 1 -- 172.21.15.46:0/3805620414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07c806ea50 msgr2=0x7f07c80cdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:31:48.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:48.127+0000 7f07cfa32640 1 --2- 172.21.15.46:0/3805620414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07c806ea50 0x7f07c80cdbd0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f07c000e960 tx=0x7f07c000ee20 comp rx=0 tx=0).stop 2023-12-02T00:31:48.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:48.127+0000 7f07cfa32640 1 -- 172.21.15.46:0/3805620414 shutdown_connections 2023-12-02T00:31:48.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:48.127+0000 7f07cfa32640 1 --2- 172.21.15.46:0/3805620414 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f07b406ae40 0x7f07b406d300 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:48.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:48.127+0000 7f07cfa32640 1 --2- 172.21.15.46:0/3805620414 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07c8160fb0 0x7f07c8156560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:48.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:48.127+0000 7f07cfa32640 1 --2- 172.21.15.46:0/3805620414 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07c80d0ed0 0x7f07c80cbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:48.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:48.127+0000 7f07cfa32640 1 --2- 172.21.15.46:0/3805620414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07c806ea50 0x7f07c80cdbd0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:31:48.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:48.127+0000 7f07cfa32640 1 -- 172.21.15.46:0/3805620414 >> 172.21.15.46:0/3805620414 conn(0x7f07c806a6b0 msgr2=0x7f07c80d12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:31:48.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:48.127+0000 7f07cfa32640 1 -- 172.21.15.46:0/3805620414 shutdown_connections 2023-12-02T00:31:48.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:31:48.127+0000 7f07cfa32640 1 -- 172.21.15.46:0/3805620414 wait complete. 2023-12-02T00:31:48.139 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":145,"stamp":"2023-12-02T00:31:47.201879+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":239244,"kb_used_data":6432,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735476,"statfs":{"total":773094113280,"available":772849127424,"internally_reserved":0,"allocated":6586368,"data_stored":3434330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.228074"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":199,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-12-02T00:31:38.974585+0000","last_change":"2023-12-02T00:28:39.379380+0000","last_active":"2023-12-02T00:31:38.974585+0000","last_peered":"2023-12-02T00:31:38.974585+0000","last_clean":"2023-12-02T00:31:38.974585+0000","last_became_active":"2023-12-02T00:28:39.378255+0000","last_became_peered":"2023-12-02T00:28:39.378255+0000","last_unstale":"2023-12-02T00:31:38.974585+0000","last_undegraded":"2023-12-02T00:31:38.974585+0000","last_fullsized":"2023-12-02T00:31:38.974585+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-02T00:28:38.177345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-02T00:28:38.177345+0000","last_clean_scrub_stamp":"2023-12-02T00:28:38.177345+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T04:21:08.582412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574889,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28260,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343580,"statfs":{"total":96636764160,"available":96607825920,"internally_reserved":0,"allocated":1269760,"data_stored":872222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.52900000000000003,"15min":0.50600000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.69399999999999995,"5min":0.75,"15min":0.75},"last":0.54000000000000004}]},{"osd":1,"last update":"Sat Dec 2 00:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.54400000000000004,"15min":0.53300000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.55400000000000005,"15min":0.54600000000000004},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.67000000000000004}]},{"osd":2,"last update":"Sat Dec 2 00:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54200000000000004,"15min":0.54400000000000004},"min":{"1min":0.39100000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.56299999999999994,"15min":0.54100000000000004},"min":{"1min":0.45000000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.68500000000000005}]},{"osd":3,"last update":"Sat Dec 2 00:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.53600000000000003,"15min":0.52400000000000002},"min":{"1min":0.41199999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.58199999999999996,"15min":0.57799999999999996},"min":{"1min":0.41199999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.56299999999999994}]},{"osd":4,"last update":"Sat Dec 2 00:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.50700000000000001,"15min":0.51100000000000001},"min":{"1min":0.32300000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.627},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.51700000000000002,"15min":0.52200000000000002},"min":{"1min":0.378,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.76500000000000001}]},{"osd":5,"last update":"Sat Dec 2 00:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.54400000000000004,"15min":0.54900000000000004},"min":{"1min":0.36199999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.78500000000000003,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56000000000000005,"15min":0.56599999999999995},"min":{"1min":0.41099999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.70299999999999996}]},{"osd":6,"last update":"Sat Dec 2 00:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.60999999999999999,"15min":0.624},"min":{"1min":0.39200000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.80600000000000005,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.747},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59699999999999998,"15min":0.60999999999999999},"min":{"1min":0.42999999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.83199999999999996,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.71699999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574889,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27684,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":675840,"data_stored":281854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.49199999999999999,"15min":0.48299999999999998},"min":{"1min":0.36499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.50700000000000001,"15min":0.501},"min":{"1min":0.40000000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.74099999999999999,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.59499999999999997}]},{"osd":1,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.51000000000000001,"15min":0.50600000000000001},"min":{"1min":0.33000000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.72599999999999998,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.53600000000000003,"15min":0.53300000000000003},"min":{"1min":0.39400000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.80900000000000005,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.48899999999999999}]},{"osd":2,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.52000000000000002,"15min":0.51200000000000001},"min":{"1min":0.38,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55500000000000005,"15min":0.55700000000000005},"min":{"1min":0.40300000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.77400000000000002,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.70099999999999996}]},{"osd":3,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57599999999999996,"15min":0.57299999999999995},"min":{"1min":0.45200000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.76300000000000001,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57199999999999995,"15min":0.56999999999999995},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.68999999999999995}]},{"osd":4,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.35399999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.71299999999999997,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52500000000000002,"15min":0.52300000000000002},"min":{"1min":0.34100000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73699999999999999,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.66000000000000003}]},{"osd":5,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.33400000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58799999999999997,"15min":0.59299999999999997},"min":{"1min":0.42399999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.752}]},{"osd":7,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.35999999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.76300000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574890,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27680,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":675840,"data_stored":281854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:31:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.52600000000000002,"15min":0.50900000000000001},"min":{"1min":0.39600000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.79300000000000004,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.52400000000000002,"15min":0.51400000000000001},"min":{"1min":0.374,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.50700000000000001}]},{"osd":1,"last update":"Sat Dec 2 00:31:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.52100000000000002,"15min":0.50700000000000001},"min":{"1min":0.39100000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.82299999999999995,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55300000000000005,"15min":0.55000000000000004},"min":{"1min":0.36799999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.745,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.59099999999999997}]},{"osd":2,"last update":"Sat Dec 2 00:31:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.55900000000000005,"15min":0.55100000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.72499999999999998,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.54600000000000004,"15min":0.53600000000000003},"min":{"1min":0.45100000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.72199999999999998,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.47899999999999998}]},{"osd":3,"last update":"Sat Dec 2 00:31:43 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58499999999999996,"15min":0.57899999999999996},"min":{"1min":0.40500000000000003,"5min":0.38,"15min":0.38},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58399999999999996,"15min":0.58199999999999996},"min":{"1min":0.41399999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.45900000000000002}]},{"osd":4,"last update":"Sat Dec 2 00:31:43 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.53800000000000003,"15min":0.54400000000000004},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76500000000000001,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.58099999999999996,"15min":0.58599999999999997},"min":{"1min":0.372,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.66900000000000004}]},{"osd":6,"last update":"Sat Dec 2 00:31:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56000000000000005,"15min":0.56899999999999995},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.70599999999999996,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58999999999999997,"15min":0.59499999999999997},"min":{"1min":0.38900000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.60599999999999998}]},{"osd":7,"last update":"Sat Dec 2 00:31:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60699999999999998,"15min":0.60799999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.78800000000000003,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.626,"15min":0.63300000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.61699999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574889,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27680,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":675840,"data_stored":281854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55300000000000005,"15min":0.54500000000000004},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.59399999999999997,"15min":0.58699999999999997},"min":{"1min":0.48699999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.83599999999999997,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.58299999999999996}]},{"osd":1,"last update":"Sat Dec 2 00:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.57999999999999996,"15min":0.56100000000000005},"min":{"1min":0.38400000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56399999999999995,"15min":0.55600000000000005},"min":{"1min":0.41099999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.73999999999999999,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.68100000000000005}]},{"osd":2,"last update":"Sat Dec 2 00:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.58399999999999996,"15min":0.57799999999999996},"min":{"1min":0.45500000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60799999999999998,"15min":0.60199999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.752,"5min":0.879,"15min":0.879},"last":0.64100000000000001}]},{"osd":3,"last update":"Sat Dec 2 00:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57399999999999995,"15min":0.56399999999999995},"min":{"1min":0.42299999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78100000000000003,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.622},{"interface":"front","average":{"1min":0.621,"5min":0.60899999999999999,"15min":0.60499999999999998},"min":{"1min":0.46800000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.80300000000000005,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.503}]},{"osd":5,"last update":"Sat Dec 2 00:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52700000000000002,"15min":0.52300000000000002},"min":{"1min":0.34899999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.79700000000000004,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.53200000000000003,"15min":0.53800000000000003},"min":{"1min":0.34399999999999997,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71399999999999997,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.72499999999999998}]},{"osd":6,"last update":"Sat Dec 2 00:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.53500000000000003,"15min":0.53400000000000003},"min":{"1min":0.32800000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.70399999999999996,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.57299999999999995,"15min":0.56899999999999995},"min":{"1min":0.41099999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72799999999999998,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.70199999999999996}]},{"osd":7,"last update":"Sat Dec 2 00:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55800000000000005,"15min":0.55500000000000005},"min":{"1min":0.40100000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.74399999999999999,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59199999999999997,"15min":0.58999999999999997},"min":{"1min":0.42599999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.70199999999999996,"5min":1.004,"15min":1.004},"last":0.41599999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31840,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340000,"statfs":{"total":96636764160,"available":96604160000,"internally_reserved":0,"allocated":675840,"data_stored":281854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35699999999999998,"15min":0.35999999999999999},"min":{"1min":0.184,"5min":0.184,"15min":0.184},"max":{"1min":0.54700000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.372,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.27700000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.56100000000000005,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.34599999999999997}]},{"osd":1,"last update":"Sat Dec 2 00:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.375,"15min":0.38200000000000001},"min":{"1min":0.249,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.64700000000000002,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.374},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.40799999999999997,"15min":0.40899999999999997},"min":{"1min":0.26300000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.64100000000000001,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.497}]},{"osd":2,"last update":"Sat Dec 2 00:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.40200000000000002,"15min":0.40799999999999997},"min":{"1min":0.27000000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.60899999999999999,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.42399999999999999,"15min":0.42199999999999999},"min":{"1min":0.33400000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.45100000000000001}]},{"osd":4,"last update":"Sat Dec 2 00:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61599999999999999,"15min":0.61399999999999999},"min":{"1min":0.38700000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.627,"15min":0.625},"min":{"1min":0.40300000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.74199999999999999}]},{"osd":5,"last update":"Sat Dec 2 00:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58599999999999997,"15min":0.58799999999999997},"min":{"1min":0.35999999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.88800000000000001,"5min":1.004,"15min":1.004},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.60799999999999998,"15min":0.61399999999999999},"min":{"1min":0.41699999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0109999999999999,"5min":1.163,"15min":1.163},"last":0.57099999999999995}]},{"osd":6,"last update":"Sat Dec 2 00:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.64300000000000002,"15min":0.64100000000000001},"min":{"1min":0.46800000000000003,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.64300000000000002,"15min":0.63900000000000001},"min":{"1min":0.55100000000000005,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.70999999999999996}]},{"osd":7,"last update":"Sat Dec 2 00:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.64700000000000002,"15min":0.65100000000000002},"min":{"1min":0.47399999999999998,"5min":0.441,"15min":0.441},"max":{"1min":1.052,"5min":1.177,"15min":1.177},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.65000000000000002,"15min":0.65200000000000002},"min":{"1min":0.44600000000000001,"5min":0.442,"15min":0.442},"max":{"1min":1.121,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.68700000000000006}]}]},{"osd":2,"up_from":13,"seq":55834574889,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31840,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340000,"statfs":{"total":96636764160,"available":96604160000,"internally_reserved":0,"allocated":671744,"data_stored":281235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.375,"15min":0.36199999999999999},"min":{"1min":0.254,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.63500000000000001,"5min":2.649,"15min":2.649},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.41299999999999998,"15min":0.40500000000000003},"min":{"1min":0.245,"5min":0.22,"15min":0.22},"max":{"1min":0.436,"5min":3.173,"15min":3.173},"last":0.27100000000000002}]},{"osd":1,"last update":"Sat Dec 2 00:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.44400000000000001,"15min":0.41899999999999998},"min":{"1min":0.29099999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.52800000000000002,"5min":4.6639999999999997,"15min":4.6639999999999997},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.433,"15min":0.39100000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.505,"5min":6.093,"15min":6.093},"last":0.35399999999999998}]},{"osd":3,"last update":"Sat Dec 2 00:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.40300000000000002,"15min":0.376},"min":{"1min":0.27100000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.59299999999999997,"5min":4.1879999999999997,"15min":4.1879999999999997},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.438,"15min":0.42999999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.54100000000000004,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":0.37}]},{"osd":4,"last update":"Sat Dec 2 00:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.61399999999999999,"15min":0.59699999999999998},"min":{"1min":0.47399999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.76500000000000001,"5min":2.6840000000000002,"15min":2.6840000000000002},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.65200000000000002,"15min":0.64400000000000002},"min":{"1min":0.40799999999999997,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.79900000000000004,"5min":3.0419999999999998,"15min":3.0419999999999998},"last":0.63200000000000001}]},{"osd":5,"last update":"Sat Dec 2 00:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.65200000000000002,"15min":0.65000000000000002},"min":{"1min":0.438,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.77700000000000002,"5min":3.093,"15min":3.093},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.63900000000000001,"15min":0.64900000000000002},"min":{"1min":0.48999999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.79200000000000004,"5min":1.111,"15min":1.111},"last":0.57299999999999995}]},{"osd":6,"last update":"Sat Dec 2 00:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.63500000000000001,"15min":0.63700000000000001},"min":{"1min":0.41899999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.78400000000000003,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.66400000000000003,"15min":0.66700000000000004},"min":{"1min":0.44,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.82899999999999996,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":0.64300000000000002}]},{"osd":7,"last update":"Sat Dec 2 00:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.64900000000000002,"15min":0.64500000000000002},"min":{"1min":0.47799999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.85599999999999998,"5min":3.1339999999999999,"15min":3.1339999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.622,"5min":0.63700000000000001,"15min":0.64500000000000002},"min":{"1min":0.41199999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81499999999999995,"5min":1.2,"15min":1.2},"last":0.60599999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31840,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340000,"statfs":{"total":96636764160,"available":96604160000,"internally_reserved":0,"allocated":675840,"data_stored":281854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.33900000000000002,"15min":0.34100000000000003},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.436,"5min":0.48099999999999998,"15min":0.48099999999999998},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.39600000000000002,"15min":0.39800000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.58799999999999997,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.35099999999999998}]},{"osd":2,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.35099999999999998,"15min":0.34899999999999998},"min":{"1min":0.249,"5min":0.246,"15min":0.246},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.314},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.36299999999999999,"15min":0.36699999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.53500000000000003,"5min":0.626,"15min":0.626},"last":0.33200000000000002}]},{"osd":3,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.40100000000000002,"15min":0.40200000000000002},"min":{"1min":0.30499999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.371,"15min":0.376},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.5,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.29499999999999998}]},{"osd":4,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.55400000000000005,"15min":0.54800000000000004},"min":{"1min":0.377,"5min":0.376,"15min":0.376},"max":{"1min":0.77100000000000002,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.55800000000000005,"15min":0.55100000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.78000000000000003,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.45300000000000001}]},{"osd":5,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.58599999999999997,"15min":0.57899999999999996},"min":{"1min":0.45900000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.96799999999999997,"5min":1.107,"15min":1.107},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57299999999999995,"15min":0.56599999999999995},"min":{"1min":0.437,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.443}]},{"osd":6,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60399999999999998,"15min":0.60199999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.76600000000000001,"5min":1.044,"15min":1.044},"last":0.503},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.61199999999999999,"15min":0.60599999999999998},"min":{"1min":0.51000000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.85399999999999998,"5min":0.998,"15min":0.998},"last":0.46999999999999997}]},{"osd":7,"last update":"Sat Dec 2 00:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58999999999999997,"15min":0.58799999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.78900000000000003,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.60599999999999998,"15min":0.60199999999999998},"min":{"1min":0.45300000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.80000000000000004,"5min":1.151,"15min":1.151},"last":0.43099999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574890,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32420,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339420,"statfs":{"total":96636764160,"available":96603566080,"internally_reserved":0,"allocated":1265664,"data_stored":871603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Dec 2 00:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.57099999999999995,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.371,"15min":0.36499999999999999},"min":{"1min":0.26700000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.54000000000000004,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.316}]},{"osd":2,"last update":"Sat Dec 2 00:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39300000000000002,"15min":0.39200000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.58499999999999996,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.40200000000000002,"15min":0.40100000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.56399999999999995,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.377}]},{"osd":3,"last update":"Sat Dec 2 00:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.39100000000000001,"15min":0.39300000000000002},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.52700000000000002,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.42599999999999999,"15min":0.42699999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63900000000000001,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.40300000000000002}]},{"osd":4,"last update":"Sat Dec 2 00:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.40200000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.61499999999999999,"15min":0.61799999999999999},"min":{"1min":0.45300000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.78900000000000003,"5min":1.002,"15min":1.002},"last":0.56299999999999994}]},{"osd":5,"last update":"Sat Dec 2 00:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.629,"15min":0.627},"min":{"1min":0.49099999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64700000000000002,"15min":0.64600000000000002},"min":{"1min":0.47199999999999998,"5min":0.433,"15min":0.433},"max":{"1min":1.0129999999999999,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.621}]},{"osd":6,"last update":"Sat Dec 2 00:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60999999999999999,"15min":0.60099999999999998},"min":{"1min":0.435,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88500000000000001,"5min":1.05,"15min":1.05},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.63300000000000001,"15min":0.63800000000000001},"min":{"1min":0.435,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.46400000000000002}]},{"osd":7,"last update":"Sat Dec 2 00:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.64000000000000001,"15min":0.63800000000000001},"min":{"1min":0.44500000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.86699999999999999,"5min":1.028,"15min":1.028},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.624,"15min":0.626},"min":{"1min":0.432,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.58599999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-02T00:31:48.143 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-12-02T00:28:38.177345+0000 time.struct_time(tm_year=2023, tm_mon=12, tm_mday=2, tm_hour=0, tm_min=28, tm_sec=38, tm_wday=5, tm_yday=336, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=12, tm_mday=2, tm_hour=0, tm_min=31, tm_sec=43, tm_wday=5, tm_yday=336, tm_isdst=0) 2023-12-02T00:31:48.143 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-12-02T00:31:51.044 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-02T00:31:51.047 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-02T00:32:08.144 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-02T00:32:08.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.262+0000 7f47ae215640 1 Processor -- start 2023-12-02T00:32:08.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.262+0000 7f47ae215640 1 -- start start 2023-12-02T00:32:08.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.263+0000 7f47ae215640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f47a8143490 0x7f47a80b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:32:08.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.263+0000 7f47ae215640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47a815e730 0x7f47a80b1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:32:08.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.263+0000 7f47ae215640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47a815e2a0 con 0x7f47a815a4e0 2023-12-02T00:32:08.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.263+0000 7f47ae215640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47a815b520 con 0x7f47a8143490 2023-12-02T00:32:08.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.263+0000 7f47ae215640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47a8143870 con 0x7f47a815e730 2023-12-02T00:32:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.263+0000 7f47a77fe640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f47a8143490 0x7f47a80b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:32:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.263+0000 7f47a6ffd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47a815e730 0x7f47a80b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:32:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.263+0000 7f47a77fe640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f47a8143490 0x7f47a80b0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.46:35508/0 (socket says 172.21.15.46:35508) 2023-12-02T00:32:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.263+0000 7f47a7fff640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f47a815a4e0 0x7f47a815a970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.46:51076/0 (socket says 172.21.15.46:51076) 2023-12-02T00:32:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.263+0000 7f47a77fe640 1 -- 172.21.15.46:0/596959197 learned_addr learned my addr 172.21.15.46:0/596959197 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:32:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.263+0000 7f47a6ffd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47a815e730 0x7f47a80b1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.46:50818/0 (socket says 172.21.15.46:50818) 2023-12-02T00:32:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.264+0000 7f47a77fe640 1 -- 172.21.15.46:0/596959197 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47a815e730 msgr2=0x7f47a80b1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:32:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.264+0000 7f47a77fe640 1 --2- 172.21.15.46:0/596959197 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47a815e730 0x7f47a80b1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:32:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.264+0000 7f47a77fe640 1 -- 172.21.15.46:0/596959197 >> v1:172.21.15.162:6789/0 conn(0x7f47a815a4e0 legacy=0x7f47a815a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:32:08.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.264+0000 7f47a77fe640 1 -- 172.21.15.46:0/596959197 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a815b240 con 0x7f47a8143490 2023-12-02T00:32:08.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.265+0000 7f47a77fe640 1 --2- 172.21.15.46:0/596959197 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47a8143490 0x7f47a80b0cf0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4798009870 tx=0x7f479802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eae3cf15f8d5e16f server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:32:08.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.265+0000 7f47a67fc640 1 -- 172.21.15.46:0/596959197 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4798030840 con 0x7f47a8143490 2023-12-02T00:32:08.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.265+0000 7f47a67fc640 1 -- 172.21.15.46:0/596959197 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47980309a0 con 0x7f47a8143490 2023-12-02T00:32:08.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.265+0000 7f47a67fc640 1 -- 172.21.15.46:0/596959197 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4798030d30 con 0x7f47a8143490 2023-12-02T00:32:08.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.265+0000 7f47ae215640 1 -- 172.21.15.46:0/596959197 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47a8143490 msgr2=0x7f47a80b0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:32:08.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.265+0000 7f47ae215640 1 --2- 172.21.15.46:0/596959197 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47a8143490 0x7f47a80b0cf0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4798009870 tx=0x7f479802b980 comp rx=0 tx=0).stop 2023-12-02T00:32:08.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.265+0000 7f47ae215640 1 -- 172.21.15.46:0/596959197 shutdown_connections 2023-12-02T00:32:08.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.265+0000 7f47ae215640 1 --2- 172.21.15.46:0/596959197 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47a815e730 0x7f47a80b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:32:08.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.265+0000 7f47ae215640 1 --2- 172.21.15.46:0/596959197 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47a8143490 0x7f47a80b0cf0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:32:08.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.265+0000 7f47ae215640 1 -- 172.21.15.46:0/596959197 >> 172.21.15.46:0/596959197 conn(0x7f47a806a4d0 msgr2=0x7f47a806a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:32:08.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.265+0000 7f47ae215640 1 -- 172.21.15.46:0/596959197 shutdown_connections 2023-12-02T00:32:08.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.266+0000 7f47ae215640 1 -- 172.21.15.46:0/596959197 wait complete. 2023-12-02T00:32:08.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.267+0000 7f47ae215640 1 Processor -- start 2023-12-02T00:32:08.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.267+0000 7f47ae215640 1 -- start start 2023-12-02T00:32:08.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.268+0000 7f47ae215640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47a8143490 0x7f47a8142140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:32:08.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.268+0000 7f47ae215640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47a815a4e0 0x7f47a813e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:32:08.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.268+0000 7f47ae215640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47a815e730 0x7f47a8140860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:32:08.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.268+0000 7f47ae215640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a8141c40 con 0x7f47a815e730 2023-12-02T00:32:08.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.268+0000 7f47ae215640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a8143010 con 0x7f47a815a4e0 2023-12-02T00:32:08.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.268+0000 7f47ae215640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47a8142770 con 0x7f47a8143490 2023-12-02T00:32:08.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.268+0000 7f47a7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47a815e730 0x7f47a8140860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:32:08.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.268+0000 7f47a7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47a815e730 0x7f47a8140860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47936/0 (socket says 172.21.15.46:47936) 2023-12-02T00:32:08.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.268+0000 7f47a7fff640 1 -- 172.21.15.46:0/2259997790 learned_addr learned my addr 172.21.15.46:0/2259997790 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:32:08.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.268+0000 7f47a6ffd640 1 --2- 172.21.15.46:0/2259997790 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47a815a4e0 0x7f47a813e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:32:08.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.268+0000 7f47a77fe640 1 --2- 172.21.15.46:0/2259997790 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47a8143490 0x7f47a8142140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:32:08.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.268+0000 7f47a7fff640 1 -- 172.21.15.46:0/2259997790 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47a8143490 msgr2=0x7f47a8142140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:32:08.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.268+0000 7f47a7fff640 1 --2- 172.21.15.46:0/2259997790 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47a8143490 0x7f47a8142140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:32:08.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.268+0000 7f47a7fff640 1 -- 172.21.15.46:0/2259997790 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47a815a4e0 msgr2=0x7f47a813e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:32:08.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.268+0000 7f47a7fff640 1 --2- 172.21.15.46:0/2259997790 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47a815a4e0 0x7f47a813e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:32:08.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.268+0000 7f47a7fff640 1 -- 172.21.15.46:0/2259997790 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47980094d0 con 0x7f47a815e730 2023-12-02T00:32:08.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.268+0000 7f47a7fff640 1 --2- 172.21.15.46:0/2259997790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47a815e730 0x7f47a8140860 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f47a000c780 tx=0x7f47a000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:32:08.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.269+0000 7f478ffff640 1 -- 172.21.15.46:0/2259997790 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a000ce90 con 0x7f47a815e730 2023-12-02T00:32:08.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.269+0000 7f478ffff640 1 -- 172.21.15.46:0/2259997790 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47a000a640 con 0x7f47a815e730 2023-12-02T00:32:08.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.269+0000 7f47ae215640 1 -- 172.21.15.46:0/2259997790 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47a8142a50 con 0x7f47a815e730 2023-12-02T00:32:08.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.269+0000 7f478ffff640 1 -- 172.21.15.46:0/2259997790 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a000a9f0 con 0x7f47a815e730 2023-12-02T00:32:08.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.269+0000 7f47ae215640 1 -- 172.21.15.46:0/2259997790 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47a8139660 con 0x7f47a815e730 2023-12-02T00:32:08.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.270+0000 7f47ae215640 1 -- 172.21.15.46:0/2259997790 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47a8139850 con 0x7f47a815e730 2023-12-02T00:32:08.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.270+0000 7f478ffff640 1 -- 172.21.15.46:0/2259997790 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f47a000ab50 con 0x7f47a815e730 2023-12-02T00:32:08.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.270+0000 7f478ffff640 1 --2- 172.21.15.46:0/2259997790 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f479406ae40 0x7f479406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:32:08.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.271+0000 7f478ffff640 1 -- 172.21.15.46:0/2259997790 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f47a0088970 con 0x7f47a815e730 2023-12-02T00:32:08.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.271+0000 7f47a77fe640 1 --2- 172.21.15.46:0/2259997790 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f479406ae40 0x7f479406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:32:08.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.271+0000 7f47a77fe640 1 --2- 172.21.15.46:0/2259997790 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f479406ae40 0x7f479406d300 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4798036000 tx=0x7f4798008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:32:08.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.275+0000 7f478ffff640 1 -- 172.21.15.46:0/2259997790 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f47a00578f0 con 0x7f47a815e730 2023-12-02T00:32:08.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.447+0000 7f47ae215640 1 -- 172.21.15.46:0/2259997790 --> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f47a80b15d0 con 0x7f479406ae40 2023-12-02T00:32:08.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.450+0000 7f478ffff640 1 -- 172.21.15.46:0/2259997790 <== mgr.4108 v2:172.21.15.162:6832/3248644722 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50866 (secure 0 0 0) 0x7f47a80b15d0 con 0x7f479406ae40 2023-12-02T00:32:08.451 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:32:08.451 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-12-02T00:32:08.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.451+0000 7f47ae215640 1 -- 172.21.15.46:0/2259997790 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f479406ae40 msgr2=0x7f479406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:32:08.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.451+0000 7f47ae215640 1 --2- 172.21.15.46:0/2259997790 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f479406ae40 0x7f479406d300 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4798036000 tx=0x7f4798008030 comp rx=0 tx=0).stop 2023-12-02T00:32:08.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.451+0000 7f47ae215640 1 -- 172.21.15.46:0/2259997790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47a815e730 msgr2=0x7f47a8140860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:32:08.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.451+0000 7f47ae215640 1 --2- 172.21.15.46:0/2259997790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47a815e730 0x7f47a8140860 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f47a000c780 tx=0x7f47a000cc40 comp rx=0 tx=0).stop 2023-12-02T00:32:08.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.451+0000 7f47ae215640 1 -- 172.21.15.46:0/2259997790 shutdown_connections 2023-12-02T00:32:08.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.451+0000 7f47ae215640 1 --2- 172.21.15.46:0/2259997790 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f479406ae40 0x7f479406d300 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:32:08.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.451+0000 7f47ae215640 1 --2- 172.21.15.46:0/2259997790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47a815e730 0x7f47a8140860 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:32:08.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.451+0000 7f47ae215640 1 --2- 172.21.15.46:0/2259997790 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47a815a4e0 0x7f47a813e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:32:08.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.451+0000 7f47ae215640 1 --2- 172.21.15.46:0/2259997790 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47a8143490 0x7f47a8142140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:32:08.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.451+0000 7f47ae215640 1 -- 172.21.15.46:0/2259997790 >> 172.21.15.46:0/2259997790 conn(0x7f47a806a4d0 msgr2=0x7f47a8071510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:32:08.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.451+0000 7f47ae215640 1 -- 172.21.15.46:0/2259997790 shutdown_connections 2023-12-02T00:32:08.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.451+0000 7f47ae215640 1 -- 172.21.15.46:0/2259997790 wait complete. 2023-12-02T00:32:08.463 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":155,"stamp":"2023-12-02T00:32:07.205754+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":239232,"kb_used_data":6432,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735488,"statfs":{"total":773094113280,"available":772849139712,"internally_reserved":0,"allocated":6586368,"data_stored":3434330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002312"},"pg_stats":[{"pgid":"1.0","version":"56'119","reported_seq":210,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-12-02T00:31:48.725011+0000","last_change":"2023-12-02T00:31:48.725051+0000","last_active":"2023-12-02T00:31:48.725011+0000","last_peered":"2023-12-02T00:31:48.725011+0000","last_clean":"2023-12-02T00:31:48.725011+0000","last_became_active":"2023-12-02T00:28:39.378255+0000","last_became_peered":"2023-12-02T00:28:39.378255+0000","last_unstale":"2023-12-02T00:31:48.725011+0000","last_undegraded":"2023-12-02T00:31:48.725011+0000","last_fullsized":"2023-12-02T00:31:48.725011+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'119","last_scrub_stamp":"2023-12-02T00:31:48.724915+0000","last_deep_scrub":"56'119","last_deep_scrub_stamp":"2023-12-02T00:31:48.724915+0000","last_clean_scrub_stamp":"2023-12-02T00:31:48.724915+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-12-03T11:54:48.074477+0000","scrub_duration":0.0050160020000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574893,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28256,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343584,"statfs":{"total":96636764160,"available":96607830016,"internally_reserved":0,"allocated":1269760,"data_stored":872222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.52900000000000003,"15min":0.50600000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.69399999999999995,"5min":0.75,"15min":0.75},"last":0.622}]},{"osd":1,"last update":"Sat Dec 2 00:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.54400000000000004,"15min":0.53300000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.746},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.55400000000000005,"15min":0.54600000000000004},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.58099999999999996}]},{"osd":2,"last update":"Sat Dec 2 00:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54200000000000004,"15min":0.54400000000000004},"min":{"1min":0.39100000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.56299999999999994,"15min":0.54100000000000004},"min":{"1min":0.45000000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.65700000000000003}]},{"osd":3,"last update":"Sat Dec 2 00:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.53600000000000003,"15min":0.52400000000000002},"min":{"1min":0.41199999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.58199999999999996,"15min":0.57799999999999996},"min":{"1min":0.41199999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.496}]},{"osd":4,"last update":"Sat Dec 2 00:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.50700000000000001,"15min":0.51100000000000001},"min":{"1min":0.32300000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.51700000000000002,"15min":0.52200000000000002},"min":{"1min":0.378,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.41699999999999998}]},{"osd":5,"last update":"Sat Dec 2 00:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.54400000000000004,"15min":0.54900000000000004},"min":{"1min":0.36199999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.78500000000000003,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56000000000000005,"15min":0.56599999999999995},"min":{"1min":0.41099999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.44}]},{"osd":6,"last update":"Sat Dec 2 00:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.60999999999999999,"15min":0.624},"min":{"1min":0.39200000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.80600000000000005,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59699999999999998,"15min":0.60999999999999999},"min":{"1min":0.42999999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.83199999999999996,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.55500000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27684,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":675840,"data_stored":281854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:31:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.51000000000000001,"15min":0.48899999999999999},"min":{"1min":0.39300000000000002,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.79500000000000004,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.51700000000000002,"15min":0.505},"min":{"1min":0.32200000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.74199999999999999,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.496}]},{"osd":1,"last update":"Sat Dec 2 00:31:49 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.51600000000000001,"15min":0.50800000000000001},"min":{"1min":0.40999999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.64600000000000002,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.54400000000000004,"15min":0.53600000000000003},"min":{"1min":0.34300000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.80900000000000005,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.41699999999999998}]},{"osd":2,"last update":"Sat Dec 2 00:31:49 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.53200000000000003,"15min":0.51600000000000001},"min":{"1min":0.436,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.70199999999999996,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55300000000000005,"15min":0.55700000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.70099999999999996,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.34200000000000003}]},{"osd":3,"last update":"Sat Dec 2 00:31:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.58499999999999996,"15min":0.57599999999999996},"min":{"1min":0.42399999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.38500000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.73799999999999999,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.46999999999999997}]},{"osd":4,"last update":"Sat Dec 2 00:31:49 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51100000000000001,"15min":0.505},"min":{"1min":0.35899999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.71099999999999997,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.52800000000000002,"15min":0.52400000000000002},"min":{"1min":0.33700000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.437}]},{"osd":5,"last update":"Sat Dec 2 00:31:49 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.51900000000000002,"15min":0.52300000000000002},"min":{"1min":0.34100000000000003,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.72499999999999998,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58499999999999996,"15min":0.59199999999999997},"min":{"1min":0.40799999999999997,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.59399999999999997}]},{"osd":7,"last update":"Sat Dec 2 00:31:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59599999999999997,"15min":0.59999999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.58899999999999997,"15min":0.59399999999999997},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78900000000000003,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.45000000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27680,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":675840,"data_stored":281854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:31:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.52600000000000002,"15min":0.50900000000000001},"min":{"1min":0.39600000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.79300000000000004,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.52400000000000002,"15min":0.51400000000000001},"min":{"1min":0.374,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.50800000000000001}]},{"osd":1,"last update":"Sat Dec 2 00:31:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.52100000000000002,"15min":0.50700000000000001},"min":{"1min":0.39100000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.82299999999999995,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55300000000000005,"15min":0.55000000000000004},"min":{"1min":0.36799999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.745,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.46400000000000002}]},{"osd":2,"last update":"Sat Dec 2 00:31:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.55900000000000005,"15min":0.55100000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.72499999999999998,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.54600000000000004,"15min":0.53600000000000003},"min":{"1min":0.45100000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.72199999999999998,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.55400000000000005}]},{"osd":3,"last update":"Sat Dec 2 00:31:43 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58499999999999996,"15min":0.57899999999999996},"min":{"1min":0.40500000000000003,"5min":0.38,"15min":0.38},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58399999999999996,"15min":0.58199999999999996},"min":{"1min":0.41399999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.61599999999999999}]},{"osd":4,"last update":"Sat Dec 2 00:31:43 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.53800000000000003,"15min":0.54400000000000004},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76500000000000001,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.495},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.58099999999999996,"15min":0.58599999999999997},"min":{"1min":0.372,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.59699999999999998}]},{"osd":6,"last update":"Sat Dec 2 00:31:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56000000000000005,"15min":0.56899999999999995},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.70599999999999996,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58999999999999997,"15min":0.59499999999999997},"min":{"1min":0.38900000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.63300000000000001}]},{"osd":7,"last update":"Sat Dec 2 00:31:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60699999999999998,"15min":0.60799999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.78800000000000003,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.626,"15min":0.63300000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.69599999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27680,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":675840,"data_stored":281854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55300000000000005,"15min":0.54500000000000004},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.59399999999999997,"15min":0.58699999999999997},"min":{"1min":0.48699999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.83599999999999997,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.70399999999999996}]},{"osd":1,"last update":"Sat Dec 2 00:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.57999999999999996,"15min":0.56100000000000005},"min":{"1min":0.38400000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56399999999999995,"15min":0.55600000000000005},"min":{"1min":0.41099999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.73999999999999999,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.48699999999999999}]},{"osd":2,"last update":"Sat Dec 2 00:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.58399999999999996,"15min":0.57799999999999996},"min":{"1min":0.45500000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60799999999999998,"15min":0.60199999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.752,"5min":0.879,"15min":0.879},"last":0.74399999999999999}]},{"osd":3,"last update":"Sat Dec 2 00:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57399999999999995,"15min":0.56399999999999995},"min":{"1min":0.42299999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78100000000000003,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.60899999999999999,"15min":0.60499999999999998},"min":{"1min":0.46800000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.80300000000000005,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.60999999999999999}]},{"osd":5,"last update":"Sat Dec 2 00:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52700000000000002,"15min":0.52300000000000002},"min":{"1min":0.34899999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.79700000000000004,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.626},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.53200000000000003,"15min":0.53800000000000003},"min":{"1min":0.34399999999999997,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71399999999999997,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.58199999999999996}]},{"osd":6,"last update":"Sat Dec 2 00:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.53500000000000003,"15min":0.53400000000000003},"min":{"1min":0.32800000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.70399999999999996,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.57299999999999995,"15min":0.56899999999999995},"min":{"1min":0.41099999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72799999999999998,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.77000000000000002}]},{"osd":7,"last update":"Sat Dec 2 00:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55800000000000005,"15min":0.55500000000000005},"min":{"1min":0.40100000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.74399999999999999,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59199999999999997,"15min":0.58999999999999997},"min":{"1min":0.42599999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.70199999999999996,"5min":1.004,"15min":1.004},"last":0.68700000000000006}]}]},{"osd":3,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31840,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340000,"statfs":{"total":96636764160,"available":96604160000,"internally_reserved":0,"allocated":675840,"data_stored":281854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35699999999999998,"15min":0.35999999999999999},"min":{"1min":0.184,"5min":0.184,"15min":0.184},"max":{"1min":0.54700000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.443},{"interface":"front","average":{"1min":0.372,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.27700000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.56100000000000005,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.58999999999999997}]},{"osd":1,"last update":"Sat Dec 2 00:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.375,"15min":0.38200000000000001},"min":{"1min":0.249,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.64700000000000002,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.40799999999999997,"15min":0.40899999999999997},"min":{"1min":0.26300000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.64100000000000001,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.50900000000000001}]},{"osd":2,"last update":"Sat Dec 2 00:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.40200000000000002,"15min":0.40799999999999997},"min":{"1min":0.27000000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.60899999999999999,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.42399999999999999,"15min":0.42199999999999999},"min":{"1min":0.33400000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.57099999999999995}]},{"osd":4,"last update":"Sat Dec 2 00:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61599999999999999,"15min":0.61399999999999999},"min":{"1min":0.38700000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.627,"15min":0.625},"min":{"1min":0.40300000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.80600000000000005}]},{"osd":5,"last update":"Sat Dec 2 00:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58599999999999997,"15min":0.58799999999999997},"min":{"1min":0.35999999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.88800000000000001,"5min":1.004,"15min":1.004},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.60799999999999998,"15min":0.61399999999999999},"min":{"1min":0.41699999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0109999999999999,"5min":1.163,"15min":1.163},"last":0.72899999999999998}]},{"osd":6,"last update":"Sat Dec 2 00:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.64300000000000002,"15min":0.64100000000000001},"min":{"1min":0.46800000000000003,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.64300000000000002,"15min":0.63900000000000001},"min":{"1min":0.55100000000000005,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.81799999999999995}]},{"osd":7,"last update":"Sat Dec 2 00:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.64700000000000002,"15min":0.65100000000000002},"min":{"1min":0.47399999999999998,"5min":0.441,"15min":0.441},"max":{"1min":1.052,"5min":1.177,"15min":1.177},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.65000000000000002,"15min":0.65200000000000002},"min":{"1min":0.44600000000000001,"5min":0.442,"15min":0.442},"max":{"1min":1.121,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.83199999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31840,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340000,"statfs":{"total":96636764160,"available":96604160000,"internally_reserved":0,"allocated":671744,"data_stored":281235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.375,"15min":0.36199999999999999},"min":{"1min":0.254,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.63500000000000001,"5min":2.649,"15min":2.649},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.41299999999999998,"15min":0.40500000000000003},"min":{"1min":0.245,"5min":0.22,"15min":0.22},"max":{"1min":0.436,"5min":3.173,"15min":3.173},"last":0.39300000000000002}]},{"osd":1,"last update":"Sat Dec 2 00:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.44400000000000001,"15min":0.41899999999999998},"min":{"1min":0.29099999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.52800000000000002,"5min":4.6639999999999997,"15min":4.6639999999999997},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.433,"15min":0.39100000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.505,"5min":6.093,"15min":6.093},"last":0.32500000000000001}]},{"osd":3,"last update":"Sat Dec 2 00:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.40300000000000002,"15min":0.376},"min":{"1min":0.27100000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.59299999999999997,"5min":4.1879999999999997,"15min":4.1879999999999997},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.438,"15min":0.42999999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.54100000000000004,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":0.439}]},{"osd":4,"last update":"Sat Dec 2 00:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.61399999999999999,"15min":0.59699999999999998},"min":{"1min":0.47399999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.76500000000000001,"5min":2.6840000000000002,"15min":2.6840000000000002},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.65200000000000002,"15min":0.64400000000000002},"min":{"1min":0.40799999999999997,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.79900000000000004,"5min":3.0419999999999998,"15min":3.0419999999999998},"last":0.66800000000000004}]},{"osd":5,"last update":"Sat Dec 2 00:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.65200000000000002,"15min":0.65000000000000002},"min":{"1min":0.438,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.77700000000000002,"5min":3.093,"15min":3.093},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.63900000000000001,"15min":0.64900000000000002},"min":{"1min":0.48999999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.79200000000000004,"5min":1.111,"15min":1.111},"last":0.53200000000000003}]},{"osd":6,"last update":"Sat Dec 2 00:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.63500000000000001,"15min":0.63700000000000001},"min":{"1min":0.41899999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.78400000000000003,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.66400000000000003,"15min":0.66700000000000004},"min":{"1min":0.44,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.82899999999999996,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":0.64800000000000002}]},{"osd":7,"last update":"Sat Dec 2 00:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.64900000000000002,"15min":0.64500000000000002},"min":{"1min":0.47799999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.85599999999999998,"5min":3.1339999999999999,"15min":3.1339999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.622,"5min":0.63700000000000001,"15min":0.64500000000000002},"min":{"1min":0.41199999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81499999999999995,"5min":1.2,"15min":1.2},"last":0.63}]}]},{"osd":1,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31840,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340000,"statfs":{"total":96636764160,"available":96604160000,"internally_reserved":0,"allocated":675840,"data_stored":281854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 2 00:31:46 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.34300000000000003,"15min":0.34200000000000003},"min":{"1min":0.23899999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.32100000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.59099999999999997,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.52800000000000002}]},{"osd":2,"last update":"Sat Dec 2 00:31:46 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.35599999999999998,"15min":0.34999999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.498,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.35899999999999999,"15min":0.36499999999999999},"min":{"1min":0.26400000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.52900000000000003,"5min":0.626,"15min":0.626},"last":0.46300000000000002}]},{"osd":3,"last update":"Sat Dec 2 00:31:46 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.40000000000000002,"15min":0.40200000000000002},"min":{"1min":0.34100000000000003,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.57099999999999995,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.371,"5min":0.37,"15min":0.376},"min":{"1min":0.254,"5min":0.253,"15min":0.253},"max":{"1min":0.55100000000000005,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.42199999999999999}]},{"osd":4,"last update":"Sat Dec 2 00:31:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.56799999999999995,"15min":0.55200000000000005},"min":{"1min":0.49099999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.78200000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.55200000000000005},"min":{"1min":0.44,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.80000000000000004,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.69099999999999995}]},{"osd":5,"last update":"Sat Dec 2 00:31:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.42999999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.78100000000000003,"5min":1.107,"15min":1.107},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.58099999999999996,"15min":0.56899999999999995},"min":{"1min":0.443,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.76200000000000001,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.67100000000000004}]},{"osd":6,"last update":"Sat Dec 2 00:31:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.59899999999999998,"15min":0.59999999999999998},"min":{"1min":0.45500000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.751,"5min":1.044,"15min":1.044},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.623,"15min":0.60999999999999999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.81999999999999995,"5min":0.998,"15min":0.998},"last":0.78500000000000003}]},{"osd":7,"last update":"Sat Dec 2 00:31:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59099999999999997,"15min":0.58899999999999997},"min":{"1min":0.46200000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.76700000000000002,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.746,"5min":1.151,"15min":1.151},"last":0.623}]}]},{"osd":0,"up_from":13,"seq":55834574894,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32412,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339428,"statfs":{"total":96636764160,"available":96603574272,"internally_reserved":0,"allocated":1265664,"data_stored":871603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Dec 2 00:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.57099999999999995,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.371,"15min":0.36499999999999999},"min":{"1min":0.26700000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.54000000000000004,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.33800000000000002}]},{"osd":2,"last update":"Sat Dec 2 00:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39300000000000002,"15min":0.39200000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.58499999999999996,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.40200000000000002,"15min":0.40100000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.56399999999999995,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.41499999999999998}]},{"osd":3,"last update":"Sat Dec 2 00:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.39100000000000001,"15min":0.39300000000000002},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.52700000000000002,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.435},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.42599999999999999,"15min":0.42699999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63900000000000001,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.505}]},{"osd":4,"last update":"Sat Dec 2 00:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.40200000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.61499999999999999,"15min":0.61799999999999999},"min":{"1min":0.45300000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.78900000000000003,"5min":1.002,"15min":1.002},"last":0.70499999999999996}]},{"osd":5,"last update":"Sat Dec 2 00:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.629,"15min":0.627},"min":{"1min":0.49099999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64700000000000002,"15min":0.64600000000000002},"min":{"1min":0.47199999999999998,"5min":0.433,"15min":0.433},"max":{"1min":1.0129999999999999,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.72599999999999998}]},{"osd":6,"last update":"Sat Dec 2 00:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60999999999999999,"15min":0.60099999999999998},"min":{"1min":0.435,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88500000000000001,"5min":1.05,"15min":1.05},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.63300000000000001,"15min":0.63800000000000001},"min":{"1min":0.435,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.80700000000000005}]},{"osd":7,"last update":"Sat Dec 2 00:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.64000000000000001,"15min":0.63800000000000001},"min":{"1min":0.44500000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.86699999999999999,"5min":1.028,"15min":1.028},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.624,"15min":0.626},"min":{"1min":0.432,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.78500000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-02T00:32:08.466 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-12-02T00:32:08.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.571+0000 7fb925436640 1 Processor -- start 2023-12-02T00:32:08.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.571+0000 7fb925436640 1 -- start start 2023-12-02T00:32:08.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.572+0000 7fb925436640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb92015d330 0x7fb9200b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:32:08.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.572+0000 7fb925436640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb9200d3900 0x7fb9200b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:32:08.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.572+0000 7fb925436640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9200b2b10 con 0x7fb9200d2910 2023-12-02T00:32:08.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.572+0000 7fb925436640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9200d1590 con 0x7fb9200d3900 2023-12-02T00:32:08.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.572+0000 7fb925436640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9200d1870 con 0x7fb92015d330 2023-12-02T00:32:08.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.572+0000 7fb91f7fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb9200d3900 0x7fb9200b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:32:08.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.572+0000 7fb91f7fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb9200d3900 0x7fb9200b6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47940/0 (socket says 172.21.15.46:47940) 2023-12-02T00:32:08.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.572+0000 7fb91f7fe640 1 -- 172.21.15.46:0/400185026 learned_addr learned my addr 172.21.15.46:0/400185026 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:32:08.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.572+0000 7fb91effd640 1 --2- 172.21.15.46:0/400185026 >> v2:172.21.15.162:3300/0 conn(0x7fb92015d330 0x7fb9200b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:32:08.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.573+0000 7fb91f7fe640 1 -- 172.21.15.46:0/400185026 >> v2:172.21.15.162:3300/0 conn(0x7fb92015d330 msgr2=0x7fb9200b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:32:08.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.573+0000 7fb91f7fe640 1 --2- 172.21.15.46:0/400185026 >> v2:172.21.15.162:3300/0 conn(0x7fb92015d330 0x7fb9200b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:32:08.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.573+0000 7fb91f7fe640 1 -- 172.21.15.46:0/400185026 >> v1:172.21.15.162:6789/0 conn(0x7fb9200d2910 legacy=0x7fb9200d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:32:08.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.573+0000 7fb91f7fe640 1 -- 172.21.15.46:0/400185026 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9200d1b50 con 0x7fb9200d3900 2023-12-02T00:32:08.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.573+0000 7fb91f7fe640 1 --2- 172.21.15.46:0/400185026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9200d3900 0x7fb9200b6390 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fb9200d0d10 tx=0x7fb91002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ba9aed353842842 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:32:08.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.574+0000 7fb91dffb640 1 -- 172.21.15.46:0/400185026 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb910031870 con 0x7fb9200d3900 2023-12-02T00:32:08.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.574+0000 7fb91dffb640 1 -- 172.21.15.46:0/400185026 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9100319d0 con 0x7fb9200d3900 2023-12-02T00:32:08.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.574+0000 7fb91dffb640 1 -- 172.21.15.46:0/400185026 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb910031d60 con 0x7fb9200d3900 2023-12-02T00:32:08.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.574+0000 7fb925436640 1 -- 172.21.15.46:0/400185026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9200d3900 msgr2=0x7fb9200b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:32:08.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.574+0000 7fb925436640 1 --2- 172.21.15.46:0/400185026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9200d3900 0x7fb9200b6390 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fb9200d0d10 tx=0x7fb91002b910 comp rx=0 tx=0).stop 2023-12-02T00:32:08.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.574+0000 7fb925436640 1 -- 172.21.15.46:0/400185026 shutdown_connections 2023-12-02T00:32:08.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.574+0000 7fb925436640 1 --2- 172.21.15.46:0/400185026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9200d3900 0x7fb9200b6390 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:32:08.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.574+0000 7fb925436640 1 --2- 172.21.15.46:0/400185026 >> v2:172.21.15.162:3300/0 conn(0x7fb92015d330 0x7fb9200b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:32:08.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.574+0000 7fb925436640 1 -- 172.21.15.46:0/400185026 >> 172.21.15.46:0/400185026 conn(0x7fb920072020 msgr2=0x7fb920072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:32:08.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.574+0000 7fb925436640 1 -- 172.21.15.46:0/400185026 shutdown_connections 2023-12-02T00:32:08.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.574+0000 7fb925436640 1 -- 172.21.15.46:0/400185026 wait complete. 2023-12-02T00:32:08.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.575+0000 7fb925436640 1 Processor -- start 2023-12-02T00:32:08.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.576+0000 7fb925436640 1 -- start start 2023-12-02T00:32:08.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.576+0000 7fb925436640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9200d2910 0x7fb92014b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:32:08.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.576+0000 7fb925436640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9200d3900 0x7fb9200cc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:32:08.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.576+0000 7fb925436640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb92015d330 0x7fb9200b2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:32:08.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.576+0000 7fb925436640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9200cb800 con 0x7fb92015d330 2023-12-02T00:32:08.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.576+0000 7fb925436640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9200cb370 con 0x7fb9200d3900 2023-12-02T00:32:08.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.576+0000 7fb925436640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb92014bd50 con 0x7fb9200d2910 2023-12-02T00:32:08.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.576+0000 7fb91f7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb92015d330 0x7fb9200b2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:32:08.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.576+0000 7fb91f7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb92015d330 0x7fb9200b2d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47954/0 (socket says 172.21.15.46:47954) 2023-12-02T00:32:08.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.576+0000 7fb91f7fe640 1 -- 172.21.15.46:0/2063370966 learned_addr learned my addr 172.21.15.46:0/2063370966 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:32:08.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.577+0000 7fb91effd640 1 --2- 172.21.15.46:0/2063370966 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9200d2910 0x7fb92014b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:32:08.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.577+0000 7fb91e7fc640 1 --2- 172.21.15.46:0/2063370966 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9200d3900 0x7fb9200cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:32:08.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.577+0000 7fb91f7fe640 1 -- 172.21.15.46:0/2063370966 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9200d2910 msgr2=0x7fb92014b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:32:08.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.577+0000 7fb91f7fe640 1 --2- 172.21.15.46:0/2063370966 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9200d2910 0x7fb92014b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:32:08.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.577+0000 7fb91f7fe640 1 -- 172.21.15.46:0/2063370966 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9200d3900 msgr2=0x7fb9200cc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:32:08.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.577+0000 7fb91f7fe640 1 --2- 172.21.15.46:0/2063370966 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9200d3900 0x7fb9200cc360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:32:08.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.577+0000 7fb91f7fe640 1 -- 172.21.15.46:0/2063370966 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9100094d0 con 0x7fb92015d330 2023-12-02T00:32:08.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.577+0000 7fb91f7fe640 1 --2- 172.21.15.46:0/2063370966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb92015d330 0x7fb9200b2d20 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fb91002b950 tx=0x7fb9100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:32:08.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.577+0000 7fb9177fe640 1 -- 172.21.15.46:0/2063370966 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91003ac60 con 0x7fb92015d330 2023-12-02T00:32:08.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.577+0000 7fb9177fe640 1 -- 172.21.15.46:0/2063370966 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb91003ade0 con 0x7fb92015d330 2023-12-02T00:32:08.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.577+0000 7fb925436640 1 -- 172.21.15.46:0/2063370966 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9200ca760 con 0x7fb92015d330 2023-12-02T00:32:08.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.577+0000 7fb9177fe640 1 -- 172.21.15.46:0/2063370966 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb910044630 con 0x7fb92015d330 2023-12-02T00:32:08.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.577+0000 7fb925436640 1 -- 172.21.15.46:0/2063370966 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9200cad20 con 0x7fb92015d330 2023-12-02T00:32:08.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.578+0000 7fb925436640 1 -- 172.21.15.46:0/2063370966 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9200b4310 con 0x7fb92015d330 2023-12-02T00:32:08.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.579+0000 7fb9177fe640 1 -- 172.21.15.46:0/2063370966 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb910044790 con 0x7fb92015d330 2023-12-02T00:32:08.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.579+0000 7fb9177fe640 1 --2- 172.21.15.46:0/2063370966 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb90006ae40 0x7fb90006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:32:08.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.579+0000 7fb9177fe640 1 -- 172.21.15.46:0/2063370966 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb91002d040 con 0x7fb92015d330 2023-12-02T00:32:08.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.579+0000 7fb91effd640 1 --2- 172.21.15.46:0/2063370966 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb90006ae40 0x7fb90006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:32:08.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.580+0000 7fb91effd640 1 --2- 172.21.15.46:0/2063370966 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb90006ae40 0x7fb90006d300 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb9200cbb90 tx=0x7fb91800b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:32:08.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.583+0000 7fb9177fe640 1 -- 172.21.15.46:0/2063370966 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb910076c30 con 0x7fb92015d330 2023-12-02T00:32:08.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.760+0000 7fb925436640 1 -- 172.21.15.46:0/2063370966 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7fb9200cb4e0 con 0x7fb92015d330 2023-12-02T00:32:08.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.763+0000 7fb9177fe640 1 -- 172.21.15.46:0/2063370966 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v14)=0 v14) v1 ==== 123+0+0 (secure 0 0 0) 0x7fb91007a250 con 0x7fb92015d330 2023-12-02T00:32:08.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.763+0000 7fb9177fe640 1 -- 172.21.15.46:0/2063370966 <== mon.0 v2:172.21.15.46:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fb910076c30 con 0x7fb92015d330 2023-12-02T00:32:08.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.765+0000 7fb925436640 1 -- 172.21.15.46:0/2063370966 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb90006ae40 msgr2=0x7fb90006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:32:08.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.765+0000 7fb925436640 1 --2- 172.21.15.46:0/2063370966 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb90006ae40 0x7fb90006d300 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb9200cbb90 tx=0x7fb91800b040 comp rx=0 tx=0).stop 2023-12-02T00:32:08.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.765+0000 7fb925436640 1 -- 172.21.15.46:0/2063370966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb92015d330 msgr2=0x7fb9200b2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:32:08.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.765+0000 7fb925436640 1 --2- 172.21.15.46:0/2063370966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb92015d330 0x7fb9200b2d20 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fb91002b950 tx=0x7fb9100046c0 comp rx=0 tx=0).stop 2023-12-02T00:32:08.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.765+0000 7fb925436640 1 -- 172.21.15.46:0/2063370966 shutdown_connections 2023-12-02T00:32:08.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.765+0000 7fb925436640 1 --2- 172.21.15.46:0/2063370966 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7fb90006ae40 0x7fb90006d300 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:32:08.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.765+0000 7fb925436640 1 --2- 172.21.15.46:0/2063370966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb92015d330 0x7fb9200b2d20 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:32:08.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.765+0000 7fb925436640 1 --2- 172.21.15.46:0/2063370966 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9200d3900 0x7fb9200cc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:32:08.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.765+0000 7fb925436640 1 --2- 172.21.15.46:0/2063370966 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9200d2910 0x7fb92014b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:32:08.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.765+0000 7fb925436640 1 -- 172.21.15.46:0/2063370966 >> 172.21.15.46:0/2063370966 conn(0x7fb920072020 msgr2=0x7fb9200cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:32:08.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.765+0000 7fb925436640 1 -- 172.21.15.46:0/2063370966 shutdown_connections 2023-12-02T00:32:08.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.765+0000 7fb925436640 1 -- 172.21.15.46:0/2063370966 wait complete. 2023-12-02T00:32:08.777 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-02T00:32:08.777 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-02T00:32:08.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.882+0000 7f718bc64640 1 Processor -- start 2023-12-02T00:32:08.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.882+0000 7f718bc64640 1 -- start start 2023-12-02T00:32:08.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.883+0000 7f718bc64640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f718406ea50 0x7f718406ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:32:08.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.883+0000 7f718bc64640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7184160fb0 0x7f7184161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:32:08.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.883+0000 7f718bc64640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7184161d20 con 0x7f71840b3ff0 2023-12-02T00:32:08.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.883+0000 7f718bc64640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71840d1ee0 con 0x7f7184160fb0 2023-12-02T00:32:08.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.883+0000 7f718bc64640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f718406f370 con 0x7f718406ea50 2023-12-02T00:32:08.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.883+0000 7f71891ff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7184160fb0 0x7f7184161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:32:08.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.883+0000 7f71891ff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7184160fb0 0x7f7184161410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47958/0 (socket says 172.21.15.46:47958) 2023-12-02T00:32:08.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.883+0000 7f71891ff640 1 -- 172.21.15.46:0/2064293488 learned_addr learned my addr 172.21.15.46:0/2064293488 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:32:08.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.884+0000 7f71891ff640 1 -- 172.21.15.46:0/2064293488 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f718406ea50 msgr2=0x7f718406ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:32:08.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.884+0000 7f71891ff640 1 --2- 172.21.15.46:0/2064293488 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f718406ea50 0x7f718406ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:32:08.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.884+0000 7f71891ff640 1 -- 172.21.15.46:0/2064293488 >> v1:172.21.15.162:6789/0 conn(0x7f71840b3ff0 legacy=0x7f71840b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-02T00:32:08.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.884+0000 7f71891ff640 1 -- 172.21.15.46:0/2064293488 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f718415e760 con 0x7f7184160fb0 2023-12-02T00:32:08.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.885+0000 7f71891ff640 1 --2- 172.21.15.46:0/2064293488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7184160fb0 0x7f7184161410 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f7174009870 tx=0x7f717402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=238bf45f6481e838 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:32:08.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.885+0000 7f71889fe640 1 -- 172.21.15.46:0/2064293488 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71740317c0 con 0x7f7184160fb0 2023-12-02T00:32:08.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.885+0000 7f71889fe640 1 -- 172.21.15.46:0/2064293488 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f7174031920 con 0x7f7184160fb0 2023-12-02T00:32:08.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.885+0000 7f71889fe640 1 -- 172.21.15.46:0/2064293488 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7174031cd0 con 0x7f7184160fb0 2023-12-02T00:32:08.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.885+0000 7f718bc64640 1 -- 172.21.15.46:0/2064293488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7184160fb0 msgr2=0x7f7184161410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:32:08.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.885+0000 7f718bc64640 1 --2- 172.21.15.46:0/2064293488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7184160fb0 0x7f7184161410 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f7174009870 tx=0x7f717402b860 comp rx=0 tx=0).stop 2023-12-02T00:32:08.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.885+0000 7f718bc64640 1 -- 172.21.15.46:0/2064293488 shutdown_connections 2023-12-02T00:32:08.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.885+0000 7f718bc64640 1 --2- 172.21.15.46:0/2064293488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7184160fb0 0x7f7184161410 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:32:08.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.885+0000 7f718bc64640 1 --2- 172.21.15.46:0/2064293488 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f718406ea50 0x7f718406ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:32:08.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.885+0000 7f718bc64640 1 -- 172.21.15.46:0/2064293488 >> 172.21.15.46:0/2064293488 conn(0x7f718406a6b0 msgr2=0x7f718406aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:32:08.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.886+0000 7f718bc64640 1 -- 172.21.15.46:0/2064293488 shutdown_connections 2023-12-02T00:32:08.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.886+0000 7f718bc64640 1 -- 172.21.15.46:0/2064293488 wait complete. 2023-12-02T00:32:08.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.887+0000 7f718bc64640 1 Processor -- start 2023-12-02T00:32:08.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.887+0000 7f718bc64640 1 -- start start 2023-12-02T00:32:08.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.887+0000 7f718bc64640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f718406ea50 0x7f718413dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:32:08.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.887+0000 7f718bc64640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71840b3ff0 0x7f71841e9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:32:08.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.887+0000 7f718bc64640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7184160fb0 0x7f718406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:32:08.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.887+0000 7f718bc64640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7184158c70 con 0x7f71840b3ff0 2023-12-02T00:32:08.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.887+0000 7f718bc64640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7184158f50 con 0x7f7184160fb0 2023-12-02T00:32:08.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.887+0000 7f718bc64640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7184159230 con 0x7f718406ea50 2023-12-02T00:32:08.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.888+0000 7f71891ff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71840b3ff0 0x7f71841e9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:32:08.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.888+0000 7f71891ff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71840b3ff0 0x7f71841e9510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47964/0 (socket says 172.21.15.46:47964) 2023-12-02T00:32:08.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.888+0000 7f71891ff640 1 -- 172.21.15.46:0/3327087263 learned_addr learned my addr 172.21.15.46:0/3327087263 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-02T00:32:08.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.888+0000 7f7189a00640 1 --2- 172.21.15.46:0/3327087263 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f718406ea50 0x7f718413dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:32:08.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.888+0000 7f718a201640 1 --2- 172.21.15.46:0/3327087263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7184160fb0 0x7f718406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:32:08.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.888+0000 7f71891ff640 1 -- 172.21.15.46:0/3327087263 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f718406ea50 msgr2=0x7f718413dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:32:08.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.888+0000 7f71891ff640 1 --2- 172.21.15.46:0/3327087263 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f718406ea50 0x7f718413dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:32:08.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.888+0000 7f71891ff640 1 -- 172.21.15.46:0/3327087263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7184160fb0 msgr2=0x7f718406c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:32:08.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.888+0000 7f71891ff640 1 --2- 172.21.15.46:0/3327087263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7184160fb0 0x7f718406c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:32:08.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.888+0000 7f71891ff640 1 -- 172.21.15.46:0/3327087263 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71740094d0 con 0x7f71840b3ff0 2023-12-02T00:32:08.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.888+0000 7f71891ff640 1 --2- 172.21.15.46:0/3327087263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71840b3ff0 0x7f71841e9510 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f717402b8a0 tx=0x7f71740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:32:08.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.888+0000 7f717a7fc640 1 -- 172.21.15.46:0/3327087263 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71740329e0 con 0x7f71840b3ff0 2023-12-02T00:32:08.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.888+0000 7f717a7fc640 1 -- 172.21.15.46:0/3327087263 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f7174032b80 con 0x7f71840b3ff0 2023-12-02T00:32:08.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.888+0000 7f718bc64640 1 -- 172.21.15.46:0/3327087263 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7184159510 con 0x7f71840b3ff0 2023-12-02T00:32:08.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.888+0000 7f717a7fc640 1 -- 172.21.15.46:0/3327087263 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71740453f0 con 0x7f71840b3ff0 2023-12-02T00:32:08.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.888+0000 7f718bc64640 1 -- 172.21.15.46:0/3327087263 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71841588d0 con 0x7f71840b3ff0 2023-12-02T00:32:08.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.889+0000 7f718bc64640 1 -- 172.21.15.46:0/3327087263 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7184159a30 con 0x7f71840b3ff0 2023-12-02T00:32:08.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.890+0000 7f717a7fc640 1 -- 172.21.15.46:0/3327087263 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f717404d3f0 con 0x7f71840b3ff0 2023-12-02T00:32:08.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.890+0000 7f717a7fc640 1 --2- 172.21.15.46:0/3327087263 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f716806ae70 0x7f716806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-02T00:32:08.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.890+0000 7f717a7fc640 1 -- 172.21.15.46:0/3327087263 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f71740ab3d0 con 0x7f71840b3ff0 2023-12-02T00:32:08.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.890+0000 7f7189a00640 1 --2- 172.21.15.46:0/3327087263 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f716806ae70 0x7f716806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-02T00:32:08.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.891+0000 7f7189a00640 1 --2- 172.21.15.46:0/3327087263 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f716806ae70 0x7f716806d330 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f718000b810 tx=0x7f7180008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-02T00:32:08.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:08.894+0000 7f717a7fc640 1 -- 172.21.15.46:0/3327087263 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7184159a30 con 0x7f71840b3ff0 2023-12-02T00:32:09.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:09.106+0000 7f718bc64640 1 -- 172.21.15.46:0/3327087263 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7184159a30 con 0x7f71840b3ff0 2023-12-02T00:32:09.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:09.107+0000 7f717a7fc640 1 -- 172.21.15.46:0/3327087263 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+3699 (secure 0 0 0) 0x7f7184159a30 con 0x7f71840b3ff0 2023-12-02T00:32:09.108 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-02T00:32:09.108 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5189,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/4047806587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":4047806587},{"type":"v1","addr":"172.21.15.162:6837","nonce":4047806587}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5620,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/1418783715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1418783715},{"type":"v1","addr":"172.21.15.162:6836","nonce":1418783715}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5979,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/452915985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":452915985},{"type":"v1","addr":"172.21.15.46:6833","nonce":452915985}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":6474,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/894879673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":894879673},{"type":"v1","addr":"172.21.15.46:6835","nonce":894879673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26}],"filesystems":[]} 2023-12-02T00:32:09.110 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2023-12-02T00:32:09.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:09.108+0000 7f718bc64640 1 -- 172.21.15.46:0/3327087263 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f716806ae70 msgr2=0x7f716806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:32:09.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:09.108+0000 7f718bc64640 1 --2- 172.21.15.46:0/3327087263 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f716806ae70 0x7f716806d330 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f718000b810 tx=0x7f7180008040 comp rx=0 tx=0).stop 2023-12-02T00:32:09.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:09.108+0000 7f718bc64640 1 -- 172.21.15.46:0/3327087263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71840b3ff0 msgr2=0x7f71841e9510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-02T00:32:09.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:09.108+0000 7f718bc64640 1 --2- 172.21.15.46:0/3327087263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71840b3ff0 0x7f71841e9510 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f717402b8a0 tx=0x7f71740046c0 comp rx=0 tx=0).stop 2023-12-02T00:32:09.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:09.108+0000 7f718bc64640 1 -- 172.21.15.46:0/3327087263 shutdown_connections 2023-12-02T00:32:09.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:09.108+0000 7f718bc64640 1 --2- 172.21.15.46:0/3327087263 >> [v2:172.21.15.162:6832/3248644722,v1:172.21.15.162:6833/3248644722] conn(0x7f716806ae70 0x7f716806d330 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:32:09.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:09.108+0000 7f718bc64640 1 --2- 172.21.15.46:0/3327087263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7184160fb0 0x7f718406c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:32:09.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:09.108+0000 7f718bc64640 1 --2- 172.21.15.46:0/3327087263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71840b3ff0 0x7f71841e9510 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:32:09.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:09.108+0000 7f718bc64640 1 --2- 172.21.15.46:0/3327087263 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f718406ea50 0x7f718413dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-02T00:32:09.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:09.108+0000 7f718bc64640 1 -- 172.21.15.46:0/3327087263 >> 172.21.15.46:0/3327087263 conn(0x7f718406a6b0 msgr2=0x7f718413f700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-02T00:32:09.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:09.108+0000 7f718bc64640 1 -- 172.21.15.46:0/3327087263 shutdown_connections 2023-12-02T00:32:09.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-02T00:32:09.108+0000 7f718bc64640 1 -- 172.21.15.46:0/3327087263 wait complete. 2023-12-02T00:32:09.120 DEBUG:tasks.cephfs.filesystem:already dead... 2023-12-02T00:32:09.121 INFO:teuthology.misc:Shutting down mds daemons... 2023-12-02T00:32:09.121 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-02T00:32:09.121 INFO:teuthology.orchestra.run:waiting for 300 2023-12-02T00:32:09.130 INFO:tasks.ceph.mds.a:Stopped 2023-12-02T00:32:09.130 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-02T00:32:09.130 INFO:teuthology.orchestra.run:waiting for 300 2023-12-02T00:32:09.179 INFO:tasks.ceph.mds.c:Stopped 2023-12-02T00:32:09.180 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-02T00:32:09.180 INFO:teuthology.orchestra.run:waiting for 300 2023-12-02T00:32:09.195 INFO:tasks.ceph.mds.b:Stopped 2023-12-02T00:32:09.195 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-02T00:32:09.195 INFO:teuthology.orchestra.run:waiting for 300 2023-12-02T00:32:09.248 INFO:tasks.ceph.mds.d:Stopped 2023-12-02T00:32:09.248 INFO:teuthology.misc:Shutting down osd daemons... 2023-12-02T00:32:09.249 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-12-02T00:32:09.249 INFO:teuthology.orchestra.run:waiting for 300 2023-12-02T00:32:09.278 INFO:tasks.ceph.osd.0:Stopped 2023-12-02T00:32:09.278 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-12-02T00:32:09.278 INFO:teuthology.orchestra.run:waiting for 300 2023-12-02T00:32:09.353 INFO:tasks.ceph.osd.1:Stopped 2023-12-02T00:32:09.353 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-12-02T00:32:09.353 INFO:teuthology.orchestra.run:waiting for 300 2023-12-02T00:32:09.422 INFO:tasks.ceph.osd.2:Stopped 2023-12-02T00:32:09.423 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-12-02T00:32:09.423 INFO:teuthology.orchestra.run:waiting for 300 2023-12-02T00:32:09.487 INFO:tasks.ceph.osd.3:Stopped 2023-12-02T00:32:09.487 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-12-02T00:32:09.487 INFO:teuthology.orchestra.run:waiting for 300 2023-12-02T00:32:09.528 INFO:tasks.ceph.osd.4:Stopped 2023-12-02T00:32:09.529 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-12-02T00:32:09.529 INFO:teuthology.orchestra.run:waiting for 300 2023-12-02T00:32:09.577 INFO:tasks.ceph.osd.5:Stopped 2023-12-02T00:32:09.577 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-12-02T00:32:09.577 INFO:teuthology.orchestra.run:waiting for 300 2023-12-02T00:32:09.619 INFO:tasks.ceph.osd.6:Stopped 2023-12-02T00:32:09.619 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-12-02T00:32:09.619 INFO:teuthology.orchestra.run:waiting for 300 2023-12-02T00:32:09.655 INFO:tasks.ceph.osd.7:Stopped 2023-12-02T00:32:09.655 INFO:teuthology.misc:Shutting down mgr daemons... 2023-12-02T00:32:09.655 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-12-02T00:32:09.655 INFO:teuthology.orchestra.run:waiting for 300 2023-12-02T00:32:09.680 INFO:tasks.ceph.mgr.y:Stopped 2023-12-02T00:32:09.680 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-12-02T00:32:09.680 INFO:teuthology.orchestra.run:waiting for 300 2023-12-02T00:32:09.723 INFO:tasks.ceph.mgr.x:Stopped 2023-12-02T00:32:09.723 INFO:teuthology.misc:Shutting down mon daemons... 2023-12-02T00:32:09.723 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-12-02T00:32:09.723 INFO:teuthology.orchestra.run:waiting for 300 2023-12-02T00:32:09.735 INFO:tasks.ceph.mon.a:Stopped 2023-12-02T00:32:09.736 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-12-02T00:32:09.736 INFO:teuthology.orchestra.run:waiting for 300 2023-12-02T00:32:09.776 INFO:tasks.ceph.mon.b:Stopped 2023-12-02T00:32:09.777 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-12-02T00:32:09.777 INFO:teuthology.orchestra.run:waiting for 300 2023-12-02T00:32:09.829 INFO:tasks.ceph.mon.c:Stopped 2023-12-02T00:32:09.829 INFO:tasks.ceph:Checking cluster log for badness... 2023-12-02T00:32:09.830 DEBUG:teuthology.orchestra.run.smithi046:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-12-02T00:32:09.865 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi046.front.sepia.ceph.com 2023-12-02T00:32:09.865 DEBUG:teuthology.orchestra.run.smithi046:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-12-02T00:32:10.066 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi046.front.sepia.ceph.com 2023-12-02T00:32:10.067 DEBUG:teuthology.orchestra.run.smithi046:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-12-02T00:32:10.138 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi046.front.sepia.ceph.com 2023-12-02T00:32:10.138 DEBUG:teuthology.orchestra.run.smithi046:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-12-02T00:32:10.195 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi046.front.sepia.ceph.com 2023-12-02T00:32:10.196 DEBUG:teuthology.orchestra.run.smithi046:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-12-02T00:32:10.254 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi162.front.sepia.ceph.com 2023-12-02T00:32:10.255 DEBUG:teuthology.orchestra.run.smithi162:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-12-02T00:32:10.384 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi162.front.sepia.ceph.com 2023-12-02T00:32:10.385 DEBUG:teuthology.orchestra.run.smithi162:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-12-02T00:32:10.477 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi162.front.sepia.ceph.com 2023-12-02T00:32:10.477 DEBUG:teuthology.orchestra.run.smithi162:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-12-02T00:32:10.527 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi162.front.sepia.ceph.com 2023-12-02T00:32:10.528 DEBUG:teuthology.orchestra.run.smithi162:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-12-02T00:32:10.584 INFO:tasks.ceph:Archiving mon data... 2023-12-02T00:32:10.585 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2023-12-01_17:57:53-fs-main-distro-default-smithi/7474581/data/mon.a.tgz 2023-12-02T00:32:10.586 DEBUG:teuthology.orchestra.run.smithi046:> mktemp 2023-12-02T00:32:10.601 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/tmp.MGvkDalNRY 2023-12-02T00:32:10.602 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.MGvkDalNRY 2023-12-02T00:32:10.752 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0666 /tmp/tmp.MGvkDalNRY 2023-12-02T00:32:10.842 DEBUG:teuthology.orchestra.remote:smithi046:/tmp/tmp.MGvkDalNRY is 347KB 2023-12-02T00:32:10.938 DEBUG:teuthology.orchestra.run.smithi046:> rm -fr /tmp/tmp.MGvkDalNRY 2023-12-02T00:32:10.952 DEBUG:teuthology.misc:Transferring archived files from smithi162:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2023-12-01_17:57:53-fs-main-distro-default-smithi/7474581/data/mon.b.tgz 2023-12-02T00:32:10.953 DEBUG:teuthology.orchestra.run.smithi162:> mktemp 2023-12-02T00:32:10.969 INFO:teuthology.orchestra.run.smithi162.stdout:/tmp/tmp.vZB7Qze8Qv 2023-12-02T00:32:10.969 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.vZB7Qze8Qv 2023-12-02T00:32:11.141 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 0666 /tmp/tmp.vZB7Qze8Qv 2023-12-02T00:32:11.225 DEBUG:teuthology.orchestra.remote:smithi162:/tmp/tmp.vZB7Qze8Qv is 458KB 2023-12-02T00:32:11.325 DEBUG:teuthology.orchestra.run.smithi162:> rm -fr /tmp/tmp.vZB7Qze8Qv 2023-12-02T00:32:11.339 DEBUG:teuthology.misc:Transferring archived files from smithi162:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2023-12-01_17:57:53-fs-main-distro-default-smithi/7474581/data/mon.c.tgz 2023-12-02T00:32:11.340 DEBUG:teuthology.orchestra.run.smithi162:> mktemp 2023-12-02T00:32:11.394 INFO:teuthology.orchestra.run.smithi162.stdout:/tmp/tmp.LfAIFlWy41 2023-12-02T00:32:11.394 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.LfAIFlWy41 2023-12-02T00:32:11.548 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 0666 /tmp/tmp.LfAIFlWy41 2023-12-02T00:32:11.633 DEBUG:teuthology.orchestra.remote:smithi162:/tmp/tmp.LfAIFlWy41 is 458KB 2023-12-02T00:32:11.730 DEBUG:teuthology.orchestra.run.smithi162:> rm -fr /tmp/tmp.LfAIFlWy41 2023-12-02T00:32:11.745 INFO:tasks.ceph:Cleaning ceph cluster... 2023-12-02T00:32:11.745 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-12-02T00:32:11.748 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-12-02T00:32:11.838 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2023-12-02T00:32:11.867 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2023-12-02T00:32:11.867 INFO:tasks.ceph:Archiving crash dumps... 2023-12-02T00:32:11.869 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-12-01_17:57:53-fs-main-distro-default-smithi/7474581/remote/smithi046/crash 2023-12-02T00:32:11.869 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-12-02T00:32:11.913 DEBUG:teuthology.misc:Transferring archived files from smithi162:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-12-01_17:57:53-fs-main-distro-default-smithi/7474581/remote/smithi162/crash 2023-12-02T00:32:11.914 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-12-02T00:32:11.940 INFO:tasks.ceph:Shutting down logrotate 2023-12-02T00:32:11.940 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-02T00:32:11.958 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-02T00:32:12.045 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-12-02T00:32:12.079 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-12-02T00:32:12.106 INFO:tasks.ceph:Compressing logs... 2023-12-02T00:32:12.106 DEBUG:teuthology.orchestra.run.smithi046:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-12-02T00:32:12.122 DEBUG:teuthology.orchestra.run.smithi162:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-12-02T00:32:12.199 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-12-02T00:32:12.199 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.95624.log 2023-12-02T00:32:12.199 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-12-02T00:32:12.199 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.3.95624.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-12-02T00:32:12.200 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89080.log 2023-12-02T00:32:12.200 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose/var/log/ceph/ceph-mds.b.log: -- /var/log/ceph/ceph-osd.4.log 2023-12-02T00:32:12.200 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.89080.log: 90.3% -- replaced with /var/log/ceph/ceph-client.3.95624.log.gz 2023-12-02T00:32:12.200 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88920.log 2023-12-02T00:32:12.201 INFO:teuthology.orchestra.run.smithi162.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.89080.log.gz 2023-12-02T00:32:12.201 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-12-02T00:32:12.201 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.88920.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-12-02T00:32:12.201 INFO:teuthology.orchestra.run.smithi162.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.88920.log.gz 2023-12-02T00:32:12.202 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.95190.log 2023-12-02T00:32:12.202 INFO:teuthology.orchestra.run.smithi162.stderr: 92.5% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-12-02T00:32:12.202 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88880.log 2023-12-02T00:32:12.202 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.1.95190.log: gzip -5 --verbose -- 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-12-02T00:32:12.202 INFO:teuthology.orchestra.run.smithi162.stderr: /var/log/ceph/ceph-client.admin.88840.log 2023-12-02T00:32:12.203 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.88880.log: gzip -5 --verbose 89.6% -- /var/log/ceph/ceph-mgr.x.log 2023-12-02T00:32:12.203 INFO:teuthology.orchestra.run.smithi162.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.88880.log.gz 2023-12-02T00:32:12.203 INFO:teuthology.orchestra.run.smithi162.stderr: -- replaced with /var/log/ceph/ceph-client.1.95190.log.gz 2023-12-02T00:32:12.203 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.88840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89040.log 2023-12-02T00:32:12.204 INFO:teuthology.orchestra.run.smithi162.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.88840.log.gz 2023-12-02T00:32:12.204 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89000.log 2023-12-02T00:32:12.204 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.89040.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-12-02T00:32:12.204 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.89000.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.89040.log.gz 2023-12-02T00:32:12.204 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-12-02T00:32:12.205 INFO:teuthology.orchestra.run.smithi162.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.89000.log.gz 2023-12-02T00:32:12.205 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88960.log 2023-12-02T00:32:12.205 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88800.log 2023-12-02T00:32:12.205 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.88960.log: 92.0% -- replaced with /var/log/ceph/ceph.log.gz 2023-12-02T00:32:12.206 INFO:teuthology.orchestra.run.smithi162.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.88960.log.gz 2023-12-02T00:32:12.206 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-12-02T00:32:12.206 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.88800.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-12-02T00:32:12.206 INFO:teuthology.orchestra.run.smithi162.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.88800.log.gz 2023-12-02T00:32:12.209 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-mon.c.log: 91.3% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-12-02T00:32:12.247 INFO:teuthology.orchestra.run.smithi162.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-12-02T00:32:12.297 INFO:teuthology.orchestra.run.smithi162.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-12-02T00:32:12.309 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100639.log 2023-12-02T00:32:12.309 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95715.log 2023-12-02T00:32:12.309 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.100639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99922.log 2023-12-02T00:32:12.309 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.95715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101053.log 2023-12-02T00:32:12.309 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.100639.log.gz 2023-12-02T00:32:12.309 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.99922.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.95715.log.gz 2023-12-02T00:32:12.310 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90502.log 2023-12-02T00:32:12.310 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.99922.log.gz 2023-12-02T00:32:12.310 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.101053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95803.log 2023-12-02T00:32:12.310 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.90502.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-12-02T00:32:12.310 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.101053.log.gz 2023-12-02T00:32:12.310 INFO:teuthology.orchestra.run.smithi046.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.90502.log.gz 2023-12-02T00:32:12.310 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109081.log 2023-12-02T00:32:12.310 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.95803.log: /var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105128.log 2023-12-02T00:32:12.310 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.95803.log.gz 2023-12-02T00:32:12.311 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.109081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108909.log 2023-12-02T00:32:12.311 INFO:teuthology.orchestra.run.smithi046.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.109081.log.gz 2023-12-02T00:32:12.311 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.105128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106234.log 2023-12-02T00:32:12.311 INFO:teuthology.orchestra.run.smithi046.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.105128.log.gz 2023-12-02T00:32:12.311 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.108909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113171.log 2023-12-02T00:32:12.311 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.108909.log.gz 2023-12-02T00:32:12.311 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.106234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107943.log 2023-12-02T00:32:12.311 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.106234.log.gz 2023-12-02T00:32:12.311 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.113171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107733.log 2023-12-02T00:32:12.312 INFO:teuthology.orchestra.run.smithi046.stderr: 89.2%/var/log/ceph/ceph-client.admin.107943.log: -- replaced with /var/log/ceph/ceph-client.admin.113171.log.gz 2023-12-02T00:32:12.312 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108645.log 2023-12-02T00:32:12.312 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.107943.log.gz 2023-12-02T00:32:12.312 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.107733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101263.log 2023-12-02T00:32:12.312 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.108645.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.107733.log.gz 2023-12-02T00:32:12.312 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109215.log 2023-12-02T00:32:12.312 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.101263.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.108645.log.gz 2023-12-02T00:32:12.312 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103597.log 2023-12-02T00:32:12.312 INFO:teuthology.orchestra.run.smithi046.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.101263.log.gz 2023-12-02T00:32:12.313 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.109215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110321.log 2023-12-02T00:32:12.313 INFO:teuthology.orchestra.run.smithi046.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.109215.log.gz 2023-12-02T00:32:12.313 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.103597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103249.log 2023-12-02T00:32:12.313 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.110321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96630.log 2023-12-02T00:32:12.313 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.103597.log.gz 2023-12-02T00:32:12.313 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.110321.log.gz 2023-12-02T00:32:12.313 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.103249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.98651.log 2023-12-02T00:32:12.313 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8%/var/log/ceph/ceph-client.admin.96630.log: -- replaced with /var/log/ceph/ceph-client.admin.103249.log.gz 2023-12-02T00:32:12.313 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108340.log 2023-12-02T00:32:12.313 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.98651.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.96630.log.gz 2023-12-02T00:32:12.314 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100581.log 2023-12-02T00:32:12.314 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.108340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95504.log 2023-12-02T00:32:12.314 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.108340.log.gz 2023-12-02T00:32:12.314 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.100581.log: 90.1% -- replaced with /var/log/ceph/ceph-client.0.98651.log.gz 2023-12-02T00:32:12.314 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114081.log 2023-12-02T00:32:12.314 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.100581.log.gz 2023-12-02T00:32:12.314 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96199.log 2023-12-02T00:32:12.314 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.95504.log: /var/log/ceph/ceph-client.admin.114081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109787.log 2023-12-02T00:32:12.314 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.95504.log.gz 2023-12-02T00:32:12.315 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.114081.log.gz 2023-12-02T00:32:12.315 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111183.log 2023-12-02T00:32:12.315 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.96199.log: /var/log/ceph/ceph-client.admin.109787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96967.log 2023-12-02T00:32:12.315 INFO:teuthology.orchestra.run.smithi046.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.96199.log.gz 2023-12-02T00:32:12.315 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.109787.log.gz 2023-12-02T00:32:12.315 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.111183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106276.log 2023-12-02T00:32:12.315 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.111183.log.gz 2023-12-02T00:32:12.315 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.96967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101935.log 2023-12-02T00:32:12.315 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.106276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110405.log 2023-12-02T00:32:12.316 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.96967.log.gz 2023-12-02T00:32:12.316 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.101935.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.106276.log.gz 2023-12-02T00:32:12.316 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110221.log 2023-12-02T00:32:12.316 INFO:teuthology.orchestra.run.smithi046.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.101935.log.gz 2023-12-02T00:32:12.316 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.110405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97297.log 2023-12-02T00:32:12.316 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.110221.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110405.log.gz 2023-12-02T00:32:12.316 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109679.log 2023-12-02T00:32:12.316 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.97297.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.110221.log.gz 2023-12-02T00:32:12.316 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108171.log 2023-12-02T00:32:12.317 INFO:teuthology.orchestra.run.smithi046.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.97297.log.gz 2023-12-02T00:32:12.317 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.109679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102733.log 2023-12-02T00:32:12.317 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.108171.log: gzip 88.5% -5 -- replaced with /var/log/ceph/ceph-client.admin.109679.log.gz --verbose 2023-12-02T00:32:12.317 INFO:teuthology.orchestra.run.smithi046.stderr: -- /var/log/ceph/ceph-client.admin.104170.log 2023-12-02T00:32:12.317 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.102733.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.108171.log.gz 2023-12-02T00:32:12.317 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99628.log 2023-12-02T00:32:12.317 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.102733.log.gz 2023-12-02T00:32:12.317 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97963.log 2023-12-02T00:32:12.317 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.104170.log: /var/log/ceph/ceph-client.admin.99628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96955.log 2023-12-02T00:32:12.317 INFO:teuthology.orchestra.run.smithi046.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.104170.log.gz 2023-12-02T00:32:12.318 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.97963.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.99628.log.gz 2023-12-02T00:32:12.318 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.104429.log 2023-12-02T00:32:12.318 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.97963.log.gz 2023-12-02T00:32:12.318 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.96955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103123.log 2023-12-02T00:32:12.318 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.104429.log: gzip 87.9% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.96955.log.gz -- 2023-12-02T00:32:12.318 INFO:teuthology.orchestra.run.smithi046.stderr: /var/log/ceph/ceph-client.admin.107775.log 2023-12-02T00:32:12.318 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.103123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101647.log 2023-12-02T00:32:12.318 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.103123.log.gz 2023-12-02T00:32:12.318 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.107775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99418.log 2023-12-02T00:32:12.319 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.107775.log.gz 2023-12-02T00:32:12.319 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.101647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104586.log 2023-12-02T00:32:12.319 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112244.log 2023-12-02T00:32:12.319 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.99418.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.101647.log.gz 2023-12-02T00:32:12.319 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.104586.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.99418.log.gz 2023-12-02T00:32:12.319 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112606.log 2023-12-02T00:32:12.319 INFO:teuthology.orchestra.run.smithi046.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.0.104429.log.gz 2023-12-02T00:32:12.319 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.104586.log.gz 2023-12-02T00:32:12.319 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104944.log 2023-12-02T00:32:12.320 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.112244.log: /var/log/ceph/ceph-client.admin.112606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113787.log 2023-12-02T00:32:12.320 INFO:teuthology.orchestra.run.smithi046.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.112244.log.gz 2023-12-02T00:32:12.320 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.112606.log.gz 2023-12-02T00:32:12.320 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5/var/log/ceph/ceph-client.admin.104944.log: --verbose -- /var/log/ceph/ceph-client.admin.96241.log 2023-12-02T00:32:12.320 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- 87.5% /var/log/ceph/ceph-client.admin.103683.log 2023-12-02T00:32:12.320 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.104944.log.gz 2023-12-02T00:32:12.320 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.113787.log: /var/log/ceph/ceph-client.admin.96241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97305.log 2023-12-02T00:32:12.320 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.113787.log.gz 2023-12-02T00:32:12.320 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.96241.log.gz 2023-12-02T00:32:12.321 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.103683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98131.log 2023-12-02T00:32:12.321 INFO:teuthology.orchestra.run.smithi046.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.103683.log.gz 2023-12-02T00:32:12.321 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96325.log 2023-12-02T00:32:12.321 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.97305.log: /var/log/ceph/ceph-client.admin.98131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107442.log 2023-12-02T00:32:12.321 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.97305.log.gz 2023-12-02T00:32:12.321 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.96325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97304.log 2023-12-02T00:32:12.321 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.96325.log.gz 2023-12-02T00:32:12.321 INFO:teuthology.orchestra.run.smithi046.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.98131.log.gz 2023-12-02T00:32:12.321 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.107442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96283.log 2023-12-02T00:32:12.322 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3%/var/log/ceph/ceph-client.admin.97304.log: -- replaced with /var/log/ceph/ceph-client.admin.107442.log.gz 2023-12-02T00:32:12.322 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99838.log 2023-12-02T00:32:12.322 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.97304.log.gz 2023-12-02T00:32:12.322 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.96283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100723.log 2023-12-02T00:32:12.322 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.99838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102913.log 2023-12-02T00:32:12.322 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.96283.log.gz 2023-12-02T00:32:12.322 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.100723.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.99838.log.gz 2023-12-02T00:32:12.322 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106050.log 2023-12-02T00:32:12.322 INFO:teuthology.orchestra.run.smithi046.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.100723.log.gz 2023-12-02T00:32:12.323 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.102913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.111010.log 2023-12-02T00:32:12.323 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.102913.log.gz 2023-12-02T00:32:12.323 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.106050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111569.log 2023-12-02T00:32:12.323 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.111010.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.106050.log.gz 2023-12-02T00:32:12.323 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104673.log 2023-12-02T00:32:12.323 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.111569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113493.log 2023-12-02T00:32:12.323 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.111569.log.gz 2023-12-02T00:32:12.323 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.104673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112732.log 2023-12-02T00:32:12.323 INFO:teuthology.orchestra.run.smithi046.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.104673.log.gz 2023-12-02T00:32:12.323 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108550.log 2023-12-02T00:32:12.324 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.113493.log: /var/log/ceph/ceph-client.admin.112732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113745.log 2023-12-02T00:32:12.324 INFO:teuthology.orchestra.run.smithi046.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.0.111010.log.gz 2023-12-02T00:32:12.324 INFO:teuthology.orchestra.run.smithi046.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.113493.log.gz 2023-12-02T00:32:12.324 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.108550.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.112732.log.gz 2023-12-02T00:32:12.324 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113367.log 2023-12-02T00:32:12.324 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.113745.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.108550.log.gz 2023-12-02T00:32:12.324 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109553.log 2023-12-02T00:32:12.324 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.113367.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.113745.log.gz 2023-12-02T00:32:12.324 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111731.log 2023-12-02T00:32:12.325 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6%/var/log/ceph/ceph-client.admin.109553.log: -- replaced with /var/log/ceph/ceph-client.admin.113367.log.gz 2023-12-02T00:32:12.325 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99460.log 2023-12-02T00:32:12.325 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.111731.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.109553.log.gz 2023-12-02T00:32:12.325 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95673.log 2023-12-02T00:32:12.325 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.111731.log.gz 2023-12-02T00:32:12.325 INFO:teuthology.orchestra.run.smithi046.stderr:gzip/var/log/ceph/ceph-client.admin.99460.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.102271.log 2023-12-02T00:32:12.325 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.95673.log: gzip 88.6% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.99460.log.gz -- 2023-12-02T00:32:12.325 INFO:teuthology.orchestra.run.smithi046.stderr: /var/log/ceph/ceph-client.admin.107985.log 2023-12-02T00:32:12.325 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4%/var/log/ceph/ceph-client.admin.102271.log: -- replaced with /var/log/ceph/ceph-client.admin.95673.log.gz 2023-12-02T00:32:12.326 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97669.log 2023-12-02T00:32:12.326 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.102271.log.gz 2023-12-02T00:32:12.326 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.107985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99880.log 2023-12-02T00:32:12.326 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.97669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110673.log 2023-12-02T00:32:12.326 INFO:teuthology.orchestra.run.smithi046.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.107985.log.gz 2023-12-02T00:32:12.326 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.97669.log.gz 2023-12-02T00:32:12.326 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.99880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108825.log 2023-12-02T00:32:12.326 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.99880.log.gz 2023-12-02T00:32:12.326 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.110673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.111990.log 2023-12-02T00:32:12.327 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.108825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102019.log 2023-12-02T00:32:12.327 INFO:teuthology.orchestra.run.smithi046.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.110673.log.gz 2023-12-02T00:32:12.327 INFO:teuthology.orchestra.run.smithi046.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.108825.log.gz 2023-12-02T00:32:12.327 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.111990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107174.log 2023-12-02T00:32:12.327 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.102019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96965.log 2023-12-02T00:32:12.327 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.107174.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.102019.log.gz 2023-12-02T00:32:12.327 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96632.log 2023-12-02T00:32:12.327 INFO:teuthology.orchestra.run.smithi046.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.107174.log.gz 2023-12-02T00:32:12.327 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.99143.log 2023-12-02T00:32:12.327 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.96965.log: 90.1% -- replaced with /var/log/ceph/ceph-client.0.111990.log.gz/var/log/ceph/ceph-client.admin.96632.log: 2023-12-02T00:32:12.328 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104860.log 2023-12-02T00:32:12.328 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.96965.log.gz 2023-12-02T00:32:12.328 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.2.99143.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.96632.log.gz 2023-12-02T00:32:12.328 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100090.log 2023-12-02T00:32:12.328 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.104860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106092.log 2023-12-02T00:32:12.328 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.100090.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.104860.log.gz 2023-12-02T00:32:12.328 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105924.log 2023-12-02T00:32:12.328 INFO:teuthology.orchestra.run.smithi046.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.2.99143.log.gz 2023-12-02T00:32:12.328 INFO:teuthology.orchestra.run.smithi046.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.100090.log.gz 2023-12-02T00:32:12.329 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.106092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104062.log 2023-12-02T00:32:12.329 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.106092.log.gz 2023-12-02T00:32:12.329 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.105924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102775.log 2023-12-02T00:32:12.329 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.104062.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.105924.log.gz 2023-12-02T00:32:12.329 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98233.log 2023-12-02T00:32:12.329 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.104062.log.gz 2023-12-02T00:32:12.329 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.102775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110631.log 2023-12-02T00:32:12.329 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.102775.log.gz 2023-12-02T00:32:12.329 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.98233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113451.log 2023-12-02T00:32:12.330 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.110631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100885.log 2023-12-02T00:32:12.330 INFO:teuthology.orchestra.run.smithi046.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.98233.log.gz 2023-12-02T00:32:12.330 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.110631.log.gz 2023-12-02T00:32:12.330 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.113451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102439.log 2023-12-02T00:32:12.330 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.113451.log.gz 2023-12-02T00:32:12.330 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.100885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100455.log 2023-12-02T00:32:12.330 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.102439.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.100885.log.gz 2023-12-02T00:32:12.330 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106318.log 2023-12-02T00:32:12.330 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.102439.log.gz 2023-12-02T00:32:12.331 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.100455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105882.log 2023-12-02T00:32:12.331 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.100455.log.gz 2023-12-02T00:32:12.331 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.106318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105212.log 2023-12-02T00:32:12.331 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.105882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101605.log 2023-12-02T00:32:12.331 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.106318.log.gz 2023-12-02T00:32:12.331 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.105882.log.gz 2023-12-02T00:32:12.331 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.105212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105966.log 2023-12-02T00:32:12.331 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.105212.log.gz 2023-12-02T00:32:12.331 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.101605.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-12-02T00:32:12.332 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.105966.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.101605.log.gz 2023-12-02T00:32:12.332 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103375.log 2023-12-02T00:32:12.332 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.105966.log.gz 2023-12-02T00:32:12.332 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99586.log 2023-12-02T00:32:12.332 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.103375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102187.log 2023-12-02T00:32:12.332 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.99586.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.103375.log.gz 2023-12-02T00:32:12.332 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111367.log 2023-12-02T00:32:12.332 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.99586.log.gz 2023-12-02T00:32:12.332 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114165.log 2023-12-02T00:32:12.332 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.102187.log: /var/log/ceph/ceph-client.admin.111367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99796.log 2023-12-02T00:32:12.333 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.102187.log.gz 2023-12-02T00:32:12.333 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.114165.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.111367.log.gz 2023-12-02T00:32:12.333 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- 89.1% /var/log/ceph/ceph-client.admin.96624.log -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-12-02T00:32:12.333 INFO:teuthology.orchestra.run.smithi046.stderr: 2023-12-02T00:32:12.333 INFO:teuthology.orchestra.run.smithi046.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.114165.log.gz 2023-12-02T00:32:12.333 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113703.log 2023-12-02T00:32:12.333 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.99796.log: /var/log/ceph/ceph-client.admin.96624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107132.log 2023-12-02T00:32:12.333 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.99796.log.gz 2023-12-02T00:32:12.333 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.96624.log.gz/var/log/ceph/ceph-client.admin.113703.log: 2023-12-02T00:32:12.334 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103207.log 2023-12-02T00:32:12.334 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.113703.log.gz 2023-12-02T00:32:12.334 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5/var/log/ceph/ceph-client.admin.107132.log: --verbose -- /var/log/ceph/ceph-client.admin.102523.log 2023-12-02T00:32:12.334 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.103207.log: gzip 88.7% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.107132.log.gz -- 2023-12-02T00:32:12.334 INFO:teuthology.orchestra.run.smithi046.stderr: /var/log/ceph/ceph-client.admin.103852.log 2023-12-02T00:32:12.334 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.102523.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.103207.log.gz 2023-12-02T00:32:12.334 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98047.log 2023-12-02T00:32:12.334 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.102523.log.gz 2023-12-02T00:32:12.334 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109637.log 2023-12-02T00:32:12.335 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.103852.log: /var/log/ceph/ceph-client.admin.98047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96969.log 2023-12-02T00:32:12.335 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.103852.log.gz 2023-12-02T00:32:12.335 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.109637.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.98047.log.gz 2023-12-02T00:32:12.335 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113002.log 2023-12-02T00:32:12.335 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.96969.log: 88.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.109637.log.gz 2023-12-02T00:32:12.335 INFO:teuthology.orchestra.run.smithi046.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.102397.log 2023-12-02T00:32:12.335 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.96969.log.gz 2023-12-02T00:32:12.335 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.113002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107400.log 2023-12-02T00:32:12.335 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.102397.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.113002.log.gz 2023-12-02T00:32:12.336 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103810.log 2023-12-02T00:32:12.336 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.107400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110505.log 2023-12-02T00:32:12.336 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.102397.log.gz 2023-12-02T00:32:12.336 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9%/var/log/ceph/ceph-client.admin.103810.log: -- replaced with /var/log/ceph/ceph-client.admin.107400.log.gz 2023-12-02T00:32:12.336 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107048.log 2023-12-02T00:32:12.336 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.110505.log: gzip 87.9% -5 -- replaced with /var/log/ceph/ceph-client.admin.103810.log.gz --verbose 2023-12-02T00:32:12.336 INFO:teuthology.orchestra.run.smithi046.stderr: -- /var/log/ceph/ceph-client.admin.109343.log 2023-12-02T00:32:12.336 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.110505.log.gz 2023-12-02T00:32:12.336 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.107048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98760.log 2023-12-02T00:32:12.336 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.109343.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.107048.log.gz 2023-12-02T00:32:12.337 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110547.log 2023-12-02T00:32:12.337 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.98760.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.109343.log.gz 2023-12-02T00:32:12.337 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100927.log 2023-12-02T00:32:12.337 INFO:teuthology.orchestra.run.smithi046.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.98760.log.gz 2023-12-02T00:32:12.337 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.110547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113997.log 2023-12-02T00:32:12.337 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.100927.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.110547.log.gz 2023-12-02T00:32:12.337 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102955.log 2023-12-02T00:32:12.337 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.100927.log.gz 2023-12-02T00:32:12.337 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.113997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106360.log 2023-12-02T00:32:12.338 INFO:teuthology.orchestra.run.smithi046.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.113997.log.gz 2023-12-02T00:32:12.338 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.102955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96157.log 2023-12-02T00:32:12.338 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.106360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113087.log 2023-12-02T00:32:12.338 INFO:teuthology.orchestra.run.smithi046.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.102955.log.gz 2023-12-02T00:32:12.338 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.96157.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.106360.log.gz 2023-12-02T00:32:12.338 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111527.log 2023-12-02T00:32:12.338 INFO:teuthology.orchestra.run.smithi046.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.96157.log.gz 2023-12-02T00:32:12.338 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.113087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113045.log 2023-12-02T00:32:12.338 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.111527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97879.log 2023-12-02T00:32:12.339 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.113087.log.gz 2023-12-02T00:32:12.339 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.113045.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.111527.log.gz 2023-12-02T00:32:12.339 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95989.log 2023-12-02T00:32:12.339 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.113045.log.gz 2023-12-02T00:32:12.339 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.97879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100969.log 2023-12-02T00:32:12.339 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.95989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113829.log 2023-12-02T00:32:12.339 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.97879.log.gz 2023-12-02T00:32:12.339 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.100969.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.95989.log.gz 2023-12-02T00:32:12.339 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100006.log 2023-12-02T00:32:12.340 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.100969.log.gz 2023-12-02T00:32:12.340 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.113829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90250.log 2023-12-02T00:32:12.340 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.100006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96409.log 2023-12-02T00:32:12.340 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.113829.log.gz 2023-12-02T00:32:12.340 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.90250.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100006.log.gz 2023-12-02T00:32:12.340 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100807.log 2023-12-02T00:32:12.340 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.90250.log.gz 2023-12-02T00:32:12.340 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.96409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95887.log 2023-12-02T00:32:12.340 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.100807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110363.log 88.4% 2023-12-02T00:32:12.340 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.96409.log.gz 2023-12-02T00:32:12.341 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.100807.log.gz 2023-12-02T00:32:12.341 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.95887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108508.log 2023-12-02T00:32:12.341 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.95887.log.gz 2023-12-02T00:32:12.341 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.110363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98089.log 2023-12-02T00:32:12.341 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.108508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108951.log 2023-12-02T00:32:12.341 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.110363.log.gz 2023-12-02T00:32:12.341 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.98089.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.108508.log.gz 2023-12-02T00:32:12.341 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109721.log 2023-12-02T00:32:12.341 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.98089.log.gz 2023-12-02T00:32:12.342 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95547.log 2023-12-02T00:32:12.342 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.108951.log: /var/log/ceph/ceph-client.admin.109721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111409.log 2023-12-02T00:32:12.342 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.108951.log.gz 2023-12-02T00:32:12.342 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.95547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109259.log 2023-12-02T00:32:12.342 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.95547.log.gz 2023-12-02T00:32:12.342 INFO:teuthology.orchestra.run.smithi046.stderr: 90.9%/var/log/ceph/ceph-client.admin.111409.log: -- replaced with /var/log/ceph/ceph-client.admin.109721.log.gz 2023-12-02T00:32:12.342 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100681.log 2023-12-02T00:32:12.342 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.109259.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.111409.log.gz 2023-12-02T00:32:12.342 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104986.log 2023-12-02T00:32:12.343 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.100681.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.109259.log.gz 2023-12-02T00:32:12.343 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99544.log 2023-12-02T00:32:12.343 INFO:teuthology.orchestra.run.smithi046.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.100681.log.gz 2023-12-02T00:32:12.343 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.104986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112690.log 2023-12-02T00:32:12.343 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.99544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96633.log 2023-12-02T00:32:12.343 INFO:teuthology.orchestra.run.smithi046.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.104986.log.gz 2023-12-02T00:32:12.343 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.112690.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.99544.log.gz 2023-12-02T00:32:12.343 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97295.log 2023-12-02T00:32:12.343 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.112690.log.gz 2023-12-02T00:32:12.344 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.96633.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-12-02T00:32:12.344 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.97295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100318.log 2023-12-02T00:32:12.344 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.96633.log.gz 2023-12-02T00:32:12.344 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-osd.1.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.97295.log.gz 2023-12-02T00:32:12.344 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107859.log 2023-12-02T00:32:12.344 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.100318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103978.log 2023-12-02T00:32:12.344 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.107859.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.100318.log.gz 2023-12-02T00:32:12.344 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111241.log 2023-12-02T00:32:12.344 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97298.log 2023-12-02T00:32:12.344 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.103978.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.107859.log.gz 2023-12-02T00:32:12.345 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.111241.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.103978.log.gz 2023-12-02T00:32:12.345 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.84178.log 2023-12-02T00:32:12.345 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.111241.log.gz 2023-12-02T00:32:12.345 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107901.log 2023-12-02T00:32:12.345 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.97298.log: /var/log/ceph/ceph.tmp-client.admin.84178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108069.log 2023-12-02T00:32:12.345 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.84178.log.gz 2023-12-02T00:32:12.345 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.97298.log.gz 2023-12-02T00:32:12.345 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109595.log 2023-12-02T00:32:12.345 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.107901.log: /var/log/ceph/ceph-client.admin.108069.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.107901.log.gz 2023-12-02T00:32:12.346 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108763.log 2023-12-02T00:32:12.346 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102649.log 2023-12-02T00:32:12.346 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.108069.log.gz 2023-12-02T00:32:12.346 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.109595.log: /var/log/ceph/ceph-client.admin.108763.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.109595.log.gz 2023-12-02T00:32:12.346 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96073.log 2023-12-02T00:32:12.346 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98005.log 2023-12-02T00:32:12.346 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.102649.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.108763.log.gz 2023-12-02T00:32:12.346 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.96073.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.102649.log.gz 2023-12-02T00:32:12.346 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103726.log 2023-12-02T00:32:12.347 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103419.log 2023-12-02T00:32:12.347 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.96073.log.gz 2023-12-02T00:32:12.347 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.98005.log: /var/log/ceph/ceph-client.admin.103726.log: gzip -5 --verbose -- 87.7% /var/log/ceph/ceph-client.admin.111485.log -- replaced with /var/log/ceph/ceph-client.admin.98005.log.gz 2023-12-02T00:32:12.347 INFO:teuthology.orchestra.run.smithi046.stderr: 2023-12-02T00:32:12.347 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101893.log 2023-12-02T00:32:12.347 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.103419.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.103726.log.gz 2023-12-02T00:32:12.347 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.111485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106008.log 2023-12-02T00:32:12.347 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.103419.log.gz 2023-12-02T00:32:12.347 INFO:teuthology.orchestra.run.smithi046.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.111485.log.gz 2023-12-02T00:32:12.348 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.101893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111325.log 2023-12-02T00:32:12.348 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.106008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90621.log 2023-12-02T00:32:12.348 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.101893.log.gz 2023-12-02T00:32:12.348 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.111325.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.106008.log.gz 2023-12-02T00:32:12.348 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108214.log 2023-12-02T00:32:12.348 INFO:teuthology.orchestra.run.smithi046.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.111325.log.gz 2023-12-02T00:32:12.348 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.90621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101095.log 2023-12-02T00:32:12.348 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103463.log 2023-12-02T00:32:12.348 INFO:teuthology.orchestra.run.smithi046.stderr: 88.0%/var/log/ceph/ceph-client.admin.108214.log: -- replaced with /var/log/ceph/ceph-client.admin.90621.log.gz 2023-12-02T00:32:12.349 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.101095.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.108214.log.gz 2023-12-02T00:32:12.349 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105296.log 2023-12-02T00:32:12.349 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.101095.log.gz 2023-12-02T00:32:12.349 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97301.log 2023-12-02T00:32:12.349 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.103463.log: /var/log/ceph/ceph-client.admin.105296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95462.log 2023-12-02T00:32:12.349 INFO:teuthology.orchestra.run.smithi046.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.103463.log.gz 2023-12-02T00:32:12.349 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.105296.log.gz 2023-12-02T00:32:12.350 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.97301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96964.log 2023-12-02T00:32:12.350 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112900.log 2023-12-02T00:32:12.350 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.95462.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.97301.log.gz 2023-12-02T00:32:12.350 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.96964.log: gzip 88.2% -- replaced with /var/log/ceph/ceph-client.admin.95462.log.gz -5 2023-12-02T00:32:12.350 INFO:teuthology.orchestra.run.smithi046.stderr: --verbose -- /var/log/ceph/ceph-client.admin.95285.log 2023-12-02T00:32:12.350 INFO:teuthology.orchestra.run.smithi046.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.96964.log.gz 2023-12-02T00:32:12.350 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101505.log 2023-12-02T00:32:12.350 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.112900.log: /var/log/ceph/ceph-client.admin.95285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105416.log 2023-12-02T00:32:12.350 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.112900.log.gz 2023-12-02T00:32:12.351 INFO:teuthology.orchestra.run.smithi046.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.95285.log.gz 2023-12-02T00:32:12.351 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.101505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112412.log 2023-12-02T00:32:12.351 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97795.log 2023-12-02T00:32:12.351 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.105416.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.101505.log.gz 2023-12-02T00:32:12.351 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.112412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106498.log 2023-12-02T00:32:12.351 INFO:teuthology.orchestra.run.smithi046.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.105416.log.gz 2023-12-02T00:32:12.351 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.112412.log.gz 2023-12-02T00:32:12.351 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.97795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103291.log 2023-12-02T00:32:12.351 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.106498.log: gzip -5 88.7% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.97795.log.gz 2023-12-02T00:32:12.352 INFO:teuthology.orchestra.run.smithi046.stderr: /var/log/ceph/ceph-client.admin.99308.log 2023-12-02T00:32:12.352 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.106498.log.gz 2023-12-02T00:32:12.352 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.103291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114039.log 2023-12-02T00:32:12.352 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95381.log 2023-12-02T00:32:12.352 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.103291.log.gz 2023-12-02T00:32:12.352 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.99308.log: /var/log/ceph/ceph-client.admin.114039.log: 88.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.99308.log.gz 2023-12-02T00:32:12.352 INFO:teuthology.orchestra.run.smithi046.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.101731.log 2023-12-02T00:32:12.352 INFO:teuthology.orchestra.run.smithi046.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.114039.log.gz 2023-12-02T00:32:12.353 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102691.log 2023-12-02T00:32:12.353 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.95381.log: /var/log/ceph/ceph-client.admin.101731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100276.log 2023-12-02T00:32:12.353 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.95381.log.gz 2023-12-02T00:32:12.353 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.101731.log.gz 2023-12-02T00:32:12.353 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.102691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112942.log 2023-12-02T00:32:12.353 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.102691.log.gz 2023-12-02T00:32:12.353 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.100276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99964.log 2023-12-02T00:32:12.353 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.112942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103768.log 2023-12-02T00:32:12.353 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100276.log.gz 2023-12-02T00:32:12.354 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.99964.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.112942.log.gz 2023-12-02T00:32:12.354 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108466.log 2023-12-02T00:32:12.354 INFO:teuthology.orchestra.run.smithi046.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.99964.log.gz 2023-12-02T00:32:12.354 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.103768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97543.log 2023-12-02T00:32:12.354 INFO:teuthology.orchestra.run.smithi046.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.103768.log.gz 2023-12-02T00:32:12.354 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95845.log 2023-12-02T00:32:12.354 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.108466.log: /var/log/ceph/ceph-client.admin.97543.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-12-02T00:32:12.354 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.108466.log.gz 2023-12-02T00:32:12.354 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.95845.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.97543.log.gz 2023-12-02T00:32:12.355 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113535.log 2023-12-02T00:32:12.355 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.95845.log.gz 2023-12-02T00:32:12.355 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100234.log 2023-12-02T00:32:12.355 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-osd.3.log: /var/log/ceph/ceph-client.admin.113535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112816.log 2023-12-02T00:32:12.355 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.100234.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.113535.log.gz 2023-12-02T00:32:12.355 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95631.log 2023-12-02T00:32:12.355 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.100234.log.gz 2023-12-02T00:32:12.355 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.112816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109427.log 2023-12-02T00:32:12.355 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.95631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102103.log 2023-12-02T00:32:12.356 INFO:teuthology.orchestra.run.smithi046.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.112816.log.gz 2023-12-02T00:32:12.356 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.109427.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.95631.log.gz 2023-12-02T00:32:12.356 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103081.log 2023-12-02T00:32:12.356 INFO:teuthology.orchestra.run.smithi046.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.109427.log.gz 2023-12-02T00:32:12.356 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.102103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113661.log 2023-12-02T00:32:12.356 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.103081.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.102103.log.gz 2023-12-02T00:32:12.356 INFO:teuthology.orchestra.run.smithi046.stderr:gzip 87.9% -- replaced with /var/log/ceph/ceph-client.admin.103081.log.gz -5 2023-12-02T00:32:12.356 INFO:teuthology.orchestra.run.smithi046.stderr: --verbose -- /var/log/ceph/ceph-client.admin.101421.log 2023-12-02T00:32:12.356 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.113661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106192.log 2023-12-02T00:32:12.357 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.101421.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.113661.log.gz 2023-12-02T00:32:12.357 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103165.log 2023-12-02T00:32:12.357 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101977.log 2023-12-02T00:32:12.357 INFO:teuthology.orchestra.run.smithi046.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.101421.log.gz 2023-12-02T00:32:12.357 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.106192.log: /var/log/ceph/ceph-client.admin.103165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112648.log 88.0% 2023-12-02T00:32:12.357 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.106192.log.gz 2023-12-02T00:32:12.357 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.101977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100048.log 2023-12-02T00:32:12.357 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.103165.log.gz 2023-12-02T00:32:12.357 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.112648.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.101977.log.gz 2023-12-02T00:32:12.357 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101463.log 2023-12-02T00:32:12.358 INFO:teuthology.orchestra.run.smithi046.stderr: 89.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.112648.log.gz 2023-12-02T00:32:12.358 INFO:teuthology.orchestra.run.smithi046.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.109385.log 2023-12-02T00:32:12.358 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.100048.log: /var/log/ceph/ceph-client.admin.101463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103936.log 2023-12-02T00:32:12.358 INFO:teuthology.orchestra.run.smithi046.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.100048.log.gz 2023-12-02T00:32:12.358 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.109385.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.101463.log.gz 2023-12-02T00:32:12.358 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114123.log 2023-12-02T00:32:12.358 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.103936.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.109385.log.gz 2023-12-02T00:32:12.358 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105338.log 2023-12-02T00:32:12.359 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.114123.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.103936.log.gz 2023-12-02T00:32:12.359 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107316.log 2023-12-02T00:32:12.359 INFO:teuthology.orchestra.run.smithi046.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.114123.log.gz 2023-12-02T00:32:12.359 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.105338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105070.log 2023-12-02T00:32:12.359 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99754.log 2023-12-02T00:32:12.359 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.107316.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.105338.log.gz 2023-12-02T00:32:12.359 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.105070.log: gzip 87.6% -- replaced with /var/log/ceph/ceph-client.admin.107316.log.gz 2023-12-02T00:32:12.359 INFO:teuthology.orchestra.run.smithi046.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.103641.log 2023-12-02T00:32:12.359 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.105070.log.gz 2023-12-02T00:32:12.360 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.99754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114225.log 2023-12-02T00:32:12.360 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.103641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114307.log 2023-12-02T00:32:12.360 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.99754.log.gz 2023-12-02T00:32:12.360 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.114225.log: gzip 87.5% -5 -- replaced with /var/log/ceph/ceph-client.admin.103641.log.gz --verbose 2023-12-02T00:32:12.360 INFO:teuthology.orchestra.run.smithi046.stderr: -- /var/log/ceph/ceph-client.admin.104902.log 2023-12-02T00:32:12.360 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.114307.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.114225.log.gz 2023-12-02T00:32:12.360 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108111.log 2023-12-02T00:32:12.360 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.114307.log.gz 2023-12-02T00:32:12.360 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.104902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101563.log 2023-12-02T00:32:12.361 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96628.log 2023-12-02T00:32:12.361 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.104902.log.gz 2023-12-02T00:32:12.361 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.108111.log: /var/log/ceph/ceph-client.admin.101563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100765.log 2023-12-02T00:32:12.361 INFO:teuthology.orchestra.run.smithi046.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.108111.log.gz 2023-12-02T00:32:12.361 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.96628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.110046.log 2023-12-02T00:32:12.361 INFO:teuthology.orchestra.run.smithi046.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.101563.log.gz 2023-12-02T00:32:12.361 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.96628.log.gz 2023-12-02T00:32:12.361 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.100765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99502.log 2023-12-02T00:32:12.361 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108867.log 2023-12-02T00:32:12.361 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.110046.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100765.log.gz 2023-12-02T00:32:12.362 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.99502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107817.log 2023-12-02T00:32:12.362 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.99502.log.gz 2023-12-02T00:32:12.362 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.108867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112328.log 2023-12-02T00:32:12.362 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 88.8%/var/log/ceph/ceph-client.admin.107817.log: -- replaced with /var/log/ceph/ceph-client.admin.108867.log.gz 2023-12-02T00:32:12.362 INFO:teuthology.orchestra.run.smithi046.stderr: --verbose -- /var/log/ceph/ceph-client.admin.107358.log 2023-12-02T00:32:12.362 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.112328.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.107817.log.gz 2023-12-02T00:32:12.362 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113619.log 2023-12-02T00:32:12.362 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.112328.log.gz 2023-12-02T00:32:12.363 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.107358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112564.log 2023-12-02T00:32:12.363 INFO:teuthology.orchestra.run.smithi046.stderr: 91.7% -- replaced with /var/log/ceph/ceph-client.0.110046.log.gz 2023-12-02T00:32:12.363 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7%/var/log/ceph/ceph-client.admin.113619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106964.log 2023-12-02T00:32:12.363 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.107358.log.gz 2023-12-02T00:32:12.363 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.112564.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.113619.log.gz 2023-12-02T00:32:12.363 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109469.log 2023-12-02T00:32:12.363 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.112564.log.gz 2023-12-02T00:32:12.363 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.106964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102481.log 2023-12-02T00:32:12.363 INFO:teuthology.orchestra.run.smithi046.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.106964.log.gz 2023-12-02T00:32:12.364 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96115.log 2023-12-02T00:32:12.364 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.109469.log: /var/log/ceph/ceph-client.admin.102481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97627.log 2023-12-02T00:32:12.364 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.109469.log.gz 2023-12-02T00:32:12.364 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.96115.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.102481.log.gz 2023-12-02T00:32:12.364 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109037.log 2023-12-02T00:32:12.364 INFO:teuthology.orchestra.run.smithi046.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.96115.log.gz 2023-12-02T00:32:12.364 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.97627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109131.log 2023-12-02T00:32:12.364 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96031.log 2023-12-02T00:32:12.364 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.109037.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.97627.log.gz 2023-12-02T00:32:12.364 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.109131.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-12-02T00:32:12.365 INFO:teuthology.orchestra.run.smithi046.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.109037.log.gz 2023-12-02T00:32:12.365 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.109131.log.gz 2023-12-02T00:32:12.365 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.96031.log: -- /var/log/ceph/ceph-client.admin.112858.log 2023-12-02T00:32:12.365 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113913.log 2023-12-02T00:32:12.365 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.96031.log.gz 2023-12-02T00:32:12.365 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.112858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96962.log 2023-12-02T00:32:12.365 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.113913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101337.log 2023-12-02T00:32:12.365 INFO:teuthology.orchestra.run.smithi046.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.112858.log.gz 2023-12-02T00:32:12.365 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.96962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103555.log 2023-12-02T00:32:12.366 INFO:teuthology.orchestra.run.smithi046.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.113913.log.gz 2023-12-02T00:32:12.366 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 87.4% -- replaced with /var/log/ceph/ceph-client.admin.96962.log.gz/var/log/ceph/ceph-client.admin.101337.log: 2023-12-02T00:32:12.366 INFO:teuthology.orchestra.run.smithi046.stderr: --verbose -- /var/log/ceph/ceph-client.admin.107090.log 2023-12-02T00:32:12.366 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.103555.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.101337.log.gz 2023-12-02T00:32:12.366 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110589.log 2023-12-02T00:32:12.366 INFO:teuthology.orchestra.run.smithi046.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.103555.log.gz 2023-12-02T00:32:12.366 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.107090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108993.log 2023-12-02T00:32:12.366 INFO:teuthology.orchestra.run.smithi046.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.107090.log.gz 2023-12-02T00:32:12.366 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.110589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96626.log 2023-12-02T00:32:12.367 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.110589.log.gz 2023-12-02T00:32:12.367 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.108993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99712.log 2023-12-02T00:32:12.367 INFO:teuthology.orchestra.run.smithi046.stderr:gzip/var/log/ceph/ceph-client.admin.96626.log: -5 89.2% -- replaced with /var/log/ceph/ceph-client.admin.108993.log.gz 2023-12-02T00:32:12.367 INFO:teuthology.orchestra.run.smithi046.stderr: --verbose -- /var/log/ceph/ceph-client.admin.107232.log 2023-12-02T00:32:12.367 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.99712.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.96626.log.gz 2023-12-02T00:32:12.367 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose 88.1% -- replaced with /var/log/ceph/ceph-client.admin.99712.log.gz 2023-12-02T00:32:12.367 INFO:teuthology.orchestra.run.smithi046.stderr: -- /var/log/ceph/ceph-client.admin.103894.log 2023-12-02T00:32:12.367 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.107232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109511.log 2023-12-02T00:32:12.367 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.107232.log.gz 2023-12-02T00:32:12.368 INFO:teuthology.orchestra.run.smithi046.stderr: 93.2% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-12-02T00:32:12.368 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.103894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103333.log 2023-12-02T00:32:12.368 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.103894.log.gz 2023-12-02T00:32:12.368 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99266.log 2023-12-02T00:32:12.368 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.109511.log: /var/log/ceph/ceph-client.admin.103333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113955.log 2023-12-02T00:32:12.368 INFO:teuthology.orchestra.run.smithi046.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.109511.log.gz 2023-12-02T00:32:12.368 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.99266.log: 89.6%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.103333.log.gz 2023-12-02T00:32:12.368 INFO:teuthology.orchestra.run.smithi046.stderr: --verbose -- /var/log/ceph/ceph-client.admin.97921.log 2023-12-02T00:32:12.368 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.113955.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.99266.log.gz 2023-12-02T00:32:12.369 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101851.log 2023-12-02T00:32:12.369 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.113955.log.gz 2023-12-02T00:32:12.369 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112774.log 2023-12-02T00:32:12.369 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.97921.log: /var/log/ceph/ceph-client.admin.101851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100413.log 2023-12-02T00:32:12.369 INFO:teuthology.orchestra.run.smithi046.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.97921.log.gz 2023-12-02T00:32:12.369 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.112774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112165.log 2023-12-02T00:32:12.369 INFO:teuthology.orchestra.run.smithi046.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.101851.log.gz 2023-12-02T00:32:12.369 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.112774.log.gz 2023-12-02T00:32:12.369 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.100413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99670.log 2023-12-02T00:32:12.370 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100413.log.gz 2023-12-02T00:32:12.370 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108424.log 2023-12-02T00:32:12.370 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.112165.log: /var/log/ceph/ceph-client.admin.99670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95759.log 2023-12-02T00:32:12.370 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.112165.log.gz 2023-12-02T00:32:12.370 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.99670.log.gz 2023-12-02T00:32:12.370 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.108424.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-12-02T00:32:12.370 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107006.log 2023-12-02T00:32:12.370 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.95759.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.108424.log.gz 2023-12-02T00:32:12.370 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101809.log 2023-12-02T00:32:12.371 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.95759.log.gz 2023-12-02T00:32:12.371 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.107006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102997.log 2023-12-02T00:32:12.371 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.101809.log: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105170.log 2023-12-02T00:32:12.371 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.107006.log.gz 2023-12-02T00:32:12.371 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.102997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111611.log 2023-12-02T00:32:12.371 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.101809.log.gz 2023-12-02T00:32:12.371 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103039.log 2023-12-02T00:32:12.371 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.105170.log: -- replaced with /var/log/ceph/ceph-client.admin.102997.log.gz 2023-12-02T00:32:12.371 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.111611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113325.log 2023-12-02T00:32:12.372 INFO:teuthology.orchestra.run.smithi046.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.105170.log.gz 2023-12-02T00:32:12.372 INFO:teuthology.orchestra.run.smithi046.stderr:gzip 88.9%/var/log/ceph/ceph-client.admin.103039.log: -- replaced with /var/log/ceph/ceph-client.admin.111611.log.gz 2023-12-02T00:32:12.372 INFO:teuthology.orchestra.run.smithi046.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.104801.log 2023-12-02T00:32:12.372 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.113325.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.103039.log.gz 2023-12-02T00:32:12.372 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103513.log 2023-12-02T00:32:12.372 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.113325.log.gz 2023-12-02T00:32:12.372 INFO:teuthology.orchestra.run.smithi046.stderr:gzip/var/log/ceph/ceph-client.admin.104801.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.108256.log 2023-12-02T00:32:12.372 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.103513.log: gzip 88.8% -- replaced with /var/log/ceph/ceph-client.admin.104801.log.gz 2023-12-02T00:32:12.372 INFO:teuthology.orchestra.run.smithi046.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.111653.log 2023-12-02T00:32:12.373 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.108256.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.103513.log.gz 2023-12-02T00:32:12.373 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106150.log 2023-12-02T00:32:12.373 INFO:teuthology.orchestra.run.smithi046.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.108256.log.gz/var/log/ceph/ceph-client.admin.111653.log: 2023-12-02T00:32:12.373 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102607.log 2023-12-02T00:32:12.373 INFO:teuthology.orchestra.run.smithi046.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.111653.log.gz 2023-12-02T00:32:12.373 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.106150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113577.log 2023-12-02T00:32:12.373 INFO:teuthology.orchestra.run.smithi046.stderr:gzip 88.7% -- replaced with /var/log/ceph/ceph-client.admin.106150.log.gz 2023-12-02T00:32:12.373 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.102607.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.97501.log 2023-12-02T00:32:12.374 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.113577.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.102607.log.gz 2023-12-02T00:32:12.374 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96367.log 2023-12-02T00:32:12.374 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.113577.log.gz 2023-12-02T00:32:12.374 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.97501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104104.log 2023-12-02T00:32:12.374 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.96367.log: gzip 88.8% -- replaced with /var/log/ceph/ceph-client.admin.97501.log.gz 2023-12-02T00:32:12.374 INFO:teuthology.orchestra.run.smithi046.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.105254.log 2023-12-02T00:32:12.374 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.104104.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.96367.log.gz 2023-12-02T00:32:12.374 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100192.log 2023-12-02T00:32:12.374 INFO:teuthology.orchestra.run.smithi046.stderr:gzip/var/log/ceph/ceph-client.admin.105254.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.101689.log 2023-12-02T00:32:12.375 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.100192.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.104104.log.gz 2023-12-02T00:32:12.375 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97165.log 2023-12-02T00:32:12.375 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.105254.log.gz 2023-12-02T00:32:12.375 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5/var/log/ceph/ceph-client.admin.101689.log: --verbose 88.2% -- replaced with /var/log/ceph/ceph-client.admin.100192.log.gz 2023-12-02T00:32:12.375 INFO:teuthology.orchestra.run.smithi046.stderr: -- /var/log/ceph/ceph-client.admin.102061.log 2023-12-02T00:32:12.375 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.97165.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.101689.log.gz 2023-12-02T00:32:12.375 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97585.log 88.4% -- replaced with /var/log/ceph/ceph-client.admin.97165.log.gz 2023-12-02T00:32:12.375 INFO:teuthology.orchestra.run.smithi046.stderr: 2023-12-02T00:32:12.375 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.102061.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-12-02T00:32:12.375 INFO:teuthology.orchestra.run.smithi046.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.102061.log.gz 2023-12-02T00:32:12.376 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.97585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107520.log 2023-12-02T00:32:12.376 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.97585.log.gz 2023-12-02T00:32:12.376 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101011.log 2023-12-02T00:32:12.376 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.107520.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.107520.log.gz 2023-12-02T00:32:12.376 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101221.log 2023-12-02T00:32:12.376 INFO:teuthology.orchestra.run.smithi046.stderr: 85.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95947.log 2023-12-02T00:32:12.376 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph.log.gz 2023-12-02T00:32:12.376 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.101011.log: /var/log/ceph/ceph-client.admin.101221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112522.log 2023-12-02T00:32:12.376 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.101011.log.gz 2023-12-02T00:32:12.377 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96629.log 2023-12-02T00:32:12.377 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.101221.log.gz 2023-12-02T00:32:12.377 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.95947.log: /var/log/ceph/ceph-client.admin.112522.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-12-02T00:32:12.377 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.95947.log.gz 2023-12-02T00:32:12.377 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.112522.log.gz 2023-12-02T00:32:12.377 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.96629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96963.log 2023-12-02T00:32:12.377 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.96629.log.gz 2023-12-02T00:32:12.377 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102145.log 2023-12-02T00:32:12.377 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.96963.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-12-02T00:32:12.377 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.96963.log.gz 2023-12-02T00:32:12.378 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.102145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102565.log 2023-12-02T00:32:12.378 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.102145.log.gz 2023-12-02T00:32:12.378 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105028.log 2023-12-02T00:32:12.378 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.102565.log: -- /var/log/ceph/ceph-client.admin.113409.log 2023-12-02T00:32:12.378 INFO:teuthology.orchestra.run.smithi046.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.102565.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101179.log 2023-12-02T00:32:12.378 INFO:teuthology.orchestra.run.smithi046.stderr: 2023-12-02T00:32:12.378 INFO:teuthology.orchestra.run.smithi046.stderr:gzip 92.5% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-12-02T00:32:12.378 INFO:teuthology.orchestra.run.smithi046.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.109301.log 2023-12-02T00:32:12.378 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.113409.log: /var/log/ceph/ceph-client.admin.101179.log: /var/log/ceph/ceph-client.admin.105028.log: 88.8% 88.6% -- replaced with /var/log/ceph/ceph-client.admin.101179.log.gz 2023-12-02T00:32:12.379 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.113409.log.gz 2023-12-02T00:32:12.379 INFO:teuthology.orchestra.run.smithi046.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.105028.log.gz 2023-12-02T00:32:12.379 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112370.log 2023-12-02T00:32:12.379 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.109301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114265.log 2023-12-02T00:32:12.379 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.112370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106456.log 2023-12-02T00:32:12.379 INFO:teuthology.orchestra.run.smithi046.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.109301.log.gz 2023-12-02T00:32:12.379 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4%/var/log/ceph/ceph-client.admin.114265.log: -- replaced with /var/log/ceph/ceph-client.admin.112370.log.gz 2023-12-02T00:32:12.379 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- 88.7% -- replaced with /var/log/ceph/ceph-client.admin.114265.log.gz 2023-12-02T00:32:12.379 INFO:teuthology.orchestra.run.smithi046.stderr: /var/log/ceph/ceph-client.admin.96968.log 2023-12-02T00:32:12.380 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.106456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113871.log 2023-12-02T00:32:12.380 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.106456.log.gz 2023-12-02T00:32:12.380 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.96968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97753.log 2023-12-02T00:32:12.380 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.96968.log.gz 2023-12-02T00:32:12.380 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97288.log 2023-12-02T00:32:12.380 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.113871.log: /var/log/ceph/ceph-client.admin.97753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110751.log 2023-12-02T00:32:12.380 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.97288.log: 88.3% 88.0% -- replaced with /var/log/ceph/ceph-client.admin.113871.log.gz 2023-12-02T00:32:12.380 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.97753.log.gz 2023-12-02T00:32:12.380 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.97288.log.gz 2023-12-02T00:32:12.381 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102229.log 2023-12-02T00:32:12.381 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100539.log 2023-12-02T00:32:12.381 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.110751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107649.log 2023-12-02T00:32:12.381 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.102229.log: 88.3% 88.1% -- replaced with /var/log/ceph/ceph-client.admin.110751.log.gz 2023-12-02T00:32:12.381 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.100539.log: -- replaced with /var/log/ceph/ceph-client.admin.102229.log.gz 2023-12-02T00:32:12.381 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 89.5% --verbose -- /var/log/ceph/ceph-client.admin.101379.log 2023-12-02T00:32:12.381 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.100539.log.gz 2023-12-02T00:32:12.381 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.107649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110279.log 2023-12-02T00:32:12.381 INFO:teuthology.orchestra.run.smithi046.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.107649.log.gz/var/log/ceph/ceph-client.admin.101379.log: 2023-12-02T00:32:12.382 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100132.log 2023-12-02T00:32:12.382 INFO:teuthology.orchestra.run.smithi046.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.101379.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111283.log 2023-12-02T00:32:12.382 INFO:teuthology.orchestra.run.smithi046.stderr: 2023-12-02T00:32:12.382 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.110279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.105675.log 2023-12-02T00:32:12.382 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.100132.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.110279.log.gz 2023-12-02T00:32:12.382 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.111283.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.100132.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101137.log 88.0% -- replaced with /var/log/ceph/ceph-client.admin.111283.log.gz 2023-12-02T00:32:12.382 INFO:teuthology.orchestra.run.smithi046.stderr: 2023-12-02T00:32:12.382 INFO:teuthology.orchestra.run.smithi046.stderr: 2023-12-02T00:32:12.382 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.105675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104630.log 2023-12-02T00:32:12.383 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.101137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96451.log 2023-12-02T00:32:12.383 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.101137.log.gz 2023-12-02T00:32:12.383 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.104630.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.104630.log.gz 2023-12-02T00:32:12.383 INFO:teuthology.orchestra.run.smithi046.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.0.105675.log.gz 2023-12-02T00:32:12.383 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95204.log 2023-12-02T00:32:12.383 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104718.log 2023-12-02T00:32:12.383 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.96451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109173.log 2023-12-02T00:32:12.383 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.95204.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.96451.log.gz 89.4% -- replaced with /var/log/ceph/ceph-client.admin.95204.log.gz 2023-12-02T00:32:12.383 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.104718.log: 2023-12-02T00:32:12.384 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97837.log 2023-12-02T00:32:12.384 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.104718.log.gz 2023-12-02T00:32:12.384 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.109173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107274.log 2023-12-02T00:32:12.384 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108382.log 2023-12-02T00:32:12.384 INFO:teuthology.orchestra.run.smithi046.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.109173.log.gz 2023-12-02T00:32:12.384 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.97837.log: /var/log/ceph/ceph-client.admin.107274.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.97837.log.gz 2023-12-02T00:32:12.384 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.107274.log.gz 2023-12-02T00:32:12.384 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.106757.log 2023-12-02T00:32:12.384 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.108382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96625.log 2023-12-02T00:32:12.384 INFO:teuthology.orchestra.run.smithi046.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.108382.log.gz/var/log/ceph/ceph-client.0.106757.log: 2023-12-02T00:32:12.385 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113129.log 2023-12-02T00:32:12.385 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.96625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100497.log 2023-12-02T00:32:12.385 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.96625.log.gz 2023-12-02T00:32:12.385 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.113129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107691.log 2023-12-02T00:32:12.385 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.113129.log.gz 2023-12-02T00:32:12.385 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.100497.log: 91.3% -- replaced with /var/log/ceph/ceph-client.0.106757.log.gz 2023-12-02T00:32:12.385 INFO:teuthology.orchestra.run.smithi046.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.100497.log.gz 2023-12-02T00:32:12.385 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110447.log 2023-12-02T00:32:12.385 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.107691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108298.log 2023-12-02T00:32:12.386 INFO:teuthology.orchestra.run.smithi046.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.107691.log.gz 2023-12-02T00:32:12.386 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.110447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102355.log 2023-12-02T00:32:12.386 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.110447.log.gz/var/log/ceph/ceph-client.admin.108298.log: 2023-12-02T00:32:12.386 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112286.log 2023-12-02T00:32:12.386 INFO:teuthology.orchestra.run.smithi046.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.108298.log.gz 2023-12-02T00:32:12.386 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102313.log 2023-12-02T00:32:12.386 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.102355.log: /var/log/ceph/ceph-client.admin.112286.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-12-02T00:32:12.386 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.102313.log: 88.6% 89.1% -- replaced with /var/log/ceph/ceph-client.admin.102355.log.gz 2023-12-02T00:32:12.387 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.112286.log.gz 2023-12-02T00:32:12.387 INFO:teuthology.orchestra.run.smithi046.stderr:gzip 89.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.102313.log.gz --verbose 2023-12-02T00:32:12.387 INFO:teuthology.orchestra.run.smithi046.stderr: -- /var/log/ceph/ceph-client.admin.95589.log 2023-12-02T00:32:12.387 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108027.log 2023-12-02T00:32:12.387 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.95589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97711.log 2023-12-02T00:32:12.387 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.108027.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.95589.log.gz 2023-12-02T00:32:12.387 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104020.log 2023-12-02T00:32:12.387 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.108027.log.gz 2023-12-02T00:32:12.388 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.97711.log: /var/log/ceph/ceph-client.admin.104020.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.97711.log.gz 2023-12-02T00:32:12.388 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.104020.log.gz 2023-12-02T00:32:12.389 INFO:teuthology.orchestra.run.smithi046.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-12-02T00:32:12.389 INFO:teuthology.orchestra.run.smithi046.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-12-02T00:32:12.389 INFO:teuthology.orchestra.run.smithi046.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-12-02T00:32:12.390 INFO:teuthology.orchestra.run.smithi162.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-12-02T00:32:12.390 INFO:teuthology.orchestra.run.smithi162.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-12-02T00:32:12.390 INFO:teuthology.orchestra.run.smithi162.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-12-02T00:32:12.390 INFO:teuthology.orchestra.run.smithi162.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-12-02T00:32:12.390 INFO:teuthology.orchestra.run.smithi162.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-12-02T00:32:12.391 INFO:teuthology.orchestra.run.smithi162.stderr: 2023-12-02T00:32:12.391 INFO:teuthology.orchestra.run.smithi162.stderr:real 0m0.219s 2023-12-02T00:32:12.391 INFO:teuthology.orchestra.run.smithi162.stderr:user 0m1.136s 2023-12-02T00:32:12.391 INFO:teuthology.orchestra.run.smithi162.stderr:sys 0m0.070s 2023-12-02T00:32:12.427 INFO:teuthology.orchestra.run.smithi046.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-12-02T00:32:12.453 INFO:teuthology.orchestra.run.smithi046.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-12-02T00:32:12.486 INFO:teuthology.orchestra.run.smithi046.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-12-02T00:32:12.487 INFO:teuthology.orchestra.run.smithi046.stderr: 2023-12-02T00:32:12.487 INFO:teuthology.orchestra.run.smithi046.stderr:real 0m0.353s 2023-12-02T00:32:12.487 INFO:teuthology.orchestra.run.smithi046.stderr:user 0m1.251s 2023-12-02T00:32:12.487 INFO:teuthology.orchestra.run.smithi046.stderr:sys 0m0.268s 2023-12-02T00:32:12.488 INFO:tasks.ceph:Archiving logs... 2023-12-02T00:32:12.488 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-12-01_17:57:53-fs-main-distro-default-smithi/7474581/remote/smithi046/log 2023-12-02T00:32:12.489 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /var/log/ceph -- . 2023-12-02T00:32:13.006 DEBUG:teuthology.misc:Transferring archived files from smithi162:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-12-01_17:57:53-fs-main-distro-default-smithi/7474581/remote/smithi162/log 2023-12-02T00:32:13.006 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar c -f - -C /var/log/ceph -- . 2023-12-02T00:32:13.176 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-12-02T00:32:13.205 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-12-02T00:32:13.206 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-12-02T00:32:13.209 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-12-02T00:32:13.290 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-12-02T00:32:13.303 INFO:teuthology.task.clock:Checking final clock skew... 2023-12-02T00:32:13.303 DEBUG:teuthology.orchestra.run.smithi046:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-02T00:32:13.308 DEBUG:teuthology.orchestra.run.smithi162:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-02T00:32:13.321 INFO:teuthology.orchestra.run.smithi046.stderr:bash: line 1: ntpq: command not found 2023-12-02T00:32:13.324 INFO:teuthology.orchestra.run.smithi162.stderr:bash: line 1: ntpq: command not found 2023-12-02T00:32:13.325 INFO:teuthology.orchestra.run.smithi046.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-02T00:32:13.325 INFO:teuthology.orchestra.run.smithi046.stdout:=============================================================================== 2023-12-02T00:32:13.325 INFO:teuthology.orchestra.run.smithi046.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 35 +317us[ +317us] +/- 100ms 2023-12-02T00:32:13.325 INFO:teuthology.orchestra.run.smithi046.stdout:^* hv02.front.sepia.ceph.com 3 6 377 38 -188us[ -177us] +/- 57ms 2023-12-02T00:32:13.325 INFO:teuthology.orchestra.run.smithi046.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 38 +6379ns[+6379ns] +/- 60ms 2023-12-02T00:32:13.325 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-12-02T00:32:13.327 INFO:teuthology.orchestra.run.smithi162.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-02T00:32:13.327 INFO:teuthology.orchestra.run.smithi162.stdout:=============================================================================== 2023-12-02T00:32:13.327 INFO:teuthology.orchestra.run.smithi162.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 35 +320us[ +320us] +/- 100ms 2023-12-02T00:32:13.327 INFO:teuthology.orchestra.run.smithi162.stdout:^* hv02.front.sepia.ceph.com 3 6 377 37 -165us[ -136us] +/- 57ms 2023-12-02T00:32:13.327 INFO:teuthology.orchestra.run.smithi162.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 36 +15us[ +15us] +/- 60ms 2023-12-02T00:32:13.327 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-12-02T00:32:13.328 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-12-02T00:32:13.339 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-12-02T00:32:13.340 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-12-02T00:32:13.352 DEBUG:teuthology.orchestra.run.smithi046:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-12-02T00:32:13.484 DEBUG:teuthology.orchestra.run.smithi162:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-12-02T00:32:13.579 DEBUG:teuthology.orchestra.run.smithi046:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-02T00:32:13.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-02T00:32:13.616 DEBUG:teuthology.orchestra.run.smithi162:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-02T00:32:13.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-02T00:32:13.648 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-12-02T00:32:13.658 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-12-02T00:32:13.669 INFO:teuthology.task.internal:Duration was 731.677441 seconds 2023-12-02T00:32:13.741 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-12-02T00:32:13.751 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-12-02T00:32:13.751 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-12-02T00:32:13.754 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-12-02T00:32:13.797 INFO:teuthology.orchestra.run.smithi162.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-02T00:32:13.802 INFO:teuthology.orchestra.run.smithi046.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-02T00:32:14.195 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-12-02T00:32:14.196 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi046.front.sepia.ceph.com 2023-12-02T00:32:14.196 DEBUG:teuthology.orchestra.run.smithi046:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-12-02T00:32:14.217 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi162.front.sepia.ceph.com 2023-12-02T00:32:14.217 DEBUG:teuthology.orchestra.run.smithi162:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-12-02T00:32:14.262 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-12-02T00:32:14.263 DEBUG:teuthology.orchestra.run.smithi046:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-02T00:32:14.265 DEBUG:teuthology.orchestra.run.smithi162:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-02T00:32:14.332 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-12-02T00:32:14.332 DEBUG:teuthology.orchestra.run.smithi046:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-12-02T00:32:14.338 DEBUG:teuthology.orchestra.run.smithi162:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-12-02T00:32:14.467 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-12-02T00:32:14.477 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-12-02T00:32:14.477 DEBUG:teuthology.orchestra.run.smithi046:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-12-02T00:32:14.508 DEBUG:teuthology.orchestra.run.smithi162:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-12-02T00:32:14.536 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-12-02T00:32:14.547 DEBUG:teuthology.orchestra.run.smithi046:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-12-02T00:32:14.551 DEBUG:teuthology.orchestra.run.smithi162:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-12-02T00:32:14.578 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern = core 2023-12-02T00:32:14.602 INFO:teuthology.orchestra.run.smithi162.stdout:kernel.core_pattern = core 2023-12-02T00:32:14.618 DEBUG:teuthology.orchestra.run.smithi046:> test -e /home/ubuntu/cephtest/archive/coredump 2023-12-02T00:32:14.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-02T00:32:14.653 DEBUG:teuthology.orchestra.run.smithi162:> test -e /home/ubuntu/cephtest/archive/coredump 2023-12-02T00:32:14.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-02T00:32:14.671 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-12-02T00:32:14.682 INFO:teuthology.task.internal:Transferring archived files... 2023-12-02T00:32:14.683 DEBUG:teuthology.misc:Transferring archived files from smithi046:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-12-01_17:57:53-fs-main-distro-default-smithi/7474581/remote/smithi046 2023-12-02T00:32:14.683 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-12-02T00:32:14.733 DEBUG:teuthology.misc:Transferring archived files from smithi162:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-12-01_17:57:53-fs-main-distro-default-smithi/7474581/remote/smithi162 2023-12-02T00:32:14.734 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-12-02T00:32:14.769 INFO:teuthology.task.internal:Removing archive directory... 2023-12-02T00:32:14.769 DEBUG:teuthology.orchestra.run.smithi046:> rm -rf -- /home/ubuntu/cephtest/archive 2023-12-02T00:32:14.772 DEBUG:teuthology.orchestra.run.smithi162:> rm -rf -- /home/ubuntu/cephtest/archive 2023-12-02T00:32:14.819 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-12-02T00:32:14.832 INFO:teuthology.task.internal:Not uploading archives. 2023-12-02T00:32:14.832 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-12-02T00:32:14.842 INFO:teuthology.task.internal:Tidying up after the test... 2023-12-02T00:32:14.842 DEBUG:teuthology.orchestra.run.smithi046:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-12-02T00:32:14.845 DEBUG:teuthology.orchestra.run.smithi162:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-12-02T00:32:14.857 INFO:teuthology.orchestra.run.smithi046.stdout: 265244 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 2 00:32 /home/ubuntu/cephtest 2023-12-02T00:32:14.873 INFO:teuthology.orchestra.run.smithi162.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 2 00:32 /home/ubuntu/cephtest 2023-12-02T00:32:14.875 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-12-02T00:32:14.885 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-12-02T00:32:14.905 INFO:teuthology.nuke:Checking targets against current locks 2023-12-02T00:32:14.932 DEBUG:teuthology.nuke:shortname: smithi046 2023-12-02T00:32:14.932 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-02T00:32:14.956 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi046.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-12-01_17:57:53-fs-main-distro-default-smithi/7474581', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-02 00:11:05.202741', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKZy/FA+EREo0z8VJi9Rp6/BsV8NFlDNytLV8JJkAdXZb9ZW5B/l6lq8qWNioT2ApsmOClzljsrv9/+hKUss4Ag='} 2023-12-02T00:32:14.960 DEBUG:teuthology.nuke:shortname: smithi162 2023-12-02T00:32:14.960 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-02T00:32:14.990 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi162.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-12-01_17:57:53-fs-main-distro-default-smithi/7474581', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-02 00:11:05.204051', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD+84dNHyVg7BDh7SUQ8Op99UGvFszg9wn8IU0wSp3Htbhuzt5b5g5WRjXDiOjAh4yuQB30H0+XZrpmNFRD4KmQ='} 2023-12-02T00:32:15.018 INFO:teuthology.orchestra.console.smithi046:Power off 2023-12-02T00:32:15.019 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-02T00:32:15.044 DEBUG:teuthology.orchestra.console.smithi046:power off output: Chassis Power Control: Down/Off 2023-12-02T00:32:15.044 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-02T00:32:15.062 INFO:teuthology.orchestra.console.smithi162:Power off 2023-12-02T00:32:15.063 DEBUG:teuthology.orchestra.console.smithi162:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-02T00:32:15.076 DEBUG:teuthology.orchestra.console.smithi046:check power output: Chassis Power is on 2023-12-02T00:32:15.091 DEBUG:teuthology.orchestra.console.smithi162:power off output: Chassis Power Control: Down/Off 2023-12-02T00:32:15.092 DEBUG:teuthology.orchestra.console.smithi162:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-02T00:32:15.118 DEBUG:teuthology.orchestra.console.smithi162:check power output: Chassis Power is on 2023-12-02T00:32:19.078 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-02T00:32:19.119 DEBUG:teuthology.orchestra.console.smithi162:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-02T00:32:19.188 DEBUG:teuthology.orchestra.console.smithi046:check power output: Chassis Power is on 2023-12-02T00:32:19.233 DEBUG:teuthology.orchestra.console.smithi162:check power output: Chassis Power is on 2023-12-02T00:32:23.190 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-02T00:32:23.234 DEBUG:teuthology.orchestra.console.smithi162:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-02T00:32:23.300 DEBUG:teuthology.orchestra.console.smithi046:check power output: Chassis Power is off 2023-12-02T00:32:23.344 DEBUG:teuthology.orchestra.console.smithi162:check power output: Chassis Power is on 2023-12-02T00:32:23.402 INFO:teuthology.orchestra.console.smithi046:Power off completed 2023-12-02T00:32:27.346 DEBUG:teuthology.orchestra.console.smithi162:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-02T00:32:27.456 DEBUG:teuthology.orchestra.console.smithi162:check power output: Chassis Power is off 2023-12-02T00:32:27.557 INFO:teuthology.orchestra.console.smithi162:Power off completed 2023-12-02T00:32:27.661 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/pool-perm} duration: 731.6774413585663 failure_reason: 'Test failure: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm)' flavor: default owner: scheduled_yuriw@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=4c1c48699b5b4fe6b722addac570329c status: fail success: false 2023-12-02T00:32:27.662 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-02T00:32:27.745 INFO:teuthology.run:FAIL